Proceedings of the CLTW 4 @ LREC2022 , pages 127–132 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 127 AAC don Ghaeilge: the Prototype Development of Speech-Generating Assistive Technology for Irish Emily Barnes, Oisín Morrin, Ailbhe Ní Chasaide, Julia Cummins, Harald Berthelsen, Andrew Murphy, Muireann Nic Corcráin, Claire O’ Neill, Christer Gobl, Neasa Ní Chiaráin Trinity College Dublin The Phonetics and Speech Laboratory ebarnes@tcd.ie, anichsid@tcd.ie, nichiarn@tcd.ie. Abstract This paper describes the prototype development of an Alternative and Augmentative Communication (AAC) system for the Irish language. This system allows users to communicate using the ABAIR synthetic voices, by selecting a series of words or images. Similar systems are widely available in English and are often used by autistic people, as well as by people with Cerebral Palsy, Alzheimer’s and Parkinson’s disease. A dual-pronged approach to development has been adopted: this involves (i) the initial short-term prototype development that targets the immediate needs of specific users, as well as considerations for (ii) the longer term development of a bilingual AAC system which will suit a broader range of users with varying linguistic backgrounds, age ranges and needs. This paper described the design considerations and the implementation steps in the current system. Given the substantial differences in linguistic structures in Irish and English, the development of a bilingual system raises many research questions and avenues for future development. Keywords: AAC, assistive technology, speech synthesis 1. Introduction The world we inhabit is largely designed to suit neurotypical and able-bodied people, often resulting in the disable-ing of those who think, learn or move differently. Opportunities – linguistic, educational, social and otherwise – of neurodivergent people are frequently curtailed. In the Irish educational context, it is not unusual to hear of professionals recommending that children speak only English at home, or that they not attend Irish immersion education. Such recommendations conflict with the concept of inclusion and are typically based on personal beliefs, rather than research and information pertaining to the person (Wight, 2015). Fortunately, attitudes towards bilingualism are changing, and neurodivergent people now make up a substantial proportion of those accessing Irish immersion education (Nic Aindriú, Ó Duibhir & Travers, 2020). Without support, however, access does not equate to opportunity (Engstrom & Tinto, 2008). Support in education takes many forms; it includes the ethos of the school, the types of teaching strategies used as well as the practical concern of assistive technology. Assistive technology can be transformative in the lives of those who use it, removing barriers to communication and education. While there is plentiful provision of assistive technology for the English language, there is little available for the Irish language (though see Section 2). As a matter of equality of opportunity for those in Irish-medium education – and particularly for those who speak Irish in their communities as a first language – it is paramount that this discrepancy be addressed. The present paper describes the prototype development of an Alternative and Augmentative Communication (AAC) system for the Irish language. AAC systems range from simple, paper-based ones to high tech speech-generating ones. The AAC system described in this paper falls into the latter category. It is a system – typically presented on a tablet – which allows the user to select a series of words/images to compose a sentence which is then read out by a synthetic voice. People use AAC systems for a variety of reasons; many autistic children and adults use AAC to communicate (Enderby et al, 2013), and people with Cerebral Palsy, Alzheimer’s and Parkinson’s Disease often use AAC to overcome communication challenges (Enderby et al, 2013). Speech-generating AAC systems contain a large number of boards, one of which is illustrated in Figure 1. Each board comprises a number of buttons representing words or phrases which are typically linked semantically. Each button contains an image which symbolises the word, as well the orthographic form of the word. As the user selects the sequence of images/words, they appear in the bar at the top of the board. The individual words at this point do not carry the grammatical inflections, which are added when the sentence is synthesised. The development of an Irish language facility within an AAC system requires expertise in a number of areas. An understanding of Irish semantics, syntax and morphology is necessary to identify issues and design solutions; knowledge and experience of AAC use and the practices of AAC users is paramount to ensure solutions are appropriate; technical expertise is necessary in order to implement the solutions. This interdisciplinarity is reflected in the number of authors who have contributed to this paper in some way or another. mailto:ebarnes@tcd.ie mailto:anichsid@tcd.ie mailto:nichiarn@tcd.ie 128 Figure 1. Image of a board in Coughdrop This development is taking place as part of the ABAIR Initiative, described in Section 2. The linguistic structures of Irish give rise to some particular challenges in AAC development, discussed in Section 3. Subsequently, the design decisions made for the initial, short-term prototype development of the AAC system are described in Section 4. The next stage of the development is outlined in Section 5, including the research questions which will be addressed and the technical work which is outstanding. 2. The ABAIR Initiative and AAC The design and development of the AAC system described in this paper is being undertaken under the umbrella of the ABAIR Initiative. ABAIR is a suite of projects which focus on the development of speech and language technologies for the Irish language (see Ní Chasaide et al, 2019 for an overview). All of the technologies developed as part of the ABAIR Project are underpinned by basic research and by linguistic resources developed by the team. The main core technologies of the ABAIR Initiative are the synthetic voices which have been developed for each of the main dialects of Irish; these voices are a key component of the AAC system described here. Automatic Speech Recognition (ASR) is currently under development, and the current Irish ASR prototype is presented in an accompanying paper. Since the earliest days of the ABAIR initiative, the needs of the language community – and particularly of disabled members of the community – were a primary catalyst for developments. Thus, the present project expands on previous work undertaken as part of the ABAIR Initiative in the area of assistive technology and access. A plugin for the Nonvisual Desktop Access (NVDA) screenreader was developed for visually-impaired people – and involved a visually-impaired researcher working with the ABAIR 1 Names have been changed to protect the children’s identities. group – in a collaboration with the National Council for the Blind in Ireland (NCBI) (McGuirk, 2015). In addition, basic research has been undertaken in the area of dyslexia assessments and literacy training (Barnes 2017, 2021), and literacy platforms are currently in development (Ní Chiaráin & Ní Chasaide, 2018; Ní Chasaide et al, 2019). The development of an AAC system has been planned as part of the ABAIR Initiative for some time, based on requests from speech and language therapists who work with people with cerebral palsy, Alzheimer’s and Parkinson’s disease, as well as survivors of stroke. ABAIR has both the linguistic expertise as well as the speech technologies available to develop such a system for the Irish language. An urgent request from a parent provided an immediate impetus which kickstarted the project. The parent who approached us needs an Irish AAC system for her children, Eoin and Máire1. Eoin and Máire who are based in Dublin, use an AAC system to communicate in English, however they do not have access to an Irish AAC system. Though English is their first language (L1), the lack of an Irish AAC system is an obstacle to them in communicating with their Irish-speaking family members (who are based in a Gaeltacht region; a region in which Irish is spoken as the language of the community), as well as in accessing the curriculum and engaging with friends and teachers in their Gaelscoil (Irish immersion school). Eoin and Máire are in primary school and are in the early stages of acquiring literacy. As in the development of the screenreader, the design of the initial AAC system has critically involved the participation of this parent, her children and her large network of friends and acquaintances who are AAC users. This is affording us much insight into their requirements and also into the usability of various AAC platforms. 129 Given the urgent need to develop an AAC system for Eoin and Máire, as well as the longer-term need to develop a bilingual AAC system for the broader population of AAC users in Ireland, a two-pronged approach has been taken to development. This involves: I. The initial, short-term prototype development of an AAC system for the Irish language, motivated specifically by Eoin and Máire. II. The longer-term proposed development of a bilingual AAC system for a broader group of users. 3. Challenges to AAC development in a Celtic Language The linguistic structures of Irish pose a number of challenges to the development of an AAC system. Many of these will be relevant to AAC development in other Celtic languages, given their similarity. Morphological complexity: Irish is an inflected language, with a number of cases for nouns and adjectives as well as inflected prepositions and verbs. This results in many more word forms than exist in the English language. However, including every form of a noun, adjective and verb on an AAC board would result in a more cluttered and clunky user experience. An additional issue is that many of the AAC users are likely to be young or in the initial stages of learning Irish and may not be well enough acquainted with reading or with the grammar to accurately select a word form for a given context. Phrasal verbs: there are many frequently-used phrasal verbs in Irish. The meaning of these verbs with and without their accompanying preposition is often vastly different (see examples 1a and 2a compared to examples 1b and 2b). (1a) ag éirí (1b) ag éirí le ‘rising’ ‘succeeding’ (‘rising’ + ‘with’) (2a) ag bualadh (2b) ag bualadh le ‘hitting’ ‘meeting’ (‘hitting’ + ‘with’) For these phrasal verbs, the verb and its accompanying pronouns (in this case le ‘with’) should ideally be placed in close proximity on boards. However, as prepositions are inflected for person in Irish (e.g. le ‘with’ + mé ‘me’ = liom ‘with me’), these prepositional pronouns would also need to be included (entailing an additional six buttons). In addition, one of the principles of AAC system design is that there should not be multiple representations of the same word in different places, and many phrasal verbs use the same preposition/prepositional pronouns. Bilingual mapping: given that Irish AAC users are highly likely to be bilingual, a system which would allow users to easily toggle between Irish and English is desirable. AAC users use motor sequences to select items and rely on visuo- 2 A set of illustrative stories explaining the specific needs of individual users. spatial representations in memory to use AAC in a fluid way (Dukhovny & Gahl, 2014). This is similar to the motor plans we rely on when typing. This raises a question in relation to bilingual motor plans, and whether each language should have separate motor plans, as opposed to a common motor plan (insofar as possible). Bilingual voices: a bilingual system would ideally be equipped with bilingual voices, which would allow the users to utilise the same voice across languages. This would also be very important given the prevalence of code- switching. Though this work is planned as part of the ABAIR Initiative, there are not yet bilingual voices available. Code-switching and productive morphology: Novel words are often created in Irish by adding an Irish morpheme to a borrowed word from English. An example of this is the colloquial use of the verbal noun “ag zoomáil”, which uses the Irish verbal noun structure with the English word ‘zoom’. Ideally, it would be desirable to allow for the production of such words within the AAC system. Again, this relies on being able to easily toggle between the two language versions of the system. 4. Design features of initial prototype This section describes the short-term, initial development of an AAC system for specific users. As mentioned, the involvement of the AAC community network, and particularly of the parent and children mentioned earlier is central to all design features adopted in this prototype. In addition to her involvement in the adaptation, this parent also tests features with her children and provides feedback which allows for the improvement of the AAC system. The development of the AAC system to its current prototype stage has involved (i) collecting user requirements and developing user stories2 (ii) selecting a platform for development (iii) adaptation of boards (words and phrases) to Irish (iv) the selection and adaptation of the ABAIR voices and (v) the technical development necessary to embed the ABAIR voices in the platform and to produce grammatically correct speech. The processes involved in each step are described in the sections that follow. 4.1 Collecting user requirements and creating user stories Initially, a survey was conducted of AAC users to investigate how they used AAC and what features they considered important (Nic Corcráin, 2021). We also considered in detail the needs of Eoin and Máire, as well as the people who use the AAC system to communicate with them including their parents and their teachers. Based on this, we drew up user stories which illustrated the main needs of the prospective AAC stakeholders targeted in the present prototype (Eoin and Máire, teachers, parents). We established that primary requirements for this iteration of the AAC system were to: I. Have very good correspondence between the Irish and English versions of the system in terms of 130 layout. In practice, this means that the buttons for words in the Irish version are in the same place as the corresponding word in the English version, insofar as possible. As mentioned above, this is in order to preserve the motor plan that Eoin and Máire are already used to in English. This is particularly important given that Eoin and Máire are not yet proficient readers; they rely on visuo- spatial memory to access words, just as we do when typing. In the short-term, as Eoin and Máire are primary targeted users, maximising the portability between the two languages is a good strategy. Different cohorts may require different strategies in this regard, and this question will be further investigated, particularly with L1 Irish speakers. II. Produce grammatically correct output, while avoiding cluttering the AAC system’s layout with buttons for every possible form of a noun, verb, or adjective which exists in Irish. Including every possible permutation of a word would (i) result in cluttered boards which would be difficult for a child to navigate, (ii) preclude correspondence between the Irish and English motor plans and (iii) might be premature for users who have not yet mastered the grammar of Irish. III. Be both available off-line and provide technical support to users. As AAC users typically rely on their devices to communicate, it is vital that it is accessible in all contexts and that technical issues that do arise can be quickly and expertly resolved. IV. Include a range of voices which are appropriate to the user’s age, gender and identity. 4.2 Platform selection With the aforementioned user requirements in mind, we investigated a variety of platform options. Initially, designing an AAC system from scratch for Irish was considered. This option would allow the greatest amount of control in relation to the design and layout of the system. However, under the ABAIR Initiatives current remit, it would not be possible to provide the necessary technical support to users over a sustained period of time. Instead, we researched existing platforms and enquired with representatives from these platforms in relation to adaptation for Irish. Following from discussion with platform representatives as well as users, the Coughdrop platform3 was selected. Coughdrop is an open-source AAC platform which is available in a large number of languages. Technical support and training is provided within the platform, and it has offline functionality. 4.3 Initial adaptation of the AAC system boards For this prototype iteration of the AAC system, the boards were created to mirror the layout of Eoin and Máire’s AAC system in English. In the case of some words and phrases, there was a straightforward mapping from English to Irish. 3 https://www.mycoughdrop.com/ For many, however, there was not. For example, the verb ‘to know’ does not map neatly onto Irish, which contains a variety of verbs and phrases depending on whether the subject of the sentence is a person, a fact or an area of knowledge (aithne, fios, ar eolas, etc). This necessitated multiple buttons corresponding to a single button in English. Similarly, additional buttons were included to represent the counting systems in Irish, which differ depending on whether people or things are being counted. As mentioned in Section 3, Irish contains many frequently- occurring phrasal verbs which require the use of prepositional pronouns. The challenge is to provide easy access to these prepositional pronouns while avoiding including them on multiple boards. The present solution to this is to include the prepositional pronouns in a sidebar which the user can open on any board; this feature will be tested to investigate its suitability. An additional challenge pertains to producing grammatically correct word forms, given that the grammatical relationships within the phrase are primarily indicated through morphological inflection, rather than word order as in English. A technical solution was sought for this particular issue, which is described in Section 4.5. The present section describes just some of the challenges that arose in adaptation, and it is expected that more will emerge as the system is tested by AAC users. 4.4 Voice selection The voice of the AAC system becomes the voice of the user, and this raises many questions of identity. This pertains in the first instance to the choice of dialect; ideally, a Conamara user will be able to use their own dialect. At present, the ABAIR Initiative has developed four synthetic adult voices (Ulster dialect – female; Connaught dialect – male; Munster dialect – male and female) and one other voice is currently under development (Connaught dialect – female). This affords a certain amount of choice to the user, but there are gaps; there is currently no Donegal male, and there are dialects for which no voice is yet available. Furthermore, even when we have a male and female representative for a given dialect, such as Conamara, there is an immediate issue of identity, when there are groups of more than one male or female user (e.g. in a classroom). A variety of voices are necessary in an AAC system, in order to allow for users to retain and express their individual identity and to avoid difficulties in communication arising from two identical voices conversing (e.g. Pullin et al, 2017). There are currently no children’s’ voices available for the Irish dialects, and this will be a priority for the future. In the absence of children’s synthetic voices, a temporary solution was sought to provide child-like voices for Eoin and Máire’s AAC systems. An online system was developed in consultation with an expert on voice synthesis within the ABAIR Project; the system allows users to change the parameters of an existing synthetic voice in order to sound more child-like, more masculine or more feminine. Effectively, this would in principle allow users to create a bespoke voice for themselves, although the current results are only partially successful. Note that voice 131 adaptation, where we control and manipulate vocal parameters, is a complex field where parallel work is ongoing as part of ABAIR-RóbóGlór (Murphy et al, 2020). In the longer term, the goal would be to have robust ways of fine-tuning the parameters of the synthetic voice for the individual user. Right now, in the current prototype we are using the female Donegal voice. It is planned to offer a fuller menu in the near future. 4.5 Technical architecture The technical architecture for the AAC system is represented schematically in Figure 2. Figure 2: Schematic representation of system architecture As explained in the Introduction, the user inputs a series of words/images by selecting a sequence of buttons. These appear in the bar at the top of the board, but do not carry the grammatical inflections. The sentence is sent to the ABAIR server, where three steps are carried out to generate the grammatically correct forms: (1) The sentence is passed through a morphological corrector, which provides a number of inflected forms, not catered for in an already existing grammar checker, An Gramadóir (Scannell, 2005). This initial processing step involved the hand coding of grammatical rules. In the second step (2) the output of 1 is fed to the grammar checker (Scannell, 2005). The final step (3) takes the output of the grammar checker and adds a further grammatical refinement in the form of a genitive case checker/corrector which has been developed within the ABAIR project. An example of the original input is provided in 3 (a). In this case, the verb is not accurately conjugated, two initial mutations are omitted - one of which results in an inaccurate interpretation of possession – and the final noun is in the nominative rather than the genitive case. After being processed by the morphological corrector and An Gramadóir, these inaccuracies are resolved with the exception of the genitive case issue, as seen in 3(b). Finally, having been processed by the genitive case corrector, the input is grammatically accurate and can be send to the synthetic voices, as evident in 3 (c). 3 (a) Original input An gheobhadh siad a bronntanais ó fear an post? ‘would they get their presents from the postman’ 3 (b) Morphological corrector & An Gramadóir output An bhfaighfidís a mbronntanais ó fhear an post? 3 (c) Genitive case corrector output An bhfaighfidís a mbronntanais ó fhear an phoist? The corrected text is then synthesised and an ABAIR voice reads the output on the user’s AAC system. 4.6 Current state of the system At present, a prototype Coughdrop-based system is available online, and contains a single ABAIR voice. The grammatical accuracy of the system is very good in the small set of sentences on which it has been tested so far. 5. The next stage of development: a bilingual AAC system Some of the design features of the current prototype AAC system were motivated by the urgent need of providing an Irish language facility that would suit Eoin and Máire’s requirements. Future iterations will aim to encompass other potential users and contexts of use. This will include people from the Gaeltacht Irish language community, where requirements may differ from those of the current prototype in certain respects. We will also be catering to people of a range of ages and with a variety of needs. Research questions. Important avenues of future research will be explored, including the following: • the suitability of current solutions that focus on specific linguistic features of Irish (e.g. the sidebar for phrasal verbs) will be examined, and other possible solutions will be explored. These issues will resonate with the structurally-similar Celtic languages, and it is hoped that the present research could be broadened and strengthened by Pan-Celtic collaboration. • the bilingual context of users (e.g. Gaeltacht native speakers and Irish speakers and learners outside the Gaeltacht) will be further explored, and the requirements of different cohorts investigated. This will include examining the needs and wants of users in relation to voice characteristics, including sociolinguistic, dialectal and voice quality factors. • research is also needed on bilingual AAC systems in other languages where insights can be gleaned from the approaches adopted. Further research is intended in relation to motor plans, and whether they should be closely modelled on the language structure or should be harmonised to facilitate the early stages of acquisition, and indeed the code-switching that is a prominent feature of spoken Irish. In this regard, it is important to note that learning the layout of an AAC system typically requires a large time investment on the part of the user and often on the part of a parent or professional too. 132 • a related question involves ways to facilitate the productive derivational morphology whereby English words are borrowed but inflected to create new Irish forms (e.g. ag zoomáil). User testing will be conducted with a broad group of users to examine their attitudes towards and opinions of the usability of the system, the quality and robustness of the synthetic voices and the grammatical accuracy of the system, among other things. The findings from this evaluation will feed back into the development of the system. The development of an offline version is planned in order to allow for use in every context and environment. Early informal feedback from users indicates that this should be a priority. Increasing morphological accuracy is also a priority. Though the morphological corrector is producing very good results at present, additional rules will be added to this system to increase the grammatical accuracy of speech output. Children’s synthetic voices are planned for the near future. This will involve the recording of corpora and subsequent development of children’s voices, and will allow for more authentic child-like speech output. Training courses for stakeholders will be developed. This is an essential accompaniment to the system which aims to support users, parents, guardians, teachers and other educational professionals in accessing and using the AAC system. 6. Acknowledgements We gratefully acknowledge the support of An Chomhairle um Oideachas Gaeltachta agus Gaelscolaíochta (COGG). We also gratefully acknowledge An Roinn Turasóireachta, Cultúir, Ealaíon, Gaeltachta, Spóirt agus Meán. which support the ABAIR-RóbóGlór Project, as part of Stráitéis 20 bliain don Ghaeilge, 2010-2030. 7. Bibliographical References Barnes, E. (2017) Dyslexia Assessment and Reading Interventions for Pupils in Irish- Medium Education: Insights into current practice and considerations for improvement, M.Phil. Dissertation, School of Linguistic, Speech and Communication Sciences, Trinity College Dublin, Ireland. Barnes. (2021). Predicting dual-language literacy attainment in Irish-English bilinguals: language specific and language-universal contributions. PhD thesis, Trinity College Dublin. Dukhovny, E., & Gahl, S. (2014). Manual motor-plan similarity affects lexical recall on a speech-generating device: Implications for AAC users. Journal of communication disorders, 48, 52-60. Enderby, P., Judge, S., Creer, S., & John, A. (2013). Examining the need for, and provision of, AAC in the United Kingdom. Research Report. Communication Matters. Engstrom, C., & Tinto, V. (2008). Access without support is not opportunity. Change: The magazine of higher learning, 40(1), 46-50. McGuirk, R. (2015). Exploration of the Use of Irish Language Synthesis with a Screen Reader in the Teaching of Irish to Pupils with Vision Impairment, M.Phil. Dissertation, School of Linguistic, Speech and Communication Sciences, Trinity College Dublin, Ireland. Murphy, A., Yanushevskaya, I., Chasaide, A. N., & Gobl, C. (2020). Testing the GlórCáil system in a speaker and affect voice transformation task. In Speech Prosody 2020 (pp. 950-954). Ní Chasaide, A., Ní Chiaráin, N., Berthelsen, H., Wendler, C., Murphy, A., Barnes, E., Gobl, C. (2019). Leveraging Phonetic and Speech Research for Language Revitalisation and Maintenance, Proceedings of ICPhS, Melbourne. Ní Chiaráin, N. & Ní Chasaide, A. (2018). An Scéalaí: synthetic voices for autonomous learning. Future-proof CALL: language learning as exploration and encounters–short papers from EUROCALL, 230-235. Nic Aindriú, S., Ó Duibhir, P., & Travers, J. (2020). The prevalence and types of special educational needs in Irish immersion primary schools in the Republic of Ireland. European Journal of Special Needs Education, 35(5), 603-619. Nic Corcráin, M. (2021). ‘AAC don Ghaeilge’: A needs analysis survey for the development of Irish language augmentative communication devices for people with speech difficulties. Unpublished M.Phil. thesis, Trinity College Dublin. Pullin, G., Treviranus, J., Patel, R., & Higginbotham, J. (2017). Designing interaction, voice, and inclusion in AAC research. Augmentative and Alternative Communication, 33(3), 139-148. Wight, M. C. S. (2015). Students with learning disabilities in the foreign language learning environment and the practice of exemption. Foreign Language Annals, 48(1), 39-55. 8. Language Resource References Scannell, K. (2005). An Gramadóir. Retrieved 7 April, 2022, from https://cadhan.com/gramadoir/