Category Archives: Research

Past or present tense, plurals and other grammatical structures

Class 2 milestones and patterns in development – Indira Cevallos (slideshare )

It seems amazing that one can spend over a month discussing the intricacies of verbs and how they should be represented in our Arabic / English Symbol dictionary. The team have reached what Speech Therapists may describe as Brown’s stage 2, where we are learning how to represent more grammatical structures from verb tenses to plurals and simple multiword phrases. These can be difficult to translate into representative symbols as well as work across two very different languages. Yet if AAC users are to be able to make use of a core vocabulary and start to develop real language and literacy skills they must be available in any symbol dictionary that is going to be useful.

In English we tend to accept that the verb will be seen initially in the present tense so one would search for ‘play’ as the root word in for example the Oxford English Dictionary . However, in Arabic dictionaries such as The Dictionary of Modern Written Arabic compiled by Hans Wehr and edited by J Milton Cowan which has been reviewed as one of the best Arabic English dictionaries by Sa’id (1962) lists of action words are based on the root of the verb usually the imperfect, as an action happening in the past ’ لعب ‘ . This root of three consonants, can then be easily conjugated (if it is regular) with a series of vowel changes and additional prefixes and/or suffixes. http://acon.baykal.be/

So the first dilemma is how do we present our lexical entries – Should the English be in the past or the present? Should the symbols all have additional elements that illustrate the past as their main entry – going from Arabic to English?

How do we represent the past and the future when the writing is going from right to left – is the arrow for past going from right to left or left to right? Widgit offer a very good example of how this is achieved in English with their symbols.

Google offer an example of how this should be achieved on applications and the web when there is a discussion about bi-directionality. In Arabic we will need to mirror all the indicators of past and future as well as any diagonal marks. So lines originally coming from top left down to right will need to change to top right down to left.

In terms of mirroring actions it will depend on what the person is doing but discussions around Arabic etiquette are being represented throughout the symbols with right hand use for anything to do with food and drink and other appropriate actions.

Verbs also have gender specific spellings so representing the word as a symbol – should the verbs have boys or men as well as girls or females carrying out the actions. It may not be possible to have just the one version or a stick figure if we are supporting literacy skills and yet for those who are just getting to grips with AAC clarity and uncluttered symbols will be essential.

There are similar issues with singular and plural – can one just add the equivalent of +s as in ARASAAC symbols or a ++ as in Widgit – neither are needed if the image illustrates the fact that there are more than one. So how relevant are these extras, do they help when teaching concepts for literacy skills? How do we show this change of status in Arabic where plurals are different for male and female individuals – teachers – معلمات معلمون and then inanimate objects for example tables – طاولات and table – طاولة

Finally (for the moment) in Arabic an adjective can change its spelling to fit the noun it follows and just as in English there are irregular words and irregular ways of changing them. So developing guidelines as to how these things will best be represented in symbols is challenging as we reach over 800 Arabic lexical entries!

References

Brown, R. (1973). A first language: The early stages. London: George Allen & Unwin.

Sa’id M, F., (1962) Review of The Dictionary of Modern Written Arabic is an Arabic-English dictionary compiled by Hans Wehr and edited by J Milton Cowan. Language, Vol. 38, No. 3, Part 1 (Jul. – Sep., 1962), pp. 328-330 Published by: Linguistic Society of America. http://www.jstor.org/stable/410799 Accessed: 23 March 2015

Designs for Symbols and Arabic Core vocabulary.

Over the past month a further 20 therapists with 13 Arabic speakers voted on the first batch of symbols with Tullah reporting that there was a preference for colour versions of male and female images for verbs rather than stick figures (3 voted for stick figures and 17 for full image with colours). Tullah also felt that gender might play a role in the symbol choices, as it appeared that the male attendees were more willing to accept stick figures in black and white. This area needs more research! Does the gender of the therapist impact on the choice of symbol they may make for their AAC users?

When it came to discussing the clothing concerns there was a general consensus that there should be a range of dress with options for traditional styles as well as westerner clothing. The final dictionary will provide access to all the ARASAAC symbols as well as the specially designed ones for Arabic culture so it is hoped that there will be sufficient choice.

Survey stick figure with Arabic modifications Because there have been some concerns about the way action symbols for verbs are portrayed and the type of clothing needed across all the symbols it was decided that we needed to increase the number of people involved in making these decisions so a survey is being sent out across several organisations with AAC users in the hope that we receive clear direction.

The results from this survey and the final votes for Batch 1 of the newly designed symbols will be discussed in the next blog. In the meantime a new collection of Arabic words lists have been gathered from the Awsaj Academy teachers, parents and speech therapists working with AAC users.

Core Vocabulary

During the year the team have kept up the discussion about core Arabic vocabularies. According to the PrAActical Blog (author Carole Zangari) the recent ATIA 2015 conference

“served as validation that core vocabulary is now a widely accepted practice in supporting language development with AAC learners. Presentation after presentation discussed the rationale, research support, and strategies for implementation.”

Early on it was the aim to have a localised core vocabulary used by AAC users in Doha with a set of words that were based around a symbol vocabulary collected from therapists, teachers and parents in specialist schools and clinics. The original list was largely based on an English core vocabulary taken from the 100 frequently used core words provided by Prentke Romich Company (PRC), with a considerable number of fringe words – mainly nouns as can be seen from the most frequently used ones.

Now an Arabic word list has been developed, built up thanks to collaboration with 8 Doha based institutions. Description of data origins for Core vocab. The AAC user lists contain around 1000 words and the initial analysis has shown an interesting set of frequently used Arabic words which still need more accurate analysis and checks against other lists. The top 20 translated into English (with frequency in brackets) are at the moment:

I (16), car (13), ball (12), on (11), banana (11), to (10), he went (10), work (10), house (9), he sat (9), bicycle (9). clock/watch (9), in (9), chair (9), I want (8), aeroplane (8), pen (8), was (8), he played (8), flower / rose (8)

Signing young in Arabic (taken from ‘ndictative dictionary’ provided by The Qatar Society for Rehabilitation of Special Needs)

We also have access to a list of words collected by the Qatar Society for Rehabilitation of Special Needs for those who are deaf or have hearing impairments and are using sign language in Qatar and we are beginning to collect words used when teaching Arabic to children and those wishing to speak Arabic as a second language.

It is felt that these three collections should be representative of the types of vocabulary needed as a basis for the dictionary. The intention is to analyse the collections looking for similarities in the words used between the lists, frequency of use and comparisons with the English core vocabularies used at the start of the project.

References

Zangari, C. (2012).Helping the general education team support students who use AAC. Perspectives on Augmentative and Alternative Communication, 21:82;91. (Accessed 22nd February, 2015 http://www.minspeak.com/students/documents/%20Zangari.pdf)

VanTatenhove, Gail M. (2013) Core Vocabulary with Emergent & Context-Dependent Communicators in Special Education Classrooms (Accessed 22nd February, 2015 http://www.vantatenhove.com/files/handouts/CoreVocabWithECDCommunicators.pdf )

VanTatenhove, Gail M. (2007 Normal Language Development, Generative Language & AAC (Accessed 22nd February, 2015 http://www.vantatenhove.com/files/papers/LangDevelopmentIntervention/NormalLanguage&AAC.pdf )

Discussions around the latest voting for newly adapted symbols.

Having finished the initial development of the new voting system it was time to trial it with the Mada team on 18th January. The view voters received looked similar to past views, but with only one set of symbols and a finally agreed version of the voting criteria, namely four choices on a scale of five in terms of acceptance plus a comments box.

Voting for an adapted symbol

Voting criteria

The main comments from the voters regarding the system was that the images were not large enough, they wanted to have less clutter around the voting area and perhaps to be asked to guess the meaning of the symbol rather than to be given the text. The latter would mean there would need to be a two fold process where the participants first see the image without the word (Guessabilty – Cairney and Sless (1982), Hanson and Hartzema (1995), Yovetich and Young (1998) (Translucency – Haupt and Alant 2003, Evans et al 2006)) and then see the symbol with the word but help the graphic designer by saying where any issues lie with the imagery. This could take rather too long in the time available, but will need to be considered.

On Sunday 18th and Monday 19th January members of the Arabic Symbol Dictionary team in Doha held the second round of voting sessions at the Mada Assistive Technology Centre with members of the AAC Forum.

Tullah takes up the reporting…

“We were joined by 13 voters in session 1 and 7 voters in session 2. These therapists represented centres including Hamad Medical Corporation Special Needs Centre, Mada Assistive Technology Center, Qatar Academy – special needs section, researchers from Qatar University working in the field of special needs, Child Development Centre and Shafallah special needs school. We had a range of opinions from males and females, Qataris and non-Qataris, Arabic speakers and non-Arabic speakers. Despite the mirage of opinions and perspectives showcased throughout the sessions, there were some general opinions that came up again and again. It is these opinions of the majority that we will utilize to make the second round of adaptations to the symbols.

“The prayer symbols”

This series of symbols have definitely been the topic of much discussion in our team internally and also at the voting sessions. The image below depicts the transitions the symbol for “Maghrib prayer” or “evening prayer” (just after sunset), has taken (and will continue to take) in order to produce a symbol which the majority of people will be happy to use. The position of the sun, the colour and contrast of the sky and mat, the body structure and depiction, the clothing and religious sensitivities have made this symbol evolve into many different forms.

A collection of prayer symbols

Here are some comments from our reviewers that helped us make the modifications…

“The symbol with the person praying is more representative of prayers. Mosque symbol seems more generalized, representing faith, Islam”

“Change the color of the sun in the sunset picture to reflect the sunset”

“Increase the color contrast and the background color”

Here is one suggestion from our forum members of how to depict all the 5 prayers at different times of the day.

Praying at different times of day

“To stick figure or not to stick figure”

Other discussions were surrounding the team’s choice to use only stick figures for verbs and adjectives. This would solve complexities within the Arabic language of having to draw many versions of the symbol to fulfill the requirements of the Arabic language i.e having female and male versions for single, dual, plural, past and present tense forms. The general consensus amongst voters was that some symbol users will prefer stick figures and some full prefer the coloured/more detailed version. In any case, users of the Arabic symbol dictionary will have access to all ARASAAC symbols which have stick and coloured symbols for all words and can simply make adjustments to the symbol based on their preferences.

“The significance of colour choice”

Other voters believed that the colour scheme of the clothing used in our new symbols needed to be more consistent. Culturally, most Qatari women wear a black Abaya (dress) and shayla/hijab (head cover) and men wear the thawb (white dress) with a Ghutra (white head piece) on their head and Iqal to keep it in place.

Qatari dress

In many of our symbols this has been depicted as seen in the symbol below.

Symbol dress codes.

However, in some symbols coloured clothing was used for several reasons;

Younger symbol users are generally attracted to colours and have a tendency to want to use them more , this may also apply to those who have cognitive impairments (Stephenson, 2007)
Children in Qatar wear Western looking coloured clothing and coloured uniforms to school
It allows for more detail to be seen
Arabic speaking countries aside from the gulf (which represent the vast majority of Arabs e.g Egypt, Syria, Iraq etc) wear coloured clothing and wearing all black is seen as depressing and for older women.

Our voters suggested that if we would like to make the symbols coloured then the symbols should reflect symbols of children with and without the shayla because children do not wear the shayla. However, if the symbol system is to be based on adults then the thawb + ghutra in white for male and abaya + shayla in black for female are necessary.

“Gestures- comprehendo or no comprehendo?”

Different gestures used to depict a concept was also a topic which facilitated a lot of discussion. The “yes” and “no” gestures were agreed not to have a universal indicator to depict these concepts. While an Arab may indicate a headshake as “no”, and an Indian might interpret this as “yes”. What ARASAAC indicates as “thank you” (hand over chest) was said by our voters to have many different meanings in the Arab culture but did not in any case mean thank you. The ARASAAC ‘thank you’ symbol (hand over chest) to them meant “count on me”/I’ll take care of it or “sorry” or “a woman’s gesture when she does not wish to shake hands with a male”.
“…and that’s a wrap”

Many of these very intriguing discussions took place and gave the Arabic Symbol Dictionary team lots to think about and our graphic designer, Dana a lot of work to do. It is amazing to see how one symbol can mean so many different things to so many people and such small details can be offensive to one ethnicity and small changes can be made to please another.

I am glad that our team has chosen a participatory approach as without our voters’ feedback this symbol dictionary would not fulfill their needs or the needs of symbol users in Qatar and the wider Arab world. It was reiterated time and time again by voters how much this symbol dictionary really is needed and how keen therapists are to get their hands on the final product.” Amatullah Kadous

References

Cairney, S., Sless, D.: Communication effectiveness of symbolic safety signs with different user groups. Applied Ergonomics, Vol. 13 (1982), 91-97

Evans D, Bowick L, Johnson M, Blenkhorn P (2006) Using iconicity to evaluate symbol use. In: Proceedings of the 10th international conference on computers helping people. Linz, Austria, pp 874–881

Hanson, E. C., Hartzema, A.: Evaluating pictograms as an aid for counselling elderly and low-literate patients. Journal of Pharmaceutical Marketing and Management, Vol. 9, No. 3 (1995), 51-55

Haupt, L., Alant, E.: The iconicity of picture communication symbols for rural Zulu children. South African Journal of Communication Disorders, Vol. 49 (2003), 40-49

Stephenson, J. (2007). The effect of color on the recognition and use of line drawings by children with severe intellectual disabilities. Augmentative and Alternative Communication, 23(1), 44–55. doi:10.1080/07434610600924457

Yovetich, W. S., Young, T. A: The Effects of Representativeness and Concreteness on the “Guessability” of Blissymbols. Augmentative and Alternative Communication, Vol. 4, (1988) 35-39.

Updates to Symbol Manager, core and fringe vocabularies in Arabic and symbol voting.

Having discussed changes to the Symbol Manager over the last month the work has been completed with categories being divided into small lists and alphabetic versions of the sub-categories used by ARASAAC. Those not needed for the Arabic Symbol Dictionary have been hidden and others have been added – mainly related to food complement some of the categories the therapists have been using with Boardmaker.

There is now a Localisation section of the Symbol Manager that allows all the elements of the site to be translated from English to Arabic with changes that can be made at any time should the need arise.

There have been discussions about adding definitions for each lexical entry as it was clear that the WordNet entries in English needed some changes to suit the audience of AAC symbol users and that an Arabic dictionary was also needed. A request was made to the Almaany Dictionary and they have kindly offered collaboration which will be incredibly helpful.

Recent visits to Awsaj Academy have resulted in more vocabulary lists being provided by the Arabic department, for which we are incredibly grateful as this has allowed us to compare the use of various parts of speech in the lists provided by English teachers and those teaching in Arabic when using symbol communication. But we need to be wary of the results as they come from a wide range of ages and this last list comes from a group of more able children compared to those in the earlier samples. However, all the centres and schools need core and fringe Arabic vocabularies to provide a base for the most commonly used symbols. So frequency of use will be of paramount importance when making the choices of which symbols to adapt to suit the culture, environment, language and personal requirements.

Percentage	Number of words	Parts of Speech
12%	54	adjectives
2%	7	adverbs
3%	15	interjections
48%	209	nouns
1%	5	prepositions
1%	6	pronouns
1%	5	question words
32%	138	verbs
Total	439

A VOIP meeting with the ARASAAC team discussed the collaboration on the adaptation of symbols. They have kindly agreed to give us some guidance as to how they develop their symbols and we will send them our list of symbol files that need to be added to the database to suit the needs of users in Doha and the Arabic culture, language and environment. Sharing .svg files will help with the development and ARASAAC are changing their website and database in the coming months.

google plus symbols One of the ways we have been working on symbols that has greatly speeded up interaction between team members has been the use of Google+ with images being uploaded and our votes and comments being monitored by Dana before she finally uploads the images to the Symbol Manager for voting by the AAC Forum.

On going research into issues around voting as to the iconicity of symbols will confirm our decisions around how the AAC Forum can vote on the final draft versions of the symbols. At present we have several options as illustrated in the slides below that are available on SlideShare.

References

Bloomberg, K., Karlan, G. R., Llloyd, L. L.: The comparative translucency of initial lexical items represented in five graphic symbol systems and sets. Journal of Speech and Hearing Research, Vol. 33 (1990), 717-725

Evans D, Bowick L, Johnson M, Blenkhorn P (2006) Using iconicity to evaluate symbol use. In: Proceedings of the 10th international conference on computers helping people. Linz, Austria, pp 874–881
Fuller, D. R.: Initial study into the effects of translucency and complexity on the learning of Blissymbols by children and adults with normal cognitive abilities. Augmentative and Alternative Communication, Vol. 7, (1997), 30-39

Haupt, L., Alant, E.: The iconicity of picture communication symbols for rural Zulu children. South African Journal of Communication Disorders, Vol. 49 (2003), 40-49

Huer, M. B.: Examining perceptions of graphic symbols across cultures: preliminary study of the impact of culture/ethnicity. Augmentative and Alternative Communication, Vol. 16 (2000), 180-185

Mizuko, M.: Transparency and ease of learning of symbols represented by Blissymbols, PCS and Picsysms. Augmentative and Alternative Communication, Vol. 3 (1987), 129-136

Musselwhite, C. R., Ruscello, D. M.: Transparency of three communication symbol systems. Journal of Speech and Hearing Research, Vol. 27, (1984), 436-443

Symbols go walkabout – preparing symbols for their different uses.

The initial research be carried out for the symbol dictionary when visiting several centres around Doha gave us insight as to how symbols were used in schools and special needs centres. In particular at the Shafallah Center were Dalal highlighted the issue of changing the size of symbols allowing them to be used on walls and doors as signs and for guidance. Sample sizes go from 6×4 inches (150 by 101cms) down to 500 pixels square which is 4.23cms or 1.67inches square.

A more recent visit to Awsaj Academy and the Step by Step Centre for Special Needs gave us the chance to collect a selection of photographs to show how they used symbols to enhance the teaching and learning environment. They have kindly allowed us to share a sample selection.

: Hallways and stairs

: Indoors/Outdoors behaviour

: Desk reminders

: Communication book

: PECS set

: PECS board of symbols

This has led to a debate about the format of symbols. At present most centres are using Boardmaker to develop their symbol sets – the software allows images or PCS symbols to be flexibly resized and copied to other programs as well as being saved as symbol boards with varying cell sizes.

At one of our AAC forum meetings therapists working at the Al Noor Institute for the Blind also highlighted the need for high contrast and black and white images. As with the ARASAAC symbols, which all come as coloured and black and white images, it is our intention to include these options.

During the recent TechshareME conference we also learned, as a result of a survey carried out during a workshop, that at least six centres were using multimodal techniques alongside their symbol use. Techniques included word segmentation with audio feedback (text to speech) to encourage phonemic awareness skills and auditory discrimination. Symbols were also being used to illustrate letter combinations, the meaning of words, parts of words and sentences. The use of Jolly Phonics was mentioned, Clicker 5 and Go Talk as well as the integration of PECS techniques, Boardmaker for development of symbol sets and Makaton for gestures.

It was clear that as discussed in our poster and paper presentations at The HMC Annual Research Day Techshare Middle East, the Qatar Foundation ARC’14 and RAatE 2014 the team need to not only include all these elements in the dictionary, but to also be aware that the way images are formatted allows for flexible adaptations to suit all needs. ARASAAC provide their symbols as .png files and Boardmaker allows users to import images in .jpg, .jpeg, .gif,.png, .bmp, wmf and .emf formats. ARASAAC offer symbols that are 500×500 pixels and this is the size the dictionary will carry on the website.

References

GoldBart, J. and Caton, S. (2010) Communication and people with the most complex
needs: What works and why this is essential Research Institute for Health and Social Change
Manchester Metropolitan University (MMU)
https://e-space.mmu.ac.uk/198309/1/Mencap%20Comms_guide_dec_10.pdf
This report is a useful resource for academic references as well as practical advice.

Millar, S. (2009) Communication Friendly Schools CALL Scotland, University of Edinburgh – http://www.educationscotland.gov.uk/Images/SallyMillarFriendlySchools_tcm4-629155.ppt PowerPoint presentation with example of symbols in use around schools in Scotland.

Dissemination, revision of the Symbol Manager System and the beginning of Dictionary building.

Virginia Creeper in October

Summer has faded in the UK with autumn bringing in rain showers, wind and turning leaves. We have begun the task of telling people about the Arabic Symbol Dictionary at conferences with paper and poster presentations. At ICCHP 2014 the Project Possibility University of Southampton students presented Symbol Dragoman using the symbol and word lists already gathered for the symbol dictionary along with the Tatoeba lists. This was followed up by Communication Matters 2014 where we had a poster and will be writing a paper. We have also just heard that we have been accepted for poster and paper presentations at TechShare Middle East, the Qatar Foundation Annual Research Conference (ARC’14) and RAATE 2014.

ICT Qatar

In Qatar, Nadine joined the research team with Mada as a speech and language therapist whilst also supporting AAC users at the Shafallah Center and Dana has come on board as a graphic designer just when we need to be thinking about logos, leaflets, updated posters and beginning the task of adapting or adding to the symbol set we wish to use. The ARASAAC team have kindly agreed to collaborate with us on the task of using their creative commons licensed symbols where appropriate for the dictionary. This was discussed as a result of the voting that took place in June and July.

During August and September Nadine and Tullah have been researching the issues around gathering core vocabularies in Arabic that are suitable for AAC users as well as considering the concerns around the enhancement of literacy skills which are challenged by the diglossic nature of Arabic. Levin et al (2008) mention the fact that “research has shown that the linguistic distance between Standard Arabic, the language of print, and spoken Arabic vernacular, the oral language of children challenges the acquisition of reading in Arabic (Abu-Rabia, 2000; Saiegh-Haddad, 2003a, 2004, 2005, 2007a).” It appears to affect all aspects including “lexicon, syntax, morphology and phonology”. It is felt that by offering sound patterns of lexical entries (with the use of recorded and synthesised text to speech) this could support carers and teachers of speech impaired individuals when working on literacy skill acquisition.

So discussions have continued around phonemic segmentation and how this will be represented in the dictionary for both Arabic and English with the result of changes being made to the symbol manager system. It appears that in Arabic the phonemic segmentation can be generated almost automatically with the help of some clever computer coding as long as the diacritics are in place – that is according to Nawar!

Levin et al have cited several researchers in their comment that the sub-syllabic level (Consonant Vowel level (CV)) in Arabic phonemic segmentation is more easily learnt compared to any other way of encouraging phonemic awareness. A study with bilingual children by i Saiegh-Haddad & Geva (2008) showed that being able to sound out parts of words when learning to read was equally important in Arabic as in English. However, when it comes to deciding which section in a word forms a phonemic segment there is a rather more torturous route for English words which will require manual entry for consonant blends and digraphs etc. Spaces between the segments will be used when adding words to the Symbol Manager.

As a result of the changes made to the system the team at Southampton university have been beta testing the latest version. Early trials have been completed and the system is ready for the addition of new lexical entries with definitions, sentences for context, categorise for browsing and searching, parts of speech and the extra field for phonemic segmentation. New symbols can be added with categories such as monochrome, colour and gender. All items will be distinguished by their language English, Modern Standard Arabic and Qatari. The latest version is faster and accesssible using mouse, keyboard only or touch screen. Nawar has worked hard to make it as flexible as possible and it is now ready for further testing using a simple check list – download MS Word doc Beta Testing Symbol Manager v1 . We have also used the SUS evaluation scale (Brooke, 1996). Taking an iterative approach with the participation of as many interested parties as possible we have setup up logins for the Symbol Manager System and will be reacting to any feedback we receive from those involved with the project.

References

Brooke, J. (1996). SUS: A “quick and dirty” usability scale. In P. W. Jordan, B. Thomas, B. A.
Weerdmeester, & A. L. McClelland (Eds.), Usability Evaluation in Industry. London: Taylor
and Francis.

Levin, I., Saiegh-Haddad, E., Hende, N., & Ziv, M. (2008). Early literacy in Arabic: An intervention study among Israeli Palestinian kindergartners. Applied Psycholinguistics. Accessed 12th October 2014, http://journals.cambridge.org/action/displayAbstract?fromPage=online&aid=1899796

Saiegh-Haddad, E., & Geva, E. (2008). Morphological awareness, phonological awareness, and reading in English-Arabic bilingual children. Reading and Writing, 21(5), 481–504. Accessed 12th October 2014 http://link.springer.com/article/10.1007%2Fs11145-007-9074-x

English and Arabic Phonic Representations to aid Literacy Skills in AAC users

Over the past few weeks we have been trying to understand the importance of the various ways phonemes are represented to support literacy skills in Arabic and English and how best to show them alongside words or multiwords that are added to the dictionary via the Symbol Management system. We have also discussed the need for recorded speech where synthesised speech or text to speech fails for both MSA and Qatari Arabic. Research has shown how important phonemic awareness skills are for AAC users who go on to develop literacy skills and it appears that listening to the sounds and seeing the text highlighted helps reading skills as well as finger pointing (Vandervelden & Siegel 1999).

One of the hardest problems in English is how to represent the sounds when the spelling of the words bears little resemblance to the spoken version. Decisions have to be made as to whether one uses a system similar to that offered by the BBC where sounds are written as a combination of vowels or consonants that represent what is said such as /th a ng k / y oo/ or stay with the original spelling and just divide the word up into segments or syllables with the various blended or individual sounds e.g. th a n k | y ou

BBC Phonics kit available at from the BBC website

Whilst discussing this matter with Professor Annalu Waller, Rolf Black, Andrea Kirton and Simon Judge at the Communication Matters Conference 2014 it was clear that the presentation should follow the way the phonics are being taught in schools by primary school teachers where the AAC users developing literacy skills could work alongside their classmates. In UK such schemes as Jolly Phonics are being used and Andrea Kirton and Simon Judge are working on a phonic screen that might well be developed further to present the sounds with speech output in a similar way to the Macmillan app developed by Vivid Interactive to provide speech therapists with the phonetic alphabet. It is possible that with the English section of the Arabic Symbol Dictionary we will need to take this further with clusters and blends being part of the segmentation to aid search and categorisation of words for example the listings provided in ‘Spotlight On Spelling: A Structured Guide To The Assessment And Teaching Of Spelling’ and the work of Cootes and Jamieson

Phonic Screen developed by Andrea Kirton and Simon Judge – Barnsley Hospital NHS Foundation

In Arabic some thought is needed as to how phonemes are represented with the various diacritical marks. However, it is felt that by offering all the movements (diacritical marks) the text to speech (TTS) voices on offer will be able to provide acceptable pronunciation for most words even if they fail on individual phonemes were there will be the need for human recordings.

Below you will find 16 rows with 28 representations of the Arabic alphabet with possible phonemic variations which can be read using the Arabic version of ATbar. As the phonemes are used in written Arabic their letter shapes will change. The shape of each letter altering depending on the position in the word and phrase. Arabic keyboards achieve this automatically! You are seeing all the letter combinations as if they are in their initial position. I should point out that corrections to this table may still need to be made by our Arabic speaking experts, but this is just to show the type of discussions taking place at this stage in the research.

ي	و	ه	ن	م	ل	ك	ق	ف	غ	ع	ظ	ط	ض	ص	ش	س	ز	ر	ذ	د	خ	ح	ج	ث	ت	ب	ا
يَ	وَ	هَ	نَ	مَ	لَ	كَ	قَ	فَ	غَ	عَ	ظَ	طَ	ضَ	صَ	شَ	سَ	زَ	رَ	ذَ	دَ	خَ	حَ	جَ	ثَ	تَ	بَ	اَ
يُ	وُ	هُ	نُ	مُ	لُ	كُ	قُ	فُ	غُ	عُ	ظُ	طُ	ضُ	صُ	شُ	سُ	زُ	رُ	ذُ	دُ	خُ	حُ	جُ	ثُ	تُ	بُ	اُ
يِ	وِ	هِ	نِ	مِ	لِ	كِ	قِ	فِ	غِ	عِ	ظِ	طِ	ضِ	صِ	شِ	سِ	ذِ	رِ	ذِ	دِ	خِ	حِ	جِ	ثِ	تِ	بِ	اِ
يّْ	وّْ	هّْ	نّْ	مّْ	لّْ	كّْ	قّْ	فّْ	غّْ	عّْ	ظّْ	طّْ	ضّْ	صّْ	شّْ	سّْ	زّْ	رّْ	ذّْ	دّْ	خّْ	حّْ	جّْ	ثّْ	تّْ	بّْ	اّْ
يَّ	وَّ	هَّ	نَّ	مَّ	لَّ	كَّ	قَّ	فَّ	غَّ	عَّ	ظَّ	طَّ	ضَّ	صَّ	شَّ	سَّ	زَّ	رَّ	ذَّ	دَّ	خَّ	حَّ	جَّ	ثَّ	تَّ	بَّ	اَّ
يُّ	وُّ	هُّ	نُّ	مُّ	لُّ	كُّ	قُّ	فُّ	غُّ	عُّ	ظُّ	طُّ	ضُّ	صُّ	شُّ	سُّ	زُّ	رُّ	ذُّ	دُّ	خُّ	حُّ	جُّ	ثُّ	تُّ	بُّ	اُّ
يِّ	وِّ	هِّ	نِّ	مِّ	لِّ	كِّ	قِّ	فِّ	غِّ	عِّ	ظِّ	طِّ	ضِّ	صِّ	شِّ	سِّ	زِّ	رِّ	ذِّ	دِّ	خِّ	حِّ	جِّ	ثِّ	تِّ	بِّ	اِّ
يَا	وَا	هَا	نَا	مَا	لَا	كَا	قَا	فَا	غَا	عَا	ظَا	طَا	ضَا	صَا	شَا	سَا	زَا	رَا	ذَا	دَا	خَا	حَا	جَا	ثَا	تَ	بَا	آ
يُو	وُو	هُو	نُو	مُو	لُو	كُو	قُو	فُو	غُو	عُو	ظُو	طُو	ضُو	صُو	شُو	سُو	زُو	رُو	ذُو	دُو	خُو	حُو	جُو	ثُو	تُو	بُو	اُو
يِي	وِي	هِي	نِي	مِي	لِي	كِي	قِي	فِي	ضِي	عِي	ظِي	طِي	ضِي	صِي	شِي	سِي	زِي	رِي	ذِي	دِي	خِي	حِي	جِي	ثِي	تِي	بِي	إِي
يَّا	وَّا	هَّا	نَّا	مَّا	لَّا	كَّا	قَّا	فَّا	غَّا	عَّا	ظَّا	طَّا	ضَّا	صَّا	شَّا	سَّا	زَّا	رَّا	ذَّا	دَّا	خَّا	حَّا	جَّا	ثَّا	تَّا	بَّا	آ
يُّو	وُّو	هُّو	نُّو	مُّو	لُّو	كُّو	قُّو	فُّو	غُّو	عًّو	ظُّو	طُّو	ضُّو	صُّو	شُّو	سُّو	زُّو	رُّو	ذُّو	دُّو	خُّو	حُّو	جُّو	ثُّو	تُّو	بُّو	اُّو
يِّي	وِّي	هِّي	نِّي	مِّي	لِّي	كِّي	قِّي	فِّي	غِّي	عِّي	ظِّي	طِّي	ضِّي	صِّي	شِّي	سِّي	زِّي	رِّي	ذِّي	دِّي	خِّي	حِّي	جِّي	ثِّي	تِّي	بِّي	اِّي
يَة	وَة	هَة	نَة	مَة	لَة	كَة	قَة	فَة	غَة	عَة	ظَة	طَة	ضَة	صَة	شَة	سَة	زَة	رَة	ذَة	دَة	خَة	حَة	جَة	ثَة	تَة	بَة	اَة
يَّة	وَّة	هَّة	نَّة	مَّة	لَّة	كَّة	قَّة	فَّة	غَّة	عَّة	ظَّة	طَّة	ضَّة	صَّة	شَّة	سَّة	زًّة	رَّة	ذَّة	دَّة	خَّة	حَّة	جَّة	ثَّة	تَّة	بَّة	اَّة

Tullah has also been carrying out research in this area and has discovered an iPad app called ‘Sawti‘ developed by Gadah Alofisan from King Saud University who has won awards for his work in this area and has presented at ICCHP . This is one of the first apps to offer Arabic AAC support with symbols and their corresponding words being said by male and female children’s voices. It offers users the chance to practice symbol / word recognition with free text being read aloud with the synthesised voice. There are some colloquial Arabic words as well as MSA and the user can choose when to use speech feedback.

The only problem we have found is that the voice changes depending on the symbol being read which can be a little distracting and sometimes the way the word is pronounced was questioned by some Arabic speakers.

Both Arabic and English have such a wide range of pronunciation that we are going to have to agree on some guidelines for the way we work with voices / TTS and the way phonemes are presented.

References

Bayan Alarifi, Arwa Alrubaian, Ghada Alofisan, Nora Alromi, Areej Al-Wabil (2013) Towards an Arabic Language Augmentative and Alternative Communication Application for Autism, In proceedings of HCI International 2013 A. Marcus (Ed.): DUXU/HCII 2013, Part II, LNCS 8013, pp. 333-341. Springer, Heidelberg (2013).

Black R, Waller A, Pullin G, Abel E. Introducing the PhonicStick: Preliminary evaluation
with seven children. Montreal, Canada: ISAAC; 2008. http://phonicstick.computing.dundee.ac.uk/publications/

Andrea Kirton, Simon Judge, P. B. (2014). Using Phonemes to Construct Utterances for Aided Communication. ISAAC 2014. doi:10.13140/2.1.3524.4162 http://openconf.faiddsolutions.com/modules/request.php?module=oc_program&action=summary.php&id=142

Trinh, H. (2011). Using a Computer Intervention to Support Phonological Awareness Development of Adults with Severe Speech and Physical Impairments. The 13th International ACM SIGACCESS Conference on Computers and Accessibility, Dundee, UK. Accessed 5th September 2014 http://src.acm.org/2012/HaTrinh.pdf

Trinh1, H. (2012). iSCAN: A Phoneme-based Predictive Communication Aid for Nonspeaking Individuals. Proceeding ASSETS ’12 Proceedings of the 14th international ACM SIGACCESS conference on Computers and accessibility. ccessed 5th September 2014 http://keithv.com/pub/iscan/iSCAN_Final.pdf

Vandervelden, M., & Siegel, L. (1999). Phonological Processing and Literacy in AAC Users and Students with Motor Speech Impairments. Augmentative and Alternative Communication, 15(September), 191–211. Accessed 5th September 2014 http://informahealthcare.com/doi/abs/10.1080/07434619912331278725

A participatory approach to symbol choices – Voting outcomes.

At the outset of this Arabic Symbol Dictionary research there was a determination to ensure that the approach would be participatory in nature as mentioned in our blog dated December 13th, 2013 ‘A Participatory Approach to Research’. In that blog we outlined the type of participatory approach being taken where those supporting the project would be involved with all the planning and take part in the decision making. Voting on symbols

To this end a voting system was set up to allow AAC experts, users and those supporting users to provide feedback about the two freely available symbol sets compared to the PCS or Widgit ones already in use.

Members of the Advisory Group had separately mentioned that they did not want to see the development of yet another set of symbols and this seemed to be an eminently sensible plan if we were to also keep to the request in the early days of the AAC Forum meetings that the dictionary should be available in English and Arabic as many of those working with AAC users and caring for them spoke better English than Arabic.

The voting system allowed users to login in with an email address to not only vote positively or negatively for a symbol but to also select a series of check boxes to say why they liked or did not like it when shown a lexicon entry on one side of the screen and a selection of symbols on the right. The symbols they voted on where only from ARASAAC and Sclera symbols in order to compare to the ones they already knew – PCS or Widgit.

The statements that were provided were positive or negative based on the initial vote so a positive up vote would result in the following phrases being presented:

Would you like to tell us the reason?

Select any number of checkboxes

The symbol represents the word/phrase/meaning
The symbol is similar to PCS/Widgit
The symbol’s colour/contrast/drawing is good
The symbol is culturally sensitive
Other reason, please specify:

The negative vote would result with the same check boxes but each phrase had the addition of the word ‘not’.

There were 562 symbols that could take a vote each with the check boxes and free text comment. 33 individuals took part from the various centres in Doha. The outcome was interesting as both sets of freely available symbol sets presented issues for those voting with a wide variation in additional comments besides those offered with the check boxes.

Both symbol sets had their compliments for clarity, meaningfulness and cultural sensitivity but when combined with the free textual comments it was the negative sentiments that were quite stark in their numbers. 777 check box comments were related to a lack of clarity in representing the word or multiword in English – the word lists used were based on those collected from English speaking therapists in Doha – the Arabic core vocabularies are to follow. 499 were related to the way the symbol was drawn, colour and contrast levels, 197 comments were checked as being culturally insensitive and 172 were not similar to PCS or Widgit. The ranking of this list follows the way voters wrote in the free text field. There were 130 additional comments about the meaning of the symbol and its representation of a word or phrase. There were comments about the way arrows and question marks represented actions or words, poor representations of important words such as ‘want’ with individuals varying in their views about text appearing in a speech bubble or near a symbol. Plurals also caused comments but it was the way the abstract items may be drawn that also caused concern. Overall there were 97 additional comments about the look of some symbols and some questioned how children would learn a shape or object. Distractable and busy drawings were described in 9 comments with facial expressions also being considered important.

Additional cultural issues were raised in 41 free text comments with ‘thumbs up and down’ being noted as an issue along with dress, female/male requirements, language and using the pointed finger plus the use of a ‘halo’ for being good. Colour is an item that will really need discussion especially for those with visual impairments. The 28 additional mentions were usually around contrast levels. There were only 8 additional points that seemed to be related to the image not suiting the environment such as “In Qatar a rainy day is a good day”! The use of text with symbols is also a debatable subject and some pictograms were just deemed to be totally unsuitable whether in English or Arabic.

Overall the ARASAAC symbols appeared to be the ones that were most similar to those already in use and the most acceptable as a collection for the symbol dictionary. A small video has been made to show how we can check the suitability of every image against future and previous votes to begin the process of making sure adaptations can be made to enhance the chances of the pictogram/symbol becoming more acceptable whilst adding the Arabic core vocabularies to the symbol manager.

Micrsoft Excel 2013 PowerView allows us to analyse entries that were made via the voting system. The latter was built using MongoDB and offers a flexible way of uploading images and lexicons with parts of speech, definitions and in the future phonemic segmentation for the Arabic Symbol Dictionary.

Thanks go to Russell Newman and Nawar Halabi from ECS University of Southampton, for their work on this part of the research project and also to Amatullah Kadous who has arranged all the voting parties to conclude this part of the research.

Phonemic awareness for literacy skills – Possible additional feature for each lexical entry

Whilst we analyse the voting for the symbols and check the results against the core English vocabulary lists provided by Tullah from the various Doha groups of AAC users we have been investigating how we could tag and store the symbols and their matching lexicons in English and Arabic.

Meetings with Professor Annalu Waller in Dundee and Simon Judge in Sheffield confirmed suspicions that if we wanted to ensure that the dictionary not only coped with the communication side of symbol use and also encouraged literacy skills there needed to be links to the way words were made up of phonemes. Research has shown that phonemic awareness can be used as a predictor for reading ability (Gillon, 2004)

Teaching phonics has become a hot topic in UK Primary schools to the extent that even the newspapers have come up with lists of resources to aid teachers and parents. An example is the Guardian article in 2013 “How to teach … phonics”

Primary children learning with a Jolly Phonics reading scheme. Photograph: Murdo Macleod for the Guardian

But the dilemma is how we make a dictionary of words / multiwords linking to symbols that can also be searched by phonemes in English and Arabic for those who can often only listen to the sounds or may find it hard to say them or recognise their significance. Clearly this aspect of the dictionary is only for certain groups of symbol users who may have the ability to learn to read and write.

‘geck’ and ‘chom’ from Year 1 Phonics Screening (Five things about phonics By Kathryn Westcott BBC News, Magazine, 2012)

‘Synthetic phonics‘ is often used with ‘made-up’ words to encourage grapheme / phoneme recognition. However, as can be seen from the example, even those words introduced at primary school level may have letter combinations that can be pronounced in different ways, for instance in ‘geck’ the letter ‘g’ can be said with a hard sound or a soft sound in front of an ‘e’ (/dz/ or /g/). Children tend to be taught around

According to teaching documents provided by the Education Department at Oxford Brookes University: “In English, much more than in other languages,

many letters or letter-combinations can commonly represent more than one sound – for example ea as in heat and head;
most sounds can be spelt in more than one way – for example the vowel sound in heat is also commonly spelt as in he, see, chief and complete;
Some very common words contain grapheme-phoneme correspondences that occur in few if any other words – for example one, two, are, said, great, people, laugh.” (TDA, 2011)

Children tend to learn the basic 44 phonemes in English which are represented below.

The Phoneme Table below lists the 44 phonemes of the English Language, from the National Strategies Standards’ phonics sounds. (DFES 2007)

The good news is that in Arabic every letter combination has a set rule and although there may be many sounds that cannot be replicated in English the grapheme/phoneme representation is stable. The image below shows where there are overlaps between Arabic and English phonemes.

Arabic Consonant Phonemes – Focus on Bilingualism: Assessing Speech Skills in Bilinguals by Ellen Kester, Ph.D., CCC-SLP and Alejandro Brice, Ph.D., CCC-SLP (2012)

Research carried out by Amor and Maad (2013) working with Arabic speaking Tunisian children has shown that: (direct quote from their article)

The best performances of Good Readers confirm the idea supported by many researchers (e.g. Byrne et al., 1992; Gombert, 1992) that reading failure may manifest itself through a lack of phonemic awareness. The lowest results of the Preliterate children suggest that phonemic awareness does not develop spontaneously, but only in the specific context of learning to read an alphabetic script at school. This phenomenon was observed in many alphabetically written languages, such as English, French (Gillon, 2004; Morais et al., 1987), and Hebrew (Bentin et al., 1991; Oren, 2001).

The researchers found there were specific difficulties with phonemic segmentation for Arabic words across the cohort of “110 Tunisian children enrolled in primary education schools and kindergartens”. All their participants scored less well than expected in comparison to research results carried out in other languages. This was felt to be due to the diglossia nature of Arabic with colloquial Arabic often being spoken at home whereas Modern Standard Arabic is used in schools. Nevertheless, the children coped better with consonants in deletion tasks, for example /k/t/b/ that make up many words around reading and writing e,g /kataba/ (he wrote), /kutiba/ (it was written), /kutubun/ (books), etc.

Those learning to read in English tend to find it easier to mark initial and final phonemes as individual sounds with the medial one proving harder to work out. Amor and Maad (2013) found that their participants had difficulties across all three positions. “Divergence between the performances of Arabic-speaking and English-speaking children confirms that the representations about the consonantal segments were not the same. “

Summary

It appears that the ability to gain phoneme / grapheme awareness is harder in Arabic than some other languages and that the total number of consonant segments in Arabic are higher than most languages but the number of vowel based segments are lower (Newman, 2002)

One idea when creating the lexical entry in the symbol dictionary is to include the representative phonemes with their diacritics, their change in shape (depending on the position in the word) and the sound they make using text to speech or recorded speech.

References

Abu-Rabia, S. (2001). The role of vowels in reading Semitic scripts: Data from Arabic and Hebrew. Reading and Writing: An Interdisciplinary Journal, 14, 39-59.

Amaryeh, M. M., Dyson, A.T. (1998). The Acquisition of Arabic Consonants. Journal of Speech, Language and Hearing Research, 41, 642

Amor P. D. M., & Maad, R. Ben. (2013). The Role of Arabic Orthographic Literacy in the Phonological Awareness of Tunisian Children (April). Retrieved from http://www.ijonte.org/FileUpload/ks63207/File/02.amor.pdf

Bentin, S., Hammer, R. & Cahan, C. (1991). The effects of ageing and first grade schooling on the development of phonological awareness. Psychological science, 2(4) 271-274.

Byrne, B., Freebody, P. & Gates, A. (1992). Longitudinal data on the relation of word-reading strategies to comprehension, reading time and phonemic awareness. Reading, Research Quarterly, 27, 141-151.

Department of Education and Skills (2007) Letters and Sounds: Principles and Practice of High Quality Phonics – Primary National Strategy ( PDF Downloaded July 30th, 2014)

Gombert, J.E. (1992). Metalinguistic development. Chicago: University of Chicago Press.
Gillon, G. (2004). Phonological awareness: From research to practice. New York: Guilford Press.

Kurtz, R. (2010). Phonemic awareness affects speech and literacy. Speech-Language-Development. Retrieved from http://www.speech-language-development.com/phonemic-awareness.html

Khomsi, A. (1993). L’epreuve collective d’identification de mots. Nantes : University of Nantes.

Liberman, I., Shankweiler, D., Fisher, F. & Carter, B. (1974). Explicit syllable and phoneme segmentation in the young child, Journal of Experimental Child Psychology, 18, 201-212.

MacDonald, G. & Cornwall, A. (1995). The relationship between phonological awareness and reading and spelling achievement eleven years later. Journal of Learning Disabilities, 28(8) 523-527.

Morais, J., Alegria, J., & Content, A. (1987). The relationship between segmental analysis and alphabetic literacy: An interactive view. Cahiers de Psychologie Cognitive, 7, 415-438.

Newman, Daniel L. 2002. The phonetic status of Arabic within the world’s languages.
Antwerp Papers in Linguistics, 100:63–75 http://uahost.uantwerpen.be/apil/apil100/Arabic1.pdf

Saiegh-Haddad, E. (2005). Correlates of reading fluency in Arabic: Diglossic and orthographic factors. Reading and Writing: An Interdisciplinary Journal: An Interdisciplinary Journal, 18, 559-582. International Journal on New Trends in Education and Their Implications April 2013 Volume: 4 Issue: 2 Article: 02 ISSN 1309-6249

The Training and Development Agency for Schools (TDA) (2011) Systematic synthetic phonics in initial teaching training: Guidance and support materials. (Word document downloaded July 30th, 2014)

Vandervelden, M., & Siegel, L. (1995). Phonological recoding and phonemic awareness in early literacy: Developmental approach. Reading Research Quarterly, 30 (4). 854-875.

Ziegler, J. & Goswami, U. (2005). Reading acquisition, developmental dyslexia and skilled reading across languages. Psychological Bulletin, 131, 3–29.

Project Possibility Symbol Dragoman to learn more about translation issues

SS12: EU Code for a Cause 2014 is supported by Project Possibility at the ICCHP 2014 conference having links with eAccess+, AAATE and ICTpsp. The organisers state that the “aim is to inspire students, faculty and other stakeholders in Computer Science/Engineering and related fields to take up accessibility and Assistive Technology as a topic. It

introduces the power and the need of eAccessibility to the next generation of software engineers
brings students in contact with the eAccessiblity and AT field and in particular with end users
contributes to easy and fast solutions of pressing AT and eAccessibility problems outlined and defined by users
releases code as open source which facilitates an accelerated path towards products and services.

Thereby a long term impact on the uptake and implementation of accessibility in practice is expected.”

Symbol dragoman overview from E.A. Draffan

The University of Southampton team from Electonics and Computer Science Web and Internet Research Group have attempted to develop a Symbol Dragoman and presented the idea to the Project Possibility (SS12) judges on 7th July with a set of PowerPoint slides and demonstration.

According to Wikipedia “A dragoman was an interpreter, translator and official guide between Turkish, Arabic, and Persian-speaking countries and polities of the Middle East and European embassies, consulates, vice-consulates and trading posts. A dragoman had to have a knowledge of Arabic, Persian, Turkish, and European languages.”

Symbol Dragoman aims to allow someone who has no spoken language and uses pictograms or images to communicate in Arabic or English. To combine chosen ‘symbols’ in any way they want to produce a sentence that can be read or heard in both languages with the potential of offering any combination of languages in the future.

Learning Curve

In this case the team had to develop a basic understanding of Augmentative and Alternative Communication (AAC) where users were learning the language of symbols.

The issues that occur in a bi-lingual environment, such as Arabic or English?

Understanding whether words above or beneath the pictogram are helpful in multilingual situations.

Coping with single and multiword differences impacting on syntaxt and semantics

Dealing with situations where several concepts were available using one symbol or several symbols could make up concept?

It is often the case that the AAC user may not be able to read Symbol Dragoman although text to speech can offer some support but there needed to be the translation to help the carer or assistant who speaks another language.

Examples

put in
‘put in’ possibly means ‘insert’ in Arabic – أدخل
This symbol also means ‘tidy up’ or نظف ويرتب
put in a safe place

But now we have ‘put in a safe place’ could mean put in a ‘secure’ place. In Arabic وضعه في مكان آمن This literally means ‘he put him in a safe place’ as there is no ‘it’ in Arabic. The Arabic verb in its most basic form is third person masculine. It should be noted that in English we would also need a pronoun such as ‘it’ in this case. In Arabic the letter that represents ‘it’ or ‘him’ is part of the verb

The problem is that we need a system that not only translates the symbol into a word or multiword in both Arabic and English but also allows the user to build a sentence in symbols that can be recognised in both languages, where order is not important but the sentence is understandable with correct grammar.

Requirements

A system that will allow Arabic symbol users to work in both languages with the symbols and words appearing from right to left for Arabic and left to right for English
A system that works out what will possibly be the meaning of a collection of symbols chosen by the AAC user in any order
Grammatically correct sentences / phrases presented in two languages

The Outcome

A open source web based application that has been designed using responsive methodologies allowing it to be used within a browser on a computer or tablet.

Yunjia writing on a white board sowing responsive design plans

It has made use of creative commons symbols from ARASAAC and The Noun Project. These symbols have been categorised and present a sample set of words provided by young AAC users and their carers in Doha who are from Arabic speaking families but are often taught or have therapy with international specialists who do not speak Arabic.

The team have drawn on Natural Language Processing and machine learning to offer users a more speedy and flexible approach to symbol choices and sentence production. TATOEBA is being used as the source for sample sentences.

TATOEBA has a database with more than 3 million sentences in 179 different languages. Among them, 440 thousand sentences in English and 15 thousand in Arabic. The database saves the translation relationships of sentences in different languages in the form of sentence 1 in language 1 is the translation of sentence 2 in language 2, and those translations are created by users in the TATOEBA community. Currently, there are more than 6 million translation relationships in the database. The sentences have been downloaded along with the translation relationships between the sentences. These have been placed in an Elasticsearch (ES) server which has be wrapped with a RESTful API for the front end to query the sentences.

When symbols are selected from the user interface, the words and phrases that the symbols represent in a certain language are constructed as a query for ES server. The response is based on all the sentences in the ES server that contains all those words and phrases in that language that will be returned with any available translations for each sentence. The fact that the sentences are generated by a community that uses the word combinations in their first language, means that accuracy rates should be higher than machine translation.

However, there are many more sentences available in English compared to Arabic so some adjustments have had to be made. It was also found that the number of sentences that the person engaging with the AAC user should be reduced to allow for a dialogue to develop that is not just based on ‘yes/no’ responses.

Sample screen shots showing a choice of English and Arabic sentences

English sentences
Arabic sentences

Code availability

The open source code is available on BitBucket

Instructions

The user is able to select a symbol from the web pages using the keyboard or mouse

A simple sentence can be built by individual symbol selection with the images appearing in the edit box above. Then using the menu buttons alterations to content and the language can be achieved

back one symbol Symbols can be deleted one by one using the orange framed back arrow

delete all The entire phrase/sentence can be deleted using the red framed trash can

sentence generator The symbols are changed into a choice of sentences by selecting the green framed rotor

return to home page Should one of the categories on the right be chosen, the user returns to the home page using blue framed arrow

switch languages The book with open pages toggles access to second language – Arabic or English.

ATbar The Text to Speech in Arabic is activated by highlighting the text and using ATbar speech output if you do not have Arabic on your computer

Keyboard and mouse accessibility include the ability to use tabbing, enter and several access keys for example numners for categories and initial letters for menu actions.

There is also the option to use the open source ATbar to provide further accessibility options such as coloured overlays and text to speech support in English and Arabic once a sentence has appeared.

Challenges Arising

Odd sentences that do not relate to the choices made – often where there are too many alternative words available.

Blank when no symbols relate to any sentences – often because too many symbols have been used in a random way.

Evaluation

At present it was felt by some testers that the accuracy rates were still a problem even though the grammar structures in both languages were often more correct when compared to word by word translation.

The system needed to be more intelligent in its choice of sentences so people’s names did not appear.

More core vocabulary options with an increased number of symbols may provide better rates of accuracy

Note

Several student groups took part in the Project Possibility SS12 competition and the well deserved winners produced a very polished application called PAVE that analysed PDF documents for their accessibility and guided users to make appropriate changes.

References

Core Vocabulary

References

Tullah takes up the reporting…

“The prayer symbols”

“To stick figure or not to stick figure”

“The significance of colour choice”

“Gestures- comprehendo or no comprehendo?”

References

Percentage

Number of words

Parts of Speech

References

References

References

References

Summary

References

The University of Southampton team from Electonics and Computer Science Web and Internet Research Group have attempted to develop a Symbol Dragoman and presented the idea to the Project Possibility (SS12) judges on 7th July with a set of PowerPoint slides and demonstration.

Learning Curve

Requirements

The Outcome

Sample screen shots showing a choice of English and Arabic sentences

Code availability

Instructions

Challenges Arising

Evaluation

Note