New York and the UNICEF AAC Cohort

David Banes and I were lucky enough to meet up with an exciting group of start-up companies in the world of open source AAC. The UNICEF AAC Cohort is made up of CIREHA (Argentina), Daokudai (China) and Ninaad Digital Technology (India) led by the UNICEF Innovation team based in New York. We met up at 101 Park Avenue for three days of intense workshops and one on one meetings. It was an exciting agenda with topics ranging from open source application development by Atul Varma to localisation, developing personas, business plans and budgets. Gabriella Levine provided a thought provoking risk analysis guide with some very helpful advice relevant to both open source hardware and software development.

The slides below about Open Source Development have been copied into Slidewiki from Atul’s github pages.

Associate Professor Ayesha Butt from University of Riphah in Islamabad has also let us show her slides. She highlighted the importance of localisation, making systems that work for the individual in their own settings with symbols that support cultural needs and suitable social settings.

Mercy Kirui, Matthew Utterback and Will Clurman from Ekitabu, Kenya were also in attendance as they were sharing what they had learnt in the set up period of their UNICEF supported open source, cross-platform e-reading system for those with print disabilities, such as visual impairment. Their company provides access to ebooks that offer an accessibility provision in schools and described how they dealt with data collection, marketing and their business model. They also mentioned their successful digital essay competitions and events such as book fairs and have now set up a Digital Literacy Trust

Each AAC company gave us a final summary of their plans for the future based on all the ideas that had been shared over the three days.

CIREHA showed us the early version of cBoard that uses Mulberry Symbols and offers support for those with complex communication needs. It is an open source online and offline dynamic symbol board system with text to speech, that aims to offer a flexible approach to the creation of communication boards so that it can be personalised to suit both children and adults. An early online version of cBoard can be be used as a demonstration of what is to come. Many ideas were discussed about the use of OpenBoard format that we have also used to produce communication charts along with the adoption of other Creative Commons symbol sets!

Daokudai (China) are developing Yuudee2, an application designed for children with Autism Spectrum Disorders (ASD) or with language disorders. They have an interesting use of animated symbols working with speech recognition to improve speech and language in social situations. Earlier versions of the app are available as Yuudee on Google Play

Ninaad Digital Technology (India) have been developing Jellow that aims to support children with speech and language difficulties. The symbols have been designed with the particular needs of the local community in mind and but are provided under a Creative Commons licence for sharing with a wider audience. They are also used in communication charts, booklets and as an app available in Hindi and UK/USA English from Google Play There were interesting discussions about data collection and Ninaad showed the power of analysing the data they have received as a result of sessions using their app. It was possible to not only see how long the app had been used but also which areas attracted most attention.

We really are looking forward to supporting the development of cBoard, Yuudee2 and Jellow as open source projects in the future. It will be exciting to see if we can can integrate their languages with translations and symbol sets with others that are available with Creative Commons licences. This would allow many more AAC users to benefit from the UNICEF Innovation funded AAC Cohort’s work. You can tell it was an inspiring week away from the day to day job!

Happy New Year and welcome to our new logo and name

As we start the new year and wish you all the best for 2018, the team behind the research and development of the Arabic Symbol Dictionary have updated their website and renamed it globalsymbols.com.

We want to continue to support communities to develop and host new symbol sets for a range of languages supporting communication and literacy for those with special needs and we hope you will join us on this journey.

Mada, the Qatar Assistive Technology Center will be maintaining Tawasol symbols (keeping the original name on their servers). However, the original Arabic symbol dictionary remains on our secure server with your data and passwords allowing you to have access to all the freely available resources and those we develop in the future alongside links to other similarly licensed materials namely Creative Commons Share Alike Licence (cc-by-40).

This blog will continue to be our way of sharing the research aspect of the international symbol dictionary development, as we explore new ideas.

friends Watch this space as we start to work with colleagues in Pakistan thanks to a travel grant from the Global Challenges Research Fund

Payment for the flights and accommodation will allow a member of the team to visit Lahore and Islamabad. It is hoped that we can find ways of applying for funding to adapt and add to the symbol dictionary to reflect local linguistic concepts and culture with AAC users, families, carers and experts in the region using the online voting systems to develop resources that suit local requirements. It will also be great to have feedback through Facebook as we have done in the past.

Using Tawasol Symbols on apps for portable technologies

Over the last few months there have been several presentations around the world about Tawasol Symbols including the GREAT Conference in Doha where Nadine talked about “Making Educational Resources with TAWASOL Symbols to Support Students with Disabilities”. By chance we will also be showcasing our work at another GREAT meeting – The World Health Organisation Global Research and Education in Assistive Technology (GREAT) Summit in August.

There we will be talking about how Tawasol symbols can be used to enhance AAC outcomes for those working in the Arabic language. But as can be seen in this Slideshare presentation there are now several ways of presenting symbols in apps with text to speech output when using portable or mobile technologies such as The Grid, CoughDrop and The Open Voice Factory (used to AzuleJoe).

AAC systems for Arabic Speakers from David Banes

Other apps that will support Tawasol Symbols with Arabic speech output include

TouchChat AAC chat app for Apple iOS systems such as the iPad and iPhone
Proloquo offers apps such as Proloquo4Text 2.0 with Arabic voices.
Go Talk Now is a very flexible app that offers other symbol sets with the Acapela voices but also allows the import of personalised images.
Colourful Semantics in Arabic comes with lots of resources to build on sentence construction and story telling skills.
The Babnoor app has been developed in Dubai to support those with Autism and provides an easy to use Arabic interface with its own symbols but others can be added along with local dialect voices.

We know there are more to come and hope to also see Tawasol symbols being used in other language such as Hindi on Jellow – also available in English.

Arabic Speech Corpus shared by Dr. Nawar Halabi

respond symbol with audio If you have been using our Arabic symbols page you will have noticed that we have made every phoneme for our lexical entries available as a sound file, so that you can hear how it is pronounced. You can see the audio links at the bottom of the symbol for ‘respond’ in the picture beside this text. This can help those who have literacy skills difficulties as well as those wish to learn Arabic.

Nawar, who has been part of our Tawasol Symbols project from the beginning at the same time as successfully completing his PhD, has made this possible with the development of an Arabic Speech Corpus with support from the University of Southampton and MicrolinkPC.

The synthesised speech output that results from this corpse is a very natural sounding voice, recorded using Levantine Arabic, as heard in and around Damascus. Levantine Arabic is considered one of the three main Arabic dialects and differs from Gulf Arabic in some aspects of grammar and pronunciation although when phonemes are read aloud, they are often nearer Modern Standard Arabic and when combined there is less dialectal impact.

The corpus has been made available for download as a zip file and is released under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. As the Arabic Speech Corpus website says the packages includes:

1813 .wav files containing spoken utterances.
1813 .lab files containing text utterances.
1813 .TextGrid files containing the phoneme labels with time stamps of the boundaries where these occur in the .wav files. These files can be opened using Praat software.
phonetic-transcript.txt which has the form “[wav_filename]” “[Phoneme Sequence]” in every line.
orthographic-transcript.txt which has the form “[wav_filename]” “[Orthographic Transcript]” in every line. Orthography is in Buckwalter Format which is friendlier where there is software that does not read Arabic script. It can be easily converted back to Arabic.
There is an extra 18 minutes of fully annotated corpus (separate from above, but with the same structure as above) which was used to evaluate the corpus (see PhD thesis). Feel free to use this in your applications.

Please contact Nawar Halabi by email for further information.

Question words – Why are they so hard to design?

question

We have been having another look at our question words and it is really hard to design ones that make sense in all contexts. If you are working in a European language such as English you will be used to the concept of the ‘wh’ words and there is a particular order in which they tend to appear as mentioned by Mira Shah “what,” “where,” “why,” “how,” “when,” then “which.” “Where” is earlier in English and “who” is earlier in Italian.”

ARASAAC offer a collection of symbols for them

what

where

why

how

when

which

But then one has to think about the context and it might be time or an action. So should one add more clues or just have what + Time as a separate question mark with a clock face?

The general consensus amongst the symbol sets seems to be that ‘what’ is just a question mark and the other question words come with a clue but that can be difficult when the word in Arabic is combined and there needs to be a difference in the gender…

what

what is your name (F)?

what is your name (M)?

where

why (F)

What time is it?

It seems that we need to have many more question words that are linked to relevant settings but they can cause confusion, such as ‘when?’ and ‘what is the time?’ or ‘what time is it?’

when

There do not seem to be any articles on this subject and as different languages have very different ways of asking questions we will go on developing more symbols so there is lots of choice!

Any ideas for ‘which’?

how are you?

Voting on symbols at Awsaj Academy

tablet with symbol It has always been felt that it was essential that AAC users and those with communication difficulties should evaluate our symbols as part our research into symbol acceptance with real participation at all stages. We felt that this would ensure that the very people who would be using the symbols would provide us with feedback so we could tailor the symbols to their needs.

The team contacted the Speech Therapy team at Awsaj Academy for students with special needs to see if we could do a voting session with some of their students. Dr. Biji Philips arranged for 11 students to vote individually, with 15 minute time slots to complete the task. 2 students; a Tobii user with Cerebral Palsy and another with severe Autism required 30-45 minutes.

student voting The students were given 20 of our symbols to vote on and a thumbs up and thumbs down symbol to communicate their like/dislike for the symbol. Some students preferred to use okay as an option as well. Based on the student’s capacity judged online, Nadine and I asked why they liked/disliked the symbols. Some of the older students were superb and gave us detailed feedback relating to the need for more detailed facial expressions, or adding context to the symbol rather than just characters. Others did not want to let us down and said they liked the majority of the symbols. Here are the results of the voting sessions:

Student comments

	Good	Ok	Bad	Comments
Hello (Assalumu alaikum)	11	1		– Not clear, waving or speaking – Saying hi and smiling – He tells how are you – Goodbye – Nice because he’s wearing Thobe – Nice clothes – Clear – Goodbye
Rice & Chicken	12			– Only chicken – Doesn’t look like our food – Chicken
Children	10	2		– Not nice hair – Put them in uniform – Boys
Dance	12			– Very nice – Sing – Nice because he’s dancing properly – Nice design
Eat	9	2	1	– Looks angry – I don’t see plate or water; I like that he’s opening his mouth and has spoon – Eat with hands – Don’t wear Thobe when you eat – Holding pen
First	11		1	– Won – Appropriate for Qatar
Friends	10		2	– Uniform – Like because it has 2^nd and 3^rd – Clear – Thobe are same – It’s nice how they hold each other – All the shoes black color – Shoes different – They shouldn’t hold their hands, it’s a shame
House	12			– White + door white/gray; It’s big – White + smaller – Two thumbs up – Like our house – Qatari houses are different
Hug	12			– Change clothes color – The girl is hugging her mother – Mom cuz wearing Abaya
I	10	1	1	– Needs arrow on top of head – Picture matches meaning – Full body – Not clear
Mother	10	2		– Add rainbow color + lighten colors to look more happy – Child holding hand + smiling – Put Abaya – Should be in Abaya. I know non-Muslims don’t wear it. Should wear Abaya wherever you go – Black Abaya
Pray	11	1		– I like he’s praying – Like him praying – Put him in the house. You don’t pray in the middle of the road – Clear
Nursery	12			– Kids are playing and smiling – Good – School – The colors are nice – Nice colors
Please	5	2	5	– Add text – She’s saying please and child should be angry – Open hands – Tilt head – Telling secrets – Not clear, I can’t see the two hands – Talk – Greeting
Souq Waqif	10	2		– Add a lot of people – It has Thobe and Abaya – Looks like the old days – Didn’t know – Change the buildings
Pray	7		5	– Needs more colors + full mosque – Not clear – Put someone praying + purple sky – There’s a bird – Didn’t know – It’s a mosque, not clear, maybe add colors. – I prefer the other praying symbol – Add colors
Thank you	8	1	3	– Hand gesture is more I love you ; handshake – Hold and shake hands – I do this for thank you – No hand on chest – He’s saying the national anthem – I don’t use this gesture for thank you – Hands greeting
Travel	10		2	– More sky + men with yellow clothes – Need stairs or bus – Terminal – Dad looks like brother – Add airport
You	10		2	– They should look at each other – Context. Add playground – Clothes are so different and shoes are different – Didn’t know – Not clear – Come
Bye	9	1	2	– Show side profile + say bye to someone else – Sad face for saying bye

Boys on a stand as winners

Overall it was a great voting session with some valuable feedback obtained. Speech therapists reinforced the need for such a project, giving the example of one student who “could not look at” a picture card used for inferencing emotions due to the image of the boys not covering their arms (picture below). Teachers also reported that students felt empowered by giving their feedback, as they had always been accustomed to receiving help, but on this occasion they felt they were able to help others.

An important visit to Cologne by team member Amatullah Kadous

autumn trees along a canal

On the 26^th October Dr. Amal Ahmad, David Banes and I headed to Cologne to meet Professor Jens Boenisch and his team from the University of Cologne. I met Jens and his colleague Lena Schmidt at ISAAC 2016 in Toronto where they were presenting on “Teaching Core and Language in Bilingual Settings “. Jens attended one of our sessions and was so impressed by the work of Tawasol. He realized the potential for future collaboration as Germany welcomed many Arab refugees. He gave me his card and we stayed in touch. Some weeks later Jens emailed me to inform me that his team had received a huge grant to help Arabic refugees and invited me to Cologne along with some of my other colleagues to discuss the possibility of a collaboration.

We arrived to Cologne with a warm welcome from Lena who picked us up from the airport and took us to our hotel. The next day was the Cologne team’s presenting day where they presented to us all the amazing research work they had done, were doing and were planning on doing with the refugee project they had recently started. They presented to us about their major core vocabulary study they had been working on for over 5 years. They collected language samples from American and German school aged children as well as typically developing and cognitively impaired students (IQ<75). They sought to see if core vocabulary was the same across language and cognitive abilities. Their research showed that there were minor differences but more than 90% of core words were the same amongst all research groups. They also asked parents/teachers to identify the most commonly used POS that the cognitively impaired group use. They said nouns and verbs. Surprisingly, the percentages of POS used by the cognitively impaired group was very similar to the typically developing children (the highest being pronouns). This conveys that sometimes what we think our cognitively impaired children know can be very different to what they actually know and do. They showed us their cologne communication binders which are a wonderful resource based on 10 years of research into core vocabularies and so important in language learning.

Cologne Communication Boards and Binder

The second day of our Cologne stay was Tawasol’s turn to present. We were keen on setting the scene with the first part – Arabic history, language and culture. I felt it was important to convey this information as in order to be able to work with a group of people that are different to you, it is important to know who they are, the things that have shaped them and the things that are important to them. Only then can understanding be fostered and a bond of trust and rapport building be facilitated. I started with our proud moments in Arab history where the Arab world was the centre of the world in terms of education, philosophy, medicine, astronomy, military strategy and much more. I talked about the complexities of the Arabic language; the syntax – noun and verb root words and how pronouns attach to indicate tense, plural, gender, pronouns etc. and the differences between Arabic and English and how this affects symbol to text translation. I discussed the Arabic letter system and the complexities of having a diacritic system and its implications on young Arabic readers. I discussed the rich culture of the Arabs including the importance of family, food, marriage/pro-creation, the home land, religion and neighbours. I also spoke about Arabic social norms and how that may affect interactions within German society.

Tawasol symbols project overview from E.A. Draffan

It was very insightful for the Cologne team and we agreed that there were many beautiful things that we could learn from each other. I then spoke about the work of Tawasol with a specific focus on the symbol designs and criteria we used in creating our symbols as well as our frequency lists.

We discussed the implications and how we could combine our work to help refugees learn the German language in a quick and effective way to help them integrate. In addition to helping Arabic AAC users learn German, but also explore a means of communication with their Arabic speaking parents. We are very thankful to the Cologne team for their exceptional hospitality and inviting us to help them help Arab refugees settle into their new homeland.

Stay posted for more updates on this collaboration…

Tawasol Symbols celebrating International AAC month October 2016

clothing This month we are celebrating International AAC month like many others around the world as can be seen from the ISAAC website

The Tawasol symbols team from Qatar ran a workshop for parents, therapists and people with Autism at the the Second National Autism Forum. The team stressed the importance of using culturally and linguistically appropriate symbols. Demonstrations were given on how to use Tawasol symbols to support communication and build vocabulary in a fun and exciting way through Tawasol game cards. The team provided the audience with free Arabic symbol resources in Arabic and English. The attendees were very excited to see and receive something that they believed represented their culture and religion and appeared very keen to start using the Tawasol resources.

In the photographs below you can see the Tawasol symbols on the attendees tables.

people attending autism forum at tables

Nadine presenting about the symbols

In the picture above Nadine is presenting whilst Dr Amal and Tullah were taking photos at this stage in the day. Everyone was given a pack of cards with Tawasol Symbols in Arabic and English, that can be downloaded from the resources page.

tawasol symbol cards

worksheets

laminated worksheets with symbols

Symbols for different settings across the world.

map of countries for Tawasol Symbols downloads

World map where Tawasol Symbols have been downloaded

There have been many debates about localisation and globalisation or internationalisation and the different requirements to support these ideas – W3C have provided definitions that fit the web and in many ways localisation can support concepts used on web pages namely customisation related to:

“Numeric, date and time formats
Use of currency
Keyboard usage
Collation and sorting
Symbols, icons and colors
Text and graphics containing references to objects, actions or ideas which, in a given culture, may be subject to misinterpretation or viewed as insensitive.
Varying legal requirements
and many more things.”

We have acted on many of the W3C ideas over the last two and half years and noticed that while we have been developing our symbols for an audience based mainly in the MIddle East many of those who have requested use of the symbols and downloads have come from as far afield as Sweden, Australia and India. We know some of these symbols have been for refugee groups and camps and others have been for religious symbols. Both these requests have led to an increase in the number of symbols we have produced in these areas and many have been seen on our Facebook pages.

But we are not the only ones making symbols for a wider audience and it is interesting to know that there have been requests that Apple should introduce emojis with women wearing the hijab with a petition gaining over 2,295 signatures seven months ago and headline news from the BBC and many others.

Many refugee organisations have booklets and charts with images to help those who do not speak the language of the country to which they are going. An example is the ICOON project which has many charts for download as PDF files. These images tend to be in monochrome but cover a wide range of topics.

Example of ICOON refugee charts freely available to download in PDF

Tawasol symbols are also available for download with charts in both Arabic and English and we have included religious settings and have an interactive version as a demonstration to illustrate the symbols in action thanks to The Open Voice Factory.

Sample prayer symbols from the interactive online communication chart

Tawasol at ISAAC

I presented “Core Vocabularies: Same or different for Bilingual Language Learning and Literacy Skill building with Symbols?” and together with Dana; “Developing an Arabic Symbol Dictionary for AAC users: Bridging the Cultural, Social and Linguistic Gap”. We received a lot of great feedback about both presentations and many people showed interest in our project. One Speech and Language Pathologist (SLP) from Malta approached us having very similar issues with the analysis of core vocabulary. She mentioned that the issues I discussed in my presentation were almost identical to the issues in Maltese and was seeking advice on how to categorise pronouns given their attachment to nouns in the Maltese language.

A few SLPs from Israel approached us seeking advice about resources and access to our symbols as 20% of the population is Arabic speaking. A team from Germany and a team from Sweden were very interested in using our symbols with refugees and the German team were interested in collaborating with us on a project that focused on a German/Arabic symbol dictionary.

Many attendees also found our second presentation very insightful as Dana and I discussed the criteria we used to adapt the symbols to be culturally appropriate. Several commented that they found the Arabic cultural and social norms as well as environmental considerations very different when compared to their personal experiences and were grateful for opening their eyes to things that would have never occurred to them as being offensive or unsuitable.

One of the highlights of this trip was the screening of our film at the ISAAC Film Festival which can now be seen on the Arabic and English home pages of the Tawasol Symbol website. Our film shared the story of Mohammed, an eye gaze user for whom we developed prayer symbols so that he could actively participate in prayer with his family. It was screened alongside approximately 10 other films from around the world and provided such a unique insight into the mix of films presented. Many people approached me after the screening and congratulated the Tawasol team on our “amazing work”, a “wonderful film” and a few took our details as they could see how the prayer symbols could benefit some of their clients.

film festival

For the rest of the week, Dana and I went to sessions with a focus on core vocabularies and where possible in bi-lingual situations. It seemed that the issue of core vocabularies in other languages being quite different to English was a global linguistic challenge; whether it was Spanish, Maltese, Hebrew or German.

We visited the exhibition and saw some great new products and services. At the Boardmaker/TobiiDynavox stand we were shown some of their new apps including SnapScene and Pathways for Snap Scene. In these apps you are able to take pictures, add voice recordings, circle and highlight objects in images as well as label them. Pathways then gives you tips and tools on how to make these pictures an opportunity for communication, social interaction and learning. We also visited VocalID who customise your speech generated device to sound just like you. We topped off our ISAAC 2016 experience by attending the BUILD meeting whose members hope to bring together people working in AAC in developing countries. It was lovely to see/hear the work being done in South Africa, Taiwan, Singapore, Africa and Eastern European countries. It really made us think about creating an ISAAC Arabia or at least get the conversation going as to how we are collectively advancing the status of AAC users in the Arab region.

Quote ISAAC

Student comments

Good

Ok

Bad

Comments

Hello (Assalumu alaikum)

11

1

– Not clear, waving or speaking

– Saying hi and smiling

– He tells how are you

– Goodbye

– Nice because he’s wearing Thobe

– Nice clothes

– Clear

– Goodbye

Rice & Chicken

Children

Dance

Eat

First

Friends

House

Hug

I

Mother

Pray

Nursery

Please

Souq Waqif

Pray

Thank you

Travel

You

Bye