Localised images to aid recognition – What about free localised AAC symbols?

rememberThanks to a note from Aejaz Zahid about an article on the BBC website we came across a wonderful example of how localised and personalised pictures can help those who find it hard to read or remember important items in their social setting.  The article reminds one of the  importance of knowing the local situation in which those with literacy difficulties or speech and language disabilities find themselves, and understanding the imagery that brings it alive.  Being aware of personalisation and social circumstances cannot be stressed more highly with regards to the way symbols are developed but sometimes it is possible to use readymade versions to speed things up.

In the article titled ‘Spanish grandson’s doodles help grandma find phone numbers’  it is interesting to see that the grandson has used images that perhaps could have been taken from the ARASAAC set of AAC symbols  which has been funded by the EU and the Government of Aragon in northeastern Spain, had he known they existed.  Examples shown in the photographs include equivalent symbols such as a red cross for medical problems and a hospital, a vet for the dog and a mobile phone number.

red cross hospital mobile phonevet

 

 

 

 

The more we can spread the news about repositories that have freely available AAC symbols such as the English language OpenSymbols.org  or the free application such as PictoSelector that offers a way of making communication boards from the symbol sets it holds, the more we can help people like Pedro Ortega’s grandmother.

We hope that our soon to be multilingual symbol dictionary, will be used by more people wanting these types of pictograms, symbols or images for supporting those with literacy difficulties and cognitive impairments as well as those who need augmentative and alternative forms of communication (AAC) in many languages.

global symbolsDid I say multilingual symbol dictionary?  Watch this space, we are developing, thanks to a UNICEF innovation fund grant,  a way of being able to add new free symbol sets that can be seen in multiple languages – think open source and ConceptNet then possibly in the future AI and machine learning, for a glimpse into the technical side of the sorts of things we want to achieve! Crowdsourcing also comes to mind when errors need correcting and additions have to be added!  These symbol sets will all be linked to their owners and we will be stressing the types of licences under which they are provided and hoping you will join us on this new adventure! Símbolos gracias a ARASAAC adventure

Classifying AAC symbols for ease of use

Low tech to high tech

With thanks to Techcess and the SCORE layout

There has been a long history and much debate about the way we classify AAC symbols for ease of use.  Do we make it easy for people to choose the symbols they want on a communication board from a label?  Should we try to provide categories and if so which ones?  Do we need to think about semantics and the meaning of words, phrases, or text in context as concepts and syntax (sequencing words into grammatically correct sentences) when we are searching?  What happens when people want to make visual displays based on different topics?

These questions become even more important when considering a range of languages which may have very few similarities in particular if they are not part of the European group of languages.  Issues around orthography as well as left to right and right to left placement if one is thinking about text to speech output on electronic devices and moving through a sentence or phrases with a minimal number of selections and distance to cover.

Once again we would like to use share our questions with others in the field in order to make decisions that suit as many AAC users, carers, families and professionals as possible.  Here are some key points in a set of slides available on SlideWiki

Global versus Universal?

“Global” reflects the nuance of culture and language, “Universal” assumes that one size fits all.

voting hands around the worldWorking with UNICEF and the AAC Cohort is one of the most exciting things we have been doing recently.  We have had telemeetings with lots of discussions about opening the world of AAC symbols to the widest possible audience.  Topics have ranged from different open licences such as Creative Commons and open source software to what it takes to develop Alternative and Augmentative Communication (AAC) symbols that can be used across the world and on to more complex ideas including Artificial Intelligence (AI), machine learning and AAC!

You might ask why are we thinking about the meaning of words such as  ‘Global’  and ‘Universal’ whilst building a symbol repository.  We feel that global recognises different languages, cultures, religions and social settings and environments.   In part this is because we have promised ourselves that we will not be introducing yet another symbol set that includes symbols that are universally recognised.   We accept that there are many symbols that can be said to be universal because they are recognised worldwide, but we are looking at the nuances that occur in different countries and where localisation is important.

what time is it?

What time is it?

drink

drink

We discussed the idea of “Symbols for different settings across the world” when we were working on the Tawasol Symbols in 2016 and looked at some of the issues that W3C highlighted for web developers thinking about  localisation and globalisation or internationalisation. such as:

  1. “Numeric, date and time formats
  2. Use of currency
  3. Keyboard usage
  4. Collation and sorting
  5. Symbols, icons and colors
  6. Text and graphics containing references to objects, actions or ideas which, in a given culture, may be subject to misinterpretation or viewed as insensitive.
  7. Varying legal requirements
  8. and many more things.”

We discovered that No.8  ‘many more things’ included  the criteria below when working on the Tawasol Symbols and that these features came about as a result of our voting sessions with AAC users, their families, carers and the professionals working with them.  criteria for symbol design

Global Symbols aims to ensure that all the open symbols we add will have been reviewed by those using AAC and those involved in supporting AAC users in the locality where they have been developed, whilst also allowing for personalisation.

Watch this spot for all the changes we plan for the Global Symbols web site in the coming months! The first group of symbols will be coming from the UNICEF AAC cohort members – Jellow designed by those developing the app in India and cBoard, developed in Argentina and Israel, at present using the Mulberry Symbols from Straight Street that were voted on by users and AAC supporters in the UK over a period of several years.

Hihello

We will be updating this blog as we add symbols and please join us on Facebook to discuss the changes!  We will also be tweeting about updates

Thank you UNICEF for this very thought provoking, challenging and interesting partnership. 

 

Celebrating Global Accessibility Awareness Day 2018

GAAD

We are celebrating Global Accessibility Awareness Day  with many others around the world! Companies and organisations are offering accessibility advice and support alongside exciting new ideas that have been filling our twitter feeds and @gbla11yday

The Australian Network on Disability has started the day by providing a useful collection of videos, articles and resources all linked to making things easier to use by those with disabilities.

In the UK the BBC have a theme of Access All Areas 2018 with talks about Accessible Gaming, Deaf awareness in the workplace, Voice assistants and spoken interfaces to name just a few of the subjects from well known experts.  The Accessible Gaming is linked to the work of Microsoft and they have launched an AI for Accessibility theme saying:

accessibility“We have started to see the impact AI can have in accelerating accessible technology. Microsoft Translator is today empowering people who are deaf or hard of hearing with real-time captioning of conversations. Helpicto, an application that turns voice commands into images, is enabling children in France with autism to better understand situations and communicate with others. And, Seeing AI and auto alt-text features are helping narrate the world for people who are blind or low vision.” (Microsoft, May 7th, 2018)

Abilitynet have blogged about their GAAD news and all the events they are involved with over the day including a fast talking video on Web site accessibility that highlights the fact cost of ignoring the ‘purple pound’.  “Disabled people represent a massive untapped market for business with a collective spending power estimated at £249 billion.” (Independent Living)

Each week Debra Ruh, Neil Milliken, and Antonio Santos host a Twitterchat providing practical advice given by experts in the field of inclusion and digital accessibility. You can find them on https://twitter.com/AXSChat. Neil will be hosting a day of talks at ATOS with live streaming of the speakers

The Paciello Group has a series of YouTube videos that cover topics from the use of screen readers to ‘The Future of Work: How Emerging Workplace Trends are Affecting People with Disabilities’. These will be streamed online in USA Eastern time from 0800.

communicationThinking about Global Symbols there is something else to look forward to The “2018 AAC In the Cloud Conference Schedule” June 26, 2018. All sessions are broadcasted live, and will be available on YouTube afterwards. You can see a link to uploaded resources/slides/handouts on the conference home page. http://aacconference.com/schedule-3/

Watch out for Coughdrop and their range of free communication boards, symbols and online support.

 

Techcess introduce SCORE (UK) working with Core vocabularies.

basic vocab charts An event held at The Barnsley Hospital Education Centre was an interesting morning concerning the translation of a core vocabulary system developed by Professor Jens Boenisch and his team from the University of Cologne.  The use of core vocabularies on a series of personalised communication charts have been translated into English from the German originals and can be provided in a folder that offers a way of encouraging the growth of language and literacy skills. Some members of the Tawasol symbols team were lucky enough to meet Professor Boenisch in 2016 when they discussed the possible translation of the system into Arabic. The visit was reported in our blog with an image of the original communication folder

Low tech to high techTechcess with Bart Noe (Jabbla) have developed a way of working with these charts on speech generating devices that speed access to symbols and therefore the creation of phrases and sentences that are read aloud with speech synthesis.  The SCORE system has been built to work with Mind Express on Windows tablet AAC systems and with the present configuration users can choose to use Symbol Stix, PCS or Widgit Symbols. MindExpress is available as a demo for 60 days and SCORE will be available for 30 days.  The system is being sold for £228 as a folder and £220 as a vocabulary that works with MindExpress.

You may have noticed that colour coding has been used with the symbols.  In this case the a modified version of the Fitzgerald key colour system has been used to encourage correct sentence structure.

“Modified Fitzgerald Key

Blue: Adjectives
Green: Verbs
Yellow: Pronouns
Orange: Nouns
White: Conjunctions
Pink: Prepositions, social words
Purple: Questions
Brown: Adverbs
Red: Important function words, negation, emergency words
Grey: Determiners ” Praactical AAC . Another example of the code is shown below.

Fitzgerald colour key

The SCORE symbol layout system illustrates how important it is to aim for the smallest number of keystrokes that can be achieved to select a required word.  So high frequency words can be reached with one stroke, most are two strokes away and fringe words should not be more than four key strokes away.  When working in different languages (for bilingual situations) it is important to not only have the core words within easy reach, but to also be aware of word placement within phrases and sentences with correct use of grammar.  This means that charts may have parts of speech appearing in different places depending on the language being used.

The SCORE vocabulary allows for the use of natural language, with a full range of correct grammar,  to create sentences with both symbols and alphabetical options so suits both children and adults.  Users can adapt charts with additional symbols and layouts, but it is important to maintain consistent word/symbol positioning for individual languages, so that predictable motor patterns can be developed to aid automaticity and speed communication.

 

 

New York and the UNICEF AAC Cohort

David Banes and I were lucky enough to meet up with an exciting group of start-up companies in the world of open source AAC. The UNICEF AAC Cohort is made up of CIREHA (Argentina), Daokudai (China) and Ninaad Digital Technology (India) led by the UNICEF Innovation team based in New York.  We met up at 101 Park Avenue for three days of intense workshops and one on one meetings.  It was an exciting agenda with topics ranging from open source application development by Atul Varma to localisation, developing personas, business plans and budgets.  Gabriella Levine provided a thought provoking risk analysis guide with some very helpful advice relevant to both open source hardware and software development.

The slides below about Open Source Development have been copied into Slidewiki from Atul’s github pages. 

Associate Professor Ayesha Butt from University of Riphah in Islamabad has also let us show her slides.  She highlighted the importance of localisation, making systems that work for the individual in their own settings with symbols that support cultural needs and suitable social settings.

Mercy Kirui, Matthew Utterback and Will Clurman from Ekitabu, Kenya were also in attendance as they were sharing what they had learnt in the set up period of their UNICEF supported open source, cross-platform e-reading system for those with print disabilities, such as visual impairment.  Their company provides access to ebooks that offer an accessibility provision in schools and described how they dealt with data collection, marketing and their business model.  They also mentioned their successful digital essay competitions and events such as book fairs and have now set up a Digital Literacy Trust

Each AAC company gave us a final summary of their plans for the future based on all the ideas that had been shared over the three days.

communication boardCIREHA showed us the early version of cBoard that uses  Mulberry Symbols and offers support for those with complex communication needs.  It is an open source online and offline dynamic symbol board system with text to speech, that aims to offer a flexible approach to the creation of communication boards so that it can be personalised to suit both children and adults.  An early online version of cBoard can be be used as a demonstration of what is to come. Many ideas were discussed about the use of OpenBoard format that we have also used to produce communication charts along with the adoption of other Creative Commons symbol sets!

Yuudee sample symbol chartDaokudai (China) are developing Yuudee2, an application designed for children with Autism Spectrum Disorders (ASD) or with language disorders.  They have an interesting use of animated symbols working with speech recognition to improve speech and language in social situations. Earlier versions of the app are available as Yuudee on Google Play

Jellow symbols Ninaad Digital Technology (India) have been developing Jellow that aims to support children with speech and language difficulties.  The symbols have been designed with the particular needs of the local community in mind and but are provided under a Creative Commons licence for sharing with a wider audience.  They are also used in communication charts, booklets and as an app available in Hindi and UK/USA English from Google Play There were interesting discussions about data collection and Ninaad showed the power of analysing the data they have received as a result of sessions using their app.   It was possible to not only see how long the app had been used but also which areas attracted most attention.

We really are looking forward to supporting the development of cBoard, Yuudee2 and Jellow as open source projects in the future.  It will be exciting to see if we can can integrate their languages with translations and symbol sets with others that are available with Creative Commons licences.  This would allow many more AAC users to benefit from the UNICEF Innovation funded AAC Cohort’s work.  You can tell it was an inspiring week away from the day to day job!

Happy New Year and welcome to our new logo and name

Global Symbols logoAs we start the new year and wish you all the best for 2018, the team behind the research and development of the Arabic Symbol Dictionary have updated their website and renamed it globalsymbols.com.

We want to continue to support communities to develop and host new symbol sets for a range of languages supporting communication and literacy for those with special needs and we hope you will join us on this journey.

Mada, the Qatar Assistive Technology Center will be maintaining Tawasol symbols (keeping the original name on their servers).  However, the original Arabic symbol dictionary remains on our secure server with your data and passwords allowing you to have access to all the freely available resources and those we develop in the future alongside links to other similarly licensed materials namely Creative Commons Share Alike Licence (cc-by-40).

This blog will continue to be our way of sharing the research aspect of the international symbol dictionary development, as we explore new ideas.

friendsWatch this space as we start to work with colleagues in Pakistan thanks to a travel grant from the Global Challenges Research Fund

Payment for the flights and accommodation will allow a member of the team to visit Lahore and Islamabad.  It is hoped that we can find ways of applying for funding to adapt and add to the symbol dictionary to reflect local linguistic concepts and culture with AAC users, families, carers and experts in the region using the online voting systems to develop resources that suit local requirements.  It will also be great to have  feedback through Facebook as we have  done in the past.

 

Using Tawasol Symbols on apps for portable technologies

Over the last few months there have been several presentations around the world about Tawasol Symbols including the GREAT Conference in Doha where Nadine talked about “Making Educational Resources with TAWASOL Symbols to Support Students with Disabilities”.   By chance we will also be showcasing our work at another GREAT meeting – The World Health Organisation Global Research and Education in Assistive Technology (GREAT) Summit in August.

There we will be talking about how Tawasol symbols can be used to enhance AAC outcomes for those working in the Arabic language.  But  as can be seen in this Slideshare presentation there are now several ways of presenting symbols in apps with text to speech output when using portable or mobile technologies such as The Grid, CoughDrop and The Open Voice Factory (used to AzuleJoe).

 

Other apps that will support Tawasol Symbols with Arabic speech output include

  •  TouchChat AAC chat app for Apple iOS systems such as the iPad and iPhone
  • Proloquo offers apps such as Proloquo4Text 2.0 with Arabic voices.
  • Go Talk Now is a very flexible app that offers other symbol sets with the Acapela voices but also allows the import of personalised images.
  • Colourful Semantics in Arabic comes with lots of resources to build on sentence construction and story telling skills.
  • The Babnoor app  has been developed in Dubai to support those with Autism and provides an easy to use Arabic interface with its own symbols but others can be added along with local dialect voices.

We know there are more to come and hope to also see Tawasol symbols being used in other language such as Hindi on Jellow – also available in English.

Arabic Speech Corpus shared by Dr. Nawar Halabi

respond symbol with audioIf you have been using our Arabic symbols page you will have noticed that we have made every phoneme for our lexical entries available as a sound file, so that you can hear how it is pronounced. You can see the audio links at the bottom of the symbol for ‘respond’ in the picture beside this text.   This can help those who have literacy skills difficulties as well as those wish to learn Arabic.

Nawar, who has been part of our Tawasol Symbols project from the beginning at the same time as successfully completing  his PhD, has made this possible with the development of an Arabic Speech Corpus with support from the University of Southampton and MicrolinkPC.

The synthesised speech output that results from this corpse is a very natural sounding voice, recorded using Levantine Arabic, as heard in and around Damascus.  Levantine Arabic is considered one of the three main Arabic dialects and differs from Gulf Arabic in some aspects of grammar and pronunciation although when phonemes are read aloud, they are often nearer Modern Standard Arabic and when combined there is less dialectal impact.

The corpus has been made available for download as a zip file and is released under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.  As the Arabic Speech Corpus website says the packages includes:

  • 1813 .wav files containing spoken utterances.
  • 1813 .lab files containing text utterances.
  • 1813 .TextGrid files containing the phoneme labels with time stamps of the boundaries where these occur in the .wav files. These files can be opened using Praat software.
  • phonetic-transcript.txt which has the form “[wav_filename]” “[Phoneme Sequence]” in every line.
  • orthographic-transcript.txt which has the form “[wav_filename]” “[Orthographic Transcript]” in every line. Orthography is in Buckwalter Format which is friendlier where there is software that does not read Arabic script. It can be easily converted back to Arabic.
  • There is an extra 18 minutes of fully annotated corpus (separate from above, but with the same structure as above) which was used to evaluate the corpus (see PhD thesis). Feel free to use this in your applications.

Please contact Nawar Halabi by email for further information.

Question words – Why are they so hard to design?

question

question

We have been having another look at our question words and it is really hard to design ones that make sense in all contexts.  If you are working in a European language such as English you will be used to the concept of the ‘wh’ words and there is a particular order in which they tend to appear as mentioned by Mira Shah  “what,” “where,” “why,” “how,” “when,” then “which.” “Where” is earlier in English and “who” is earlier in Italian.”

ARASAAC offer a collection of symbols for them

what

what

where

where

why

why

how

how

when

when

 

which

which

But then one has to think about the context and it might be time or an action.  So should one add more clues or just have what + Time as a separate question mark with a clock face?

 

 

The general consensus amongst the symbol sets  seems to be that ‘what’ is just a question mark and the other question words come with a clue but that can be difficult when the word in Arabic is combined and there needs to be a difference in the gender…

what

what

what is your name (F)?

what is your name (F)?

what is your name?

what is your name (M)?

where

where

why

why (F)

What time is it?

What time is it?

 

It seems that we need to have many more question words that are linked to relevant settings but they can cause confusion, such as ‘when?’ and ‘what is the time?’ or ‘what time is it?’

 

when

when

There do not seem to be any articles on this subject and as different languages have very different ways of asking questions we will go on developing more symbols so there is lots of choice!

Any ideas for ‘which’?

 

how are you

how are you?