Category Archives: Evaluations

REALISE Transferability Workshop

building 32

©2010 François-Xavier Beckers (CC-BY)

The Realise project held its Transferability Workshop on June 7th at the University of Southampton in Building 32.  Thirty brave souls, all experts in their own fields accepted our invitation. (Word doc download) We are incredibly grateful for their time and energy as the day was spent discussing elements of the REALISE market place, its use, community involvement and improvements that could be made to ensure its sustainability.  There were five groups of six, each table taking on different ideas with the notion that they could become open innovation projects thanks to community involvement, mentoring and online guidance.  But what of the transferability!

“Transferability refers to the degree to which the results of qualitative research can be generalized or transferred to other contexts or settings.” (Research Methods Knowledge Base)  With this in mind those attending the workshop came from different sectors of both education and health with links to assistive technology (AT).  Experts came from the two regions of Southampton and Sheffield as well as other parts of the country.  There were several individuals from other universities and organisations already involved with open innovation projects such as the JISC funded ‘Rave in Context‘ (Oxford University) and the Widget Design Authoring Toolkit (WIDGaT) (Teeside University).  Plus those from three companies who are members of the British Assistive Technology Association with developers from the NHS, education and business such as TDM

Attendees came from Devices for Dignity who have been involved with the project from the outset along with colleagues from Sheffield 50+ and REMAP (Southampton) and Paxton Crafts Charitable Trust providing the open source Straight Street Mulberry symbol system added their user knowledge to the discussions.  Our critical friend Nigel Spencer from the British Library mentioned his EU funded Open innovation project about ‘Working with others to make new ideas fly’ and Andy Heath brought his experience of the EU4ALL project to the table.

sound levels

Thanks to Deafness Research UK

The discussions topped the 100 decibel level at times and the team had to keep a strong handle on the time table to allow for a few speaking slots!  Punam Khosla from TechDis talked about her BCE and Stakeholder work and ACUMEN work with small and medium sized enterprises (SMEs), which was mentioned at our last Support and Synthesis group meeting. Dr Nasser Siabi as CEO of MicrolinkPC talked about his work with open source tools in South Africa and the transfer of applications such as ATbar to other sectors such as employment and charitable organisations e.g.  Fix the Web.

The team also had the support of two researchers to collect data and recordings of the hands-on sessions around ideas, incubation and projects which each ended with a little form filling to provide feedback. Points of importance were mentioned in whispered tones by Will Allen to some us seeking advice and networking was well underway in the intervals. The plan for the day, questionnaires plus results were made available on paper and as a set of slides (download .ppt) at the end of the day. We have to thank Peter Cudd for this output, Lauren for organising all the food plus all those who came from near and far to make it an amazingly useful day for us all. Thank you very much.

Join the REALISE Market Place! Open innovation for Assistive Technologies

Screen shot RealiseThe team have been working on the final phase of the REALISE Market Place. Recent changes to the website have been the result of our evaluation and discussions both off and on the project Google Group .  Do come and join us there!

The team’s 2 day meeting in March has resulted in the addition of the OSS Watch Openness Rating that is really an experts view on how to rate progress in readiness for a project’s launch onto the Open Source market.   You can now follow this progress once you have signed in.  The latter is possible through a Linkedin account or you can register separately to join REALISE.

Do you have an Idea?

If you have an idea that might help someone who has a disability or help those professionals working in this area of assistive technology – Add it now! Comments can be made about submitted ideas as well as votes if you think there is a good one.  Anyone can take an idea through into the incubator and there you will discover more advice about how to build a community and good governance around the project.

Projects in the Incubator may need both financial and programming skills to take them into the final section under the Projects Tab – Here you will see their Openness Rating once you have selected an individual project,  if it is being carried out.  Code maybe available, licences discussed and websites developed for each project with their own discussion forums and bug tracking sites.  At any stage you can follow progress and become involved whatever your skills. You may be a user offering advice about features, a developer solving particular problems or even someone with a commercial eye who can help sustain a project.

question markFinally we want to generate a series of FAQs to help users.  A small sample of the planned questions are already available under the Resources tab.  Send your queries to the Google Group or add them as comments under the REALISE idea on the actual site or at each stage as we push it into the Incubator and onwards as a living project!

REALISE – Web Services Evaluation


This document is an evaluation of four different software packages that have been proposed as the foundation for the REALISE website. Two, Drupal and ATutor, are content management systems (CMS), and two, myExperiment and CloudWorks, are communities based on open-source software that would be available for us to re-use to create our own community.


Drupal is an open-source content management system that has been in development for nearly a decade. It runs on PHP and has a large development community who have produced a range of third-party add-on modules.
Drupal appears to score well in accessibility based on the Web2Access test criteria, with minor issues noted in only four of the fifteen test categories (see It is also extensible, with a wide range of third-party modules available enabling it to provide a broad range of features (including all of the requirements of the REALISE project).
Installation is simple for a UNIX administrator – one simply downloads and extracts the files, sets appropriate permissions, creates an empty database and runs Drupal’s self-installation script. It is unlikely that anyone without technical experience would be able to install this system.
A fresh installation of Drupal uses only the most basic of modules required to run the system. Many other “official” modules, including forum and blog support, are inactive by default. However, one of the main drawbacks of Drupal is its immensity; the admin menu system is labyrinthine, and it can take a long time to locate any given option or feature. The system also looks and feels bloated. Having said this, it took only a couple of hours to have a dummy REALISE site (still available at up and running.
As detailed here, Drupal is known for using large amounts of memory. Fatal errors related to this were encountered whilst creating even this simple dummy site. It also has a reputation for security flaws and poorly written code, which could cause us expensive difficulties further down the road.
Overall, it may be simpler and more effective to create our own content management system using a framework such as Ruby on Rails or Python Django rather than to use a ready-made but buggy solution like Drupal.


ATutor is another open-source web-based content management system. It describes itself as a “Learning Content Management System”, and is designed for use in an education setting. Content is built up into units called “Courses”, which are administered by “Instructors” and used by “Students”.
An immediate difficulty which arises with ATutor is the difficulty of mapping the requirements of the REALISE project (e.g. accessibility products) to ATutor modules (i.e. courses). ATutor is designed for a very specific purpose, and would need modification to be suitable for REALISE.
ATutor, despite having been designed with accessibility in mind, does not score as well for accessibility under the Web2Access test criteria as Drupal (see It performs solidly nevertheless, with notable deficiencies in only two areas; its rich text editor is completely inaccessible without a mouse, and the page layout breaks when using the zoom feature on any major web browser. This makes it unlikely to be suitable for visually impaired users.
As with Drupal, ATutor installation was simple enough for a UNIX administrator, but would be difficult if not impossible for an average user.
Being designed for a specific purpose rather than as a generic solution, ATutor is noticeably smaller and easier to navigate than Drupal. The admin menu systems could still be very confusing for a novice user, however. ATutor is also much less widely used than Drupal, and thus has far less third-party support and extensions. A dummy site was not created in ATutor due to the mapping difficulties mentioned above.
Overall, ATutor has few advantages over Drupal and does not appear to be suitable for use in the REALISE project.


myExperiment is a collaborative working environment designed to enable scientists to share their workflows and experiment plans. It was built using the open source Ruby on Rails, and ECS was involved in its development. Once logged in, a user can upload workflows and other files, and create “packs” that tie them together.
As with ATutor, this presents a mapping problem. In this case, the problem is almost certainly insurmountable, as the site does not provide all of the features necessary for the REALISE project; chiefly, it lacks a suitable method for submitting a product idea for review by the community.
Installation of myExperiment is a lengthy process, requiring existing knowledge of Ruby on Rails. The software has a number of dependencies on other packages, and would not be possible for someone without considerable knowledge of UNIX administration.
The site is seems quite easy to use from an end-user perspective, particularly if the user has a scientific background (in line with the site’s target demographic), but conversely, the average person would probably not understand what the term “workflow” means, let alone how to create one.
In terms of accessibility, myExperiment performs fairly well under the Web2Access tests (see The rich text editor is inaccessible with a keyboard, as is often the case, and the large amount of content displayed on each page makes navigation with the keyboard cumbersome, which is particularly evident when the stylesheet is removed. The tag clouds in particular take a long time to tab through.
In conclusion, myExperiment does not seem appropriate for use in the REALISE project.


CloudWorks is an online community for sharing ideas about learning and teaching. Content is posted in the form of “Clouds”, and can consist of anything that can be marked up in HTML or that can be embedded from another site such as YouTube. Other users can comment on each Cloud, and can add links to other clouds and web pages as well as references to academic papers. Clouds with common themes can be grouped together into sets called “Cloudscapes”. There is a potential mapping between Clouds in CloudWorks and product ideas in REALISE. Other requirements of the REALISE process are also present, including tagging, text search, and discussion (though this is not threaded and cannot be sorted).
At present, ease of installation is unknown. This is because the source code is not yet available publicly. ECS will be able to obtain this in a couple of weeks, but this would slow the project down considerably. We will ideally need to set up our own instance of CloudWorks in order to apply REALISE branding to the project,, rather than simply creating our own Cloudscape on the existing site.
CloudWorks is easy to use on the whole, and the average user would likely be able to grasp how to use it in a fairly short space of time. The interface is very clean, with little clutter on the screen. Online help (including an optional introductory video) and e-mail support is available.
Accessibility according to the Web2Access guidelines is very good, with the only notable issue being the lack of on-screen feedback after content is submitted. It has performed better under these tests then the other three products that we are evaluating (see
In essence, CloudWorks may be the most appropriate solution out of the four services evaluated, but it will be difficult to know for sure until we can acquire the source code.


The LinkedIn API is being considered for use in the REALISE project. This would enable users to log in to the REALISE site with their LinkedIn credentials, and would allow the REALISE site to make use of LinkedIn users’ profiles and networks.
The accessibility of is important if we are to require users of REALISE to have a LinkedIn account. LinkedIn was checked against the Web2Access accessibility criteria in August 2009. The results obtained then still apply, though it has been noted that the sign-in form for the site is not located on the home page and must be navigated to, which is unusual and potentially confusing for some users. The only other problems with the site, as noted in the original review, are that it makes use of font sizes that are too small to read comfortably, that form feedback is sometimes incompatible with some screen readers, and that some non-critical page elements can only be used with a mouse.
The LinkedIn API could be a useful addition to the REALISE site as it would enable us to draw on LinkedIn’s social networking features and may encourage LinkedIn users to make accessibility projects a part of their LinkedIn portfolio. However, there is a potential caveat – some users, particularly those of a more casual nature, may be put off by having to make a LinkedIn account in order to use the REALISE site. It may be possible to work around this by making it optional to link one’s REALISE profile to one’s LinkedIn account.