This document is an evaluation of four different software packages that have been proposed as the foundation for the REALISE website. Two, Drupal and ATutor, are content management systems (CMS), and two, myExperiment and CloudWorks, are communities based on open-source software that would be available for us to re-use to create our own community.
Drupal is an open-source content management system that has been in development for nearly a decade. It runs on PHP and has a large development community who have produced a range of third-party add-on modules.
Drupal appears to score well in accessibility based on the Web2Access test criteria, with minor issues noted in only four of the fifteen test categories (see http://www.web2access.org.uk/product/233/). It is also extensible, with a wide range of third-party modules available enabling it to provide a broad range of features (including all of the requirements of the REALISE project).
Installation is simple for a UNIX administrator – one simply downloads and extracts the files, sets appropriate permissions, creates an empty database and runs Drupal’s self-installation script. It is unlikely that anyone without technical experience would be able to install this system.
A fresh installation of Drupal uses only the most basic of modules required to run the system. Many other “official” modules, including forum and blog support, are inactive by default. However, one of the main drawbacks of Drupal is its immensity; the admin menu system is labyrinthine, and it can take a long time to locate any given option or feature. The system also looks and feels bloated. Having said this, it took only a couple of hours to have a dummy REALISE site (still available at www.realisepotential.org) up and running.
As detailed here, Drupal is known for using large amounts of memory. Fatal errors related to this were encountered whilst creating even this simple dummy site. It also has a reputation for security flaws and poorly written code, which could cause us expensive difficulties further down the road.
Overall, it may be simpler and more effective to create our own content management system using a framework such as Ruby on Rails or Python Django rather than to use a ready-made but buggy solution like Drupal.
ATutor is another open-source web-based content management system. It describes itself as a “Learning Content Management System”, and is designed for use in an education setting. Content is built up into units called “Courses”, which are administered by “Instructors” and used by “Students”.
An immediate difficulty which arises with ATutor is the difficulty of mapping the requirements of the REALISE project (e.g. accessibility products) to ATutor modules (i.e. courses). ATutor is designed for a very specific purpose, and would need modification to be suitable for REALISE.
ATutor, despite having been designed with accessibility in mind, does not score as well for accessibility under the Web2Access test criteria as Drupal (see http://www.web2access.org.uk/product/235/). It performs solidly nevertheless, with notable deficiencies in only two areas; its rich text editor is completely inaccessible without a mouse, and the page layout breaks when using the zoom feature on any major web browser. This makes it unlikely to be suitable for visually impaired users.
As with Drupal, ATutor installation was simple enough for a UNIX administrator, but would be difficult if not impossible for an average user.
Being designed for a specific purpose rather than as a generic solution, ATutor is noticeably smaller and easier to navigate than Drupal. The admin menu systems could still be very confusing for a novice user, however. ATutor is also much less widely used than Drupal, and thus has far less third-party support and extensions. A dummy site was not created in ATutor due to the mapping difficulties mentioned above.
Overall, ATutor has few advantages over Drupal and does not appear to be suitable for use in the REALISE project.
myExperiment is a collaborative working environment designed to enable scientists to share their workflows and experiment plans. It was built using the open source Ruby on Rails, and ECS was involved in its development. Once logged in, a user can upload workflows and other files, and create “packs” that tie them together.
As with ATutor, this presents a mapping problem. In this case, the problem is almost certainly insurmountable, as the site does not provide all of the features necessary for the REALISE project; chiefly, it lacks a suitable method for submitting a product idea for review by the community.
Installation of myExperiment is a lengthy process, requiring existing knowledge of Ruby on Rails. The software has a number of dependencies on other packages, and would not be possible for someone without considerable knowledge of UNIX administration.
The site is seems quite easy to use from an end-user perspective, particularly if the user has a scientific background (in line with the site’s target demographic), but conversely, the average person would probably not understand what the term “workflow” means, let alone how to create one.
In terms of accessibility, myExperiment performs fairly well under the Web2Access tests (see http://www.web2access.org.uk/product/234/). The rich text editor is inaccessible with a keyboard, as is often the case, and the large amount of content displayed on each page makes navigation with the keyboard cumbersome, which is particularly evident when the stylesheet is removed. The tag clouds in particular take a long time to tab through.
In conclusion, myExperiment does not seem appropriate for use in the REALISE project.
CloudWorks is an online community for sharing ideas about learning and teaching. Content is posted in the form of “Clouds”, and can consist of anything that can be marked up in HTML or that can be embedded from another site such as YouTube. Other users can comment on each Cloud, and can add links to other clouds and web pages as well as references to academic papers. Clouds with common themes can be grouped together into sets called “Cloudscapes”. There is a potential mapping between Clouds in CloudWorks and product ideas in REALISE. Other requirements of the REALISE process are also present, including tagging, text search, and discussion (though this is not threaded and cannot be sorted).
At present, ease of installation is unknown. This is because the source code is not yet available publicly. ECS will be able to obtain this in a couple of weeks, but this would slow the project down considerably. We will ideally need to set up our own instance of CloudWorks in order to apply REALISE branding to the project,, rather than simply creating our own Cloudscape on the existing site.
CloudWorks is easy to use on the whole, and the average user would likely be able to grasp how to use it in a fairly short space of time. The interface is very clean, with little clutter on the screen. Online help (including an optional introductory video) and e-mail support is available.
Accessibility according to the Web2Access guidelines is very good, with the only notable issue being the lack of on-screen feedback after content is submitted. It has performed better under these tests then the other three products that we are evaluating (see http://www.web2access.org.uk/product/236/).
In essence, CloudWorks may be the most appropriate solution out of the four services evaluated, but it will be difficult to know for sure until we can acquire the source code.
The LinkedIn API is being considered for use in the REALISE project. This would enable users to log in to the REALISE site with their LinkedIn credentials, and would allow the REALISE site to make use of LinkedIn users’ profiles and networks.
The accessibility of LinkedIn.com is important if we are to require users of REALISE to have a LinkedIn account. LinkedIn was checked against the Web2Access accessibility criteria in August 2009. The results obtained then still apply, though it has been noted that the sign-in form for the site is not located on the home page and must be navigated to, which is unusual and potentially confusing for some users. The only other problems with the site, as noted in the original review, are that it makes use of font sizes that are too small to read comfortably, that form feedback is sometimes incompatible with some screen readers, and that some non-critical page elements can only be used with a mouse.
The LinkedIn API could be a useful addition to the REALISE site as it would enable us to draw on LinkedIn’s social networking features and may encourage LinkedIn users to make accessibility projects a part of their LinkedIn portfolio. However, there is a potential caveat – some users, particularly those of a more casual nature, may be put off by having to make a LinkedIn account in order to use the REALISE site. It may be possible to work around this by making it optional to link one’s REALISE profile to one’s LinkedIn account.