Category Archives: Meetings

Show and Tell – Reflections on Design and Issues arising.

This was an instant meeting with Yunjia, Mike and E.A. Yunjia wanted to explain some important issues that were arising and to show us where these were taking the project in terms of the responsive design (link provides a good example when you reduce the size of your browser window).

The metadata about the video is very important if you want to show the video and audio in the right way on each device…. Lower level metadata shows format and the kind of player needed. Youtube provides the right sort of medadata, Kaltura provides metadata

For responsive design you need metadata – 2 categories lower level – encoding, framerate etc and this can also be divided into two categories one for multimedia services like Youtube or Vimio and Kaltura which have transcoded services with the metadata already available. Delivery is speedy and accurate when it comes player and device choices.

Then the other lower level category is a web server that just delivers the mulltimedia files without explicit metadata – in other words the metadata has to be checked before delivery and the entire file has to be downloaded to be able to jump to sections. This takes up bandwidth and slows response times at the delivery point with no guarantee of correct player or device choice. There needs to be another service to carry out the transcoding on the fly. This might be possibly implemented by cloud to cloud service composition.

The higher level metadata is about how the user interacts with the annotations and how the user sees the media with standards such as HTML 5 tracks, accessibility information, schema.org and open graph plus information around the user annotations e.g. notes, comments, captions etc.

If we can combine all the elements discussed regarding the metadata categories and their content in theory we should have a truly responsive design for Synote mobile. At present collecting annotations is possible but some aspects of display are only half way there due to incomplete standards in particular with HTML 5. The same can be said about the metadata standards. However, the pre-transcoding is all possible but at a cost if you want to run your own service – using YouTube has no cost but there is a copyright problem and some confidentiality issues may arise.

There is a lack of mutual understanding between APIs for the services – this means it is hard to collect media from different services – each has their own semantic descriptions for their API and therefore each link from Synote mobile would have to have its own API to get access to that specific service. Synote mobile would need to tell users which services it can work with rather than allowing users to take any media from anywhere.

At present Synote mobile prototype works on an iPad and 7 inch Android tablet with YouTube and could work on a hosted Kaltura server but at a cost. A future approach could be a responsive delivery on the fly from a cloud storage system to a cloud transcoding service – this would once again come at a cost.

There is also the fall forward approach when it comes to media players and it is intended that should the player presented (which is an HTML 5 player fail) on the device Synote mobile will present the user with Silverlight or Flash called Media element GS and vice versa in a fall back situation.

Responsive Design – Minutes from the 3rd team meeting

As mentioned in the previous blog this meeting was attended by Dr Mike Wald, Yunjia Li and E.A.Draffan – Gary Wills sent his apologies.

Responsive Design

It all started on the white board where Yunjia explained how he was adapting his design to fit with the problem of 7 inch plus screens versus the average smart phone screen. He demonstrated the ever narrowing of the viewing area and how the video, annotation and transcription windows could shrink and eventually offer access via tabs. But when it came to mobile phone users there was a need to rearrange the view to be totally linear. In other words two versions of Synote mobile but with changes that would happen automatically depending on the metadata received.

Here are four views of Synote mobile on the different sized screens

Screen size 1200 pixels and above

screen size 980 to 1200

screen size 768 to 979

screen size 480 to 767

It will be very important to gather the metadata from the media so that the correct view and player can be selected. At present there are 3 useful approaches for this demonstrator – ffmpeg and YouTube has an api with metadata such as title, resolution etc and finally the link to the file format itself offers metadata which can be drawn out by a commercial service and this would need to be re-created if we did not want to pay for the service. The minimum amount of data needed is the duration of the media, the format and coding and whether it is video or audio.

It just so happened that there was a blog on this subject that fitted the discussion around screen size issues. Titled “What’s next for mobile now that adaptive design has failed?” it seemed like an appropriate article to read before the meeting.

“Desktop web browsers, tablets, and mobile devices are fundamentally different and are used in very different ways. Across our properties at CBS Interactive, we have experimented with a variety of adaptive and direct designs and are learning the hard way that a one-size-fits-all solution delivers a subpar user experience.” Peter Yared (VentureBeat MobileBeat blog)

Annotations / Captions

Captions can be displayed on a desktop with the video but there appear to be no standards for displaying captions within web pages across all tablets and mobiles alongside transcriptions and annotations.

At present if someone is deaf they have to read the caption and watch the video and then scroll down to the note taking mode. It is not possible on the iPhone to display both the transcript window and the video due to the size of the screen. However, it will be possible to capture an image from the video and annotate this as part of the note taking process.

3rd Team Meeting – Wednesday 23rd May, 2012

This month seems to have gone by rather quickly and this team meeting will be a small one with Dr Gary Wills away at the moment. Dr Mike Wald, Yunjia Li and E.A Draffan will meet to hear about Yunjia’s “responsive design for Synote Player and the pages he will be presenting using wireframework’.

He has set the discussion going with the following statement:

“This project will use basic principles of responsive web design to create a new mobile HTML5 version of Synote, able to replay Synote recordings on any student’s mobile device capable of connecting to the Internet.”

Item 2 will be how this is achieved on the various sizes monitors such as those offered on the iPad, Galaxy Tab and iPhone etc.

Item 3 will include the issues that have arisen with the player and being able to read annotations simultaneously whilst watching videos.

As Dr Mike Wald said in a recent email to the team “One issue with captioning overlayed on the video (as suggested for Synote Mobile) rather than in a separate panel (as we have currently in Synote) is how can a hearing impaired person read the captions when the video is a thumbnail or not displayed at all? There needs to be a way the captions can be displayed separately … Matterhorn designers are now thinking about ways to do this …