Tag Archives: HTML5

HTML 5 Video and Support in Mobile Browsers – Lessons Learnt

Yunjia Li wrote on his personal blog the following article after our last Synote Mobile meeting with Wei Jing who has been helping us develop the tables you see below.

“Since proposed by Opera Software in 2007, the HTML5 video tag has been around for a pretty long time. Video tag is a great step forward to make video the first class citizen of the Web. Unfortunately, different browsers understand the citizenships in different ways. The citizenship of AVI is terminated. WMV needs to hold Sliverlight Visa in order to get the entry clearance to browsers other than IE. FLV’s visa application for iOS has been Denied by Apple Empire and Adobe has declared that he’s not going to help FLV to re-apply the visa again. H.264 is like a mafia, who asks for protection fees from browsers. You see, there is no Federal Government in HTML5.

Since the smart phone joined the Web, things are getting worse again. It seems to me that there is no systemetic tests for the video compitablity on different mobile browsers. So the goal of our test is to find out which video format could be played in which browsers, or which major versions of the browsers.

Our Test

We need to emphasis something before going into the test.

All the browsers in the test are on mobile phones. They are not desktop versions of the browsers.
We only tested on android and iphones. Browsers for Windows Phone, Blackberry and Palm WebOS are not tested.
All the tests are carried out in mobile phones. Well, yes, you can make a phone call using 7 inch Samsung Galaxy tablets if you don’t feel it’s too big. But the mobile phones used in this test are 3.0 to 4.8 inches for the screensize.
Not all browsers, as well as a specific version of the browser, could be used in all different mobile devices. For example, there is no Opera Mobile browser on iPhone iOS 5. It is the same case for Firefox. But at the time of testing, we chose the latest version of the browser in that OS.
We suppose that the server which hosts the video file is properly configured in order that the file could be properly delivered.

Finally, we have tried our best to keep the data correct on the phones we have tested. But it’s quite likely that some of the test results are not accurate. If you find anything that is not correct, please feel free to let me know. I would be glad to test it again. OK. Let’s go!

Test 1, Playable

This test shows if a video file with a certain codec could be played in a certain browser on a mobile phone. We test mp4, ogv and webm on Android 2.3, 4.0 and iOS 5. We designed a test page, which embeds mp4, ogg and Webm videos in one page. Then we open this page in different mobile browsers on different mobile OSs. Table 1,2 and 3 shows the results.

**Table 1.**HTM5 Video Compatibility in Android 2.3
Browser	MP4, H.264	Ogg, Theora	WebM, VP8
Opera Mobile 12.0	Yes	No	No
Opera Mini 7.0.29	No	No	No
Firefox 14.0.1	No	Yes*	Yes*
Android Native Browser	Yes	No	No

*Unlike other browsers, the video will not enter full screen when playing, and there is no full screen control in the player in Firefox 14.0.1.

Test Phones for Table 1:

HTC Design S, Android 2.3.5, 3.7-incn touch screen
Sony SK17i, Android 2.3.4, 3.0-inch touch screen
Samsung Galaxy S II, Android version 2.3.3, 4.3-inch touch screen
HTC G10, Android 2.3.5, 4.3-incn touch screen

P.S. Google Chrome is not compatible with Android 2.3.

**Table 2.**HTM5 Video Compatibility in Android 4.0.4
Browser	MP4, H.264	Ogg, Theora	WebM, VP8
Opera Mobile 12.0	Yes	No	No
Opera Mini 7.0.29	No	No	No
Chrome 18.0	Yes	No	Yes
Firefox 14.0.1	No	Yes	Yes
Android Native Browser	Yes	No	No

Test Phones for Table 2:

Samsung Galaxy S III, 4.8-inch touch screen

**Table 3.**HTM5 Video Compatibility in iOS 5.1.1
Browser	MP4, H.264	Ogg, Theora	WebM, VP8
Safari 5	Yes	No	No
Chrome 21.0	Yes	No	No
Opera Mini 7.0.29	No	No	No

Test Phones for Table 3:

iPhone 4S, 3.5 inches screen
iPhone 4, 3.5 inches screen

As a summary, here are some interesting findings:

There is no “killer” video codecing format for mobile browsers currently. But MP4 seems compatible with most “default” browsers, i.e. Android Native browser in Android phons and Safari in iPhones.
Opera Mini is pretty disappointing as no video could be played.
Different from desktop browsers (see this table), WebM is not well supported by the native players on mobile browsers. Interestingly, WebM is developed (or more actually, sponsored) by Google, but the Android Native Browser, which is also developed by Google, can’t play WebM
Most mobile phones play the video in full screen mode by default and users cannot quit the full screen unless stopping the video. But Firefox is an exception. What’s more, if the screen size of the mobile phone is larger than 4 inches, The full screen, if screensize is bigger than 4.3 inches, sometimes, video won’t be played in full screen mode automatically.

WebVTT support in Mobile Browsers

HTML5 video tag not only brings video playing natively, but also adds some cool features alongside the video. One of them is the track tag for subtitles. Subtitles are very important for the accessiblity of videos on the Web. Many standards and de facto standards have been around for a long time. SubRip (SRT), Timed Text Markup Language and .sub, to name a few. I personally think the most promising one is Web Video Text Track (WebVTT). WebVTT is specially tailored for videos on the Web and you can also use it for audio description, chapter navigation, etc. Silvia Pfeiffer has given an very good presentation about WebVTT.

WebVTT is still a moving target and there are many things which need to be nailed before the first release. Except for the WebVTT specification itself, as developers, we are really care about the technical support of WebVTT in different browsers and are there any tools out there we can easily use? Silvia summarised the current situation of WebVTT support in different desktop browsers.

A further question we ask is that is there any mobile browsers which support WebVTT natively? After our test, the answer is NO! Table 4 is our test result. “N/A” means a certain format of the video file cannot be played in that browser, so it is meaning less to say if the WebVTT can be displayed or not.

**Table 4.**WebVTT Support in Different Mobile Browsers
OS	Browser	MP4, H.264	Ogg, Theora	WebM, VP8
Android 2.3	Opera Mobile 12.0	No	N/A	N/A
	Opera Mini 7.0.29	N/A	N/A	N/A
	Firefox 14.0.1	N/A	No	No
	Android Native Browser	No	N/A	N/A
Android 4.0.4	Opera Mobile 12.0	No	N/A	N/A
	Opera Mini 7.0.29	N/A	N/A	N/A
	Chrome 18.0	No	N/A	No
	Firefox 14.0.1	N/A	No	No
	Android Native Browser	No	N/A	N/A
iPhone iOS 5	Safari 5	No	N/A	N/A
	Chrome 21.0	No	N/A	N/A
	Opera Mini 7.0.29	N/A	N/A	N/A

Of course, there are many HTML5 video players on the Web, VideoJS, PopcornJS, MediaEleemntJS, JWPlayer, Kaltura Player, etc. Some of indeed support WebVTT on desktop browsers. But they are all “polyfills”, which means the support is not native and they use javascript and CSS to somehow present WebVTT together with the video. We did test MediaElementJS on iphones, but unfortunately, it still failed to display the subtitle. However, MediaElementJS can sucessfuly display WebVTT on iPad.

Conclusions and Future Directions

The test for HTML5 video tag and WebVTT on mobile browsers is important. Developers should be clear what video could be played in the target device. On the server side, the web applications need to host the best format that compitable with the target device. Video sharing services, such as YouTube, must be adaptive enough to deliver videos to different devices and browsers. Subtitles are always important for videos, especially on mobile platforms, where the support for subtitle is not thorough yet.

With each updates or release of new versions of browsers, new codecs might be supported and old formats might be abandoned. WebVTT is currently is not well supported on mobile browsers and we are still waiting for some break-through.

The tests tell us the HTML5 video support is on the way. In the tests above, we are trying to keep things easy, i.e. we didn’t bring in a lot of variables and we just used the most popular phones and most popular browsers. However, the real world is not that simple. Between mobile phones and desktop machine, we have got tablets, 7 inches to 10 inches. The screensize of mobile phones are getting bigger and bigger. I am very curious on how different browser kernels choose the display mode for these screensizes. Is there a borderline or threshold? So in the future, it might be necessary to compare the HTML5 video tag in a single browser across different size of devices (phones and tablets). We are also expecting more documents could be released or discussed regarding this aspect.

Again, if you find anything wrong or anything new, please feel free to leave your comments.”

Feedback on prototype Synote Mobile

comparing Galaxy and iPad presentation of Synote mobile

This meeting provided Yunjia with the first chance for a small group of users to provide feedback on the prototype of Synote Mobile using 7 and 9.7 inch tablets with Safari, Opera and Chrome browsers.

Yunjia is using CSS3 media query and viewing port size to decide when you are working with multimedia materials and how they will be displayed on devices – so it is hoped the user automatically experiences a comfortable presentation. In terms of Synote Mobile this means that when the viewing window is below 768 pixels the two columns that hold the transcript and synmarks (annotations) collapse into one. When a video or audio is played the synmarks will be on one tab with the closed captions allowing you to see the transcript across the window. The user can also choose just to read the transcript via that tab.

One issue that has been discovered by taking screen size as a method for changing presentation of videos and transcripts is that the CSS3 media query is dependent on screen size and videos are dependent on pixel widths. It is a tricky problem to explain as it appears that CSS media queries result in different behaviours for continuous media and paged media. There is also an inconsistency in behaviour across devices and browsers.

The controls are dependent on viewing modes, devices and browsers – so when using landscape mode on the iPad it senses that it is in desktop mode. When playing the video and viewing the transcript the player controls are always seen but when using Portrait mode the CSS media query senses the reduced width and represents the web page without the player controls as you scroll down. Yunjia says it is possible to solve the problem on some devices, but because this does not work for all devices it has been decided that ‘width’ is probably the best method for defining when the controls will be shown – Yunjia is using ‘Bootstrap’ from Twitter at present, which is considered the most consistent environment across all devices when it comes to presenting media data.

When discussing other user issues such as the look and feel, the Synote Researcher website used as the basis for viewing recordings was considered to be easy to use and attractive but there were one or two login issues on the iPhone.

One user suggested that the highlights of synmarks (annotations) should be different from the transcript highlights as the video or audio plays – the example provided shows a YouTube video working with Synote Mobile.

If there is a large chunk in the transcript when viewed on the video the caption will overlay too much of the picture. YouTube have set their automatic speech recognition captioning to always generate 2 lines – if the transcript is put in manually the size can be reduced to two lines for each timed section so it appears in a similar fashion but work is needed in this area to ensure this occurs in Synote mobile. It would be possible to edit the transcripts with an autosync system that divides longer sections into 80 character chunks that display as two lines. This cannot be done on the fly but would need to occur as part of the process in the database where the separate caption record is kept. This would not be necessary for audio as can be seen from the image as the caption can be displayed on a plain background.

The mobile phone version is still being adapted with single column viewing and the transcript appearing as captions – at present it has reached the viewing stage but not editing mode. A further meeting with the developer is planned on August 3rd, 2012

Discussions around embedding media players

There has been much discussion over the last few weeks about embedding media players within web pages that will automatically adapt to the user’s chosen device. The responsive design needs to be more than just responsive to one or two issues as we have discovered – there needs to be fall forward and fall back options! As Yunjia has said

The challenge for HTML5 video/audio in Synote is we need to embed different players not only based on the media type, but also the platform. Obviously, Flash is not well supported on mobile platform. So we must consider the html5 native player and control it through javascript.

MediaElement.js is a “fallforward” html player, which means it is based on HTML5 native player. However, if the browser doesn’t support HTML5, mediaelement.js will embed the self-developed flash and silverlight player. Hopefully this option is going to provide us with a solution to the issue!

The website MediaElement.js has provided a comparison of codecing and applicable browsers, including mobile devices.

There is also a large table providing a useful comparison of the HTML5 video player with available features.