HTML5 vs. Flash Part II [Video Benchmark on All Browsers, WebM, OGG, MP4, Flash]

By Jonathan Bowen

HTML5 vs Adobe Flash, Part II

Hello, and welcome to the second part of a three part article on HTML5 vs Adobe Flash.  In the first part we laid down the basics of both technologies and went over some of the pros and cons of each as well.  In this part, we will benchmark the HTML5 video tag against using a flash video player.  In part three we will continue by comparing and contrasting the canvas tag in HTML5 with the drawing inside Flash.  So that full disclosure is satisfied, I am an open source sort of guy and have pretty much resented Adobe since they bought Macromedia and my beloved Dreamweaver passed away.  That being said, I believe I have approached this in an objective fashion. Even when I wish it were not true, I will freely admit which certain aspects are better with the Adobe product than the open source technology.

What was tested

When endeavoring to design a test for video playback, I decided that converting the videos to the proper format needed to be the first step.  After the videos were converted, I developed three web pages using the HTML5 video tag to embed the video, and then wrote a fourth page to embed the video using flash.  Each HTML5 page uses a different codec; one for each that is supposed to be supported.  I was going to time them, in terms of how long it took them to load and play, but it turns out they were all pretty much instantaneous. So, I checked CPU usage for both technologies, with surprising results – to me, anyway. I tested all files in Opera, Firefox, Chrome, and Internet Explorer 8 on Windows.  I also tested them in Firefox, Opera and Chrome on 64 bit Fedora 14 Linux.  Internet Explorer 9 is in RC, but we tried it and it does not support the HTML5 video tag yet.  The following two matrices depict which browsers support which video delivery technology.

Windows XP Support for Embedded Video Technologies by Browser

64 bit Fedora 14 for Embedded Video Technologies by Browser

Support for Codes in HTML5 Video Tag by Browser

Prepping the videos

First of all, I had to prepare the videos to be shown with each technology.  This is because, as you are probably aware, Flash video needs an FLV file and HTML5 needs an OGG or OGV file.  I also tested using the VP8 codec with WebM.  I know that HTML5 is supposed to be able to render an H.264 encoded file as well, so I also tested with it, even though it is not truly a free codec.  Anyway, I used two videos to test the conversion characteristics, and a third for the CPU tests.  The first was a 64MB MP4 file, and the second was a 470 MB AVI file. Yes, the second one was an entire movie.  This may seem extreme to some readers, but with the growing trend of renting movies to watch on your computers, this is not as far fetched as it would have been 5 years ago.  Obviously, as developers, we need to discover the most efficient way to deliver these videos to our clients. The third was a 2.71GB AVI clip. I wanted to try to make the resultant videos have the same bitrate, aspect ratio, and frames per second, so that these variables would be eliminated.  I chose to simulate a medium quality video and set the bitrate to 1024.  I set the frames per second to be 25, and the aspect to be 352×264 or nearly cif(352×288) To convert the videos, I used ffmpeg which is a great command line utility for manipulating videos. To convert to FLV, I used these commands:

ffmpeg -i test1.mp4 -b 1024k -s 352×264 -r 25 -acodec copy test1.flv

ffmpeg -i test2.avi -b 1024k -s 352×264 -r 25 -acodec copy test2.flv

ffmpeg -i Produce_1.avi -b 1024k -s 352×264 -r 25 -acodec libmp3lame -ar 44100 test.flv

To convert them to OGG, I used the following commands which are obviously quite similar:

ffmpeg -i test1.mp4 -b 1024k -s 352×264 -r 25 -acodec libvorbis test1.ogg

ffmpeg -i test2.avi -b 1024k -s 352×264 -r 25 -acodec libvorbis test2.ogg

ffmpeg -i Produce_1.avi -b 1024k -s 352×264 -r 25 -acodec libvorbis -ar 44100 test.ogv

By the way, I also did the first two videos using the OGV extension.  It made no difference to any of the findings.  OGG and OGV are completely interchangeable for the purposes of this study.

Converting to WebM:

ffmpeg -i test1.mp4 -b 1024k -s 352×264 -r 25 -acodec libvorbis test1.webm

ffmpeg -i test2.avi -b 1024k -s 352×264 -r 25 -acodec libvorbis test2.webm

ffmpeg -i Produce_1.avi -b 1024k -s 352×264 -r 25 -acodec libvorbis -ar 44100 test.webm

Lastly, converting the AVI’s to MP4(H.264):

ffmpeg -i test2.avi -b 1024k -s 352×264 -r 25 -acodec libvorbis test2.mp4

ffmpeg -i Produce_1.avi -b 1024k -s 352×264 -r 25 -acodec libvorbis -ar 44100 test.mp4

There was a great disparity in conversion times.  The small FLV took about 3 minutes to convert as opposed to 8 minutes for the small OGG.  So the OGG took about 2.6 times longer to complete.  I was thinking that this was a linear difference and surmised that the larger video would take 2.6 times as long as well.  I was wrong.  The large FLV took about 14 minutes to convert, whereas the large OGG took about 59 minutes to convert.  This second OGG took about 4.2 times as long to convert.  From this, one can only conclude that as the file to be converted gets bigger the time to convert to OGG grows exponentially in reference to the time taken to convert the same file to FLV.  I suspect that this has to do with the audio codec, but that is just an educated guess.  MP4 times were similar to FLV, but WebM took the longest by far with the large file taking about an hour and a half to convert.

While the conversion was orders of magnitude longer for the OGG and WebM conversions, OGG seemed to have better compression while webM compression seemed exactly the same as FLV.  The first file increased to 122MB with OGG, and 129MB with FLV and WebM.  The second increased to about 998 MB for OGG and about 1.04 GB for FLV and WebM.  MP4 had the best compression of all as the chart below shows.  However, I do not think that the differences in compression are statistically significant.  Oddly enough, the file to test CPU compressed to about 105MB with every encoding scheme.

Video File Conversion Times

All in all, it was generally less time consuming to convert a clip to FLV than to any other format, although MP4 was had very similar times.

Developing in HTML5

Using the video tag in HTML was a breeze.  Here is the source of the web pages I developed to test it:

<html>
        <head>
                <title>HTML5 Video Test Using MP4</title>
        </head>
        <body>
                The video is right below this line.<br /><hr />
                <video controls='controls' height="480" width="640"
autoplay='autoplay' src='../videos/test.mp4'>
Your browser does not support video</video><br />
        </body>
</html>

OGV Page:

<html>
        <head>
                <title>HTML5 Video Test Using MP4</title>
        </head>
        <body>
                The video is right below this line.<br /><hr />
                <video controls='controls' height="480" width="640"
autoplay='autoplay' src='../videos/test.ogv'>
Your browser does not support video</video><br />
        </body>
</html>

WebM Page:

<html>
        <head>
                <title>HTML5 Video Test Using MP4</title>
        </head>
        <body>
                The video is right below this line.<br /><hr />
                <video controls='controls' height="480" width="640"
autoplay='autoplay' src='../videos/test.webm'>
Your browser does not support video</video><br />
        </body>
</html>

That is it! Less than 10 lines of code, only 5 of which are necessary.  It worked perfectly the first time! Well, almost perfectly.  Internet Explorer version 8 does not support video tags. IE 9 was purported to support the video tag, but does not yet. Safari 5.0.3 for Windows does not support video tags, either.  It worked beautifully in all the other browsers, including the 64 bit Linux ones.  I will tell you that there are not many less complex programming tasks than using the HTML5 video tag.  The only difference between the three pages is the extension on the file.  As I had said earlier, the videos starting playing virtually instantaneously.  Really, the only drawback I saw was that there was no support for full screen mode.  One could make it take up most of the screen by using the height and width parameters, though, as sort of a workaround, but that really is not the same.  In my opinion, the reason there is no full screen support is simply because HTML5 is an emerging standard and therefore not a mature technology like Flash is.  The HTML5 player included volume controls and a elapsed time widget, so I would be shocked if the future does not include a full screen mode for it.

Developing for Flash

If the ease of programming for the HTML5 video tag is one on a scale of one to ten, I would rate Flash an 11! To start with, I was determined to use an open, or at least free flash tool to design this.  This is not unreasonable, as I used VIM to code the HTML5 video tag.  I could not find one.  Not only that, I had trouble finding examples in my research that showed how to code flash video from a text editor.  I did find a product called FlashDevelop that seemed promising, but it lacked a design mode.  Then I thought that I would use a Flash player that was already designed, so I downloaded that one.  It worked, but I could not get my web server to serve it, then it stopped working as a standalone file on Windows.  So, I finally bit the bullet and went to Adobe and downloaded Flex4.  This led to two irritations for me.  One was that when you download Flex4, you are evidently downloading FlashBuilder4 which is NOT free.  The second was that now my Eclipse IDE, which I use for Android development has a spam message about registering FlashBuilder.  Once I got past this annoyance, though, I was able to go to design mode, drag a video player onto the work area, set the properties and make the SWF file. I was now able to watch my videos, although I was never able to get them to serve right from the server. I am certain that is not the fault of Adobe, but probably some setting I was missing.  Finally, I gave up trying to get it to serve on my server and used a ready made Flash Player called jwplayer. Jwplayer was quite simple to use, although a bit more complex than the video tag, but it served the video fine.  At any rate, here is the HTML code generated to play the flash movie:

< !DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>

        <title>Flash Test</title>
</head>
<body>
        The video should play below this line<br /><hr />

        <object id="player"
classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000"
 name="player" width="640" height="480">
                <param name="movie" value="player.swf" />
                <param name="allowfullscreen" value="true" />
                <param name="allowscriptaccess" value="always" />
                <param name="flashvars" value="file=test.flv" />
                <embed type="application/x-shockwave-flash"
                        id="player2"
                        name="player2"
                        src="player.swf"
                        width="640"
                        height="480"
                        allowscriptaccess="always"
                        allowfullscreen="true"
                        flashvars="file=test.flv"
                />
        </object>

</body>
</html>

This is in addition to having the SWF file and an ancillary JavaScript file.  Compare this to the code needed to play a video in HTML5.  So, Flash has a lot of complexity and four files versus two for HTML5.  I would say that the HTML5 video tag beat Adobe Flash in ease of programming hands down!  I will say, though, that the Adobe Flash Player has a full screen mode that is intrinsic to the player whereas the HTML5 video object does not. There is also another issue with Flash that I need to mention here.  There is a distinct lack of support for 64 bit Linux systems.  I tried using Adobe’s “no guarantees” 64 bit plugin, and while it worked for Opera, I could never get it to work for Firefox.  This is one of the issues that can arise when a company owns a technology and dictates what it can and cannot support.  IE9 only running on Windows 7 is another example of the same phenomenon.  I do not think Microsoft or Adobe are evil or anything of that nature, but just that lack of support for certain technologies can be one of the issues when a single company owns a technology.

CPU Usage

When I ran these files, I also ran a utility to monitor CPU usage.  The Linux system I used to test this is a Fedora 14 x86-64 Symmetric Multi-Processing on 2 Quad Core 2 Ghz Xeon E5405 processors with 850GB of RAID5 HDD space and 16GB of RAM.  The Windows system I used is a Windows XP SP3 with Flash Player 10 on a 2.9 Ghz AMD AthonII x2 245 with 2 GB of RAM and a 700 GB HDD with about 600MB free.  I was sure that the Flash Player would eat up all the CPU, because all my research had lead me to believe that.  Imagine my surprise when HTML5 video and Flash video were fairly comparable in terms of CPU usage! I was shocked.  I do not think that most of these differences in CPU usage were statistically significant, but it did show that Flash 10 is not the resource hog that others may have thought!  I did notice that when I made the Flash full screen, it averaged around 25% CPU.  However, that is not apples to apples because HTML5 is not full screen.  I wonder if the increased CPU usage for full screen mode is why some people hold the opinion that Flash is a resource hog? I made the videos the same size and the CPU load was pretty comparable between Flash and HTML5.  It was interesting to note that HTML5 used slightly less CPU on the Linux platform and slightly more CPU on the Windows platform.  Also, it was pretty glaring that Firefox for Windows used a whopping 54% of CPU to render the HTML5 video as opposed to around 10% for Opera and Chrome.

CPU Load by Browser

Anyone who wishes can run these tests for themselves.  Simply navigate to http://projects.bowentech.net/Articles/html5 and follow the directions.  It would be interesting to see any differing results or tests on MAC OS.

Final Thoughts

So, how do we summarize this?  What does it all mean?  Well, in my opinion, HTML5 will eventually replace Flash as the main video delivery system on the internet.  This will not happen immediately.  It will take years.  However, it is a natural evolution.  The HTML5 video tag is so easy to use, and as it becomes a more mature standard, it will catch up to the capabilities of the flash player and still be easier to code!  Converting the videos does take longer for HTML5, but this would probably be a batch process anyway.  The fact that you can just type a line into a html document and have a nice video rendered as opposed to having to enroll a Flash developer to do the same thing will eventually make HTML5 the standard of video delivery.  Stay tuned for the last part of this series wherein we will compare the canvas tag to drawings in Flash.

Tags: Featured flash html5 web Web Performance webm