23 Jul 2014

WebVTT Released in Firefox 31

If you haven't seen the release notes WebVTT has finally been released in Firefox 31. I'm super excited about this as it's the culmination of a lot of my own and countless others' work over the last two years. Especially since it has been delayed for releases 29 and 30.

That being said, there are still a few known major bugs with WebVTT in Firefox 31:

  • TextTrackCue enter, exit, and change events do not work yet. I'm working on getting them done now.
  • WebVTT subtitles do not show on audio only elements yet. This will probably be what is tackled after the TextTrackCue events (EDIT: To clarify, I meant audio only video elements).
  • There is no support for any in-band TextTrack WebVTT data yet. If your a video or audio codec developer that wants in-band WebVTT to work in Firefox, please help out :-).
  • Oh, and there is no UI on the HTML5 Video element to control subtitles... not the most convenient, but it's currently being worked on as well.
I do expect the bugs to start rolling in as well and I'm actually kind of looking forward to that as it will help improve WebVTT in Firefox.

- Tagged in mozilla and open-source.
15 Apr 2014

Getting the number of lines of text in an Element

One of the biggest problems I faced when developing vtt.js is that a lot of the layout algorithm depends on being able to know the line height of the subtitle text. This boils down to being able to know the line height of the div within which the subtitle text sits. A lot of the time this is easy to get:

  var lineHeight = div.style.lineHeight;

But, what if you haven't set a line height? Then you would need to get the computed value of the line height:

  var lineHeight = window.getComputedStyle(null, div).getPropertyValue("lineHeight");

This works... some of the time. On some browsers if you try to get the computed value of the line height and you haven't explicitly set a line height, the computed property will return back as the value normal. That's helpful...

After much searching I found out that you if you use getClientRects on an inline element it will return you a TextRectangle box for each line of text in the inline element. At that point you can either assume that each line has the same height and get just use the height property of the first TextRectangle or to get a somewhat more accurate number you can take the height of the inline element and divide it by the number of TextRectangles you have.

  var inlineElement = document.getElementById("myInlineElement"),
      textRectangles = inlineElement.getClientRects(),
      container = inlineElement.getBoundingClientRect(),
      lineHeight = container.height / textRectangles.length;

  alert("The average line height is: " + lineHeight);

This works really well for the amount of actual code you need to write. I've read about more accurate methods, but they take some serious coding. Like walking through each character in the text and tracking when overflow happens serious.

Now back to my original question which was how to get the number of lines of text in a div (block level) element. The way I did this was to wrap my div which has my content in another div, and set the inner div's display property to inline. Then you can calculate the line height/number of lines of text of the inner div since it has inline display. This way you retain your contents block level layout while being able to figure out how many lines of text it is.

This is it all put together:

  <div>
    <div id="content" style="display:inline;">
      This is all my content in here. I wonder how many lines it is?
    </div>
  </div>
  var inlineElement = document.getElementById("content"),
      textRectangles = inlineElement.getClientRects(),
      container = inlineElement.getBoundingClientRect(),
      lineHeight = container.height / textRectangles.length;

  alert("The average line height is: " + lineHeight);
- Tagged in mozilla, seneca college, open-source, and cdot.
11 Apr 2014

Hosting your JavaScript library builds for Bower

A while ago I blogged about the troubles of hosting a pre-built distribution of vtt.js for Bower. The issue was that there is a build step we have to do to get a distributable file that Bower can use. So we couldn't just point Bower at our repo and be done with it as we weren't currently checking in the builds. I decided on hosting these builds in a separate repo instead of checking the builds into the main repo. However, this got troublesome after a while (as you might be able to imagine) since I was building and commiting the Bower updates manually instead of making a script like I should have. It might be a good thing that I didn't end up automating it with a script since we decided to switch to hosting the builds in the same repo as the source code.

The way I ended up solving this was to build a grunt task that utilizes a number of other tasks to build and commit the files while bumping our library version. This way we're not checking in new dist files with every little change to the code. Dist files which won't even be available through Bower or node because they're not attached to a particular version. We only need to build and check in the dist files when we're ready to make a new release.

I called this grunt task release and it utilizes the grunt-contrib-concat, grunt-contrib-uglify, and grunt-bump modules.

  grunt.registerTask( "build", [ "uglify:dist", "concat:dist" ] );

  grunt.registerTask( "stage-dist", "Stage dist files.", function() {
    exec( "git add dist/*", this.async() );
  });

  grunt.registerTask("release", "Build the distributables and bump the version.", function(arg) {
    grunt.task.run( "build", "stage-dist", "bump:" + arg );
  });

I've also separated out builds into dev builds and dist builds. This way in the normal course of development we don't build dist files which are tracked by git and have to worry about not commiting those changes. Which would be the case because our test suite needs to build the library in order to test it.

  grunt.registerTask( "build", [ "uglify:dist", "concat:dist" ] );
  grunt.registerTask( "dev-build", [ "uglify:dev", "concat:dev" ])
  grunt.registerTask( "default", [ "jshint", "dev-build" ]);

Then when we're ready to make a new release with a new dist we would just run.

  grunt release:patch // Or major or minor if we want too.
- Tagged in mozilla, seneca college, open-source, and cdot.
28 Mar 2014

WebVTT (vtt.js) on Nodejs

Good news! You can now use vtt.js on Nodejs. There are two different ways you can get it on npm:

  1. You can install the basic vtt.js package which just exports the WebVTT, VTTCue, and VTTRegion implementations into a nice Node package for you. Using this method you will have to provide your own window object for vtt.js to run on. See the docs for more information.
  2. You can install node-vtt which is a Node wrapper for vtt.js. node-vtt runs vtt.js on an instance of PhantomJS. This means that you don't have to provide your own window object for vtt.js since it will already be running in the context of a web page. It also provides a couple of convenience functions such as parseFile, processFile, and others. See the docs for more information.
- Tagged in mozilla, seneca college, open-source, and cdot.
26 Feb 2014

Positioning WebVTT In Firefox

The other day I landed the final patch to get positioning WebVTT in Firefox working. I'd like to share a demo today that I put together to showcase some of the different position settings that Firefox now supports. Note that you'll need to download Firefox Nightly for this to work correctly.

You can checkout the WebVTT file used to make this demo here.

Enjoy!

Thank you to Ali Al Dalla for the video ;-).

The part of the spec that deals with positioning WebVTT cues is called the processing model algorithm and it specifies where to position the cues based on the cue settings that the author has set for each cue and also how to avoid overlap if two or more cues overlap on the video. All this needs to be done manually—positioning boxes absolutely—because the algorithm needs to behave deterministically across browsers.

The algorithm gets the positions for the various cues from the cue settings which are:

  • Position—The position from the cue box anchor to where the cue sits on the video.
  • Size—The size of the cue in percent of the video.
  • Line—The line position on the video; Can be a percent or integer value.
  • Align—The text alignment of the cue.
  • Vertical—Specifies vertical or horizontal text.
Checkout the cue setting portion of the spec if you'd like to know more about the various cue settings.

Interestingly enough the hardest part about implementing this part of the standard wasn't the spec itself, but getting the test suite working correctly. The main problem for this is that our test suite isn't hosted in Gecko, but in the vtt.js repo and the algorithm for positioning relies heavily on the positions of computed DOM elements. Since we aren't running the test suite in Gecko we don't have access to a CSS layout engine, which... was a problem.

I tried a lot of different solutions for getting the tests working in vtt.js and settled on running our test suite with PhantomJS. In the future we'd like to move it over to using something like Testling so that we can test vtt.js as a polyfill for older browsers at the same time.

Next up is getting some kind of UI that can control subtitles and the resource selection algorithm.


Edit:

Šime Vidas put together a picture of the WebVTT markup in the VTT file and what the video looks like at the same time. It really helps visualize how this works all together. Thanks Šime ☺. Check it out!

- Tagged in mozilla, seneca college, open-source, and cdot.

Wow coding Copyleft Richard Eyre 2013

Code for this site can be found on GitHub.