Web Subtitling: How Hard Can It Be?

The apparent simplicity of creating subtitles and captioning for web-based video belies the real complexities of this relatively new endeavour, writes John Birch of Screen Systems.

Ask most broadcasters and Internet video users about how to implement subtitles (or captions) on web distributed video and chances are you'll get an answer that it's relatively easy to do; that all that is needed is a simple text file with some timing and perhaps style information included (e.g. an SRT, DFXP, WebVTT, SMIL or SAMI file).
And they would be right, at least as long as they were not too concerned about how the text looked or if the text was presented with the same degree of timing accuracy as is the norm for conventional TV broadcasting. Ask them if this works for live web video (e.g. simulcasts or web-only live broadcasts) and a greater level of uncertainty will kick in.
In this article the term 'subtitling' includes captioning. However, the priorities of the caption user are different to those of a subtitle user, and consequently technologies that have been developed to support caption provision in web video distribution do not typically provide all the features needed for high quality translation subtitling.
The reality is that subtitling of video on the internet is in its infancy. There is a wide range of competing video distribution standards, most of which are proprietary and offer differing mechanisms and levels of provision for subtitles. Unlike the world of traditional broadcast, there are no clear standards for subtitle files, and no consistent strategies for subtitle presentation.
This presents a number of problems to many broadcasters of video content online. In general, online video has a wider demographic audience, with a larger range of translation requirements. Creating a wide range of translations for your web video (plus at least one caption service) is problem enough, without having to create those translations and captions in many different formats to match all the different distribution mechanisms your content may be delivered through.
Add to that the issue of inconsistent presentation and varying user interfaces to select and enable captions or subtitles and web subtitling becomes very hard indeed.
To address this situation, Screen Systems has developed technology which can add high quality image based subtitles to web video. The subtitles are NOT burned in and can be switched on and off by the viewer using controls in the player. Delivering the text as bitmap images eliminates all text rendering issues and is key to providing a consistent high quality service across diverse video player platforms.
The images are not sent within the video stream and are delivered by separate HTTP download. The bandwidth required compared to a text-based mechanism is larger, but is still very small compared to the video itself and only subtitles for the selected language are ever sent to the player. With subtitles switched off, there is no overhead at all.
For video on demand (VOD) subtitles, all that is needed is a player environment that supports scripting (or plugins) and transparent overlay, plus a mechanism to report the current play position within the video. The video stream is not modified at all to add subtitles and additionally languages can be easily added to the subtitle server without any changes to the video distribution.
This leads on to the thornier problem of subtitling live web video.
In a live web video, the player cannot correctly report the current play position within the video, since this differs for each viewer depending on when they 'switched on'. Additionally, each viewer may choose to pause or rewind live video stream (if this feature is provided).
Screen therefore developed a different synchronisation mechanism, while still using image based subtitles on a separate server. The principal difference is in the player 'script' or plugin. In our solution, some low bandwidth timing information is added to the video stream and used to synchronise subtitles to the frames of video the viewer is watching. The 'script' at the viewer's device can obtain the current frame's time value from the video stream and use it to determine which subtitle, if any, should be displayed from the subtitle server.
The need to develop player components (plugins or scripts) might be seem as a limitation of this solution compared to using the 'off the shelf' functionality provided by a player (if any). However, in reality, almost every web video service has elements of custom functionality added when they are deployed (e.g. DRM). An advantage of the Screen solution is that the same general mechanism of image based subtitles held on a separate server can be used for consistent broadcast quality subtitling of both Video on Demand and live web video.

The author wishes to acknowledge contributions from Roland Boorman.

The article is also available to read in BFV online.

(IT/JP)