The Silverlight HVP Spec

by jesseliberty via Jesse Liberty - Silverlight Geek on 11/15/2009 5:43:00 AM

The Silverlight HyperVideo Player: A Community Project

The Silverlight HVP Elevator Pitch:

The Silverlight HyperVideo Player (HVP) will do for Silverlight How Do I Videos what Hypertext did for the Internet: it will provide simple, fast, and easy to use links to additional, supplementary or related information.

As an example, the user might be watching a video on data binding and the term “Dependency Property” is mentioned and at the same time a link appears. If the user clicks on that link, the video is paused and an explanation of Dependency Properties is provided.

Update 11/20: We have received our MS-PL allowing me to move this project to CodePlex. I will update again with a link once it is in place. Other updates in-line in red.

Summary of Purpose

While I have covered HyperVideo in numerous blog postings and videos in the past, this project will follow the creation and delivery of a complete application. spinning out mini-tutorials and videos on an extremely wide range of topics.

The project will be designed, developed and delivered in collaboration with many others, both inside of Microsoft and out.

Scope and Deliverables

This document will serve as the draft “spec” for the first phases of development. I have intentionally under-documented the spec because of the immutable law that the likelihood of a spec being followed is inversely proportional to its size and (b) the requirements and design will evolve as the community adds ideas and refinement.

Contributing to the Spec and to the Project

Contributions, refinements and critiques of this document are not only welcome, they are its lifeblood. For now, I’ll ask that you respond in the comments, , as things evolve I’ll roll out a better mechanism for tracking all of the feedback.

Contributions to the project will be described by the end of the month; for now, if the comments section is insufficient, please feel free to write to me directly.

Core Functionality

The Silverlight HVP will be a modular, extensible, and robust Rich Internet Application that can deliver HyperVideo on any platform that is supported by Silverlight. Initial requirements include:

Present any existing Silverlight How Do I Videos as is.
- That is, the Silverilght HVP will be backwards compatible with the existing player
- Any existing video can be enhanced to provide a Table of Contents and/or Hyperlinks either through encoding or marker injection at which point it will be indistinguishable from videos created specifically for the Silverlight HVP
Display a Table of Contents (TOC) for each video, with affordances for users to move from one topic to the next
- Current plan is to separate the TOC from the links, presenting the complete Table of Contents when the video begins, but revealing the links as their context arises.
- For sufficiently complex Tables of Content we may want to provide the ability to collapse and expand subtopics
Display a list of topics as they are mentioned in the video, with each topic’s name acting as a HyperVideo Link.
Full support for Closed Captioning and all standard video controls (e.g.,, stop, pause, skip etc.)
TOC and Hypervideo Links will be decoupled from the markers in the video so that the links can be modified without changing or re-encoding the video

Potential Features

There are literally dozens of potential additional features, some of which are key to adoption of the player in various settings; others either won’t make it into early versions of the product or will be developed by the community once the open source mechanism is in place.

[Images in this spec are for illustration purposes and are not final visualizations of the application]

Not Just An Application

A primary goal of this project is not only the end product, but the opportunity for discussing Silverlight skills in the context of a real-world application. Building the Silverlight HyperVideo Player will force consideration of a number of interesting aspects of Silverlight and related technologies.

Some Thoughts on Implementation

There are numerous ways to add markers to video in Silverlight; in the course of developing the Silverlight HVP we will consider two. The easier approach is to use Expression Encoder to do the initial heavy lifting. Encoder is designed to record four pieces of information about each marker, the first two of which matter to us.

The elapsed time into the video at which the marker appears
A “value” that can have an intrinsic meaning or not as you choose (more about this below)

The other two pieces of meta-data in a marker are the thumbnail and keyframe for each marker; these are typically used to create and manage “chapters” in long videos.

Working With Markers

The Media Element (or a control derived from the Media Element) will fire an event each time a marker is encountered. The HyperVideo player will respond to that event by following the link or opening the associated video as indicated in the metadata that relates a given marker to a given action.

During development of the Silverlight HVP we will explore at least two ways of managing the metadata: storing it in a relational database and storing it in an XML file.

As an off-the-cuff example, the MetaData might be stored in an XML file that looks more or less like this:

<VidepPlayerMetaData>
   <VideoInfo>
     <Title="Data Binding" />
     <URL="http://foo.bar.com" />
   </VideoInfo>
   <Markers>
      <Marker Val="DBX09-001" Action="Play" URL="http://silverlight.net/learn/videos/v1027" />
      <Marker Val="DBX09-002" Action="Play" URL="http://silverlight.net/learn/videos/h1375" />
      <Marker Val="DBX09-003" Action="Link" URL="http://en.wikipedia.org/LCA" />
      <Marker Val="DBX09-004"  />
      <Marker Val="DBX09-005" Action="Play" URL="http://silverlight.net/learn/videos/v1288" />
   </Markers>
</VidepPlayerMetaData>

Encoder can create a working player to get us started.

At its heart, the player is a control or a set of controls that can be skinned to change its appearance and can be subclassed to change its behavior. Doing so demonstrates best practices, and significantly simplifies the development effort.

The UI Sketch

Of course, the player created by Encoder is just the starting point it will serve as an essential component within a larger application. The UI design at this point is only a first approximation, subject to significant change, but the rough sketch is enough for me to get started on development.

“You start coding, I’ll go find out what they want.”

The sketch calls for four windows and a menu.

The primary player (top center)
The Table of Contents (top left)
The Links window (top right)
The linked-to (target) window (bottom)

Each window will be resizable, movable, and dockable.

Deliverables and Due Dates:

It is tempting to say that too little is known to create a set of milestones and delivery dates; but projects with no dates have a known finish time: never.

Been there, done that, still have the scars.

Far better to stake yourself to a timetable, then be held accountable; if not to hitting the dates, then to explaining what has changed that justifies a change.

Thus, after many seconds of careful consideration, here is the timeline for the first iteration of this project.

Task Due

This overview spec completed & published 11/15

Proof of concept deployed on SilverlightGeek.me 1/1

Delivery of final UI Assets 1/21

First Release Mix10

My plans of course go well beyond Mix, but this is, I hope, a good start. I welcome and urge you to join me in an open technical discussion of any and all of this project.

The fine print:

Project Turing, and the feedback it engendered, served as a great testing ground for building this kind of application. It was not intended that Turing would be completed, and I’ll suspend it indefinitely. It is my intent that this project will roll out as an application.
I believe this project has tremendous potential, and I’m committed to it, but I’ve been on enough great projects that were stopped before seeing the light of day, that I make these commitments knowing that not everything is under my control (the great delusion is I still think anything is under my control)