January 09, 2013, 8:00 AM —
Source: NBC Sunday Night Football
As we near Super Bowl XLVII in New Orleans, I'm reminded of a technology that has vastly improved the football viewing experience - the yellow first down line. It might not seem like much but the science required to produce that visual aid is very complex and really interesting.
Remember watching games without this simple place marker? Many plays felt arbitrary because you had no sense of what the team was aiming for. You basically had to wait for the next play to find out what the offense had just accomplished. It was barbaric. Today (and since 1998) we've been graced with the fluorescent yellow line that seems painted on the field when we watch games. It seems so simple, but the truth is that a great deal goes into producing that effect. Indeed when it began it required a tractor trailer full of equipment, at four to five people, and about eight servers to operate.
Sportvision and its founder Stan Honey (who is worth an article in his own right) launched the 1st & Ten® line just months after leaving News Corp where he worked on the NHL puck tracer visualization. Honey, who was recently inducted into the National Sailing Hall of Fame, has leveraged this technology into just about every sport including baseball, soccer, tennis, NASCAR, sailing, and the Olympics. So how does it work?
Sportvision is not the only player in the sports augmented reality game. SportsMEDIA Technology - who has the contract for the on-screen graphics during Sunday Night Football on NBC - has been producing their own version of the first down line for nearly 15 years now. They also have the contract for this years Super Bowl on CBS.
With such a simple concept, it's helpful to outline the challenges that need to be overcome in order to illustrate its masked complexity.
Each camera location has a unique field orientation and perspective. This perspective must be calculated in order to drawn the line onto the field within that perspective.
Each camera can pan, tilt, and zoom which changes the perspective. The line must be recalculated and redrawn at least 60 times per second taking into account these changes.
The system must know the location of each individual yard marker on the field, taking into account the current perspective.
There are multiple cameras shooting the game, the system must perform all of these calculations for each camera.
The field is not flat, it curves subtly to allow water to run off.
Each stadium has different camera locations.
The system must sense when a player, object, or network graphic is on top of the line so it doesn't draw over it.
Team Jerseys and field surfaces vary in color.
Digitizing Camera Mount
Probably the most important piece to this system is a special camera mount used to hold the broadcast cameras. The mount records and encodes all of the cameras movements including pan, tilt, zoom, and focus. This data is fed back to the systems computers where it's used to calculate the lines orientation based on a 3-D computer model of the stadium.
3-D Computer Model
A computerized 3-D model of every stadium has to be created for the system. This model will blueprint the exact locations of each camera in the stadium along with every yard marker and even the crest of the field. Using these reference points combined with the camera data being fed from the mount, the system can accurately calculate how the first down line should be displayed for each camera.
Specialized Color Palettes
The system relies on the color palette loaded for the game to differentiate between the grass and everything else like uniforms, referees, the ball etc. Using a form of chroma key and the game's specific color palette, the system paints the line on the grass below every other object.
Even with all this technology the system still needs to be told where the line of scrimmage is and what yard line the first down is on. A spotter is perched in the press box relaying this data to an operator in the production truck who manually keys this information into the system. There are usually at least two additional operators on hand to perform tasks such as reconfiguring the color palette based on weather conditions like snow, or just general technical support.
Drawing the Line
Using the camera mount and its data, the virtual field for reference, and the color palettes for the field and the uniforms, the system is able to determine the exact pixels that need to be painted yellow on the screen. This pixel data is sent along with the raw camera video into a computer which draws the yellow line 60 times per second. The line data is then sent to a video keyer which superimposes the line frames onto the raw video. Live footage is purposely delayed to allow for the line data and camera footage to synchronize before being broadcast.
The Tip of the Iceberg
The NFL first down line is routinely taken for granted, despite that fact that it is essential to today's viewing experience. The understanding of each play in the game is increased for each viewer, just like Olympic swim races are made more clear with the leader lines. The technology is even more useful in other sports, particularly sailing. Sportvision's LiveLine technology uses helicopters and super precise GPS to overlay a grid of meter lines along with boat positions on the screen making it actually exciting to watch on TV. Check it out in the 2013 America's Cup which begins April 16th to see it in action.
When you're watching Super Bowl 47 on February 3rd, keep in mind everything that's involved in producing that yellow first down line. If the game gets boring, at least you'll have something interesting to talk about.