By Phil Ventre, VP Sports and Broadcast, Ncam
While augmented reality for broadcast is still in a relatively nascent stage, its usage is finally moving away from the gimmicky ‘new toy’ stage; AR graphics are becoming an integral part of programme content, and sophisticated audiences are demanding hyper-real, believable graphics that are fully immersed in the real-world environment.
The sports broadcast sector in particular has embraced AR graphics to enhance programming. In broadcast studios, the challenge is to present graphics such as sports data and statistics in new and visually creative ways; with marker-less camera tracking technology like Ncam Reality, they can now enable presenters in the studio to appear to be walking around a player on a golf course, and can even ‘teleport’ athletes into awards presentations from thousands of miles away – all with hyper-realistic graphics.
One of the last limitations that camera tracking has faced in a live broadcast environment is working wirelessly. Our AR solution had the camera and sensor bar feeding environmental data back to our software server via a tethered cable; while this is fine for studio work and fixed positions on an OB, it is restrictive for more creative live broadcasts.
When CBS Sports approached us with the idea to put live graphics on the pitch at the Super Bowl, the challenge was to make our data travel via RF from a Steadicam rig on the pitch back to the production truck. We needed to find a way to free up the camera operation, allowing for free flowing camera movements reactive to the players.
The solution was to mount the sensor bar to a Steadicam RF rig as normal, but rather than tethering the rig to the large Ncam server, a mini computer was substituted. Although the computer remained tethered to the Steadicam rig, it was far more mobile and easily transported by an assistant who could move alongside the Steadicam operator and manipulate the software on the spot. The RF signal could then be sent back wirelessly to the production truck.
As well as the RF Steadicam which was run into the centre of the field before the game, CBS Sports also deployed two further Ncam AR tethered rigs: a wired Steadicam was positioned on the GameDay Fan Plaza (an outdoor studio in front of the Mercedes Benz stadium), while another wired Technojib was positioned on the field. All graphics were created by The Future Group.
Everything worked perfectly on the day and the broadcast was a great success, with implications far beyond a single event in Atlanta – albeit a mammoth event viewed by millions around the world.
Since then, we have been working with R&D teams at several broadcasters and production companies, sharing their plans and our roadmaps to refine this technology to allow them to do more with AR. By making the server computer even smaller and more lightweight, camera operators can now carry it themselves, giving them far more freedom of movement. In addition, a marker-less camera tracking system allows broadcasters to use AR in virtually any space, whether indoors or outdoors, as it picks up natural points in its environment rather than having to rely on placed markers.
In parallel, the availability of 5G is set to play a major role in the way sport is delivered. BT Sport is at the forefront of this new technology, and has been successfully trialling live remote production over 5G in recent months. Production teams will be able to capture live images with significantly reduced latency almost anywhere from a sports event – the team bus, the tunnel, the stands, the centre of the pitch – giving directors a bigger canvas than ever before to tell their stories.
Add untethered real-time AR graphics to the production, and the creative freedom now available to broadcasters is astonishing.
While sport is the main driver for these technologies, they will also benefit any live event, from election coverage (imagine the Swingometer situated outside 10 Downing Street!) to major public broadcasts.
By opening up the possibilities of augmented reality with marker-less untethered live camera tracking, we will be seeing more broadcasters embrace the opportunities of what can be achieved.