At the 2017 U.S. Open in New York, IBM’s Watson extracted key moments from the 26,000 points scored by tennis stars such as Canadian Denis Shapovalov.Corinne Dubreuil
If there's one thing sports fans take for granted, it's highlight reels. After all, just about no one who enjoys watching their team's best plays is liable to think about the video editors who spend countless hours assembling such clips.
Fortunately for those editors, artificial intelligence might make their thankless jobs a little easier.
"It really allows the editorial and video production team to focus on creating more custom highlight packages," says Stephen Hammer, chief technology officer for IBM's sports and entertainment division. "They don't have to spend any time at all on the highly repetitive video-editing task associated with the match summary highlights."
IBM is actively deploying its artificial intelligence video highlight editor after trying it out at the U.S. Open tennis tournament in New York this past summer.
Using Watson, the super-computer that famously beat its human opponents on the game show Jeopardy! in 2011, the system can scan dozens of hours of footage and pick out the most pertinent moments for inclusion in highlight packages.
At the U.S. Open, Watson looked at more 26,000 points scored. In many cases, it was creating reels within two minutes of a match ending, then posting them to Facebook and U.S. Open apps.
"For a video editor to go through and watch that many points and pick out exciting moments, that would be a huge task," Mr. Hammer says. "It really helped them focus on other things."
To achieve its speedy results, Watson first had to be trained. Before turning to tennis, the IBM team created a proof-of-concept using footage from the Masters golf tournament this year.
They taught the computer to recognize certain elements of the game, including crowd noise, excitement from commentators and body language from players – fist pumps and high-fiving caddies, especially.
Any of those marked the particular moment as important to Watson. With a full complement of highlights, the computer then ranked them against each other to come up with a distilled set of the best. As with all AI, the more data the machine digests, the more accurate its results.
IBM then trained the system on footage from the Wimbledon tennis tournament in July before launching its Watson Media business and partnering with the U.S. Tennis Association for the U.S. Open.
Watson Media isn't just aiming for further deployments in sports, however – the broader entertainment field is a target, too, owing to the system's origins.
IBM first applied AI to highlights in the form of a trailer for Morgan, a horror movie released last year. Fittingly, the movie is about an artificially intelligent entity.
As with its sports training, Watson studied footage of previous horror movies to learn how to identify key moments. Screams, tense music and what Mr. Hammer calls "highly emotional moments" eventually made the cut. Further movie trailers are possible, he adds.
IBM is far from the only technology company applying AI to video sorting and editing.
Adobe, the maker of graphics editing program Photoshop, this summer announced a collaboration with Stanford University on a program that automatically studies footage and suggests edits. The system can be taught the user's editing style and, like Watson, promises to save copious hours of grunt work.
Also this summer, Google announced it is using AI to scrub extremist content from YouTube. The company says the new system is more efficient than humans at the task, removing hate-filled videos at double the rate.
"Over 75 per cent of the videos we've removed for violent extremism over the past month were taken down before receiving a single human flag," a YouTube spokesperson said in a blog post.
Media experts say such efforts are just the beginning. The application of AI to highlights and trailers is a natural and logical step because it will free up human editors from the tedious work of poring over footage.
Their jobs are likely to be elevated into more of an oversight role as a result.
"Humans still have to put the coherence to it, they have to give it a flow and tell a story," says Robert Clapperton, assistant professor at Ryerson University's school of professional communication in Toronto.
"AI is going to be a team member within a group of human beings that does specific work. Maybe there are three editors that put together a sports highlights show and one of them is now going to be AI whose job is to do the rote stuff."