EXPERT COMMENT: How AI can dramatically reduce the time it takes to make VAR offside calls in Premier League football matches

In this article originally written for The Conversation*, from Northumbria University, Shanfeng Hu, Assistant Professor, Computer and Information Sciences, and Mark Middling, Assistant Professor of Accounting, discuss the effect of AI on VAR offside calls in Premier League matches.

The start of the new English Premier League (EPL) football season is seeing further changes to the way VAR (the video assistant referee) is operated. VAR was introduced to the EPL in 2019 in an attempt to reduce the number of incorrect refereeing decisions, including offsides – but its use is still much criticised.

Video replays of refereeing decisions are reviewed in real time by referees and other officials based in Stockley Park, west London. The advice of these officials and video of the key moment are then fed back to the referee on the pitch, to help ensure the accuracy of their decisions.

The EPL claims the correct decision is now made 96% of the time, up from 82% before the introduction of VAR. Academic research has also found an improvement in decision-making following the introduction of VAR, which last season reviewed around 1,300 EPL match events.

One of the biggest criticisms of VAR is the time taken to carry out a review, and the EPL is about to start employing artificial intelligence (AI) to address this.

Semi-automated offside technology (SAOT), as was already used in the 2022 Qatar World Cup and 2024 Euros, employs AI for high-speed tracking of players with high levels of detail.

It has been estimated that SAOT can reduce the time taken for offside decisions by up to 31 seconds. It also provides a clearer image for TV spectators viewing these offside offences. Rather than two lines of blue and either green or red being manually placed across an image of the pitch, SAOT imposes a virtual vertical curtain showing exactly which part of the player’s body is offside.

The technology is based on computer vision systems that can track up to 10,000 surface body points of each player and the ball. It can track players with a speed of 200 updates per second, according to Genius Sports, the data company that won the EPL’s SAOT contract.

Such high-detail and high-speed tracking is made possible through modern AI and deep learning algorithms – technology that simulates the way the human brain makes decisions. It can digest vast amounts of video data that has been captured by dozens of cameras installed at each EPL football ground.

How it works

Convolutional neural networks (CNNs) are a type of deep learning algorithm specifically designed to process and comprehend the contents of images and videos. They play a key role in the operation of SAOT.

CNNs analyse images by running them through what’s called a multi-stage hierarchical process. Layers at the beginning stages of the processing extract basic features from the image, such as edges, blobs and corners, while layers further downstream aggregate and assemble these features into spatially larger, conceptually more meaningful categories.

When applied to surface point tracking, as in the case of Genius Sports’ computer vision technology, the CNNs are tailored and trained to produce an estimate of the spatial coordinates of each point on a player’s body and the ball for each streamed image.

But first the AI system needs to be “trained” – in other words, fed multiple examples of what it is going to be looking for, in order to improve how it performs these tasks. This training process, which happens largely automatically, is demanding on computers – but once completed, the system can deliver real-time predictions for SAOT.

VAR may never be perfect

Continuous advancements in AI research and engineering are pushing the boundaries of many aspects of sports data analytics, with the application of SAOT in top-level football matches a prominent example of the latest technology being utilised.

The introduction of SAOT is expected to start in October 2024, once EPL chiefs are confident it is up to scratch. The EPL’s chief football officer, Tony Scholes, said that VAR may “never be perfect”. However, the introduction of SAOT will probably get it closer, and this may help the technology be more widely accepted by the majority of fans.

Many fans say they are in favour of VAR if it is improved. In addition to the time taken for decisions, criticisms include reducing the enjoyment of fans, reducing the spontaneity of goal celebrations, and perceived pedantry over marginal decisions.

VAR’s use by the EPL was even tabled to be scrapped by Wolverhampton Wanderers at a meeting of Premier League clubs at the end of the 2022-23 season – but Wolves lost this vote 19-1.

*This article was originally published by The Conversation. Please see here for republishing guidelines.