Design meets artificial intelligence to create new visual search engine

Novel methods of searching the nation’s gallery, library and museum collections could soon be revolutionised by a visual search platform designed in collaboration with Northumbria University.

As the sector worldwide moves towards presenting collections online, the Deep Discoveries project was launched to explore ways of creating a computer vision search platform that can identify and match images across digitised collections on a national scale.

The expertise of Dr Jo Briggs and Associate Professor Jamie Steane, from Northumbria School of Design, were enlisted to help deliver the collaboration between The National Archives, the University of Surrey and the V&A Museum.

Rather than typing a keyword into an empty search box, visual search uses a query image and computer vision artificial intelligence (AI), to match similar images from across digitised collections based on properties such as colour, pattern and shape.

The Northumbria design team - made up of Jo, Jamie and talented graduate Andy Cain - joined the project at a later stage to help with information sharing and developing the user experience.

Their challenge was to find ways to visually demonstrate the AI reasoning and explain the search criteria of the platform.

On the design process, Jo said: “We developed animations to share over video calls to inform discussions on how best to represent quite complex search functions within the interface designs.”

Bernard Ogden, Research Software Engineer at The National Archives, built the prototype platform.

He said: “The design process allowed us to reach a shared understanding that permitted us to create a live prototype, accommodating the different points of view between in teams involved in this multi-disciplinary project.”

Jo and Bernard recently attended the Digital Research Infrastructure for the Arts and Humanities (DARIAH) annual conference, this year on the theme of ‘Interfaces’, to present their findings.

Project lead, Dr Lora Angelova, Head of Conservation Research and Audience Development at The National Archives, added: “The energy and innovative design vision brought to the project by the design team, sparked new ideas and possibilities around visually communicating a progressive way of deploying AI for the benefit of our audiences.”

Deep Discoveries is one of eight projects funded by the Arts and Humanities Research Council’s (AHRC) ‘Towards a National Collection: Opening UK Heritage to the World’ programme. The five-year initiative aims to benefit public audiences and researchers by making heritage collections more accessible for all.

A short video to explain the design process has been created and a prototype of the search platform has been developed.

Find out more about Northumbria School of Design research, courses and study options at northumbria.ac.uk/design