OWR’s audio deepfake technology aces the UK Home Office-led ‘Deepfake Detection Challenge’
Below is the abstract of the talk:
Voice recognition is the process by which distinctive characteristics of an individual’s speech are used to identify or verify who they are. As lay listeners, humans recognise familiar voices intuitively in an everyday sense and may also find themselves being ‘earwitnesses’ to a crime, albeit rarely. When carried out by trained practitioners using specialised methodologies and tools, voice recognition, comparing often unknown speech samples, can play an important role in investigative and forensic contexts.
This talk will consider the landscape of forensic voice recognition, encompassing auditory analysis by trained listeners, acoustic-phonetic measurements of perceptually salient features, and automatic speaker recognition using signal processing and modelling algorithms that are statistical or based on deep neural networks. The Bayesian likelihood ratio framework will be critically examined as a means of evaluating the strength of evidence using any voice analysis methodology. The importance of validation of the prevalent and emerging approaches, to understand their limitations and to provide reliable and transparent reports to the courts, will be discussed.
Additionally, the varying acceptance of voice recognition evidence in different parts of the world will be explored. Anticipating the new challenges posed by machine-created spoofed speech, this talk also will reflect on the risks, mitigations and, more optimistically, emerging opportunities afforded by using both human- and machine-based analysis.
- A convincing voice clone? Automatic voice similarity assessment for synthetic speech samples
Linda Gerlach, Finnian Kelly, Kirsty McDougall, and Anil Alexander- PASS (Phonetic Assessment of Spoofed Speech): Towards a human-expert-based framework for spoofed speech detection
Daniel Denian Lee, Kirsty McDougall, Finnian Kelly, and Anil Alexander- CON(gruence)-plots for assessing agreement between voice comparison systems
Michael Jessen, Anil Alexander, Thomas Coy, Oscar Forth, and Finnian Kelly- Automatic Speaker Recognition: does dialect switching matter?
Marlon Siewert, Linda Gerlach, Anil Alexander, Gea de Jong-Lendle, Alfred Lameli, and Roland Kehrein- Impact of the mismatches in long-term acoustic features upon different-speaker ASR scores
Chenzi XU, Paul Foulkes, Philip Harrison, Vincent Hughes, Poppy Welch, Jessica Wormwald, Finnian Kelly, and David van der Vloed- Effects of vocal variation on the output of an automatic speaker recognition system
Vincent Hughes, Jessica Wormwald, Paul Foulkes, Philip Harrison, Poppy Welch, Chenzi Xu, Finnian Kelly, and David van der Vloed
“We’re really pleased to have started the project and to be working with Oxford Wave Research. The VOCALISE software allows us to answer big questions around the use of automatic speaker recognition systems in new and exciting ways. We can’t wait to see what comes out of the work!”
Dr Vincent Hughes, Principal Investigator of the project
“We were delighted to be able to spend time with the York University team recently, getting them started with VOCALISE and discussing their exciting plans for the project. They have a really experienced team and have lots of interesting ideas in the works – we are looking forward to seeing where it all goes!”
Dr Finnian Kelly, OWR Principal Research Scientist and Lead Scientific advisor on the project
Amelia’s research to date has focused on the anatomical bases of speaker identity, and particularly how individual differences in vocal tract shape affect the speech signal. For this work she was awarded a British Academy Postdoctoral Fellowship. She holds a PhD in Electronic Engineering and an MSc in Digital Signal Processing, both from the University of York, as well as a BSc (Hons) in Audio Technology from the University of Salford.“I am delighted to be joining the team at Oxford Wave Research, where I can put my acoustics and signal processing experience to work addressing real problems for customers, and contribute to exciting technological developments in the field of audio forensics.”
Dr Amelia Gully
“I am excited to welcome Amelia to OWR – with her expertise in acoustics and signal processing, and enthusiasm for all-things audio, she will be a valuable addition to the research team!” Dr Finnian Kelly, Principal Research ScientistAmelia joins us remotely from York where she lives with her partner and two rescue dogs. When not engaged in audio and speech research, she can be found playing video games or pottering around on her allotment.
Oxford Wave Research are delighted to be collaborators with the University of York and the Netherlands Forensic Institute in a recently awarded ESRC-funded project (£1,012,570) ‘Person-specific automatic speaker recognition: understanding the behaviour of individuals for applications of ASR’ (ES/W001241/1). This is a three year project running from 2022 to 2025 led by Dr Vincent Hughes (PI), Professor Paul Foulkes (CI) and Dr Philip Harrison in the Department of Language and Linguistic Science at the University of York. The project is due to start in summer 2022 and will run for 3 years. OWR will be providing our expertise and consultancy in automatic speaker recognition and our flagship VOCALISE forensic speaker recognition system.
Automatic speaker recognition (ASR) software processes and analyses speech to inform decisions about whether two voices belong to the same or different individuals. Such technology is becoming an increasingly important part of our lives; used as a security measure when gaining access to personal accounts (e.g. banks), or as a means of tailoring content to a specific person on smart devices. Around the world, such systems are commonly used for investigative and forensic purposes, to analyse recordings of criminal voices where identity is unknown.
The aim of this project is to systematically analyse the factors that make individuals easy or difficult to recognise within automatic speaker recognition systems. By understanding these factors, we can better predict which speakers are likely to be problematic, tailor systems to those individuals, and ultimately improve overall accuracy and performance. The project will use innovative methods and large-scale data, uniting expertise from linguistics, speech technology, and forensic speech analysis, from the academic, professional, and commercial sectors. This has been made possible via the University of York’s strong collaboration with Oxford Wave Research and two project partners including the Netherlands Forensic Institute (NFI).
The University of York and OWR teams are looking forward to a very fruitful collaboration that will undoubtedly further the state of the art in forensic speaker recognition.
Dr Vincent Hughes, Principal Investigator, University of York says “We are delighted to be working so closely on this project with Oxford Wave Research, who are world leaders in the field of automatic speaker recognition and speech technology. We hope that our research will deliver major benefits to the fields of speaker recognition and forensic speech science”.
Dr Anil Alexander, CEO, Oxford Wave Research says “Our team led by Dr Finnian Kelly is thrilled to contribute to this in-depth study of the individual specific-factors affecting speaker recognition, with the accomplished research team led by Dr Hughes from the University of York who are at the forefront of this space, and real-word end-users like the Netherlands Forensic Institute who have been driving research and innovation in this space for many years”.
Content creators of all kinds, such as the musician Aphex Twin, have long used hidden secret patterns and text in the audio that can be observed in their spectrograms. More recently, video game developers have hidden Easter eggs in the spectrograms of their game soundtracks for their more inquisitive players to find. For example, among Minecraft’s sound effects, the face of a Creeper, one of the game’s enemies, can be seen in the spectrogram of the audio heard in a cave, as SpectrumView user “Musix200” discovered. See if you can spot it too!
Taking this idea a step further, alternate reality games (ARGs) are a modern spin on the traditional scavenger hunt in which participants scour websites, social media, and videos looking for clues. These games have taken social media sites like YouTube and Reddit by storm over the last few years. Organisers, often video game developers, will bury information in all sorts of places, like images, website code, and audio. Whole communities have been formed in order to find out secret stories and previews for their favourite video game, or just to have some cooperative fun while solving a digital mystery.
Epic Games created ARG content in the run-up to the Season 5 release of their famous multiplayer online game, Fortnite Battle Royale. They staged a rocket launch within the video game itself, during which some of the audio played was slightly odd. Gamers quickly realised that there was probably more to the audio clip than what could be just heard. Looking for patterns within the audio led them to visualising the frequencies in the audio in a spectrogram. One such example of using SpectrumView to analyse the audio clip by player “Rockin Thomas86” is shown in the video below.
On the spectrogram, you can see pixelated skulls at the start and end of the audio, and, in the middle, a list of letters and numbers. According to the Game Detectives Wiki, the skull shapes were shown on television screens within the game before the rocket launch, while the letters and numbers could be decoded as ASCII values to produce in-game coordinates. Some time after the rocket launch, dimensional rifts opened up at these coordinates, causing locations to appear and disappear on the game map. Players were primed to check the locations, having teased out the message hidden in the rocket launch audio.
Spectrum analysers like our iOS app SpectrumView can open up a whole new dimension of information in audio, and we are excited to see what more our users can find hidden away in the audio of all sorts of ARG content.
SpectrumView and SpectrumView Plus 2.4.1 have been released today, providing a range of bug fixes and ensuring complete compatibility with new devices and iOS 15! This free update can be downloaded from the App Store at the links below, or will have already installed if you have automatic updates turned on in Settings.
SpectrumView: https://apps.apple.com/gb/app/spectrumview/id472662922
SpectrumView Plus: https://apps.apple.com/gb/app/spectrumview-plus/id571455198
Team work makes dream work!
Oxford Wave Research staff are very excited to be attending the upcoming virtual IAFPA Conference, organised this year by Philipps-Universität Marburg. We are delighted to have a number of papers representing the results of our latest research in the field of voice biometrics and audio processing, accepted for presentation at the conference Just to give a sneak peek of what you will be seeing, here is a list of the presentations co-authored by the OWR researchers in collaboration with distinguished academicians and forensic scientists:
The European Network of Forensic Science Institutes (ENFSI) is doing it once more!
European Academy of Forensic Science Conference, EAFS 2022, being organised by Swedish National Forensic Centre (NFC), will take place in Stockholm, Sweden, on May 30 – June 3, 2022.
We look forward to sharing the latest exciting research developments in the field of forensic speaker recognition and audio processing . We will also be showcasing our product range, including the most recent updates and features of our flagship software VOCALISE forensic speaker recognition software at one of the largest and most prestigious European forensic events.