OWR’s audio deepfake technology aces the UK Home Office-led ‘Deepfake Detection Challenge’
![image_123650291 image_123650291](http://oxfordwaveresearch.com/wp-content/uploads/2024/08/image_123650291-scaled.jpg)
Below is the abstract of the talk:
Voice recognition is the process by which distinctive characteristics of an individual’s speech are used to identify or verify who they are. As lay listeners, humans recognise familiar voices intuitively in an everyday sense and may also find themselves being ‘earwitnesses’ to a crime, albeit rarely. When carried out by trained practitioners using specialised methodologies and tools, voice recognition, comparing often unknown speech samples, can play an important role in investigative and forensic contexts.
This talk will consider the landscape of forensic voice recognition, encompassing auditory analysis by trained listeners, acoustic-phonetic measurements of perceptually salient features, and automatic speaker recognition using signal processing and modelling algorithms that are statistical or based on deep neural networks. The Bayesian likelihood ratio framework will be critically examined as a means of evaluating the strength of evidence using any voice analysis methodology. The importance of validation of the prevalent and emerging approaches, to understand their limitations and to provide reliable and transparent reports to the courts, will be discussed.
Additionally, the varying acceptance of voice recognition evidence in different parts of the world will be explored. Anticipating the new challenges posed by machine-created spoofed speech, this talk also will reflect on the risks, mitigations and, more optimistically, emerging opportunities afforded by using both human- and machine-based analysis.
- A convincing voice clone? Automatic voice similarity assessment for synthetic speech samples
Linda Gerlach, Finnian Kelly, Kirsty McDougall, and Anil Alexander- PASS (Phonetic Assessment of Spoofed Speech): Towards a human-expert-based framework for spoofed speech detection
Daniel Denian Lee, Kirsty McDougall, Finnian Kelly, and Anil Alexander- CON(gruence)-plots for assessing agreement between voice comparison systems
Michael Jessen, Anil Alexander, Thomas Coy, Oscar Forth, and Finnian Kelly- Automatic Speaker Recognition: does dialect switching matter?
Marlon Siewert, Linda Gerlach, Anil Alexander, Gea de Jong-Lendle, Alfred Lameli, and Roland Kehrein- Impact of the mismatches in long-term acoustic features upon different-speaker ASR scores
Chenzi XU, Paul Foulkes, Philip Harrison, Vincent Hughes, Poppy Welch, Jessica Wormwald, Finnian Kelly, and David van der Vloed- Effects of vocal variation on the output of an automatic speaker recognition system
Vincent Hughes, Jessica Wormwald, Paul Foulkes, Philip Harrison, Poppy Welch, Chenzi Xu, Finnian Kelly, and David van der Vloed
“We’re really pleased to have started the project and to be working with Oxford Wave Research. The VOCALISE software allows us to answer big questions around the use of automatic speaker recognition systems in new and exciting ways. We can’t wait to see what comes out of the work!”
Dr Vincent Hughes, Principal Investigator of the project
“We were delighted to be able to spend time with the York University team recently, getting them started with VOCALISE and discussing their exciting plans for the project. They have a really experienced team and have lots of interesting ideas in the works – we are looking forward to seeing where it all goes!”
Dr Finnian Kelly, OWR Principal Research Scientist and Lead Scientific advisor on the project
Amelia’s research to date has focused on the anatomical bases of speaker identity, and particularly how individual differences in vocal tract shape affect the speech signal. For this work she was awarded a British Academy Postdoctoral Fellowship. She holds a PhD in Electronic Engineering and an MSc in Digital Signal Processing, both from the University of York, as well as a BSc (Hons) in Audio Technology from the University of Salford.“I am delighted to be joining the team at Oxford Wave Research, where I can put my acoustics and signal processing experience to work addressing real problems for customers, and contribute to exciting technological developments in the field of audio forensics.”
Dr Amelia Gully
“I am excited to welcome Amelia to OWR – with her expertise in acoustics and signal processing, and enthusiasm for all-things audio, she will be a valuable addition to the research team!” Dr Finnian Kelly, Principal Research ScientistAmelia joins us remotely from York where she lives with her partner and two rescue dogs. When not engaged in audio and speech research, she can be found playing video games or pottering around on her allotment.
Oxford Wave Research are delighted to be collaborators with the University of York and the Netherlands Forensic Institute in a recently awarded ESRC-funded project (£1,012,570) ‘Person-specific automatic speaker recognition: understanding the behaviour of individuals for applications of ASR’ (ES/W001241/1). This is a three year project running from 2022 to 2025 led by Dr Vincent Hughes (PI), Professor Paul Foulkes (CI) and Dr Philip Harrison in the Department of Language and Linguistic Science at the University of York. The project is due to start in summer 2022 and will run for 3 years. OWR will be providing our expertise and consultancy in automatic speaker recognition and our flagship VOCALISE forensic speaker recognition system.
Automatic speaker recognition (ASR) software processes and analyses speech to inform decisions about whether two voices belong to the same or different individuals. Such technology is becoming an increasingly important part of our lives; used as a security measure when gaining access to personal accounts (e.g. banks), or as a means of tailoring content to a specific person on smart devices. Around the world, such systems are commonly used for investigative and forensic purposes, to analyse recordings of criminal voices where identity is unknown.
The aim of this project is to systematically analyse the factors that make individuals easy or difficult to recognise within automatic speaker recognition systems. By understanding these factors, we can better predict which speakers are likely to be problematic, tailor systems to those individuals, and ultimately improve overall accuracy and performance. The project will use innovative methods and large-scale data, uniting expertise from linguistics, speech technology, and forensic speech analysis, from the academic, professional, and commercial sectors. This has been made possible via the University of York’s strong collaboration with Oxford Wave Research and two project partners including the Netherlands Forensic Institute (NFI).
The University of York and OWR teams are looking forward to a very fruitful collaboration that will undoubtedly further the state of the art in forensic speaker recognition.
Dr Vincent Hughes, Principal Investigator, University of York says “We are delighted to be working so closely on this project with Oxford Wave Research, who are world leaders in the field of automatic speaker recognition and speech technology. We hope that our research will deliver major benefits to the fields of speaker recognition and forensic speech science”.
Dr Anil Alexander, CEO, Oxford Wave Research says “Our team led by Dr Finnian Kelly is thrilled to contribute to this in-depth study of the individual specific-factors affecting speaker recognition, with the accomplished research team led by Dr Hughes from the University of York who are at the forefront of this space, and real-word end-users like the Netherlands Forensic Institute who have been driving research and innovation in this space for many years”.
Content creators of all kinds, such as the musician Aphex Twin, have long used hidden secret patterns and text in the audio that can be observed in their spectrograms. More recently, video game developers have hidden Easter eggs in the spectrograms of their game soundtracks for their more inquisitive players to find. For example, among Minecraft’s sound effects, the face of a Creeper, one of the game’s enemies, can be seen in the spectrogram of the audio heard in a cave, as SpectrumView user “Musix200” discovered. See if you can spot it too!
Taking this idea a step further, alternate reality games (ARGs) are a modern spin on the traditional scavenger hunt in which participants scour websites, social media, and videos looking for clues. These games have taken social media sites like YouTube and Reddit by storm over the last few years. Organisers, often video game developers, will bury information in all sorts of places, like images, website code, and audio. Whole communities have been formed in order to find out secret stories and previews for their favourite video game, or just to have some cooperative fun while solving a digital mystery.
Epic Games created ARG content in the run-up to the Season 5 release of their famous multiplayer online game, Fortnite Battle Royale. They staged a rocket launch within the video game itself, during which some of the audio played was slightly odd. Gamers quickly realised that there was probably more to the audio clip than what could be just heard. Looking for patterns within the audio led them to visualising the frequencies in the audio in a spectrogram. One such example of using SpectrumView to analyse the audio clip by player “Rockin Thomas86” is shown in the video below.
On the spectrogram, you can see pixelated skulls at the start and end of the audio, and, in the middle, a list of letters and numbers. According to the Game Detectives Wiki, the skull shapes were shown on television screens within the game before the rocket launch, while the letters and numbers could be decoded as ASCII values to produce in-game coordinates. Some time after the rocket launch, dimensional rifts opened up at these coordinates, causing locations to appear and disappear on the game map. Players were primed to check the locations, having teased out the message hidden in the rocket launch audio.
Spectrum analysers like our iOS app SpectrumView can open up a whole new dimension of information in audio, and we are excited to see what more our users can find hidden away in the audio of all sorts of ARG content.
SpectrumView and SpectrumView Plus 2.4.1 have been released today, providing a range of bug fixes and ensuring complete compatibility with new devices and iOS 15! This free update can be downloaded from the App Store at the links below, or will have already installed if you have automatic updates turned on in Settings.
SpectrumView: https://apps.apple.com/gb/app/spectrumview/id472662922
SpectrumView Plus: https://apps.apple.com/gb/app/spectrumview-plus/id571455198
Team work makes dream work!
Oxford Wave Research staff are very excited to be attending the upcoming virtual IAFPA Conference, organised this year by Philipps-Universität Marburg. We are delighted to have a number of papers representing the results of our latest research in the field of voice biometrics and audio processing, accepted for presentation at the conference Just to give a sneak peek of what you will be seeing, here is a list of the presentations co-authored by the OWR researchers in collaboration with distinguished academicians and forensic scientists:
The European Network of Forensic Science Institutes (ENFSI) is doing it once more!
European Academy of Forensic Science Conference, EAFS 2022, being organised by Swedish National Forensic Centre (NFC), will take place in Stockholm, Sweden, on May 30 – June 3, 2022.
We look forward to sharing the latest exciting research developments in the field of forensic speaker recognition and audio processing . We will also be showcasing our product range, including the most recent updates and features of our flagship software VOCALISE forensic speaker recognition software at one of the largest and most prestigious European forensic events.
Oxford Wave Research is pleased to announce the appointment of Bounga Informatics Pte Ltd as our distributor in Singapore. Bounga Informatics is a well established and respected provider of forensic products in Singapore and we are delighted to have their support for the increased interest in our products, including our flagship VOCALISE forensic voice biometric software, in Singapore. We look forward to a fruitful collaboration with Bounga Informatics in the months and years to come.
“We are honoured that a world-renowned company such as Oxford Wave has appointed Bounga as their Singapore distributor. We look forward to working with them in this exciting sphere.”
Frank Butler, Managing Director at Bounga
This is the fascinating use case for which our partner WhoIAM has been using our voice biometrics technology. This has just been featured in the latest edition of the Microsoft Azure Identity partner integration video that uses our speaker biometrics-based authentication to make identity security design more inclusive. We at Oxford Wave Research support this laudable goal all the way!
As Ajith Alexander, head of product management at WhoIAM, writes in the Microsoft Azure AD identity blog:
“Using voice biometrics for verification is also a powerful tool for implementing inclusive security. Human voices are readily available, can be recorded in a contactless way, and do not require specialized hardware. Our voice carries an imprint of our identity that comes through regardless of what we’re saying, what language we’re speaking, or where we’re speaking from. This makes voice biometrics an ideal choice for catering to users who are visually impaired, have difficulty reading or understanding a language, don’t have access to a dedicated personal device (residents at assisted-living communities, shift-workers), or live in less developed areas that rely on fixed phone lines. …Creatively solving for flexible, inclusive user verification ensures we can log in previously marginalized customers securely without identity verification being a frustrating experience.”
Ajith Alexander, Head of Product Management, WhoIAM
The OWR team, Nikki, Ekrem, Oscar and Anil warmly welcome you to join us at the first ever virtual Security & Policing event taking place 9-11 March 2020.
This year’s event coincides with our 10 year anniversary and we are proud to be sharing with you our state-of-the-art desktop and ‘on-device’ speaker recognition and audio processing software in use at the forefront of the voice biometric field.
We are delighted that Oxford Wave Research Ltd and the University of Cambridge, in collaboration with the Cambridge Trust, have established a new PhD studentship based at Selwyn College, Cambridge. The award enables a student to undertake a PhD in Theoretical and Applied Linguistics, commencing in October 2020. The studentship was open to UK and EU candidates of outstanding academic potential, and covers tuition fees and maintenance for three years.
The studentship is in the area of forensic phonetics, the application of phonetic analysis to criminal cases, often where the identity of a speaker is in question, either due to an incriminating recording (e.g. hoax call, ransom demand, telephone threat, etc.) or due to a witness having heard a speech event at a crime scene. Forensic phonetics uses both traditional phonetic and automatic (machine-based) techniques.
The PhD project aims to consider the relationships between traditional phonetic analyses and automatic speaker recognition (computer-based identification and recognition of the identity behind a voice). The studentship will include collaborative opportunities for the student to gain industry experience and to conduct research in conjunction with Oxford Wave Research, an audio processing and voice-biometrics company which specialises in developing solutions for law enforcement agencies in forensic voice comparison. The student’s research will consider both human and machine-based, algorithmic selection of different groups of speakers for various forensic analyses based on different criteria and the implication of the selections of these groups in the evaluation of the strength of evidence. These criteria include voice similarity perceived by human listeners and demographic features such as gender, language, age, regional accent. Further, the research will attempt to evaluate how the human or automatic, machine-based selection of databases can result in algorithmic bias.
“We are delighted to be working with a leading audio-processing and voice biometrics company that has such a strong track record of developing solutions in the forensic speech and audio arena. Cambridge has a well-established tradition of research excellence and innovation in forensic phonetics and the opportunity to bring automatic speaker recognition techniques to complement our acoustic-phonetic and perceptual approaches represents an exciting new line of investigation for our Phonetics Lab.”
Dr Kirsty McDougall, University of Cambridge
“This studentship overseen by Dr McDougall at the Phonetics Laboratory in Cambridge represents an incredible opportunity for us to formally collaborate with one of the best-regarded forensic phonetics research groups in the country, with an enduring legacy of fundamental and important research work. We look forward to the exciting research collaboration planned with the laboratory in this studentship that has important implications for how forensic casework involving speech is done in the future and which will help the legal system by providing timely, just and balanced analysis.”
Dr Anil Alexander, CEO of Oxford Wave Research
The recipient of this studentship in 2020 is Ms Linda Gerlach. Linda obtained her undergraduate degree in Language and Communication at Philipps University Marburg, Germany, and went on to complete her masters degree in Speech Science with a focus on phonetics at the same university. For her masters thesis titled “A study on voice similarity ratings: humans versus machines”, she worked in collaboration with the University of Cambridge during an internship at Oxford Wave Research (2018-2019).
The University of Cambridge Phonetics Laboratory is based in the university’s Theoretical and Applied Linguistics Section, Faculty of Modern and Medieval Languages, and accommodates a strong community of teaching and research staff, research students, a number of affiliated researchers in phonetics, and a lab manager. As well as hosting an extensive programme of research in forensic phonetics, the lab fosters research in phonetics and phonology across a diverse range of topics including speech production and perception, language acquisition, psycholinguistics, prosody, tone, sociophonetics, and language variation and change. Recent funded projects in forensic phonetics include DyViS, VoiceSim and IVIP.
Oxford Wave Research (OWR) is a specialised audio R&D company with expertise in voice biometrics, speaker diarization, audio fingerprinting and audio enhancement. The OWR team have contributed to major government projects, nationally and internationally. OWR has been particularly successful in bringing practical applications of state-of-the-art academic research algorithms to usable commercial products for law enforcement, military and other agencies. OWR’s solutions are used by law enforcement and forensic laboratories across the world including the UK, Germany, Netherlands, France, Canada, Switzerland. OWR are the creators of the well-established forensic voice comparison system ‘VOCALISE‘, used in forensic audio labs across the world, as well as ‘WHISPERS’ which is a powerful networked ‘one to many’ voice comparison system.
Two of our collaborative papers, one on voice spoofing detection, and the other on the effects of device variability on forensic speaker comparison, are appearing at this week’s virtual ODYSSEY 2020 Speaker and Language Recognition Workshop. Video presentations for both papers are now available on the workshop website: http://www.odyssey2020.org/
The full papers, along with the rest of the conference proceedings, can be found at: https://www.isca-speech.org/archive/odyssey_2020/index.html
In our paper with Bence Halpern (PhD student, University of Amsterdam), “Residual networks for resisting noise: analysis of an embeddings-based spoofing countermeasure,” we propose a new embeddings-based method of spoofed speech detection using Constant Q-Transform (CQT) features and a Dilated ResNet Deep Neural Network (DNN) architecture. The novel CQT-GMM-DNN approach, which uses the DNN embeddings with a Gaussian Mixed Model (GMM) classifier, performs favourably compared to the baseline system in both clean and noisy conditions. We also present some ‘explainable audio’ results, which provide insight into the information the DNN exploits for decision-making. This study shows that reliable detection of spoofed speech is increasingly possible, even in the presence of noise.
See a blog post from Bence (including some explainable audio examples) here: https://karkirowle.github.io/publication/odyssey-2020
In our paper with David van der Vloed (from the Netherlands Forensic Institute), “Exploring the effects of device variability on forensic speaker comparison using VOCALISE and NFI-FRIDA, a forensically realistic database,” we investigate the effect of recording device mismatch on forensic speaker comparison with VOCALISE. Using the forensically-realistic NFI-FRIDA database, consisting of speech simultaneously-recorded on multiple devices (e.g. close-mic, far-mic, and telephone intercept, as seen in the data collection image), we demonstrate that while optimal performance is achieved by matching the relevant population recording device to the case data recording device, it is not necessary to match the precise device; broadly matching the device type is sufficient. This study presents a research methodology for how a forensic practitioner can corroborate their subjective judgment of the ‘representativeness’ of the relevant population in forensic speaker comparison casework.