Voice cloning and liveness bypass

Voice cloning and liveness bypass

TL;DR: Modern voice-cloning models (ElevenLabs, OpenAI Voice, open-source XTTS / Tortoise / Bark / Vall-E variants) can clone a voice from seconds of audio with quality that fools both humans and most voice-biometric systems. Bypasses target call-center voice authentication, voice-print “passwords”, and family-impersonation scams. Defenders adopt liveness challenges, multi-factor verification, and provenance tracking. Companion to deepfake-assisted-phishing and mfa-fatigue-tradecraft.

Why voice cloning matters

  • Cost has collapsed: ElevenLabs and similar consumer-grade tools clone voices from ~10 seconds of audio.
  • Quality rivals human recordings for short utterances.
  • Real-time synthesis is now practical — voice phone calls with cloned voice.
  • Voice biometrics (call-center authentication) widely deployed and increasingly broken.
  • Family / executive impersonation scams (“Hi mom, I lost my phone, send money to this account”) have caused real losses.

Real attack scenarios

Call-center authentication bypass

Many banks / utilities use “voice print” as MFA or sole auth for phone customer service:

  • Caller speaks pre-set phrase.
  • System compares to stored voiceprint.
  • Authenticated.

Attack:

  • Obtain victim audio from social media, podcasts, YouTube, voicemail.
  • Clone with consumer tool.
  • Synthesise the auth phrase; play during call.

Documented success against several major bank voice systems in 2023–2025.

Family / executive scam

  • Attacker scrapes target’s voice from social media.
  • Clones.
  • Calls family member or finance department posing as target.
  • “I’m in trouble, please send money / wire transfer / pay this invoice.”

The Hong Kong “$25M deepfake video call” scam (2024) used cloned video and voice of CFO and others.

CEO fraud

Variant of executive scam targeting business email compromise → voice call to confirm the wire.

Government / official impersonation

  • Cloned voice of politicians for disinformation.
  • Robocall campaigns with cloned candidate voices.

How cloning works

Modern systems:

  • Voice encoder maps a reference audio to a speaker embedding.
  • TTS model generates speech conditioned on text + speaker embedding.
  • Vocoder converts to waveform.

Few-shot zero-shot cloning: 5–30 seconds reference suffices. Real-time inference on consumer GPU.

Liveness defences

Liveness challenges aim to make replays / clones harder:

  • Random phrase challenges — system asks for a unique phrase the attacker hasn’t synthesised yet. Defeated by real-time TTS.
  • Background noise analysis — check for compression artefacts, vocoder fingerprints.
  • Audio watermarking — embed inaudible signal during generation; detector identifies. See ai-model-watermark-bypass — survives only some edits.
  • Behavioural analysis — speech patterns, prosody, dialogue flow over multi-turn.
  • Call metadata — IP, geo, device fingerprint.

Modern defences specifically against voice cloning

  • Pindrop, Nuance, ID R&D — commercial deepfake-detection services for telephony.
  • Statistical detectors — looking for vocoder-specific artefacts.
  • Phase-coherence anomalies — voice produced by typical vocoders has phase patterns differing from human-speech recording.

These detect most deepfakes but accuracy on cutting-edge synthesis declines.

Defensive baseline

For organisations relying on voice authentication:

  • Don’t use voiceprint as sole authentication.
  • Multi-factor: voice + something else (account knowledge, second channel verification).
  • Random-challenge phrases plus deepfake-detection.
  • Call-back verification to registered numbers.
  • Wire-transfer hold periods with second-channel confirmation.

For executives / high-risk individuals:

  • Code word with family members for emergency identification.
  • Public profile audio discipline — minimise sample-rich content.
  • Multi-channel verification for financial requests.

Workflow to study (research / red team)

  1. Install an open-source voice clone (XTTS, Tortoise, Bark).
  2. Clone your own voice from a 10-second sample.
  3. Test against a (consensual) voice authentication system.
  4. Try real-time TTS chained with a phone call.
  5. Evaluate deepfake-detection tools on the generated audio.

For research, use only your own voice or with explicit consent.

Forensic / detection

For investigating a suspected voice fraud incident:

  • Pull the recording (call-center systems often keep recordings).
  • Run deepfake-detection tools (multiple, ensemble).
  • Check the originating number’s history / spoofing indicators.
  • Cross-check with the supposed speaker’s known availability.

Regulatory landscape

  • FCC ruled robocalls with AI-generated voices illegal under TCPA (2024).
  • Several US states have introduced AI-impersonation bans.
  • EU AI Act labels generated audio as a regulated category.
  • ITU and telco-industry standards emerging for STIR/SHAKEN-equivalent voice-authenticity.

Real-world incidents

  • Hong Kong $25M scam (2024) — deepfake video call of CFO + colleagues; finance staff wired funds.
  • CEO voice scam documented by Forbes and others (multiple cases 2019–2024).
  • Family-grandparent scams rampant; widely reported by FTC.
  • Bank voice-print bypass demonstrations at Black Hat / DEF CON.

What’s hard about defending

  • Voice quality of attacks ratchets up; detectors race to keep up.
  • False-positive rate on liveness checks frustrates legitimate users.
  • Detection signal may not survive call-quality compression (PCM → narrowband → recording).
  • Older PSTN infrastructure has minimal authenticity guarantees.

References