Audio de-identification — from redaction to voice anonymization

MDIG2026 Summer School · Masking day · audio track

Video masking hides the face; but the voice is a biometric too — a speaker embedding can re-identify a person from a few seconds of speech. So an anonymized video with an untouched audio track is not anonymized. This notebook is the audio counterpart to the video masking you did earlier, and it walks from the simplest, most transparent methods to a real voice-anonymization baseline.

The tension is the same privacy ↔︎ utility trade-off as everywhere on masking day:

we want to KEEP (utility)	we want to REMOVE (privacy)
what was said (words)	who said it (speaker identity / voice timbre)
how it was said — prosody, timing, turn-taking	recognisable vocal-tract signature

We go Basic → Advanced:

Part A — Basic: redaction, pitch-shift, telephone band-limiting. Transparent, one-liners, but disguise rather than strong de-identification.
Part B — Advanced: McAdams-coefficient anonymization — shift the vocal-tract resonances (formants) while keeping the excitation, so words and prosody survive but the voice changes. This is the VoicePrivacy 2020 baseline B2.
Part C — Evaluate: did the content survive (ASR) and did the voice actually change? With an honest look at what our metrics do and don’t prove.

Setup (CPU-only): pip install librosa soundfile scipy numpy matplotlib. Optional: faster-whisper (Part C intelligibility check) and resemblyzer (Part C real speaker-embedding check). Both are optional — the notebook runs without them.

0 · Setup & a sample recording

import os, glob, numpy as np
# use only locally-cached models if faster-whisper/resemblyzer pull anything
os.environ.setdefault("HF_HUB_OFFLINE", "1"); os.environ.setdefault("TRANSFORMERS_OFFLINE", "1")
import librosa, librosa.display, soundfile as sf
from scipy.signal import lfilter, butter, sosfiltfilt
import matplotlib.pyplot as plt
from IPython.display import display, Audio

SR = 16000  # everything runs at 16 kHz mono — plenty for speech, and what ASR/embeddings expect

def find_sample():
    """Walk up from the notebook to find a BalanceCorpus .wav; None if not in the repo."""
    d = os.path.abspath(os.getcwd())
    for _ in range(8):
        cand = os.path.join(d, "Datasets", "BalanceCorpus", "audios")
        if os.path.isdir(cand):
            ws = sorted(glob.glob(os.path.join(cand, "*", "*.wav")))
            if ws:
                return ws[0]
        p = os.path.dirname(d)
        if p == d:
            break
        d = p
    return None

def synth_voice(sr=SR, dur=4.0, f0=120.0):
    """Fallback 'voice-like' signal: a glottal pulse train shaped by three formants.
    Only used if no corpus audio is found, so the notebook runs anywhere."""
    n = int(dur * sr); t = np.arange(n) / sr
    # buzzy source: pulse train at f0 with a few harmonics
    src = np.zeros(n)
    for k in range(1, 25):
        src += (1.0 / k) * np.sin(2 * np.pi * f0 * k * t)
    # three formant resonators (a schematic /a/-ish vowel)
    y = src.copy()
    for F, BW in [(700, 90), (1220, 100), (2600, 120)]:
        w0 = 2 * np.pi * F / sr; r = np.exp(-np.pi * BW / sr)
        b = [1 - r]; a = [1, -2 * r * np.cos(w0), r * r]
        y = lfilter(b, a, y)
    env = np.clip(np.sin(np.pi * t / dur), 0, None)  # fade in/out
    y = y * env
    return (y / (np.abs(y).max() + 1e-9) * 0.9).astype(np.float32)

path = find_sample()
if path:
    orig, _ = librosa.load(path, sr=SR, mono=True, duration=5.0)
    print("Loaded real sample:", os.path.basename(path))
else:
    orig = synth_voice()
    print("No corpus audio found — using a synthetic voice-like fallback.")
orig = orig[: int(5.0 * SR)]
print("duration: %.1fs   sr: %d" % (len(orig) / SR, SR))

def plot_spec(y, sr, title, ax):
    D = librosa.amplitude_to_db(np.abs(librosa.stft(y, n_fft=512, hop_length=128)), ref=np.max)
    librosa.display.specshow(D, sr=sr, hop_length=128, x_axis="time", y_axis="hz", ax=ax, cmap="magma")
    ax.set_ylim(0, 5000); ax.set_title(title)

def compare(a, b, sr, label_b, label_a="original"):
    """Side-by-side spectrograms + both audio players."""
    fig, ax = plt.subplots(1, 2, figsize=(11, 3))
    plot_spec(a, sr, label_a, ax[0]); plot_spec(b, sr, label_b, ax[1])
    plt.tight_layout(); plt.show()
    print(label_a + ":"); display(Audio(a, rate=sr))
    print(label_b + ":"); display(Audio(b, rate=sr))

# the original, to hear and see once
fig, ax = plt.subplots(figsize=(6, 3)); plot_spec(orig, SR, "original", ax); plt.tight_layout(); plt.show()
display(Audio(orig, rate=SR))

Part A · Basic de-identification (transparent DSP)

Three one-liners. Each is easy to explain to an ethics board — and easy for an adversary to reason about. Watch the spectrogram (the horizontal bands are formants — the vocal-tract fingerprint) and note what each method does to content vs identity.

A1 · Redaction — mute or beep a known-sensitive span

The bluntest tool: when you know exactly when something identifying is said (a name, an address), delete it. Perfect privacy for that span, zero utility for it. Everything else is untouched.

def redact(y, sr, spans, mode="silence"):
    """spans: list of (t0, t1) seconds. mode: 'silence' | 'beep' | 'noise'."""
    z = y.copy()
    for t0, t1 in spans:
        i0, i1 = int(t0 * sr), int(t1 * sr)
        i0, i1 = max(0, i0), min(len(z), i1)
        if i1 <= i0:
            continue
        if mode == "silence":
            z[i0:i1] = 0.0
        elif mode == "beep":
            t = np.arange(i1 - i0) / sr
            z[i0:i1] = 0.2 * np.sin(2 * np.pi * 1000 * t)
        elif mode == "noise":
            z[i0:i1] = 0.05 * np.random.default_rng(0).standard_normal(i1 - i0)
    return z.astype(np.float32)

red = redact(orig, SR, spans=[(1.0, 1.8)], mode="beep")   # beep out 1.0-1.8 s as a demo
compare(orig, red, SR, "redacted (beep over 1.0-1.8s)")
print("Keeps: everything outside the span.  Removes: the span entirely (content AND identity).")

A2 · Pitch shift — a quick voice disguise

Shift the whole signal up/down in pitch. It sounds like a different person, but note two things on the spectrogram: the formant bands move too (so vowels can sound ‘chipmunk’/‘giant’), and — critically — it is invertible: shift back by the same amount and you largely recover the original. So this is disguise, not de-identification.

pitch = librosa.effects.pitch_shift(orig, sr=SR, n_steps=-4)   # 4 semitones down
compare(orig, pitch, SR, "pitch-shifted (-4 semitones)")
print("Keeps: words, timing.  Weakens: pitch cue.  BUT invertible (+4 semitones) -> disguise only.")

A3 · Telephone band-limiting

Keep only the 300–3400 Hz ‘telephone band’. Speech stays intelligible (phones work!), but the very low and high frequencies — which carry some speaker-specific colour — are gone. Mild privacy, mild utility cost.

def telephone(y, sr, lo=300, hi=3400):
    sos = butter(6, [lo, hi], btype="band", fs=sr, output="sos")
    return sosfiltfilt(sos, y).astype(np.float32)

tel = telephone(orig, SR)
compare(orig, tel, SR, "telephone band (300-3400 Hz)")
print("Keeps: intelligibility.  Removes: sub-300 / super-3400 Hz colour.  Weak on its own.")

Takeaway for Part A. These are transparent and cheap, but each is disguise: pitch-shift is invertible, band-limiting leaves the formant pattern intact, and redaction only helps where you already know the sensitive moment. For genuine voice anonymization we need to change the thing a speaker-recogniser actually keys on — the vocal-tract resonances (formants) — while leaving the words and prosody alone.

Part B · Advanced — McAdams-coefficient voice anonymization

Speech follows a source–filter model: a source (the vocal-fold excitation, which carries pitch and prosody) passed through a filter (the vocal tract, whose resonances are the formants — the speaker-identifying part). If we can change the filter without touching the source, the words and the melody survive but the voice changes.

The recipe (per short frame): 1. Fit an LPC all-pole filter \(1/A(z)\) to the frame — this is the vocal-tract estimate. 2. Inverse-filter to get the residual (the source/excitation) — keeps pitch, timing, energy. 3. Find the filter’s poles and shift each pole’s angle \(\theta \mapsto \theta^{\,\alpha}\) by the McAdams coefficient \(\alpha\) (pole magnitudes unchanged → still stable). This moves the formant frequencies. 4. Re-synthesise the residual through the new filter.

\(\alpha < 1\) pushes formants up, \(\alpha > 1\) pulls them down; \(\alpha = 1\) is a no-op. This is the VoicePrivacy 2020 baseline B2 (Patino et al., 2021).

def mcadams(y, sr, coef=0.8, order=16, frame=0.025):
    """McAdams-coefficient anonymization. Keeps the LPC residual (source), shifts formant
    (pole) angles by `coef`, re-synthesises. Overlap-add with a Hann window at 50% hop."""
    n = int(frame * sr); hop = n // 2; win = np.hanning(n)
    out = np.zeros(len(y) + n); norm = np.zeros(len(y) + n)
    for st in range(0, len(y) - n, hop):
        seg = y[st:st + n] * win
        if np.sum(seg ** 2) < 1e-9:
            norm[st:st + n] += win; continue
        a = librosa.lpc(seg, order=order)          # A(z), a[0] == 1
        res = lfilter(a, [1.0], seg)               # residual = whiten with the inverse filter
        r = np.roots(a); ang = np.angle(r)
        mag = np.minimum(np.abs(r), 0.999)   # clamp poles inside the unit circle -> filter always stable
        new_ang = np.sign(ang) * (np.abs(ang) ** coef)     # raise |angle| to the McAdams power
        new_a = np.real(np.poly(mag * np.exp(1j * new_ang)))
        rec = lfilter([1.0], new_a, res)           # re-synthesise through the shifted filter
        if not np.all(np.isfinite(rec)):
            rec = seg                              # safety net: keep the original frame if it still diverges
        out[st:st + n] += rec * win; norm[st:st + n] += win
    norm[norm < 1e-9] = 1.0
    z = (out / norm)[:len(y)]
    return (z / (np.abs(z).max() + 1e-9) * 0.9).astype(np.float32)   # normalise for playback

anon = mcadams(orig, SR, coef=0.8)
compare(orig, anon, SR, "McAdams anonymized (coef=0.8)")
print("Look: the formant bands have SHIFTED, but the vertical pitch striations (timing/prosody) line up.")

B2 · The coefficient is your privacy↔︎utility dial

Lower coef = a bigger formant shift = a more different voice, but also more distortion (less natural, harder to trust downstream analysis). Sweep it and listen.

for coef in (0.7, 0.85, 0.95):
    z = mcadams(orig, SR, coef=coef)
    print(f"coef = {coef}")
    display(Audio(z, rate=SR))

Part C · Did it work? Utility vs privacy

Anonymization is only useful if it does both: keep the content (utility) and change the speaker (privacy). We check each — and are honest about what each check proves.

C1 · Utility — does the content survive? (ASR)

If a speech recogniser still reads the same words off the anonymized audio, the linguistic content survived. (Optional: needs faster-whisper.)

try:
    from faster_whisper import WhisperModel
    asr = WhisperModel("small", device="cpu", compute_type="int8")
    def transcribe(y):
        segs, _ = asr.transcribe(np.ascontiguousarray(y, dtype=np.float32), language="en", beam_size=1)
        return " ".join(s.text.strip() for s in segs).strip()
    print("ORIGINAL :", transcribe(orig) or "(no speech detected)")
    print("ANONYMIZED:", transcribe(anon) or "(no speech detected)")
    print("\nIf the words broadly match, McAdams kept the content while changing the voice.")
except Exception as e:
    print("Skipping ASR check (install faster-whisper to enable). Reason:", type(e).__name__, e)

C2 · Privacy proxy — how much did the timbre change?

A dependency-free proxy: the cosine between the two clips’ average MFCC (spectral-envelope) vectors. Read this carefully — it measures how much the spectral colour changed, not verified speaker identity. It is a distortion/timbre signal, not a security guarantee.

def envelope_vec(x):
    return librosa.feature.mfcc(y=x.astype(float), sr=SR, n_mfcc=20)[1:].mean(axis=1)  # drop c0 (energy)
def cos(a, b):
    return float(a @ b / (np.linalg.norm(a) * np.linalg.norm(b) + 1e-9))

print("MFCC-envelope cosine  original vs anonymized:  %.3f  (lower = timbre changed more)"
      % cos(envelope_vec(orig), envelope_vec(anon)))
print("\nCaveat: this shows the VOICE COLOUR moved. It does NOT prove a speaker-verification system\n"
      "could no longer identify the person. For that you need a real speaker embedding -> C3.")

C3 · Privacy, properly — a speaker-embedding check (optional)

The honest way to measure de-identification is with the same weapon an attacker uses: a speaker embedding (d-vector). We compare the anonymized clip to the original speaker. A good anonymizer pushes that similarity down toward the level of an unrelated speaker. (Optional: needs resemblyzer, which pulls a small PyTorch model.)

try:
    from resemblyzer import VoiceEncoder, preprocess_wav
    enc = VoiceEncoder(verbose=False)
    def emb(y): return enc.embed_utterance(preprocess_wav(np.asarray(y, dtype=np.float32), source_sr=SR))
    e_orig, e_anon = emb(orig), emb(anon)
    sim = float(np.dot(e_orig, e_anon))
    print("speaker-embedding cosine  original vs anonymized: %.3f" % sim)
    other = find_sample()  # a different recording, if available, as a 'different speaker' reference
    ws = sorted(glob.glob(os.path.join(os.path.dirname(os.path.dirname(other)), "*", "*.wav"))) if other else []
    if len(ws) > 1:
        y2, _ = librosa.load(ws[len(ws)//2], sr=SR, mono=True, duration=5.0)
        print("reference cosine          original vs a DIFFERENT recording: %.3f" % float(np.dot(e_orig, emb(y2))))
    print("\nGoal: the anonymized cosine drops toward the different-speaker level. If it stays high,\n"
          "the anonymization is weak against this attacker.")
except Exception as e:
    print("Skipping speaker-embedding check (pip install resemblyzer to enable).")
    print("Reason:", type(e).__name__, e)

Honest limitations (say these out loud to students)

Baselines are beatable. McAdams is a baseline; strong speaker-verification systems, and attackers who know you used McAdams, can partly re-identify. Treat it as one layer, not a guarantee.
Pitch-shift is reversible — never rely on it alone for privacy.
Prosody is itself identifying. Timing, rhythm, and idiolect can leak identity even after the timbre changes; some studies deliberately alter prosody too.
Utility is task-specific. McAdams preserves words and gross prosody, but if your research measures fine formant dynamics or voice quality, this transform destroys exactly your signal — pick the method against your dependent variable.
De-identification must be irreversible and consented. Keep no key that undoes it; document what you changed.
Measure privacy with the attacker’s tool (a speaker embedding / ASV), not a convenience proxy.

Recap & references

Basic methods (redact, pitch-shift, band-limit) are transparent but are disguise: reversible or leave the formant fingerprint intact.
Advanced McAdams anonymization changes the vocal-tract filter while keeping the source, so content and prosody survive while the voice changes — the VoicePrivacy baseline.
Always evaluate both sides: content survival (ASR) and identity change (a real speaker embedding), and read every metric for what it actually proves.

References (Crossref-verified DOIs):

Tomashenko, N., et al. (2020). Introducing the VoicePrivacy Initiative. Interspeech 2020. https://doi.org/10.21437/interspeech.2020-1333
Patino, J., Tomashenko, N., Todisco, M., Nautsch, A., & Evans, N. (2021). Speaker Anonymisation Using the McAdams Coefficient. Interspeech 2021. https://doi.org/10.21437/interspeech.2021-1070

Method background: the McAdams coefficient originates in S. McAdams (1984), Spectral fusion, spectral parsing and the formation of auditory images (PhD thesis, Stanford) — no Crossref DOI.