You can't test a medical speech-to-text API on real patient recordings until you have a BAA, patient consent and somewhere safe to keep the audio. You need test audio long before any of that exists. That gap is why we put together a free set of 329 doctor-patient conversation recordings, about 60 hours in total, each with a transcript typed by a person.

Every visit was role-played, so there's no patient data in it, and it still sounds like a real visit: filler words, interruptions, lay terms and people correcting themselves. It's on Hugging Face and in the open Clinical Scribe Kit on GitHub. Sully maintains it and sells a clinical speech and notes API, so the code below uses Sully, but the audio works with any vendor.

Key Takeaways

• 329 recordings, about 60 hours. 272 Canadian history-taking interviews (CC0) and 57 UK remote GP consults (CC BY 4.0), all openly licensed for commercial use.

• Every recording has a human transcript. That's your answer key for word error rate. The 57 UK consults also come with the note the clinician wrote.

• Use the big set for accuracy and the small one for speaker labels. The UK consults have timed turns, which lets you check who said what, not only which words were said.

• It won't replace your own pilot audio. The recordings are quiet, one-on-one and not American. Use them to choose a vendor and catch regressions, then test on audio from your real setting.

What's in the Audio Set

The recordings come from two published research datasets. We converted both into one format: 16 kHz mono MP3, with transcripts that use the same Clinician: and Patient: labels.

The median visit runs to about 1,300 words in the Fareez set and 1,500 in PriMock57. PriMock57 patients say "paracetamol" and "diarrhoea", which is useful if your users aren't all American and a trap if you only expect US drug names.

What it sounds like

Here's the opening of one of the Fareez recordings, as transcribed by the dataset's authors:

Notice the repeated "I I" and the "um" before the age. A speech API that cleans those up will score worse against this transcript than one that keeps them, even if its output reads better. That's a scoring choice you need to make on purpose, covered below.

How to Download the Recordings

The fastest way is to pull only the files you need from Hugging Face. This downloads the 57 PriMock57 recordings (about 190 MB) and the metadata file that holds every transcript:

from huggingface_hub import snapshot_download

path = snapshot_download(
    "SullyHQ/open-clinical-conversations",
    repo_type="dataset",
    allow_patterns=["audio/pm57-*", "audio/metadata.jsonl"],
)

from huggingface_hub import snapshot_download

path = snapshot_download(
    "SullyHQ/open-clinical-conversations",
    repo_type="dataset",
    allow_patterns=["audio/pm57-*", "audio/metadata.jsonl"],
)

from huggingface_hub import snapshot_download

path = snapshot_download(
    "SullyHQ/open-clinical-conversations",
    repo_type="dataset",
    allow_patterns=["audio/pm57-*", "audio/metadata.jsonl"],
)

Change the pattern to audio/osce-* for the Fareez recordings (about 1.1 GB), or drop allow_patterns to take everything. Each line of audio/metadata.jsonl has the file_name, encounter_id, source, license and the full transcript.

If you'd rather work from files on disk, clone the GitHub repo. It ships 20 recordings so you can start straight away, and scripts/download_audio.py fetches the rest from the original sources. The repo also has transcript.json for each visit, where PriMock57 turns carry start and end times in seconds.

How to Measure Word Error Rate on Medical Audio

Word error rate (WER) counts the words a system got wrong, added or dropped, divided by the words in the reference transcript. It's the standard first check, and it's easy to get a misleading number if the two transcripts don't follow the same rules.

Send the audio to a speech API

Sully's transcription endpoint is asynchronous: you upload a file, get an ID back, and poll until it's done. PriMock57 is British English, so pass en-GB for those files.

import os, time, requests

BASE = "https://api.sully.ai"
HEADERS = {
    "X-Account-Id": os.environ["SULLY_ACCOUNT_ID"],
    "X-Api-Key": os.environ["SULLY_API_KEY"],
}

def transcribe(mp3_path, language="en"):
    with open(mp3_path, "rb") as f:
        r = requests.post(
            f"{BASE}/v2/audio/transcriptions",
            headers=HEADERS,
            files={"audio": (os.path.basename(mp3_path), f, "audio/mpeg")},
            data={"language": language},
        )
    r.raise_for_status()
    tx_id = r.json()["data"]["id"]

    while True:
        data = requests.get(
            f"{BASE}/v2/audio/transcriptions/{tx_id}", headers=HEADERS
        ).json()["data"]
        if data["status"] in ("completed", "failed"):
            return data
        time.sleep(3)

import os, time, requests

BASE = "https://api.sully.ai"
HEADERS = {
    "X-Account-Id": os.environ["SULLY_ACCOUNT_ID"],
    "X-Api-Key": os.environ["SULLY_API_KEY"],
}

def transcribe(mp3_path, language="en"):
    with open(mp3_path, "rb") as f:
        r = requests.post(
            f"{BASE}/v2/audio/transcriptions",
            headers=HEADERS,
            files={"audio": (os.path.basename(mp3_path), f, "audio/mpeg")},
            data={"language": language},
        )
    r.raise_for_status()
    tx_id = r.json()["data"]["id"]

    while True:
        data = requests.get(
            f"{BASE}/v2/audio/transcriptions/{tx_id}", headers=HEADERS
        ).json()["data"]
        if data["status"] in ("completed", "failed"):
            return data
        time.sleep(3)

import os, time, requests

BASE = "https://api.sully.ai"
HEADERS = {
    "X-Account-Id": os.environ["SULLY_ACCOUNT_ID"],
    "X-Api-Key": os.environ["SULLY_API_KEY"],
}

def transcribe(mp3_path, language="en"):
    with open(mp3_path, "rb") as f:
        r = requests.post(
            f"{BASE}/v2/audio/transcriptions",
            headers=HEADERS,
            files={"audio": (os.path.basename(mp3_path), f, "audio/mpeg")},
            data={"language": language},
        )
    r.raise_for_status()
    tx_id = r.json()["data"]["id"]

    while True:
        data = requests.get(
            f"{BASE}/v2/audio/transcriptions/{tx_id}", headers=HEADERS
        ).json()["data"]
        if data["status"] in ("completed", "failed"):
            return data
        time.sleep(3)

Normalize both sides, then score

The reference transcripts keep fillers like "um", use digits for some numbers and mark unclear speech as [inaudible]. Strip the speaker labels, fillers and punctuation from both sides before you compare, or you'll be measuring formatting instead of hearing.

import json, re
import jiwer  # pip install jiwer

FILLERS = r"\b(um+|uh+|ah+|er+|mm+|hmm+)\b"

def clean(text):
    text = re.sub(r"^(Clinician|Patient|Family member):\s*", "", text, flags=re.M)
    text = text.replace("[inaudible]", " ").lower()
    text = re.sub(r"[^a-z0-9' ]+", " ", text)
    text = re.sub(FILLERS, " ", text)
    return " ".join(text.split())

rows = [json.loads(line) for line in open(f"{path}/audio/metadata.jsonl")]

for row in [r for r in rows if r["file_name"].startswith("pm57")][:10]:
    result = transcribe(f"{path}/audio/{row['file_name']}", language="en-GB")
    if result["status"] != "completed":
        print(row["encounter_id"], "failed")
        continue
    hypothesis = " ".join(c["transcript"] for c in result["result"]["channels"])
    score = jiwer.wer(clean(row["transcript"]), clean(hypothesis))
    print(row["encounter_id"], round(score, 3))

import json, re
import jiwer  # pip install jiwer

FILLERS = r"\b(um+|uh+|ah+|er+|mm+|hmm+)\b"

def clean(text):
    text = re.sub(r"^(Clinician|Patient|Family member):\s*", "", text, flags=re.M)
    text = text.replace("[inaudible]", " ").lower()
    text = re.sub(r"[^a-z0-9' ]+", " ", text)
    text = re.sub(FILLERS, " ", text)
    return " ".join(text.split())

rows = [json.loads(line) for line in open(f"{path}/audio/metadata.jsonl")]

for row in [r for r in rows if r["file_name"].startswith("pm57")][:10]:
    result = transcribe(f"{path}/audio/{row['file_name']}", language="en-GB")
    if result["status"] != "completed":
        print(row["encounter_id"], "failed")
        continue
    hypothesis = " ".join(c["transcript"] for c in result["result"]["channels"])
    score = jiwer.wer(clean(row["transcript"]), clean(hypothesis))
    print(row["encounter_id"], round(score, 3))

import json, re
import jiwer  # pip install jiwer

FILLERS = r"\b(um+|uh+|ah+|er+|mm+|hmm+)\b"

def clean(text):
    text = re.sub(r"^(Clinician|Patient|Family member):\s*", "", text, flags=re.M)
    text = text.replace("[inaudible]", " ").lower()
    text = re.sub(r"[^a-z0-9' ]+", " ", text)
    text = re.sub(FILLERS, " ", text)
    return " ".join(text.split())

rows = [json.loads(line) for line in open(f"{path}/audio/metadata.jsonl")]

for row in [r for r in rows if r["file_name"].startswith("pm57")][:10]:
    result = transcribe(f"{path}/audio/{row['file_name']}", language="en-GB")
    if result["status"] != "completed":
        print(row["encounter_id"], "failed")
        continue
    hypothesis = " ".join(c["transcript"] for c in result["result"]["channels"])
    score = jiwer.wer(clean(row["transcript"]), clean(hypothesis))
    print(row["encounter_id"], round(score, 3))

Numbers are the next thing to normalize. One system writes "52", another writes "fifty two", and both are right. A shared normalizer such as the one in OpenAI's open-source Whisper package handles numbers and spellings, as long as you run it on both sides.

Look past the average

An overall WER hides the errors that matter. Missing "um" costs the same as turning "metformin" into "met for men", and only one of those ends up in a chart. Pull a list of the clinical terms in each reference transcript and check them on their own. The article on WER and drug names covers how to build that term-level check.

Testing Speaker Labels

A transcript with every word right can still put "no chest pain" in the clinician's mouth. Speech APIs return numbered speakers, like speaker: 0, not roles, so you have to test the mapping from speaker to clinician or patient yourself.

PriMock57 is the part of the set built for this, because its turns have timestamps. For each word your API returns, find the reference turn that covers that moment, then count how many words land on the wrong role after your mapping step. Keep the short answers in view: a review of systems is mostly "no" and "mm-hm", and those are the words diarization gets wrong most often.

The speaker diarization guide covers mapping labels to roles and why a low diarization error rate can still hide swapped roles. Five visits in the wider dataset include a family member speaking, which is a good stress test for any two-speaker assumption.

What This Audio Won't Tell You

Role-played audio is the right place to start, and a poor place to stop. Know where it differs from your users before you trust a number from it.

• It's quiet. No exam-room fans, no masks, no phone left on a desk across the room. Expect worse accuracy in a real clinic.

• The accents are Canadian and British. There are no US regional accents and no patients speaking English as a second language.

• It's mostly one clinician and one patient. There are no interpreters and no visits that switch language. The multilingual transcription guide covers that case.

• The topics lean respiratory. 213 of the 272 Fareez interviews are respiratory cases, so specialty vocabulary like oncology drug names is thin.

Use this set to shortlist vendors, set a baseline and catch regressions when you change models. Then confirm on recordings from your own setting, under a BAA, before you go live. The HIPAA medical transcription API comparison lists which vendors sign one and on which plans.

How Sully Fits

The Sully API covers the steps after the audio. Recorded and streaming transcription return word timings and optional speaker labels. Set multichannel to true when you have separate tracks, and dictation for dictated notes. The transcript then goes to Note Generation with your template, and Text to JSON turns the note into structured fields for the EHR.

So the same 57 PriMock57 visits can test the whole chain: audio to transcript to note, with the clinician's own note to compare against. The companion guide to the transcripts and notes covers that half.

To try it, create an account and get keys on the Sully API page, then run the scripts above on ten recordings. It takes a few minutes, and you'll have your first WER baseline on medical audio.

Frequently Asked Questions

Is there a free dataset of doctor-patient conversation audio?

Yes. The Open Clinical Conversations dataset has 329 role-played doctor-patient recordings, about 60 hours, with human transcripts. It combines the Fareez et al. OSCE interviews (CC0) and PriMock57 (CC BY 4.0).

Can I use this audio in a commercial product?

Both licenses allow commercial use. CC0 needs no attribution, and CC BY 4.0 needs credit to the original authors. The attribution file has the wording, and each row carries its own license.

Does the audio contain real patient data?

No. Every visit was role-played by clinicians with actors, trained patients or staff working from case cards. That's why it can be shared openly. It isn't medical advice and shouldn't be used for clinical decisions.

Can I train a speech model on it?

The licenses allow it, but 60 hours is small for training and the accents are narrow. It's better used as a held-out test set, so the scores you see aren't inflated by a model that has already heard the files.

Do the recordings have separate tracks for doctor and patient?

The original PriMock57 release has separate doctor and patient channels. In this dataset they're mixed to one mono track, so you can test diarization on a single microphone. Get the originals from the PriMock57 repo if you want to test channel separation.

What's a good word error rate for medical audio?

There isn't one number that holds across recordings, accents and normalization rules. Compare vendors on the same files with the same normalizer, and weight errors on drug names, doses and negations more than errors on filler words.

Sources

1. Open Clinical Conversations on Hugging Face

2. Clinical Scribe Kit on GitHub

3. Fareez et al., A dataset of simulated patient-physician medical interviews with a focus on respiratory cases, Scientific Data (2022)

4. PriMock57: A Dataset of Primary Care Mock Consultations (Korfiatis et al., ACL 2022)

5. jiwer, word error rate in Python

6. Sully transcription docs

7. Sully streaming transcription docs

8. Sully Note Generation docs

9. Sully Text to JSON docs

10. Sully API page