A multilingual medical transcription API that advertises 50 languages has usually told you one thing: you can pick one of them per request. It hasn't told you what happens when a patient starts a sentence in Spanish, names her medication in English, and the clinician answers in both. Those are separate features, vendors document them very differently, and a mixed-language visit can post a low overall error rate while losing the words that matter.

This guide separates the features, shows what each vendor documents as of November 2026, and gives you a test to run on your own recordings. Sully wrote it, and it says when another vendor fits better.

Key Takeaways

• Three features get sold as one. Per-request selection, automatic detection and code switching are different things. Only the third handles a language change inside the audio, and not every vendor means inside a sentence.
  
  

• The medical tier is usually narrower. Deepgram's medical model switches across 10 languages against 55 for its general model. AssemblyAI's Medical Mode covers four languages. Speechmatics' healthcare switching model has no custom dictionary yet.
  
  

• Overall word error rate hides switching failures. In one paper, fine-tuning improved WER on Spanish-English audio by about 5% while errors on the switched words got about 39% worse.
  
  

• Speech systems can translate when you didn't ask. One clinical evaluation recorded English "wrist" coming back as the Mandarin word for elbow. Translation also has its own review rule, so keep it apart from transcription.
  
  

• Don't send every visit through switching mode. Fix the language when a visit stays in one, and switch only when it doesn't.

Multilingual Medical Transcription API Features, and Who Documents What

Per-request selection

You send a language code and the model expects that language. When a visit stays in one language, this is the cleanest option, because there is no detector to get it wrong. The failure is the default: Sully's language defaults to en, so an integration that forgets the parameter sends a Spanish visit as English. Wire the value to the patient's preferred-language field and log it.

Automatic detection

The system works out the language, then transcribes. Often that means one answer per file. Speechmatics' auto option detects "the predominant language" and asks for at least 60 seconds of speech in it. Run a visit that is 70% English and 30% Spanish through a detector like that and it can correctly answer "English". That answer does nothing for the Spanish.

Code switching

Here the language may change within the audio. Then comes a distinction most marketing pages skip: between turns or within a sentence.

Gladia defines code switching as a language change "between turns" and runs detection on each utterance. AssemblyAI says its model follows speakers "as they shift mid-sentence". Neither is a test result. But a clinician speaking English and a patient answering in Spanish is an easier job than "me duele aquí, on the left side, desde el martes", and you should know which one a vendor is claiming.

What each vendor documents

This table reports documentation read on 3 November 2026. None of it was benchmarked for this article, and Sully is one of the vendors in it.

Three things in that table should change how you evaluate.

The medical tier and the switching tier don't fully overlap. A Vietnamese-English visit can get Deepgram's general multilingual model but not its medical one. For some language pairs you are choosing between clinical vocabulary and switching.

Fallbacks are quiet. AssemblyAI returns "Skipped medical-v1 domain correction because the language is not supported" as a warning alongside a normal transcript. A pipeline that doesn't read warnings believes it ran Medical Mode. Log what ran.

Switching has costs. Gladia says it can reduce partial transcript accuracy. Speechmatics recommends its single-language Enhanced Medical model for single-language audio that needs full features. Deepgram's multilingual rate is higher than its monolingual rate. So route by visit.

What Published Research Says About Code-Switching Accuracy

Evidence is thin. A search for this article found no peer-reviewed head-to-head of commercial APIs on clinical code-switched audio.

The closest clinical study is a preprint. A UC Irvine team evaluated Abridge, a scribe product rather than an API, on 24 reenacted primary care conversations. It hasn't been peer reviewed. Median mixed error rate was about 4% for Spanish-English and about 9% for Mandarin-English, with a worst case of 67%. Switches were generally detected reliably, though deletions were frequent at switch points in Mandarin-English. The errors underneath those numbers are the useful part:

• losartan came back as "los zartan", with "los" read as the Spanish article.
  
  

• cystatin C came back as "cistatin sí", with the letter C read as the Spanish word for yes.
  
  

• English "creatinine" came back as "creatinina", a translation nobody asked for.
  
  

• English "wrist" came back as the Mandarin word for elbow.

The authors call reenactments relatively ideal conditions, so real clinic audio may do worse.

WER can improve while switching gets worse. Researchers at Karlsruhe proposed Point-of-Interest Error Rate, which counts errors only at the embedded-language words. On Spanish-English audio, fine-tuning whisper-large on monolingual data improved WER by 5.06% and worsened PIER by 39.47%, from 36.41 to 50.78.

The arithmetic explains why. Take an illustrative visit of 1,500 tokens with 60 English terms inside Spanish speech. Get all 60 wrong and everything else right:

• WER: 60 / 1,500 = 4%
  
  

• PIER: 60 / 60 = 100%

A vendor could report the first number honestly while every embedded term in the visit is wrong.

Embedded medical English is a known weak spot. A 2022 nursing record study tested Mandarin-English speech full of drug names and abbreviations. The Azure cloud API scored 59.20% WER overall and 95.57% on the English portion. That says nothing about Azure today, but the pattern is the point.

The largest vendor comparison isn't clinical. ServiceNow's benchmark ran seven systems on 918 synthetic HR and IT records. ElevenLabs Scribe V2 and AssemblyAI Universal-3 Pro led on WER. Whisper Large V3 Turbo came last, and the authors report that without a language parameter it defaults to translating code-switched audio into English.

Transcription vs Translation, and the Rule That Cares

Transcription writes down what was said, in the language it was said in. Translation changes the language. The preprint above shows why that line needs checking: a translated word can be right or wrong, and either way the source words are gone from the transcript. Store the verbatim transcript and any translation as separate fields, each with the model that produced it and who reviewed it.

There is also a rule. Under 45 CFR 92.201(c)(3), if a covered entity uses machine translation when the text is critical to the rights, benefits or meaningful access of a person with limited English proficiency, when accuracy is essential, or when the source has complex, non-literal or technical language, "the translation must be reviewed by a qualified human translator". Section 92.4 defines machine translation as automated, text-based translation without such review, "sometimes with an option for audio input or output". The 2024 final rule's preamble also recommends warning patients about possible errors when review isn't required. Separately, a covered entity must offer a qualified interpreter when interpretation is required, and words on a screen aren't that.

The duty sits with the covered entity, and your customers will ask how your product supports it. Two cases to plan for:

• A Spanish after-visit summary generated from an English note is the obvious case for human review. One study of 76 computer-generated Spanish medicine labels found a 50% error rate, including "once a day" rendered as eleven times a day.
  
  

• An English note drafted from a Spanish transcript is a question for counsel, not something an API doc can settle.

This is not legal advice. The rule text was read at eCFR on 3 November 2026, and counsel should confirm how it applies to you.

Interpreter-Led Visits, Drug Names and Regional Variants

Interpreter-led visits

An interpreted visit says everything twice, and the two versions can disagree. Flores and colleagues taped 13 pediatric encounters and counted 396 interpreter errors, a mean of 31 per encounter. 63% had potential clinical consequences: 77% of errors by ad hoc interpreters and 53% by professional ones. It is small and old, but if you keep only the English, you've recorded the interpreter's account of what the patient said.

So transcribe both sides, which creates its own problems:

• A note generator sees each symptom twice and may count it twice.
  
  

• Speaker labels don't map cleanly to roles when an interpreter speaks in the first person for both sides.
  
  

• The audio can't tell you who is interpreting. The rule treats a qualified interpreter, an accompanying adult and a minor child differently, so capture that as visit metadata.
  
  

• Phone interpreters often arrive as narrowband audio. Test at the real sample rate.
  
  

• Relay interpretation runs through an intermediate language, often for select indigenous languages. Part of that audio may be in a language your model doesn't list, so find out what it does then.

Drug names inside non-English speech

The preprint notes that medication brand names often stay in English when the surrounding conversation doesn't. Add local forms like metformina and you have two spellings and two pronunciations for one drug in one visit.

Check whether vocabulary tools survive switching mode. Deepgram's keyterm prompting works with Nova-3 multilingual. Gladia recommends custom vocabulary per language. Speechmatics' Oak 1 doesn't take a custom dictionary yet. Sully's word_boost is a streaming option.

Then there's the case for language labels. In Spanish, once means eleven. A transcript containing "take it once" is ambiguous unless something records which language that word was in, and a note generator reading plain text has to guess. Test label accuracy on the words where it matters.

Regional variants

A locale code is a routing label, not evidence of accuracy. Gladia says so directly: supported languages "describe coverage, not guaranteed accuracy on your production audio".

Deepgram's table lists zh-Hant as Mandarin and zh-HK as Cantonese, both in traditional script, so "Chinese" is at least two spoken languages and two scripts. Its docs also say a regional hint with no matching tag collapses to the base language, en-GB to en. Sully's list has es and es-419, pt-BR and pt-PT, fr and fr-CA, and no es-MX or es-US. No list tells you how a model handles your clinic's Puerto Rican, Mexican or Dominican patients. Recordings of those patients do.

A Test Plan to Run on Your Own Recordings

Before any recording leaves your system, have a BAA in place with each vendor under test. If you can't yet, reenact the visits, as the UC Irvine team did.

1. Build six buckets from your real population. Single language, for each language you serve. Clinician in English and patient in another language, switching between turns. Switching inside sentences. Professional interpreter in the room. Phone or video interpreter. Family member interpreting.

2. Write reference transcripts with bilingual reviewers. Keep them verbatim, tag every token's language, and mark switch points. Flag critical tokens (drugs, doses, numbers, labs, body parts, negation) and have a second reviewer check them.

3. Run several configurations per vendor. Fixed primary language, fixed secondary language, detection, switching without hints, and switching with hints and vocabulary. Run recorded and streaming, and score partials separately from finals. Log the model name, every warning and each returned language field.

4. Score these separately, per bucket.

5. Never blend critical-term recall across language contexts. An illustrative example, not data:

• 60 drug mentions in English sentences, 58 correct: 96.7%
  
  

• 60 drug mentions in Spanish sentences, 47 correct: 78.3%
  
  

• Blended: 105 / 120 = 87.5%

The blended figure describes no patient you have.

6. Collect enough mentions to tell vendors apart. At around 90% recall, the 95% margin is 1.96 x √(0.9 x 0.1 / n). That is about ±8.3 points at 50 mentions and ±4.2 at 200. Fifty mentions can't separate 85% from 92%.

7. Set pass and fail thresholds before you see results, decide which errors route to human review, and rerun when a vendor changes its model. For general accuracy, see why drug names break word error rate and the medical transcription API guide.

Where Sully Fits

• Languages: 53 language values across 33 base languages, which fits the API page's "30+". Arabic, Tagalog, Persian, Hebrew and Haitian Creole aren't on the list. Deepgram's Nova-3 multilingual list includes the first four.
  
  

• One value per request: language is singular on uploads and defaults to en. Multichannel uploads have one language field too.
  
  

• Streaming: language is a connection parameter. Reconnecting creates a new session, and billable time starts when the server sends connected.
  
  

• Word labels: streaming word objects can include a language field.

So Sully fits visits that stay in one language you know in advance. Set the value per visit and test it like any other vendor.

If mid-sentence switching is a hard requirement today, shortlist a vendor that documents it. Deepgram's nova-3-medical and Speechmatics' Oak 1 are aimed at clinical audio, and AssemblyAI's mid-sentence claim is the most explicit. You can still mix parts: Note Generation v3 takes a transcript string plus your template. Buying parts versus one API has its own tradeoffs.

Don't mix this up with the separate product in Sully's help center: a medical interpreter feature with real-time translation, whose list includes Arabic and Filipino. Neither its list nor its translation applies to these transcription APIs.

FAQ

Can a speech-to-text API handle two languages in one conversation?

Some can, through a code-switching mode. Deepgram, AssemblyAI, Speechmatics and Gladia document one. A language list alone doesn't tell you, so look for the setting and the field that labels each word or turn.

Is automatic language detection the same as code switching?

No. Detection often picks one language for a whole file. Code switching lets the language change during the audio. A detector can be right about a visit and still do nothing for the minority language in it.

Does a multilingual transcription API translate speech into English?

It shouldn't unless you ask, but a clinical preprint recorded words and whole sentences translated during transcription. Test for it, because overall error rates don't separate it from other mistakes.

Do machine-translated patient instructions need human review?

Under 45 CFR 92.201(c)(3), when a covered entity uses machine translation for critical text, where accuracy is essential, or for complex or technical material, a qualified human translator must review it. Ask counsel how that applies to your product. This isn't legal advice.

To try it on a single-language visit, get an API key from the Sully API page and start with the transcription docs.

Sources

1. Deepgram, Multilingual Code Switching

2. Deepgram, Models and Languages Overview

3. Deepgram, Keyterm Prompting

4. AssemblyAI, Code Switching (pre-recorded)

5. AssemblyAI, Multilingual Transcription (streaming)

6. AssemblyAI, Medical Mode

7. Speechmatics, Models

8. Speechmatics, Languages

9. Speechmatics, Language Identification

10. Gladia, Code switching in multilingual audio

11. ElevenLabs, Speech to Text API reference

12. 45 CFR 92.201, Meaningful access for individuals with limited English proficiency

13. 45 CFR 92.4, Definitions

14. HHS, Nondiscrimination in Health Programs and Activities, final rule, May 6, 2024

15. Hu et al., Ambient AI Documentation in Mixed-Language Encounters, medRxiv preprint, 2026

16. Ugan et al., PIER: A Novel Metric for Evaluating What Matters in Code-Switching, arXiv, 2025

17. Code-Switching Automatic Speech Recognition for Nursing Record Documentation, JMIR Nursing, 2022

18. ServiceNow AI, Can Voice Agents Handle Bilingual Customers?, 2026

19. Flores et al., Errors in medical interpretation and their potential clinical consequences in pediatric encounters, Pediatrics, 2003

20. Sharif and Tse, Accuracy of Computer-Generated, Spanish-Language Medicine Labels, Pediatrics, 2010

21. Sully, Transcription docs

22. Sully, Streaming transcription docs

23. Sully, Note Generation v3 docs

24. Sully, API page

25. Sully, What languages does Sully.ai support (help center)

26. Sully, Word Error Rate in Medical Transcription

27. Sully, Deepgram Alternatives for Medical Transcription

28. Sully, Medical Transcription API guide

29. Sully, Speaker Diarization API guide

30. Sully, Speech-to-Text API for Medical Ambient Scribes