Automated medical scribe

Not to be confused with medical scribe, a human scribe.

Automated medical scribes (also called artificial intelligence scribes, AI scribes, digital scribes, virtual scribes, ambient AI scribes, AI documentation assistants, and digital/virtual/smart clinical assistants) are tools for transcribing medical speech, such as patient consultations and dictated medical notes. Many also produce summaries of consultations. Automated medical scribes based on Large Language Models (LLMs, commonly called "AI", short for "artificial intelligence") increased drastically in popularity in 2024. There are privacy and antitrust concerns. Accuracy concerns also exist, and intensify in situations in which tools try to go beyond transcribing and summarizing, and are asked to format information by its meaning, since LLMs do not deal well with meaning (see weak artificial intelligence). Medics using these scribes are generally expected to understand the ethical and legal considerations, and supervise the outputs.

The privacy protections of automated medical scribes vary widely. While it is possible to do all the transcription and summarizing locally, with no connection to the internet, most closed-source providers require that data be sent to their own servers over the internet, processed there, and the results sent back (as with digital voice assistants). Some say they use zero-knowledge encryption (meaning that the service provider can't access the data). Others explicitly say that they use patient data to train their AIs, or rent or resell it to third parties; the nature of privacy protections used in such situations is unclear, and they are likely not to be fully effective.

Most providers have not published any safety or utility data in academic journals, and are not responsive to requests from medical researchers studying their products.

Privacy

Some providers unclear about what happens to user data. Some may sell data to third parties. Some explicitly send user data to for-profit tech companies for secondary purposes, which may not be specified. Some require users to sign consents to such reuse of their data. Some ingest user data to train the software, promising to anonymize it; however, deanonymization may be possible (that is, it may become obvious who the patient is). It is intrinsically impossible to prevent an LLM from correlating its inputs; they work by finding similar patterns across very large data sets. Some information on the patient will be known from other sources (for instance, information that they were injured in an incident on a certain day might be available from the news media; information that they attended specific appointment locations at specific times is probably available to their cellphone provider/apps/data brokers; information about when they had a baby is probably implied by their online shopping records; and they might mention lifestyle changes to their doctor and on a forum or blog). The software may correlate such information with the "anonymized" clinical consultation record, and, asked about the named patient, provide information which they only told their doctor privately. Because a patient's record is all about the same patient, it is all unavoidably linked; in very many cases, medical histories are intrinsically identifiable. Depending on how common a condition and what other data is available, K-anonymity may be useless. Differential privacy could theoretically preserve privacy.

Data broker companies like Google, Amazon, and Microsoft have produced or bought up medical scribes, some of which use user data for secondary purposes, which has lead to antitrust concerns. Transfer of patient records for AI training has, in the past, prompted legal action.

Open-source programs typically do all the transcription locally, on the doctor's own computer. Open-source software is widely used in healthcare, with some national public healthcare bodies holding hack days.

Multifactor authentication for access to the data is expected practice.

Platforms

Scribes may operate on desktops, laptop, or mobile computers, under a variety of operating systems. These vary in their risks; for instance, mobiles can be lost. The underlying mobile or desktop operating systems are also part of the trusted computing base, and if they are not secure, the software relying on them cannot be secure either.

Confabulation, omissions, and other errors

Like other LLMs, medical-scribe LLMs are prone to confabulation, where they make up content based on statistically associations between their training data and the transcription audio. LLMs do not distinguish between trying to transcribe the audio and guessing what words will come next, but perform both processes mixed together. They are especially likely to take short silences or non-speech noises and invent some sort of speech to transcribe them as.

LLM medical scribes have been known to confabulate racist and otherwise prejudiced content; this is partly because the training datasets of many LLMs contain pseudoscientific texts about medical racism. They may misgender patients. A survey found that most doctors preferred, in principle, that scribes be trained on data reviewed by medical subject experts. Relevant, accurate training data increases the probability of an accurate transcription, but does not guarantee accuracy. Software trained on thousands of real clinical conversations generated transcripts with lower word error rates. Software trained on manually-transcribed training data did better than software trained with automatically transcribed training data (such as YouTube captions).

Autoscribes omit parts of the conversation classes as irrelevant. The may wrongly classify pertinent information as irrelevant and omit it. They may also confuse historic and current symptoms, or otherwise misclassify information. They may also simply wrongly transcribe the speech, writing something incorrect instead. If clinicians do not carefully check the recording, such mistakes could make their way into their medical records and cause patient harms.

Patient consent

Professional organizations generally require that scribes be used only with patient consent; some bodies may require written consent. Medics must also abide by local surveillance laws, which may criminalize recording private conversations without consent. Full information on how data is encrypted, transmitted, stored, and destroyed should be provided. In some jurisdictions, it is illegal to transmit the data to any country without equivalent privacy laws, or process or store the data there; vendors who cannot guarantee that their products won't illegally send data abroad cannot be legally used.

Some vendors collect data for reuse or resale. Medical professionals are generally considered to have a duty to review the terms and conditions of the user agreement and identify such data reuse. General practices are generally required to provide information on secondary uses to patients, allow them to opt out of secondary uses, and obtain consent for each specific secondary use. Data must only be used for agreed-upon purposes.

Technology and market

The medical scribe market is, as of 2024, highly competitive, with over 50 products on the market. Many of these products are just proprietary wrappers around the same LLM backends, including backends whose designers have warned they are not to be used for critical applications like medicine. Some vendors market scribes specialized to specific branches of medicine (though most target general practitioners, who make up about a third of doctors). Increasingly, vendors market their products as more than scribes, claiming that they are intelligent assistants and co-pilots to doctors. These broader uses raise more accuracy concerns. Extracting information from the conversation to autopopulate a form, for instance, may be problematic, with symptoms incorrectly auto-labelled as "absent" even if they were repeatedly discussed. Models failed to extract many indirect descriptions of symptoms, like a patient saying they could only sleep for four hours (instead of using the word "insomnia").

LLMs are not trained to produce facts, but things which look like facts. The use of templates and rules can make them more reliable at extracting semantic information, but "confabulations" or "hallucinations" (convincing but wrong output) are an intrinsic part of the technology.

Pricing

With the exception of fully open-source programs, which are free, medical scribe computer programs are rented rather than sold ("software as a service"). Monthly fees vary from mid-two figures to four figures, in US dollars. Some companies run on a freemium model, where a certain number of transcriptions per month are free.

Scribes that integrate into Electronic Health Records, removing the need for copy-pasting, typically cost more.

Fully open-source scribes provide the software for free. The user can install it on hardware of their choice, or pay to have it installed. Some open-source scribes can be installed on the local device (that is, the one recording the audio) or on a local server (for instance, one serving a single clinic). They can typically be set not to send any information externally, and can indeed be used with no internet connection.

See also

• Medical scribe
• Large Language Model
• AI hallucinations
• List of open-source health software