AI-built physician portraits from clinical searches identify 78.4% of physicians, but raw search text reaches 85.0%

Zenodo 2026 · 1 min read
Top-1 identification · portrait (TF-IDF)
78.4%
Matching two halves of each physician's history; chance 0.3%, raw search text 85.0%
Self-declared specialty identified
49.9%
169 of 339 physicians, versus 41.3% from raw text
Physicians in the cohort
339
34,241 searches in the Arkangel AI clinical search engine

Arkangel AI whitepaper: four LLM agents turned 34,241 searches by 339 physicians into portraits; raw search text still identified more.

This whitepaper explores whether the history a physician leaves in a clinical search engine can be turned into a portrait that describes that specific person rather than a physician type. Four mutually blind language-model agents read each history from the perspective of a clinician, an educator, an information-behavior researcher and a skeptic, producing portraits with a median of about 10,700 tokens. The portraits were tested on 339 physicians and 34,241 searches from Arkangel AI with three evaluations: identification, specialty and future query generation.

When each history was split in two, the portrait from one half matched the same physician's other half in 78.4% of cases (chance 0.3%), but simply concatenating the raw search text reached 85.0% and beat the portrait in every volume stratum. The portrait identified the self-declared specialty in 49.9% of physicians versus 41.3% from raw text, an advantage that came from generalists, and in query generation it ranked the correct physician higher for 157 physicians versus 96 (p < 0.0001), although the top-1 difference was not significant (p = 0.17). Agreement between portraits from different halves was limited (median ICC 0.496 across 22 dimensions), and a later code review found a train-test overlap affecting 3.7% of the holdout. All findings are exploratory; it is a whitepaper, not peer reviewed.

Results at a glance

The headline numbers from the paper, in plain language.

How the model performed

Higher is better

Data: Matching two halves of each physician's history; chance 0.3%, raw search text 85.0% · 169 of 339 physicians, versus 41.3% from raw text — Zenodo 2026

Show the numbers as a table
MetricValueWhat it means
Top-1 identification · portrait (TF-IDF)78.4%Matching two halves of each physician's history; chance 0.3%, raw search text 85.0%
Self-declared specialty identified49.9%169 of 339 physicians, versus 41.3% from raw text
Physicians in the cohort33934,241 searches in the Arkangel AI clinical search engine

How to cite

Katherine Monsalve, Natalia Castaño-Villegas, María Camila Villa, Laura Velásquez, Jose Zea. Physician Portraits Built with AI from Their Clinical Searches: An Exploratory Evaluation. Zenodo. 2026. https://doi.org/10.5281/zenodo.22926613

@article{monsalve2026physician,
  title     = {{Physician Portraits Built with AI from Their Clinical Searches: An Exploratory Evaluation}},
  author    = {Katherine Monsalve and Natalia Castaño-Villegas and María Camila Villa and Laura Velásquez and Jose Zea},
  journal   = {Zenodo},
  year      = {2026},
  doi       = {10.5281/zenodo.22926613},
  url       = {https://doi.org/10.5281/zenodo.22926613}
}

More research from Arkangel AI

All research →

For hospitals and insurers

Validate these results at your institution

We run the model on your own clinical records with your team, under your privacy and governance rules.

Start a research partnership
Read paperCite