Arkangel AI, OpenEvidence, ChatGPT, and Medisearch evaluated against medical standards
Peer-reviewed 2026 study compares four conversational agents across 128 Q/A pairs and 1024 expert evaluations.
Arkangel AI, OpenEvidence, ChatGPT, and Medisearch evaluated against medical standards
Original title: Arkangel AI, OpenEvidence, ChatGPT, Medisearch: Are They Objectively up to Medical Standards?
Authors: Arkangel AI and collaborators
Venue: Mayo Clinic Proceedings: Digital Health, 2026
Status: Peer-reviewed, published
Headline metrics: Overall satisfaction range across agents 71.1% to 93% (Kruskal-Wallis P<.001); reference authenticity reached 100% satisfaction for Arkangel AI-Deep, ChatGPT-Deep, and OpenEvidence; ChatGPT modes rated unsatisfactory in 75–97% of evaluator pairs; median response times 18 seconds to 12.8 minutes (Wilcoxon P<.05)
This peer-reviewed Mayo Clinic Proceedings: Digital Health study compared Arkangel AI, OpenEvidence, ChatGPT, and Medisearch across four clinical vignettes and four question types. The design produced 128 Q/A pairs and 1024 expert evaluations using an eight-criterion Likert scale.
Overall satisfaction ranged from 71.1% to 93% across agents (Kruskal-Wallis P<.001). Reference authenticity reached 100% satisfaction for Arkangel AI-Deep, ChatGPT-Deep, and OpenEvidence. ChatGPT modes were rated unsatisfactory in 75–97% of evaluator pairs, and median response times ranged from 18 seconds to 12.8 minutes (Wilcoxon P<.05).
Source
https://www.mcpdigitalhealth.org/article/S2949-7612(26)00022-2/fulltext