This methodological study describes Hippocrates, a governed AutoML methodology for building tabular screening models from electronic health records in Colombian and Latin American health systems. It splits development into 13 phases, from data ingestion and mandatory leakage gates to isotonic calibration, threshold selection, subgroup assessment and TRIPOD+AI documentation. A calibration slope gate of 0.85–1.15 must be passed before a model is eligible for deployment, 18 human pause-points stop automation at decisions such as target definition and deployment scope, and every decision goes into an append-only audit log. The methodology runs as reusable skill files executed by a large language model agent, and was tested on six synthetic edge-case datasets.
Across five real-world sessions covering CKD screening, COPD screening and workforce retention, it produced calibrated models with complete governance trails. In a retrospective evaluation across four institutions (more than 53,000 patients), the new CKD model consistently improved on a previously deployed model with near-chance discrimination (AUROC about 0.58) and a calibration slope of 0.04; across five random seeds it reached an AUROC of 0.7479 ± 0.0013, with inter-operator variability (about 0.029 AUROC) as the main source of variation. It also corrected miscalibration in COPD screening and caught temporal leakage in the workforce session. The work is a preprint not yet peer reviewed, and the authors call for prospective studies of reproducibility and clinical utility.
