Benchmark Tasks

EHRSHOT contains 15 clinical prediction tasks across 4 categories, designed for few-shot evaluation of EHR foundation models.

Download Dataset Code Paper

Long Length of Stay

Predict whether a patient's total length of stay during a visit will be at least 7 days. Prediction time is at 11:59pm on the day of admission.

Binary 3,855 patients 25.3% prevalence

30-day Readmission

Predict whether a patient will be re-admitted to the hospital within 30 days after discharge. Prediction time is at 11:59pm on the day of admission.

Binary 3,718 patients 13.0% prevalence

ICU Transfer

Predict whether a patient will be transferred to the ICU during a hospital visit. Prediction time is at 11:59pm on the day of admission.

Binary 3,617 patients 4.5% prevalence

Labels are normal, mild, moderate, or severe. Treated as binary (normal vs. abnormal) for benchmarking. Prediction time is immediately before the lab result is recorded.

Thrombocytopenia

Platelet count: normal (≥150), mild (100–150), moderate (50–100), or severe (<50) ×10⁹/L.

Multiclass 6,063 patients 33.2% prevalence

Hyperkalemia

Potassium: normal (≤5.5), mild (5.5–6), moderate (6–7), or severe (>7) mmol/L.

Multiclass 5,931 patients 2.4% prevalence

Hypoglycemia

Blood glucose: normal (≥3.9), mild (3.5–3.9), moderate (3–3.5), or severe (<3) mmol/L.

Multiclass 5,974 patients 1.5% prevalence

Hyponatremia

Sodium: normal (≥135), mild (130–135), moderate (125–130), or severe (<125) mmol/L.

Multiclass 5,921 patients 28.5% prevalence

Anemia

Hemoglobin: normal (≥120), mild (110–120), moderate (70–110), or severe (<70) g/L.

Multiclass 6,086 patients 69.0% prevalence

Prediction time is at 11:59pm on the day of discharge from an inpatient visit. Positive outcome = first diagnosis within 365 days post-discharge.

Acute MI

First diagnosis of acute myocardial infarction (SNOMED/57054005).

Binary 3,834 patients 6.8% prevalence

Hypertension

First diagnosis of essential hypertension (SNOMED/59621000).

Binary 2,328 patients 13.7% prevalence

Hyperlipidemia

First diagnosis of hyperlipidemia (SNOMED/55822004).

Binary 2,650 patients 12.7% prevalence

Pancreatic Cancer

First diagnosis of pancreatic cancer (SNOMED/372003004).

Binary 3,864 patients 3.8% prevalence

Celiac Disease

First diagnosis of celiac disease (SNOMED/396331005).

Binary 3,899 patients 1.3% prevalence

Lupus

First diagnosis of lupus (SNOMED/55464009).

Binary 3,864 patients 2.2% prevalence

Prediction time is 24 hours before the radiology report. Labels derived from CheXpert NLP labeler on radiology report text (text not released).

Chest X-Ray Findings

Identify which of 14 findings appear in a chest X-ray report: No Finding, Enlarged Cardiomediastinum, Cardiomegaly, Lung Lesion, Lung Opacity, Edema, Consolidation, Pneumonia, Atelectasis, Pneumothorax, Pleural Effusion, Pleural Other, Fracture, Support Devices.

Multilabel 1,045 patients 65.5% prevalence

Label Counts

Total patients and labels across all splits. Each patient can have multiple labels.

Task	Patients	Positive	Labels	Pos. Labels	Prevalence
Long LOS	3,855	1,271	6,995	1,767	25.3%
30-Day Readmission	3,718	474	7,003	911	13.0%
ICU Transfer	3,617	266	6,491	290	4.5%
Thrombocytopenia	6,063	2,566	179,618	59,718	33.2%
Hyperkalemia	5,931	1,289	200,170	4,769	2.4%
Hypoglycemia	5,974	1,379	318,164	4,721	1.5%
Hyponatremia	5,921	3,692	212,837	60,708	28.5%
Anemia	6,086	4,271	184,880	127,496	69.0%
Hypertension	2,328	386	3,764	516	13.7%
Hyperlipidemia	2,650	410	4,442	566	12.7%
Pancreatic Cancer	3,864	214	7,011	264	3.8%
Celiac	3,899	69	7,129	94	1.3%
Lupus	3,864	122	7,038	157	2.2%
Acute MI	3,834	357	6,837	464	6.8%
Chest X-Ray	1,045	996	26,275	17,203	65.5%

Numbers may differ slightly from the paper due to dataset preparation changes for public release.

For full details, see the EHRSHOT paper (NeurIPS 2023). Questions? Open an issue on GitHub.

EHRSHOT