How AI Models Choose Doctors for Patients
Listen to the summary
Uses a voice available on your device
Audio options
On this page 4 sections
Key Takeaways
- Physician ratings have a massive impact, with an increase from 3.9 to 4.7 stars boosting recommendation probability by 31.4 percentage points.
- Models exhibit demographic biases, favoring female-signaled names by 2.5 percentage points.
- Hispanic, South-Asian, and Black-signaled names received a 1.3 to 2.9 percentage point advantage compared to White-signaled names.
- Models frequently fail to align their actual recommendation logic with their self-reported reasoning when audited.
Summary & Methodology Analysis
The researchers employed a randomized choice-based conjoint design to isolate the variables influencing AI-driven physician recommendations. By presenting models with synthetic profiles where attributes like fees, hospital affiliation, and demographics were independently randomized, they generated 40,068 scored responses across seven specific models: llama3.2:3b, qwen2.5:3b, phi3:mini, mistral:7b-instruct, gemma3:4b, llama3.1:8b, and gpt-4o-mini. The team estimated average marginal component effects (AMCEs), which quantify the causal influence of specific features on the model output, using linear probability models to map these signals to choice behavior.
Interactive System Flowchart
Cross-Examination & FAQs
A deeper dive clarifying mechanics, constraints, and baseline evaluations.
Q1. What is the primary goal of this research?
The study aims to uncover how large language models weigh factors like physician reputation and demographic identity when acting as AI infomediaries to recommend doctors.
Q2. How do patient ratings influence AI recommendations?
Ratings are highly influential. Moving a physician from a 3.9 to a 4.7 rating increases the probability of being recommended by 31.4 percentage points.
Q3. Does the AI demonstrate a bias toward specific names?
Yes. The models showed a 2.5 percentage point preference for female-signaled names and a 1.3 to 2.9 percentage point preference for Hispanic, South-Asian, and Black-signaled names over White-signaled names.
Q4. How did the researchers test for demographic parity?
They used two one-sided tests (TOST) with a smallest-effect-size-of-interest of plus or minus 1.5 percentage points.
Q5. Did the models accurately report why they chose a specific physician?
No. The researchers compared the models' revealed weights against their self-reported reasoning using a validated dictionary and an independent judge model to identify clear discrepancies.
Q6. What specific models were included in the audit?
The audit included llama3.2:3b, qwen2.5:3b, phi3:mini, mistral:7b-instruct, gemma3:4b, llama3.1:8b, and gpt-4o-mini.
Q7. How did the study handle the relationship between physician attributes?
The study used synthetic physician profiles, which is a limitation because it does not account for real-world correlations between attributes like appearance or specific practice environments.
Q8. How many total responses were collected during the audit?
The researchers collected 40,068 scored responses across three patient personas and nine prompt paraphrases.
Q9. Are the results applicable to any AI model?
No. The study notes that findings are model-specific and time-dependent snapshots, meaning they do not necessarily generalize across all architectures or deployment timelines.