Capability deep-dive
Conceptual, prototyped on synthetic data, not a production systemFinding the rare-disease patients hiding in plain sight.
They spend years undiagnosed while their data already shows the pattern. Here is how the capability works end to end, the kind of model behind it, and where it stops being true.
How it works, end to end
Start here. Data on the left feeds a model in the lakehouse; the model ranks who is likely undiagnosed; the field and marketing reach them the only lawful ways; and every confirmed diagnosis flows back to sharpen the next cycle. One boundary governs all of it: the field never sees a patient name.
My synthesis of a standard commercial data flow and public HIPAA / HHS-OIG / PhRMA constraints, not a description of any company's actual internal routing. Vendor systems are generalized.
The model
It learns the footprint a disease leaves before anyone names it.
Years before a diagnosis, a rare disease leaves a faint trail in de-identified claims, procedures and labs. The model learns that trail and ranks everyone by how likely they are to be an undiagnosed case. The metric people ask for first is AUROC (0.86 here). But the curve that actually sizes a screening program is this one: how many true cases you catch for the share of patients you screen.
Why does 0.86 flatter it? When the disease sits at roughly one in forty thousand, a model can rank well and still hand you a list that is almost entirely non-cases. The two numbers that decide anything are enrichment, how many more cases per person tested than picking at random, and ultimately confirmed-diagnosis yield from a real prospective run.
What that delivers, in numbers
A target list you can actually work, and the yield to expect.
Put real prevalence to it. On a 1-million-patient claims database, at a rare-disease rate of about one in forty thousand:
- ~25 true cases exist across the population, and only about 13 of them are findable at all.
- Screen the top 1% by score, 10,000 patients, and you expect ~7 true cases among them, about one in every 1,500 screened.
- Those roll up to the providers who treat them, so the field works a ranked HCP list, not a raw patient list. How many providers that is depends on how concentrated cases are, which you measure rather than assume.
- The confirmatory budget, genetic tests and chart reviews, decides how many of the 7 you confirm.
And the net matters more than the model: cast wider, over more of the population and richer data, and you find more of them. Yield scales with coverage.
Real published prevalence, illustrative model performance; the disease is left unnamed by design.
The honest part
Where it works, where it breaks, and how you would know.
- It succeeds when the footprint separates from the lookalikes. Where the data covers the right patients and their pre-diagnosis trail is distinct, a good model surfaces them. That is the roughly half of cases worth chasing.
- It fails when the trail is too sparse. About half of true cases look like everyone else at any threshold. Better modeling does not fix that, only better data does. And precision stays low by construction: at the top 1% the flagged list is still around 99.9% non-cases, so the confirmatory testing budget, not the model score, is the real constraint.
- The unlabeled pile is not the healthy pile. In an underdiagnosed disease, the people worth finding sit in records nobody has flagged yet. Treat every unlabeled record as a confirmed negative and you train the model to miss exactly the patients you built it for.
- Keep the giveaways out of the model. If it can see the confirmatory test, the specialist, or the disease-specific drug, it stops predicting undiagnosed and starts recognizing already diagnosed. A strong headline number can just mean it memorized who the system already found; the discipline is holding those signals back so the prediction is earned.
- Strong on a rich sample, humble on the real one. You train where cases are dense enough to learn from, then translate that back to a population where they are one in tens of thousands. Only reporting performance at true prevalence tells you which you actually have.
- You cannot verify it from this. Synthetic AUROC proves the reasoning, not real-world performance. The only number that would count is confirmed-diagnosis yield from a prospective run, which is exactly what the feedback loop in the diagram exists to measure.
- It has not run in production. Designed and prototyped on synthetic data. A real deployment meets data-sharing agreements, vendor coverage gaps, and a clinical workflow that has to absorb a flag.
What I would do with real data: back-test recall against known patients before trusting any prospective precision number, and instrument the activation path from day one rather than bolting attribution on later.