vizmaya desk · 4 Nov 2026

Can a computer replace the lottery that tests job training?

Researchers built a digital twin for 105,216 job trainees, using tax records. Then they checked the twins against 37 real trials.

Typical error: twins $2,073. Real trials $2,126.

Start with a number that should bother you.

Source: Chetty, Fogel, Katz, Noray, Porter and Reisinger, arXiv:2611.03938 (preprint, not peer reviewed), p. 23, Table 1.

A 1936 print of people waiting in a crowded employment office
Artwork: Jolán Gross-Bettelheim, "In the Employment Office" (1936), Wikimedia Commons, public domain.

The problem

1%

Fewer than 1 in 100 job training programs in the U.S. has ever faced a real trial.

There are about 5,000 programs. That is fewer than 50 tested with a lottery (computed: 1% of 5,000).

For the rest, we have no lottery-based answer.

So how do we judge the rest?

Source: paper p. 1 (cites Gable et al., 2023).

The old way: compare people who signed up with people who did not.

Studies since the 1980s did exactly that. They kept finding that the answers did not match what real trials showed.

Why? People who sign up tend to be more driven. They might earn more even without the program.

How do you make the two groups fair?

IllustrationMade-up dots to show the idea. Source: paper p. 1 to p. 2.

A coin flip makes the groups alike.

In a randomized trial, a lottery decides who gets the program. Chance spreads motivation evenly across both groups.

So any pay gap later comes from the program. The U.S. Education Department's evidence site reserves its "strong" rating for trials. But trials are costly and hard to run.

What if a computer could stand in for the lottery?

IllustrationMade-up dots to show the idea. Source: paper p. 1.

Old computer guesses missed by about $4,000 a year.

The authors reran two famous old recipes on the same 37 trials. Typical miss: $4,485 and $3,960 in yearly pay.

Even the trials wobble by $2,126, from the random luck of who was picked.

What changes if the guess gets far more to work with?

Source: paper p. 4, p. 24, Table 1 p. 49, Table 2 p. 50. Miss = error in yearly pay 2 to 5 years after sign-up.

Old studies had 15,992 people to compare with. This one has 327 million.

The new data are U.S. tax records from 1999 to 2025. They are linked to 105,216 people who went through job training programs.

Every W-2 form an employer files adds one more paycheck to the picture.

What does a twin look like?

Sign outside the Internal Revenue Service building in Washington, D.C.
Photo: G. Edward Johnson / Wikimedia Commons, CC BY 4.0.
A blank U.S. Form W-2 wage and tax statement
Form W-2 (2012), IRS / Wikimedia Commons, public domain.

Source: paper p. 1, p. 2, p. 8.

A twin is a guess about the road not taken.

For each trainee, a model reads five years of past paychecks, plus age and neighborhood. It predicts the pay that person would have had without the program.

The gap between real pay and the twin's pay is the program's effect.

Can you trust a guess like that?

IllustrationMade-up numbers to show the idea. Source: paper p. 11 to p. 14.

Then they checked the twin against a real trial.

Year Up is a one-year program for young adults. In the trial, trainees later earned $9,627 a year more than the control group. The twin said $10,116.

The twin model never saw the trial's control group.

Was that one lucky match?

Source: paper p. 4, p. 15, p. 17; Figure 1B p. 42 (dots read off the figure, about $0.5K).

37trials

The twins landed about where the trials did.

Twin and trial results line up, with a correlation of 0.93. Typical error: $2,073 for twins, $2,126 for trials.

Contested"As good as a trial" is the authors' own bar.

What made it work this time?

Source: paper p. 3, p. 18, p. 23; Table 1 rows 7 and 8, p. 49; Table 2, p. 50.

No single upgrade did it. All three together did.

The upgrades: a smarter model, a bigger and better-matched pool of people, and years of paycheck history.

Any one alone left the miss at $3,365 or more. Any two left it at $2,698 or more. All three: $1,717.

So what can twins show that one trial cannot?

Source: paper p. 24 to p. 27; Table 2, p. 50. Error from bias alone, noise removed.

The same program paid more than twice as much ten years later.

Year Up trainees who signed up in 2007 gained $5,198 a year, two years on. Those who signed up in 2017 gained $12,000. For 2022, the gain was $9,077.

That is 2.31 times (computed: 12,000 ÷ 5,198). The gap is too big to blame on noise or bias (p = 0.01).

Is that true in every city?

Source: paper p. 4, p. 5, p. 30; Figure 5A p. 46 (unlabeled dots read off the figure).

Two cities, one program, two stories.

In City A, the gain climbed from $3,781 to $13,247. In City B, it stayed flat, near $6,783 a year.

For a site one standard deviation above average, a pooled 2013-14 trial would miss by $7,050. The twin's miss: $2,073, or 3.4 times smaller.

What do the winning programs have in common?

Source: paper p. 30 to p. 31; Figure 5B p. 46 (dots read off the figure). The paper hides the city names.

Programs that place trainees into jobs in their field pay off most.

Where under 31% of trainees landed a job in the field they trained for, the gain was $3,040 a year. Where 80% or more did, $10,163.

Each extra point of placement went with $121 more a year. Worth asking any program: what share of graduates land jobs in their field?

ContestedA link, not proof of cause.

And does the gain last?

Source: paper p. 33; Figure 6 p. 47 (bins read off the figure). 271 site-and-year groups, 20 bins.

90%

The gain stays even after people change jobs.

Nine in ten Year Up trainees changed employer within five years. Those who switched still earned $5,762 a year more than their twins.

A wooden ladder standing against a wall
Photo: Nadinviki / Wikimedia Commons, CC BY-SA 4.0.

So where does all this break?

Source: paper p. 6, p. 34; Figure 7 p. 48 (dots read off the figure). The authors call it a career ladder.

Twins are a tool, not a replacement. Here is where they break.

  • They are not unbiased. The paper's own test rejects it (p < 0.001).
  • They miss the pay dip in the sign-up year.
  • Most of the measured bias sits in two kinds of trials, welfare-to-work and subsidized jobs. Leave those out and it is $0.
  • At the trials' own sample sizes, twin error is $2,257, above the trials' $2,126.
  • Built for adults with work histories. For a new program, the bias is assumed to match the 37 trials. Nobody can test that.
  • Raj Chetty leads Opportunity Capital, which invests in training programs using these methods.

What is still missing?

Source: paper p. 6, p. 17 to p. 18, p. 22 (footnote 25), p. 23, p. 28, title page footnote. The $2,257 figure is Table 1 row 6, p. 49.

Credit, caveats and sources

Read this first. This is a preprint from 2 Nov 2026, not yet peer reviewed. We found no replication or outside coverage as of 4 Nov 2026.

The data are confidential Census records, so outsiders cannot rerun them. The paper promises more programs and longer follow-up in later versions. This deck uses only what this version shows.

Research: Raj Chetty, Jamie Fogel, Lawrence F. Katz, Kadeem Noray, Sonya Porter and James Reisinger. Trial data from MDRC, Economic Mobility Corporation, Abt Associates, Public/Private Ventures, Year Up United, Per Scholas and Project QUEST.

Funders: Andrew Carnegie Foundation, Chan Zuckerberg Initiative, Gates Foundation, Overdeck Family Foundation, Samvid Philanthropies, Walton Family Foundation.

vizmaya desk · 4 Nov 2026 · More vizmaya stories

  1. Chetty et al., "Experimentally Validated Digital Twins," arXiv:2611.03938v1, 2 Nov 2026.
  2. LaLonde (1986), American Economic Review 76(4).
  3. Painting: J. Gross-Bettelheim, Wikimedia Commons, public domain.
  4. IRS sign: G. Edward Johnson, Wikimedia Commons, CC BY 4.0.
  5. Form W-2: IRS, Wikimedia Commons, public domain.
  6. Ladder: Nadinviki, Wikimedia Commons, CC BY-SA 4.0.
  7. Dots marked "read off the figure" are approximate readings of the paper's charts.