Pickup recommender: the 2026 backtest, scored in categories

If you came from the front page. The figure quoted there, +0.15 categories a week on 547 of 602 decisions, is the row "streaming, after" in section 0.2 below, the replay run on September 24, 2026, and four in five of the moves it named finished at or above zero (82.4%). This page is the working record of that measurement, kept whole: each revision is written above the one before it with its own figures, so the first table you meet (section 0, +0.13 on 531) is an earlier run, not a different claim. A number that moved is shown moving. The words are the builders' own, and name files and functions; the front page's sentence is its summary.

one real league, 2026. Generated by scripts/pickups_backtest.py. 14 teams x 44 decision points, tilt on, off, 5602s.

0. Revision, 2026-09-23: the weekly spread was widened by two

What changed. app.pickups.stream.SPREAD_SCALE is 2.0 since that morning. The league's measured spread is one team's total over a period; a category is decided by the difference between two of them, and the model had been using the first for the second (docs/projected_record.md section 0, docs/spread_revision.md). Every chance in the recommender comes from that one function, so every net in the tables below is a different number from the run before.

Declared when. The factor was the owner's, chosen on 2026-09-22's published evidence and fixed before this replay started. No hurdle moved with it, and this script still changes no constant.

At the owner's own hurdles (stream 0.20, season 0.20/0.10), before and after:

The streaming half got slightly worse and the season half slightly better, and neither is claimed as a finding. The bar names about the same number of moves; what changed is which move the search puts first, because the ranking reads the net and the net moved. On this replay the moves it now names delivered +0.13 a matchup against +0.16, at a win rate of 80.8% against 82.3%. Over roughly 530 moves this run does not separate that from noise, and it is published because it is what the run says rather than because it means anything. The claim fell with it -- +0.07 against +0.09 -- so the calibration ratio went the other way, 1.82 against 1.75: the tool is still delivering more than it promises on a stream, and promises less than it did.

What the sweep now picks is what it picked before, and what is shipped. Streaming: no setting qualifies, for the same reason as ever -- an empty day filled is recommended whenever it helps at all, so the no-move rate never clears 20% at any hurdle. Rest of season: 0.20 paid / 0.10 free, which is the pair already set. Nothing here asks the owner to move anything.

The baseline is identical, as it must be: 1,120 of the league's own swaps, +0.050 a week, -0.558 over thirty days, 84.2% at or above zero. It is real moves scored by the replay and reads no model at all, so it is the check that the scoring machinery did not move underneath the comparison.

What a manager sees on the page is in scripts/spread_census.py, which re-runs these same 602 decision points at both scales and counts rather than scores. Of the five moves the search ranks, 3.99 cleared the 0.20 bar per team-week before and 4.02 after -- the same list. The numbers beside them are a third smaller: mean net +0.287 to +0.221, median +0.172 to +0.107, ninetieth percentile +0.434 to +0.289. And the week itself compresses toward a coin: the spread of expected_wins falls from 1.16 to 0.91 and its average distance from 4.5 from 0.88 to 0.61.

6532s against 5138s, on a machine also running the census.

0.2. Revision, 2026-09-24: a replayed morning reads the NBA's own injury report

What changed. Revision 0.1 below came back byte-identical and the reason was the source, not the rule: a replayed morning read its injury status from player_status_snapshots, which the listener writes only for the season in progress, so every man in a replayed 2026 was counted fit and the rule declared the day before never fired. The engine now applies a declared source order (docs/replay_status.md): ESPN's snapshot on a morning the listener had already taken one by, and otherwise the NBA's own official report as of ten o'clock Eastern that morning, read point-in-time, with the league's five words mapped onto ESPN's. Out alone is ruled out, as before, and nothing else moved.

Declared when. The source order was written down before any of the three calibrations was re-run and was not tuned afterwards. No constant moved and no hurdle moved.

What it covers, before any number below is read. On the 44 decision mornings the league's report was there on 43, with a mean of 42.5 placed lines and 14.8 names that could not be placed on a player. 7.9% of rostered man-mornings carry a status at all and 3.3% carry Out, because the league names only players whose team plays that day and this database holds one snapshot a game date. Everything that moved, moved on one man in ten.

What cannot move, and it is the check. The delivered side of every number here is seated from the stored box scores -- Replay.resolved reads what each man actually posted on each day and knows nothing about a status -- so the baseline of the league's own 1,120 swaps and the categories any named move delivered are arithmetic about what happened. What the injury report can move is the recommender's own half: which move the search ranks first, and what it claims the move is worth.

At the owner's own hurdles (stream 0.20, season 0.20/0.10), before and after. Tilt on and tilt off are the same run here, so one table serves both:

The streaming half got better on moves named, categories delivered, win rate and no-move rate at once, which is the first unambiguous improvement any of the three records has shown. The bar names sixteen more moves of the same 602 decisions, they delivered +0.15 categories a matchup against +0.13, and the win rate rose from 80.8% to 82.4%. The direction is what a status ought to buy a one-week question: a man the league had Out is no longer seated, so he neither displaces a man who could play nor is proposed as a pickup, and the search spends its ranking on men who were going to be on the floor. The no-move rate fell with it, 11.8% to 9.1%, which is the same fact read the other way -- a roster with a hole in it has something worth doing.

The rest-of-season half named a third more moves and delivered a little less each. 325 against 289 at the same 80.3% win rate, +1.25 over thirty days against +1.32. Both halves of that are the return prior: an OUT man is now worth a fraction rather than nothing, so a swap that takes him on clears the bar where it used to be refused outright, and the moves it adds are by construction the marginal ones. The total delivered went up (325 x 1.25 = 406 against 289 x 1.32 = 381) and the mean went down, which is what widening a filter does.

The claim rose faster than the delivery, and the calibration ratio fell. Streaming 1.82 to 1.94 -- still promising less than it delivers -- and rest of season 0.81 to 0.74, which is the direction to watch: the season side already claimed more than it delivered, and counting an OUT man for his expected games claims more still. It is not tuned here and nothing was moved on it.

The baseline is identical, as it must be: 1,120 of the league's own swaps, +0.050 a week, -0.558 over thirty days, 84.2% at or above zero. The sweep's pick is identical too: no streaming setting qualifies, for the same reason as ever, and rest of season is 0.20 paid / 0.10 free, the pair already shipped. Nothing here asks the owner to move anything. Of the 188 lines the before-and-after pair each wrote -- before this section was added to the script, so neither carried it -- 138 are equal in place, and every difference is inside the four hurdle-grid tables and the two sentences that quote them.

Wall times. The before-and-after pair were run concurrently on one machine and took 6,840s each, a third longer than the 5,773s of the run before them, which had the machine to itself. The time in this document's own first line is a third run, made after this paragraph was written so that the document would be generated by the script rather than have the paragraph pasted into a document the next run overwrites. It is the same code and the same rows as the after arm.

0.1. Revision, 2026-09-24: a man who is out is counted for the games he is expected to play

What changed. app.pickups.state.playable_days gave a man ESPN had ruled out zero games for the rest of the season, because it waited for an expected_return_date ESPN's basketball API does not have a field for. Every rest-of-season caller now counts him for his expected games instead -- each remaining game day weighted by the chance he is back by it, from the box-score return prior of docs/stashes.md section 2a, times the ramp of its section 3 (docs/stash_mode.md). The week's own seating is untouched: a man out tonight is out tonight.

Declared when. The rule was written down before any of the three calibrations was re-run and was not tuned afterwards. No constant moved and no hurdle moved.

This record cannot see it, and that is a fact about the database. player_status_snapshots holds the season in progress only, so a replay of 2026 reads no injury status for anybody and every man in it is counted as fit. The whole written run came back line for line identical to the one before the change -- both tilt settings, all twelve hurdle pairs, the sweep's own pick, the baseline, the per-team table and the worst misses -- and the only difference between the two documents is the runtime in the sentence above (5,883s before, 5,773s after).

That is a guard rather than a calibration: it says the healthy path did not move, which is exactly what RosteredPlayer.season_games claims -- that it equals the whole days for everybody who is not ruled out. It does not test the rule. What does is section 7 of docs/stashes.md, re-scored in docs/stash_mode.md: the engine's mean error on 2026's ninety-one claimed stashes falls from +1.04 categories a stash to +0.05.

1. What is measured, and in what

Every number in this document is categories won, not composite production. A move is scored by replaying the matchup that actually happened with the swap in it. Day by day over the window, the roster the team really held is taken from daily_lineup_slots, the swap applied to it, and the ten starting slots filled by the recommender's own seating rule; the day's line is the seated men's real box scores. The same is done without the swap. Both lines are counted against the opponent's real period totals and subtracted. +1 means the swap flipped one category.

Both sides are re-solved -- the roster without the swap is seated by the same rule -- so none of a move's credit is really credit for setting a better lineup. How far the re-solve is from what the manager actually did is reported as drift in section 4 and enters no score.

The composite-scored run of 2026-09-18 read -3.2 composite a day at a 20% win rate. That was a measurement of composite, not of the recommender: the objective spends points to win the categories that are close, and composite charges it for every point it spends. The two scores disagree by construction, and only one of them is the league's scoring.

2. The headline

n is decisions where a move cleared the hurdle; the mean and the win rate are over those moves. The streaming report is judged on the matchup period the decision fell in, the rest-of-season report over the next 30 days. claimed is the mean net the recommender said the move was worth (app.pickups.judge), so delivered/claimed is its calibration.

Tilt on

Tilt off

Against the league's own moves

The baseline is scored by the same replay, run backwards: a real move is undone (the dropped man goes back in, the added man comes out) and the sign flipped, so a recommendation and a real claim are the same quantity. It is the same population docs/acquirable_value.md measured -- executed transactions with exactly one add and one drop -- in a different currency, so that note's composite figure of +0.57 a day is not comparable to the number above and is not quoted as if it were.

3. Calibration: claimed against delivered

A recommender that says +0.5 and delivers +0.1 is a different tool from one that says +0.5 and delivers +0.5, at the same win rate. The claim is rescaled to the window the delivery is measured over -- one matchup for a stream, 30 days for a season move -- because the raw net spans every week left in the year. The ratio is delivered over claimed. zero is the share of named moves that changed no category at all -- now a fact about the move rather than about the scorer, since nothing caps the replay.

4. The tuning rule, and what it chose

A setting qualifies when it still says no move on more than 20% of decisions -- a tool that always names a pickup makes its user worse, which is the league's own finding -- and when the categories it delivers beat the league's own moves (+0.05 a week). Among the qualifying settings, the best mean wins.

• Tilt on, streaming: no setting qualifies. The no-move rate never clears the bar, because a move that seats a man on a day a slot was going empty is recommended whenever it helps at all (Move.clears), whatever the hurdle; the hurdle only decides the rest. STREAM_HURDLE is left where the design note put it.
• Tilt on, rest of season: best qualifying hurdles 0.20 paid / 0.10 free (+1.25 categories over 30 days, 325 moves, 46.0% no-move).
• Tilt off, streaming: no setting qualifies. The no-move rate never clears the bar, because a move that seats a man on a day a slot was going empty is recommended whenever it helps at all (Move.clears), whatever the hurdle; the hurdle only decides the rest. STREAM_HURDLE is left where the design note put it.
• Tilt off, rest of season: best qualifying hurdles 0.20 paid / 0.10 free (+1.25 categories over 30 days, 325 moves, 46.0% no-move).

Applied 2026-09-18, Patrick's decision: STREAM_HURDLE 0.10 -> 0.20 (the least churn for no loss, with a finite add budget and finite FAAB) and SEASON_HURDLE_PAID 0.05 -> 0.10. That run's grid ran the free bar above the paid one, which the design rejects, and adds into an open place are rare enough that the cell measured the paid bar alone; SEASON_HURDLE_FREE follows the paid bar at half, 0.02 -> 0.05, unmeasured on its own. The grid has since been corrected to run the free bar below the paid one and extended to 0.20 paid, so the season rows above are not the same cells that decision was taken on.

Applied 2026-09-21, Patrick's decision: SEASON_HURDLE_PAID 0.10 -> 0.20 and SEASON_HURDLE_FREE 0.05 -> 0.10, the pair the corrected sweep measured at +1.21 categories over 30 days on 307 moves, saying "no move" 49% of the time. He was shown the other side of it -- the 0.10 bar names 450 moves at +1.05, which is more in total -- and chose fewer and better: adds are seven a matchup period and shared with streaming, and this backtest does not charge a season move for the stream it displaces. STREAM_HURDLE stays at 0.20, which the same run now shows as the best of the four settings and not only the most frugal.

This script changes no constant. The run covers 14 teams; the settings above are its recommendation, and the two hurdle constants are changed by hand with this table quoted beside them.

Tilt on and tilt off are the same run here. The minutes tilt reads the listener's minutes events (player_status_events), and this season holds none: the listener began the season after. Every number above is therefore the untilted recommender, and the tilt is unmeasured, not measured as worthless.

Drift: +0.02 categories a period. That is how much the re-solved lineup wins that the manager's own starts did not, averaged over the periods replayed. A number near zero says the seating model is close to what this league actually does, so the counterfactual both sides of every move rest on is not a fantasy roster.

5. One defect in the stored data, and one leak now closed

The 2026 schedule is reconstructed, and no longer because it is missing

app/pickups/state.py counts a player's remaining games from pro_team_games, which when this was written held no 2026 rows at all (only 2027 had any; the listener that writes it started in 2027). With no schedule every player has no game days, stream._Week.project seats nobody, and every move's change in expected wins is exactly 0.0. Workaround: the script reconstructs a schedule from the box scores -- a played line on scoring period N is a game on N, and the player's NBA team comes from the weekly roster row, the same fallback state.build_players uses. Verified: 30 NBA teams, 2,461 team-game slots against the 2,460 a full 82-game season needs.

The stored 2026 schedule has since been backfilled -- 2,468 rows, and it is what season_calendar reads to turn a scoring period into the calendar date the injury report is asked about (section 0.2). The substitution is kept anyway and is now a choice rather than a necessity: the reconstruction is keyed on the player and the stored table on his team, which are different facts about a man who was traded mid-season, and swapping them is a change to the games every number here rests on. It is not one to make inside a run that is scoring something else.

The matchup totals no longer leak the rest of the week

state.load_team_week used to read my_totals and opp_totals from matchup_team_stats, the period's final total, with no day column to cap it by, and this script substituted a capped sum of the started lines for the length of a run. state._posted takes the day itself now: it keeps ESPN's row only on a genuinely live morning -- one where the database holds no box score on or after today -- and otherwise sums the started lines on the days of this period before today, which over a whole period reproduces matchup_team_stats exactly. This run therefore measures the product's own totals, with nothing substituted.

It moved the streaming numbers, and almost nothing else. The cap this script used to install was inclusive of day N, so it counted day N's lines as already posted while the projection was adding that same day on top; 584 of the 616 team-decision points had a different posted total afterwards, and the whole run's posted PTS fell from 125,125 to 77,483. Against the 2026-09-18 run: the streaming mean fell from +0.17 categories a matchup at every hurdle to +0.14 to +0.16, the win rate from 82.6-83.1% to 81.4-82.3%, and the calibration ratio from about 2.0 to about 1.7; the number of moves named and the no-move rate barely moved (574 to 573 at the 0.05 bar, 538 to 536 at 0.20). The rest-of-season side is unchanged where the two grids overlap -- the 0.05 paid cell 524 moves at +0.98 against 525 at +0.98, the 0.10 paid cell 453 at +1.06 against 450 at +1.05 -- which is what it should be, since only that report's week half reads the posted totals. The baseline is identical to three decimals (1,120 moves, +0.050 a week, -0.558 over 30 days, 84.2%), which is the check that the scoring machinery itself did not move.

Verified no leak in the two places the design note asks about: app/scoring/knowable.py filters to scoring_period < day and the projection's minutes tilt does the same. app/pickups/bids.bid_fit reads the whole season's transactions, which would be a look-ahead if a bid were taken as advice; the replay never scores a bid.

6. Caveats

• The replay assumes a perfectly set lineup, on both sides. Every day is seated from scratch by the same rule with the swap and without it, so the difference is the move and not the lineup; but neither side is the lineup the manager actually set. The gap between the re-solve and his own starts is printed beside the run and enters no score.
• One morning snapshot a day of injury news. Since 2026-09-24 a replayed morning reads the NBA's own official report as of ten o'clock Eastern (docs/replay_status.md), so a man the league had Out is no longer counted fit. What that covers is 7.9% of rostered man-mornings, 3.3% of them Out: the league names only players whose team plays that day, and the database holds one snapshot a game date. A man whose team was idle reads as fit.
• 44 decision points a team, 14 teams. Small. One season, one league.
• A move is replayed against the roster the team really held, so a man the manager dropped for other reasons later in the window leaves the counterfactual roster one place larger than thirteen for the rest of it.
• A move's week score is zero on a bye, and on a decision day in a period with no recorded opponent, since there are no categories to win.
• The add budget never binds here. The replay takes two decision points a matchup period (its first and fourth days) and scores the moves found at each, so no team ever approaches the one-add-a-day-of-the-period budget the reports carry (ADDS_PER_PERIOD_DAY, docs/pickups.md section 4.3). Nothing here measures a plan's second move, or a day with no adds left; the hurdles were fitted on one move a decision. Before 2026-09-21 it could bind spuriously, because the adds a team had spent were counted over the period's whole span rather than through the decision day.
• Ten 2026 days carry no box scores at all, and daily_lineup_slots stops at day 160 while stats run to 174. Decision points falling on an empty day yield no pool and are counted separately as pool_empty.

7. Decisions taken in this script

• Moves are scored in categories, by replaying the real matchup with the swap in it, and composite is gone. The recommender optimises categories; scoring it in anything else measures the scorer.
• The lineup is re-solved day by day for the swapped roster, and for the unswapped one as well. Scoring a re-solved lineup against the manager's own starts would pay every move the difference between a well-set lineup and a badly-set one.
• The seating order is each man's form before that day, not his line on it. A manager sets the lineup in the morning; hindsight seating would flatter every pickup and would not be a lineup anybody could have set.
• The earlier rule -- cap the added man at the dropped man's started days -- is gone. It scored 60% of named streaming moves at exactly zero, because an empty-day pickup replaces a man who was not starting, so the measurement could not tune the streaming hurdle at all.
• The baseline is the league's own swaps replayed backwards and negated, rather than quoted from docs/acquirable_value.md. The note's +0.57 is composite a day and is not the same quantity as a category.
• A tie in a category counts half to each side, so a swap that changes nothing scores exactly zero rather than flipping on a rounding.
• A move is a win at >= 0 rather than > 0, since a swap that costs no categories costs nothing but the transaction.
• The season's played rows are loaded once into Replay -- four queries -- because the category replay asks the same questions of the same rows hundreds of times a team. SeasonBook was not reused: it aggregates started lines per matchup period, and the replay needs them per day.
• The schedule was reconstructed from box scores rather than declared a dead end, because without it the measurement is identically zero.
• The matchup totals are the product's own now, not a substitution: state._posted takes the day and sums the started lines before it, verified exact against ESPN at the full period. The cap this script used to install was inclusive of day N, which counted day N once as posted and again as projected.
• The hurdle grid is applied after one evaluation per decision point, not by re-running per cell. The hurdle is a reporting filter in both modules, so this is exact and not an approximation.
• The projection entry points are memoized per decision point. Verified identical move deltas to nine decimals, 11.6x faster; the cache is keyed on the day and rebuilt per decision point, so it cannot carry an answer across days.
• The one remaining workaround, the reconstructed schedule, is installed by attribute substitution and removed in a finally, so nothing on disk changes.
• The top 5 moves per decision are scored, not only the recommended one, so the top-ranked move's return is visible whether or not it cleared the bar.

8. A most-categories league: the sweep graded in matchups

Declared 2026-09-30 (Job AS), before the code and before any run; the whole declaration is docs/most_categories.md, "The bars, measured in matchups". The same replay, on the matchup recommender (Jobs AL and AP), with one change: each re-solved line is counted as the league counts a week, the matchup won (1), tied (1/2) or lost (0) against the opponent's real totals, and a move's grade is the result with it less the result without it. The week grade is the period the decision fell in; the season grade the periods of the next thirty days, summed. The league's own swaps are graded the same way.

• The grid is in matchups: 0.005 to 0.10 on the net for the stream bar; the FAAB bar the same values per week, the free bar at half of it.
• The tuning rule carries over unchanged (more than a fifth no-move, beat the league's own moves, best mean), with one addition on the stream side: when nothing qualifies there, the highest bar whose mean is within one standard error of the best, among the bars that beat the league's own.
• The minimums are derived from the grade's noise at the pick on the fitting seasons, not copied from section 4's 200.
• The split: fit 2020-2023, test 2024-2026, refit 2020-2026 for the default a most-categories league reads, labelled so.

Code: Replay(majority=True) grades by the matchup; the intake runs it through app.intake.measure.sweep_one_season(..., majority=True); the study is scripts/matchup_bars.py. Results: "Measured in matchups", below.

Measured in matchups, 2026-09-30

The run is docs/runs/2026-09-30-matchup-bars.txt; the reading is in docs/most_categories.md, "Measured: the bars in matchups". In short, on the owner's league read as a matchup league:

• Fitted on 2020-2023 (1,928 decision points): stream 0.10 by the least-churn addition, nothing qualifying (no-move 2.2% to 8.7% at every bar, as in categories); season 0.10 FAAB / 0.05 free, qualifying, +0.2393 matchups over thirty days (se 0.0260) on 861 moves, 53.5% no-move. Both are the grid's top; past it the kept moves show nothing better by more than its noise.
• Held out, 2024-2026: the season bar separates (+0.1576 matchups over thirty days between the top moves over and under it, se 0.0479); the stream bar does not (-0.0003, se 0.0202).
• Refitted on 2020-2026, the default: stream 0.10, season 0.10 / 0.05, 3,640 decision points; the league's own swaps +0.0193 a week and -0.0765 over thirty days (n 5,948).
• Minimums: 1,550 decision points for the stream bar, 400 for the season bars, from the grade's noise at the fitted picks.