Environment: Node.js 20, Claude Code (Opus). The allocation script is 184 lines with no dependencies.
A drill book has ten chapters. Every chapter draws its questions from the same reference word list — 3,400 words and 1,026 idioms for the grade I’ll use as the example here.
Do nothing about this and you get two problems, both of which surface late.
Repeats. Chapter 9 tests advocate and so did chapter 2. Nobody notices during writing, because chapter 2 is no longer in the context that wrote chapter 9.
Gaps. A book that claims to cover the grade’s vocabulary ends up covering maybe 60% of it, clustered around whatever the model reaches for first. The other 40% is not missing in any visible way. It is simply never asked about.
The obvious fix is to check afterwards: collect every word used across all ten chapters, find the duplicates, patch them. We did that first. This post is about why we stopped.
Why the after-the-fact check is the expensive one
A duplicate found at chapter 9 is not one edit.
The word has to be replaced. The replacement has to be a word that isn’t used anywhere else in the book — which means consulting the global usage set again. The question around it has to be rewritten, because a fill-in-the-blank sentence is built for its answer. The three distractors have to be reconsidered, because they were chosen to be plausible against that answer. Then the whole question goes back through every check it already passed: level, answer uniqueness, choice length, answer distribution.
And the check that found it can only run once all ten chapters exist. So it fires at the end, when the manuscript feels finished, which is exactly when nobody wants to hear it.
The check itself is also global by nature. "Is this word used elsewhere in the book?" cannot be answered from one file. Every chapter’s verification depends on every other chapter, which means no chapter is ever really done.
Deal the cards first
So the allocation happens once, before any chapter is written:
$ node scripts/allocate-vocab-pools.mjs --level=p1
Pool allocation (per chapter):
┌─────────┬───────┬───────┬────────────┬─────────┬─────────────────┬───────┬────────┬───────┐
│ chapter │ verbs │ nouns │ adjectives │ adverbs │ idiom_in_vocabs │ other │ idioms │ total │
├─────────┼───────┼───────┼────────────┼─────────┼─────────────────┼───────┼────────┼───────┤
│ 'ch01' │ 72 │ 138 │ 70 │ 8 │ 54 │ 1 │ 103 │ 446 │
│ 'ch02' │ 72 │ 138 │ 70 │ 8 │ 53 │ 1 │ 103 │ 445 │
│ 'ch03' │ 72 │ 138 │ 70 │ 8 │ 53 │ 1 │ 103 │ 445 │
│ 'ch04' │ 72 │ 138 │ 70 │ 8 │ 53 │ 1 │ 103 │ 445 │
│ 'ch05' │ 72 │ 137 │ 69 │ 8 │ 53 │ 1 │ 103 │ 443 │
│ 'ch06' │ 72 │ 137 │ 69 │ 8 │ 53 │ 0 │ 103 │ 442 │
│ 'ch07' │ 71 │ 137 │ 69 │ 8 │ 53 │ 0 │ 102 │ 440 │
│ 'ch08' │ 71 │ 137 │ 69 │ 8 │ 53 │ 0 │ 102 │ 440 │
│ 'ch09' │ 71 │ 137 │ 69 │ 8 │ 53 │ 0 │ 102 │ 440 │
│ 'ch10' │ 71 │ 137 │ 69 │ 8 │ 53 │ 0 │ 102 │ 440 │
└─────────┴───────┴───────┴────────────┴─────────┴─────────────────┴───────┴────────┴───────┘
Written: books/p1-drill/vocab-allocation.json(idiom_in_vocabs is for entries the reference list files under a phrase part of speech; other catches the handful that fit none of the buckets.)
Every word in the reference list lands in exactly one chapter’s pool. Cross-chapter repetition is not checked and found to be zero — it is structurally impossible. Coverage is not measured at 94% — every entry is somewhere, by construction.
The generator’s instruction changes accordingly. Not "choose from the grade’s word list" but "choose from books/p1-drill/vocab-allocation.json, chapter ch01". Same principle as handing over the allow-list by path, one level narrower.
Two details that turned out to matter
Group by part of speech before dealing
The naive version is: shuffle 3,400 words, cut into ten slices of 340. It fails on the thin categories.
This grade’s list has exactly 80 adverbs out of 3,400 entries. Ten random slices will hand one chapter fourteen adverbs and another chapter three, purely by chance — and a chapter with three adverbs cannot write an adverb-focused exercise. The same applies to any question type keyed to word class.
So the script buckets by part of speech first, shuffles each bucket, and deals round-robin:
for (const [pos, list] of Object.entries(byPos)) {
const shuffled = shuffle(list);
for (let i = 0; i < shuffled.length; i++) {
const chIdx = i % NUM_CHAPTERS; // deal one card at a time
chapterPools[chIdx][posToKey[pos]].push(shuffled[i]);
}
}Which is why every chapter in the table above gets 8 adverbs and either 137 or 138 nouns. The ±1 is just the remainder.
Eight adverbs per chapter is thin, and that is worth knowing before the plan is written rather than discovering it in chapter 6. The allocation is an honest inventory of what the grade’s list can actually support.
Fix the seed
// Per-level seed so the two books get different shuffles.
const SEED = level === 'p2' ? 20260419 : level === '2q' ? 20260420 : 20260426;
function makeRng(seed) {
let state = seed;
return () => {
state = (state * 1664525 + 1013904223) >>> 0; // numerical recipes LCG
return state / 0x100000000;
};
}Twelve lines instead of a dependency. The quality of the randomness is irrelevant here; what matters is that it is reproducible.
Without a fixed seed, re-running the allocation reshuffles everything, and every chapter written against the old pools is now failing a check against pools it never saw. The allocation would become a moving target that silently invalidates finished work. With the seed, re-running is a no-op, and the file can be regenerated from scratch at any time to confirm nothing has drifted.
Different seeds per grade, so that two books built from overlapping lists don’t end up with suspiciously parallel chapters.
What the pool file actually contains
{
"meta": {
"level": "p1",
"created_at": "2026-04-26T05:24:02.244Z",
"source_vocabs": "knowledge/p1/reference/vocabs.json (3400 語)",
"source_idioms": "knowledge/p1/reference/idioms.json (1026 句) + specs §4.4 whitelist",
"num_chapters": 10,
"seed": 20260426,
"policy": "品詞別に shuffle し round-robin で 10 章に配分。章 pool 外の語を正解・誤答に使うのは禁止。"
},
"pools": {
"ch01": {
"verbs": [
{ "english": "stagger", "pos": "動詞", "meaning": "よろめく，よろよろ歩く",
"source_id": "vocabs#1226" },
{ "english": "perplex", "pos": "動詞", "meaning": "を困らせる，混乱させる",
"source_id": "vocabs#1532" }
],
"idioms": [
{ "phrase": "rank and file", "meaning": "兵卒，一般人", "source_id": "idioms" }
]
}
}
}Each entry keeps its source_id — the exact row it came from in the reference list. When a check reports a violation, the trail back to the source is one lookup, not an argument.
The file is committed to the repository. The allocation is data, not a decision made at run time.
The payoff is in the check, not the allocation
Because the pools are disjoint, verification stops being global:
$ node scripts/checks/check-choices-in-list.mjs books/p1-drill/chapters/ch01.md
✓ PASS [p1] 章 ch01 pool (4099 items incl. inflections) vs 18 問 × 4 = 72 slots; pool 外 0 件One file in, one verdict out. No other chapter is consulted. Chapter 1 can be finished — genuinely finished, gate passed, moved on from — before chapter 2 starts. That is the property the whole per-chapter workflow depends on, and it comes from the allocation rather than from the checker.
The dedup check we used to run no longer exists. There is nothing for it to find.
The cost, stated plainly
The pools constrain what each chapter can be about.
Chapter 1 was dealt stagger, perplex, despise, and the idiom rank and file. Those words have to appear in chapter 1 or nowhere. You cannot pick a theme and then find words to fit it; the plan has to work with the hand it was dealt, and sometimes that means a chapter’s passages accommodate a set of words that share no natural topic.
That is a real editorial cost and it is worth being explicit about it. We took the trade because the alternative — deciding themes first, drawing words to match, and reconciling ten chapters afterwards — put the discovery of every conflict at the end of the project, where fixing it costs the most.
Move the constraint upstream and the check gets cheaper, smaller, and earlier. That generalises well beyond word lists.
Next: the two-context review gate — why the agent that writes a chapter is never the agent that grades it, and what changes when the reviewer is not allowed to edit.
