The Randomness Audit
This is a member-only chapter. Log in with your Signal Over Noise membership email to continue.
Log in to readModule 5 · Section 5 of 9
The Randomness Audit
One of the more interesting sections of the skill file is a pre-analysis check for randomness contamination — based on Nassim Taleb’s work. Before evaluating any options, Claude is instructed to ask whether the evidence itself is trustworthy:
| Check | Question |
|---|---|
| Survivorship bias | Am I only seeing the winners? |
| Narrative fallacy | Am I constructing a tidy story to explain an outcome randomness explains equally well? |
| Attribution bias | Am I attributing this to skill when it could be luck? |
| Sample path overfitting | Is this strategy successful only because the specific sequence of events happened to be benign? |
That check runs before any model gets applied. The point is that even structured analysis produces bad results if the input data is noise masquerading as signal.