All 23 published skills carry the label not yet measured. The harness below is real, it holds 140 graded runs, and it has already deleted 6 methods that failed it.
All 23 skills published here, across 5 categories, have been written, checked for accuracy and shipped. Not one of them has been run against a control. There is no number on any card, no p value on any page, and no claim anywhere on this site about what these files do to an output, because we have not shown one. Every card says so in the grid, every page says so in a grey box above everything else, and each page names the test it is waiting for. The library is filed under LinkedIn, paid ads and growth and launch, and the label is the same on all three.
That is the entire state of the evidence, and it is the first thing on this page for a reason.
Unmeasured is not a quieter way of saying tested, and it is not a way of saying failed either. It means the experiment has not been run, so we are not going to tell you the result.
Because the harness is real, it has been run, and it has deleted skills for failing it.
It has judged 10 methods and failed 6 of them, and every one of those failures still has its row in the table below. The run file behind this page holds 140 runs across 14 cases, every output graded blind.
That history is the only reason a label is worth anything. A directory that says unmeasured and means it is worth more than one that says tested and does not, and from the outside those two look identical unless somebody writes the method down and publishes the failures alongside it. So the rest of this page is the method in full, then the record of what the method did when we pointed it at the library we used to have.
This is the part we would want to read on somebody else's directory, so here it is whole.
One skill costs about twenty model sessions. That is 5 runs with it, 5 runs without it on the identical task, and every output graded on its own afterwards. Runs execute on the opus tier and grading on the sonnet tier. That price is the reason nothing here is measured yet, and it is a reason rather than an excuse.
Both arms run with every customisation disabled. No user instruction files, no plugins, no installed skills, no hooks, no MCP servers. The only difference between the two arms is the skill body.
We did not do that at first, and it cost a whole round of results. A run started from a normal working machine silently inherits that machine's standing instruction file. Ours contained a rule about verifying that a database field really exists before writing code against it, which is precisely the thesis of one of the skills being tested. It was sitting inside the control arm of its own experiment. You can check this on your own setup in about ten seconds: ask a question only your instruction file can answer, then ask it again with customisations disabled and watch the answer disappear.
The second contamination was worse, because it looks like it cannot happen. Install the skill the normal way, in the project folder, and disable auto-loading, and the control arm finds the file by listing the directory and follows it anyway. We only caught it by planting a nonsense constant in a test skill and watching a control run cite it by name. Skills are now injected directly into the arm that is supposed to have them, and the file never exists anywhere the other arm can read.
Grading is blind. Each output is scored on its own, by a fresh session that is not told another arm exists, against a fixed rubric written by an author who was forbidden from reading any skill file. Every criterion is binary and needs a quoted piece of evidence. Roughly a third of every rubric is negative criteria, the form does NOT report a problem on this page that is genuinely fine, because without those a skill wins simply by being noisier. Before grading, the outputs are scrubbed of anything identifying which arm produced them.
The decision rule was written down before any run happened. A skill ships if it moves the median by 2 or more, or the worst case by 3 or more, at p below 0.10. That file was committed before the first case existed. Moving the line after seeing the number is the one thing that would make every figure on this page worthless.
The p value is an exact permutation test, not an approximation. With 5 runs an arm there are only 252 ways to split the ten scores between the two labels, so the whole distribution is computable and there is nothing left to estimate.
The material is real and the prompts are vague on purpose. Cases run against open-source repositories at real size, pinned to a commit. Where a case needs ground truth we derive it mechanically from the repository itself, the other places a pattern genuinely occurs, the identifiers the schema genuinely does and does not define. More than half the prompts are the vague thing a person actually types, have a look before I ship this, rather than a well-formed question with the answer implied in it.
The first harness was one run with the skill, one run without, against fixtures with defects we had planted ourselves. Five of the six methods tied their control and were cut.
That test was wrong in a way worth explaining, because the same mistake is in almost every prompt comparison published anywhere. One run per arm can only see the ceiling. A skill that makes a good run slightly better and a bad run dramatically better scores identically to a skill that does nothing, because a single sample has no floor to measure. And planted defects flatter the control: a bug placed to be found sits in a small file, in a directory built for the test, described the way the scoring criteria are described. Finding it is a much easier job than noticing something wrong in code nobody wrote as an exercise.
So the harness was rebuilt to the method above and the five were run again. Two of them came back.
The table is unedited and includes the failures, which is the point of keeping it. No row in it links anywhere. A method that failed was never published, and every method that passed belonged to the previous library, which was withdrawn in full for a reason that had nothing to do with its numbers. That is the section after this one.
| Skill | Worst without | Median without | Worst with | Median with | p | Verdict |
|---|---|---|---|---|---|---|
| Blast radius review (withdrawn) | 9 of 21 | 11 | 13 | 14 | 0.0079 | Revived |
| Contract drift review | 11 of 18 | 14 | 11 | 14 | 0.5635 | Still cut |
| Design system conformance | 15 of 21 | 17 | 17 | 18 | 0.2778 | Still cut |
| SERP-first content brief | 16 of 19 | 17 | 18 | 18 | 0.0317 | Still cut |
| Pre-publish page audit (withdrawn) | 7 of 22 | 9 | 10 | 12 | 0.0238 | Revived |
| Email deliverability audit (withdrawn) | 13 of 22 | 16 | 18 | 19 | 0.0079 | Passed |
| Form design audit (withdrawn) | 18 of 24 | 19 | 21 | 22 | 0.0079 | Passed |
| Cold email audit | 19 of 22 | 20 | 19 | 21 | 0.5714 | Cut |
| ATS-safe resume build | 12 of 22 | 15 | 15 | 16 | 0.1349 | Cut |
| Spreadsheet handoff build | 15 of 22 | 17 | 15 | 17 | 0.754 | Cut |
Worst and median are out of that case's rubric total, across 5 runs per arm.
This is the strongest thing on the page and it is the reason the rule gets written down first.
A content brief method moved its median from 17 to 18 out of 19, and its worst case from 16 to 18, at p=0.0317. That is a real effect and it is not plausibly noise. It is also a median moved by one point and a worst case moved by two, and the rule fixed before the first run asked for two and three. It was cut. Every argument for keeping it was available and none of them was allowed to matter, because a threshold you are willing to move once is not a threshold, it is a preference with arithmetic attached.
The contract drift method checks that every identifier a change names actually exists. It failed its first test against a control that scored suspiciously well, and we later found out why: that was the round the standing instruction file had leaked into. The finding was invalid. The answer was not. Re-run against a genuinely clean control it scores 14 of 18 without and 14 with, p=0.5635. The number was untrustworthy and the conclusion was right anyway.
Line the cases up by how well the control did and the pattern is not subtle. Where the assistant working alone already scored near the top of the rubric, no skill moved anything, and one made things slightly worse. Where the control had real room to fail, on a vague prompt against a messy codebase, the surviving methods moved the floor a long way.
Put every round together and the rule turns out not to be about vagueness at all. A method works when it carries a fact the model does not have, and does nothing when it carries a discipline the model already has. The methods that failed asked for rigour, and rigour is something a capable assistant already brings. The ones that passed carried a 1024-bit key length, a testing flag, a token vocabulary and a coercion that eats a leading zero. There is no amount of trying harder that produces those.
One of the passes also came with a cost nobody predicted. A form audit found four whole classes of defect the control never found, and was measurably worse at saying which single defect was the release blocker, three of five runs without it against one of five with it. Breadth bought at the price of priority. Nobody would have guessed the sign of that in advance, which is the entire argument for measuring anything.
In August 2026 the whole thing was withdrawn. All 133 published skills, sources archived, this directory rebuilt from an empty folder.
No number failed. Nothing in that withdrawal was cut, and cut stays a word reserved for a method the harness ran against a control and found wanting. 6 methods have ever earned it and they are all in the table above. The rest went on judgement, which is a different claim resting on a different kind of evidence, and letting one word cover both would drain the meaning out of every figure in that table. Evidence cut some skills. Taste withdrew the rest. We did not run the rest, so we are not entitled to the other word for them.
Here is the judgement. The library had filled up with audits: files that read something you had already made and handed back a list of findings. Several were good. Almost none was a thing a person would go looking for, because an audit is an errand you run once rather than a job you do again on Tuesday. A directory is judged on whatever a first-time reader happens to click, and a library where most of the entries are things nobody seeks out is worse than a small one where none of them are, because the filler is doing the introducing.
Nobody decided to build a review shop. What happened is more ordinary and harder to see. An audit emits a findings list. A findings list can be scored against mechanical ground truth: the identifier is in the schema or it is not, the pattern occurs at eleven call sites or it does not. So an audit is cheap to measure, which means an audit can carry a number, which means an audit clears the bar this page exists to enforce. A skill that produces an artefact emits something whose quality is partly a matter of judgement, so it is expensive to measure and often cannot carry a number at all.
We built a harness, we made it the gate, and the gate started choosing the work. Forty-three of the first fifty-four skills were audits. Eighty per cent. That is the streetlight effect with a directory attached to it. We were not writing what was most useful, we were writing what our own instrument could see, and nobody noticed for two rounds.
The rebuilt library is 0 per cent audits. 19 of the 23 produce an artefact that did not exist before and 4 hands back a rewritten version of one you bring, which is 100 per cent on the reading that counts an edit as production. Those figures are computed from a classification of every published file rather than typed in, because the last hand-maintained percentage on this page had already drifted by the time anyone checked it.
The cost of that correction is the honest half of it. Almost none of this work can be measured the way an audit can, and none of it has been. We traded measurability for usefulness, on the grounds that a skill nobody wants is not improved by having a p value attached to it.
What did not change is the bar on knowledge. A skill still has to carry a procedure, an ordering, a set of thresholds or a catalogue of named failures that a capable model does not already have. "Write a great landing page" is not a skill and was not accepted as one. What earns a place is the kind of thing that turns out to be true and is not guessable: that the clause after "but" in a damaging admission is the one the reader believes, so the order is load-bearing; that a translated page whose locale segment sits after the section name returns a 404 in every language; that charging by a unit which does not move when the customer succeeds caps you at your worst customer's willingness to pay.
There is no date here, because a date would be the one unearned number on the page.
The order is set by which skills can carry an objective spine. The ones built on a published specification or a documented threshold go first, because the pattern above says those are the ones with a real chance, and because a rubric can score them against something checkable rather than against an opinion. Each unmeasured page already names the test it is waiting for, so the queue is readable rather than filed in a spreadsheet nobody publishes.
Several of these skills produce prose, and a rubric applied to prose is judgement wearing a number. For those the honest route is a blind preference comparison rather than a scored rubric. That is weaker evidence, and the first page to carry one will say so in the same place the grey box sits now.
Whatever comes back gets published. A method that fails is deleted and keeps its row in the table above. That is the arrangement, it has already cost 6 methods, and it is the only thing that will make the word measured worth printing when there is finally something to print it on.
Nothing here needs an account or our app. Copy the file or download it and it works today. Locul just makes it one click and puts the file where each assistant expects it.