Skills/Data analysis
6 skills · 0 measured · no API keys, no install, no scraping

Claude data analysis skill, for the part before the analysis

Six skills for the unglamorous half: the cleaning order, the metric definition two teams can agree on, the query that counts correctly, the retention spec, the checks, and the experiment readout. How we test →

All 6 shown

6 skills we have not measured yet

Written, reviewed and free to take. No run behind them, so no claim about what they do to an output. What that means.

Data analysisNot yet measured Analytical query build Question in, query plus runnable proof out. The assertion block states the grain and the row accounting, so the join that doubled every row fails loudly instead of returning a believable total. skill · 3,736 words · MIT Read the write-up
Data analysisNot yet measured Cohort retention spec Retention names three different calculations. The spec fixes which one, the cohort key, the grain, the activity event, the censoring rule and the minimum cohort age a cell may be plotted at. skill · 4,133 words · MIT Read the write-up
Data analysisNot yet measured Data quality check suite Pipeline in, suite specification out. Ten check classes with what each one silently misses, thresholds derived from observed history, and a BLOCK, WARN, LOG ladder with owners. skill · 3,610 words · MIT Read the write-up
Data analysisNot yet measured Dataset cleaning run Dirty file in, cleaned file plus a cleaning log out. The content is the ordering: which steps destroy information a later step needs, and which two cannot be undone at all. skill · 4,124 words · MIT Read the write-up
Data analysisNot yet measured Experiment readout build Experiment data in, readout document out. The sample ratio check runs before the lift is allowed to be a number, and a ratio metric gets the variance calculation it actually needs. skill · 3,941 words · MIT Read the write-up
Data analysisNot yet measured Metric definition spec One metric in, one specification out. Grain, eligibility, day boundary, window, dedup key, late arrival, denominator and restatement, each written down with the choice actually made. skill · 3,302 words · MIT Read the write-up
no API keys, no install, no scraping

The list

Ask a strong model to analyse a dataset and it will do a decent job of the analysis. The failures almost never happen there. They happen two steps earlier, in the part nobody writes about: a join that quietly doubled every row before anything was counted, a date column where a third of the rows were read as day-month and the rest as month-day, and two teams reporting the same metric with different numbers because they never agreed on what a day is.

That is the ground these skills stand on. They are all generative, which is deliberate. The library already carries the judging half of this work and there was no point writing it twice.

The skill that is not here

We wrote a seventh file for this category, on how a spreadsheet corrupts a data handoff: precision limits, leading zeros, locale-dependent separators, the formula-injection surface in a free-text column. It was dense and every number in it was sourced.

Then we tested it against a control, on a fixture and a rubric built by an author who had never seen it, and it did nothing. Not a small effect we are choosing to call nothing: the middle of the distribution did not move, the worst run did not move, and the difference between the two arms was comfortably inside what you would expect from noise alone. The control already found the important problems on its own.

So it is not published, in any state. That rule was fixed before the run, and the honest reading of this particular result is the most useful thing we can tell you about the topic: on a task like this one, a capable model does not appear to need the catalogue. The figures sit in the results table on the method page with every other method we have ever run, including the ones that worked.

Where this category stops and the neighbours start

If you have a dataset and need it to become charts somebody will read, that is the data story build, which maps each claim to an encoding before anything is drawn. If a chart already exists and you want a verdict on it, that is the chart honesty audit. If you have a finding and want to know whether it survives contact with base rates, mix shift and survivorship, that is the analysis sanity check, which is ten named reversals with the arithmetic under each one.

The six

Dataset cleaning run is about order rather than technique. Every operation in a cleaning pipeline is easy on its own and several of them destroy information if run before another one, so the skill fixes the sequence and says which step is irreversible.

Metric definition spec produces the document that stops two dashboards disagreeing. Its content is the list of decisions a metric name hides: the grain, the timezone the day boundary sits in, what happens to events that arrive late, which duplicates get collapsed and what the denominator counts. It is built on public company filings, because the clearest possible evidence that "active user" specifies almost nothing is that the largest companies in the world define it incompatibly in their own annual reports.

Analytical query build is not about performance. It is about the small number of ways a query returns a confident wrong number: the join that fans out, the null that swallows a filter, the window frame that silently sums a whole partition.

Cohort retention spec exists because retention names at least three different calculations that give different curves from identical data, and because most retention charts bend at the right-hand end for a reason that has nothing to do with users.

Experiment readout build carries the checks that run before a result is allowed to be a result, including the one most readouts skip, which is whether the two groups were even the right size. Data quality check suite produces the tests that run on every load rather than the investigation you do after somebody notices.

None of the six carries a number yet. Each card says so and each page names the test it is waiting for.

What the two labels mean

Some of these carry a number. Most do not, and they say so.

Measured means the skill was given a realistic task on real material, then the identical task was run again with the skill removed, five runs each way. Each output was graded alone, against a rubric written by someone who had never seen the skill, by a session that was not told the other arm existed. Whether it passed was decided by a rule written down before any run executed. Those pages carry the worst case, the median, the p value, and what the skill costs you as well as what it buys.

Not yet measured means exactly that. It is written, it has been read, it is free to take, and we have run no experiment on it, so we make no claim about what it does to an output. It is not a skill that failed. Skills that failed are not published at all, in either state, and their numbers are in the results table.

Measuring one skill properly costs roughly twenty model sessions. We are working down the queue and moving skills from the second group into the first. Read the full method, or go back to all skills.