Standards of evidence

An analysis specified before the data exists is a test. The same analysis specified afterwards is a description. Both are useful and they are not interchangeable, and the difference cannot be recovered later by anyone's assurance that they would have chosen the same threshold.

Every measurement this project intends to treat as a test is preceded by a written procedure: what will be recorded, under what conditions, how it will be processed, what quantity will be computed, and what value of that quantity counts as a departure. The document is dated and circulated before the run, and it is not amended afterwards.

Nothing is interesting on its own. A recording is interesting relative to other recordings made the same way, and an instrument's behaviour is interesting relative to its own behaviour under conditions that differ in one declared respect.

In practice this means the dull half of the archive is the important half: the same bell on an ordinary evening, the same room with the window shut, the same procedure run on a day when nobody expected anything.

If eleven strikes are examined and one is unusual, the relevant question is how many strikes would have to be examined before one unusual strike appeared by chance. That number is the denominator, and a result quoted without one is not a result.

This applies with particular force to anything found by looking. A feature noticed while listening to a recording has an unknown denominator: nobody counted the things that were looked at and dismissed, including the ones dismissed before they were consciously noticed.

Nulls are data. They are filed at the same prominence as anything else, they are never removed to tidy the archive, and a procedure that produced nothing is described in the same detail as one that did not.

The reason is arithmetical rather than ethical. A collection from which uninteresting outcomes have been quietly dropped cannot support any inference at all, and the dropping does not have to be dishonest to have that effect. It only has to be tidy.

A single recorder, held by a single person, in a single room, produces errors that are correlated with everything else that recorder ever does. Two chains that share a component share its faults.

Where a result is ever to be taken seriously, the measurement chain must be duplicated in a way that does not share equipment, software, or the person operating it. This project does not currently meet that standard and says so on every page where it matters.

Every file in the archive carries where it came from, who made it, what was done to it, and what has not been checked. Original files are preserved unmodified; anything derived from them is labelled as derived and never replaces the original.

Where a record depends on somebody else's claim - a date on a catalogue entry, a label inside an instrument, a parish's account of a recasting - the record states that this is what the source says, and it does not quietly promote that into something the project has verified.

The order is fixed: observe, describe, characterise ordinary causes, repeat, have somebody else repeat, and only then discuss what it might mean. Skipping to the last step is not a shortcut; it is a different activity that resembles this one.

In practice the discipline required is small and constant. It consists of refusing to write the interesting sentence until the boring ones are finished.

Transformations that make data easier to inspect are welcome, including ones that turn a measurement into something audible. They are inspection, not decoding. A transformation that makes a structure easier to perceive has not shown that the structure carries information, and the human ear will find pattern in almost anything given a rendering that flatters it.

Sonification is a way of inspecting data. It is not a method of decoding messages.