Experiments

An experiment here has a question, a dataset, and a stated way of being wrong. It is run to find out rather than to demonstrate, which means roughly half of them should return an unremarkable result.

What is in this group

Reading a recommender notebook in the open is the run itself: collaborative filtering evaluated against a music listening dataset, published as a notebook so the steps can be checked rather than described. The popularity baseline was harder to beat than expected is what it showed, recorded separately as a finding.

The run and the finding are separate pages

An experiment and its conclusion are kept apart on purpose. The notebook can be re-run and criticized on its own terms; the finding states what the run supports and no more. Each names the other, so a reader arriving at the conclusion can reach the evidence in one step.

An inconclusive run is still published

The finding above is that a simple popularity baseline was not beaten by more than the evaluation’s own noise. That is a real result and a dull one, and publishing it is the point. Work that only appears when it succeeded is a selected sample.