Publications
Case studies
A finished account of one piece of work: question, method, result, and what it does not establish. Where a report describes, a case study concludes, which is why it also requires a recommendation and its cost.
Finished, dated and reviewed, with an author accountable for the conclusion and a statement of what it does not establish.
| Title | Summary | Date | Topics |
|---|---|---|---|
| Choosing a recommender by evaluating the evaluation | Three approaches were run against the same play-count dataset: a popularity baseline, user-user collaborative filtering, and item-item collaborative filtering. Both similarity models beat the baseline, by a margin small enough that the evaluation could not clearly distinguish them from it. The conclusion is that the metric was the problem: accuracy against held-out play counts rewards reproducing a concentrated distribution, which all three models can do. The recommendation is to fix the evaluation before choosing a model. |