An Offline Metric for the Debiasedness of Click Models
Click models aim to identify and extract biases in search data, but we do not evaluate whether they actually do so. We propose a metric based on relevance annotations that measures the debiasedness of click models, and consequently their usefulness in learning-to-rank from implicit feedback.