Motivation!¶
Note
Why do we want Equal Treament?
We already have a lot of tools to audit models for fairness, but most of them are focused on the notion of equality of opportunity or in demographic parity
Equal Treatment vs Equal Opportunity¶
From a technical perspective, Equal Opportunity requires labeled data which in practice, after deployment is hard to obtain. Equal Opportuntiy forces to meet some quotas (True Positive Rate), this introduces a second order discrimination. Equal Treatment has no quotas, it is fine to “hire” the best, while the decision is taken independently from the protected attribute.
From a philosophical perspective, much is discussed in the literature. This commentary [:link:`Cavanagh`] about [1] provides some hints. We have conducted a technical mapping, considering other possibilities.
Note that previous translations, such as Equal Opportunity as TPR difference, do not always align with nuanced philosophical notions.
[1] Against Equality of Opportunity, Matt Cavanagh (Link to the paper: https://link.springer.com/content/pdf/10.1057/palgrave.cpt.9300060.pdf)
Equal Treatment vs Demographic Parity¶
Note
Is “equal treatment” a superior notion of fairness than demographic parity?
We make reference to our paper “Beyond Demographic Parity: Redefining Equal Treatment”
To define a metric, one must collect domain requirements and formalize the metric such that its mathematical properties match the requirements. For the metric of fairness, the domain requirements are given by philosophical foundations (section 2.2). For the egalitarian school of thought, we find that Hardt et al 2016 propose a metric for equal opportunity, but it has the shortcoming that it requires labellings of the true outcomes - which are hardly ever available in practice - which renders equal opportunity often unusable.
For the equal treatment of individuals, all available metrics violate philosophical requirements from the liberalism school of thought. Thus, we find that our suggestion of equal treatment is a significant improvement from the domain requirements point of view. From the mathematical point of view, we show its desired behaviour from a theory perspective in section 4 and from an experimental perspective in section 5.
It is a particular strength of this paper that philosophical domain requirements are recognised and matched. This is the only way to avoid that one defines a metric that measures the wrong thing. Thus, indeed, we argue that equal treatment is a superior notion of fairne