026 ICML 2018 Paper Review - A Discussion on the Fairness Issues of Machine Learning Algorithms #

In our previous review, we introduced a best paper from this year’s ICML conference, which is an excellent combination of machine learning and computer security. This paper analyzed how attackers can bypass a technique called “gradient obfuscation” to achieve effective attacks in white-box attack scenarios.

Today, we will share another best paper from ICML 2018, titled “Delayed Impact of Fair Machine Learning.” This paper mainly explores the application of “fairness” in machine learning. The five authors of this paper are all from the University of California, Berkeley.

Background of the Paper #

The topic explored in this paper is the issue of “fairness” in machine learning. In recent years, this topic has received increasing attention from the academic community. However, for ordinary AI engineers and data scientists, this issue still appears unfamiliar and distant. So, let me first outline the core ideas of research in this area.

Machine learning has an important application in various decision-making scenarios, such as loan applications, university admissions, and police duties. One undeniable characteristic is that these decisions may have significant irreversible consequences for society or individuals. One important consequence is the possibility of unexpected “unfair” situations for different populations. For example, some commonly used algorithms may judge that African Americans are more likely to be criminals than Caucasians when assisting police in determining whether someone is likely to be a criminal. This judgment obviously poses certain problems.

Machine learning researchers have recognized the issue of “fairness” in these algorithms and have started to explore whether unrestricted machine learning algorithms can lead to unfair decision-making for underrepresented groups. Based on these explorations, researchers have proposed a series of algorithms that add fairness-related constraints to various existing machine learning models, hoping to address decision-making problems under various definitions of unfairness.

Main Contributions of the Paper #

This paper discusses, from a theoretical perspective, machine learning algorithms with fairness properties that can truly contribute to the long-term well-being of minority groups in decision-making scenarios, based on certain assumptions and conditions. It is worth noting that the term “minority groups” here is an abstract concept, referring to a relatively small number or a group of data that is relatively small in a certain characteristic. This paper does not directly discuss the definition of minority groups in the sociological sense.

The authors mainly compare two groups, A and B, and examine how the difference in a certain “utility” under different fairness conditions may change. This difference can be positive, unchanged, or negative.

The main conclusion of the paper is that under different fairness conditions, the difference in utility can have various possibilities. This is actually a very important finding. There are some fairness conditions that intuitively we feel would promote the utility of minority groups. However, this paper shows us that even if the starting point is good, in certain situations, the difference in utility may be negative.

In addition, this paper explores the impact of “measurement error” on the difference in utility. The authors believe that measurement error should also be taken into account when considering fairness in the overall system.

It should be pointed out that the analytical method of the paper is mainly based on the “one-step prediction” of the temporal relationship. In other words, we use current data and models to analyze the decision-making in the next step, without including predictions for all future time periods. Theoretically, if there were an infinite future time frame, the conclusion could potentially change.

Core Method of the Paper #

The core idea of this article is to explore how a “policy” (Policy) adopted for the populations A and B affects the difference in utility between these two groups. If a certain policy leads to a negative difference in utility for a group, we say that this policy has an “active harm” effect on the group. If the difference in utility is zero, it indicates that the policy has a “stagnation” effect on the group. If the difference in utility is positive, it indicates that the policy has an “improvement” effect on the group.

In addition, we believe that there is a strategy called “MaxUtil” that maximizes expected utility without considering the specific characteristics of populations A and B. This strategy is actually the effect achieved using general machine learning algorithms in the absence of constraints. We need to compare the new strategy to this strategy. If the new strategy is better than this strategy, it is referred to as “relative improvement”; otherwise, we say the new strategy has a “relative harm” effect.

To further analyze the topic, the authors introduce a tool called “Outcome Curve” to visualize the relationship between strategies and differences in utility. Specifically, the x-axis of the curve represents the probability of selecting a certain group due to the strategy, and the y-axis represents the difference in utility. With this curve, we can intuitively observe the changes in the difference in utility.

From this curve, we can see that the difference in utility indeed shows “relative improvement” in one interval and “relative harm” in another interval, and in the rightmost interval, there is “active harm”. This breaks our previous belief that certain selection strategies consistently lead to a unique result.

On this basis, we specifically examine these two special strategies. The first one is called “Demographic Parity”, which aims to maintain the same selection probability in both groups. The other strategy is called “Equal Opportunity”, which aims to make the probability of success in a certain group (such as loan application, school admission, etc.) independent of the group. Both of these strategies are typical attempts to achieve fairness using constraints. The main goal is to compare these two strategies and the previously mentioned strategy of maximizing utility, and draw the following three major conclusions.

The first unexpected conclusion is that the strategy of maximizing utility does not necessarily lead to “active harm”. This means that, contrary to people’s previous beliefs, maximizing utility may also improve or keep the utility of minority groups the same.

The second conclusion is that both of these fairness strategies may cause “relative improvement”. This is also the original intention of proposing these two strategies, hoping to make adjustments in the selection probability to improve the utility of minority groups.

The third conclusion is that both of these fairness strategies may cause “relative harm”. This is an important finding of this paper, formally proving that fairness strategies in a certain interval do not bring positive “improvement” but rather “harm” to minority groups. The authors further compared the “Demographic Parity” and “Equal Opportunity” strategies and found that while “Equal Opportunity” can avoid “active harm”, “Demographic Parity” cannot achieve this.

Summary #

Today I presented another best paper from ICML this year.

Let’s review the key points together: First, this paper discusses the fairness issues of computer algorithms. Second, we provided a detailed introduction to the two strategies proposed in the paper and the main conclusions derived from them.

Finally, I’ll leave you with a question to ponder: What inspirations can studying algorithm fairness bring to our daily applied work?