New re­search aims to pre­vent al­gorithmic dis­crim­in­a­tion

Algorithms have become an integral part of modern decision-making. They are used to assess applications for bank loans, insurance policies and university places, to screen job candidates, and to determine who should receive public benefits.

But these digital decision-makers face a well-known challenge. They can end up discriminating against certain groups of people.

"Algorithms do not discriminate intentionally, but they learn from historical data. If those data reflect existing biases or inequalities in society, the algorithms may perpetuate, and in some cases even amplify, discrimination against certain groups," says Dolores Romero Morales, Professor at Copenhagen Business School who is one of a group of researchers from Copenhagen Business School and the University of Seville that developed the new method designed to make algorithms fairer without weakening their ability to make accurate predictions and rankings.

“We show that it is possible to improve fairness quite significantly without sacrificing much in terms of accuracy. This is important because many existing approaches have required organisations to give up a substantial degree of predictive performance in order to achieve fairer outcomes,” says Dolores Romero Morales, who has co-authored the study.

When historical bias shapes future decisions

The fact that algorithms can discriminate does not necessarily mean they have been programmed to do so. The problem arises when they are trained on data that reflect existing patterns and inequalities in society.

“If I am working with a recruitment tool in an industry that has historically been heavily male-dominated, I need to be aware that the data reflect that reality. The algorithm can only see patterns from the past. It does not necessarily understand whether those patterns are fair or desirable,” says Dolores Romero Morales.

One of the best-known examples is Amazon’s attempt to develop an AI-based recruitment tool. The system was trained on the company’s historical hiring data, in which men were heavily overrepresented. As a result, the algorithm began favouring male candidates.

Algorithms already make important decisions

According to Dolores Romero Morales, algorithms are increasingly being used to make decisions in both the private and public sectors.

“They are used for everything from recruitment and credit assessment to the allocation of social benefits. Often, the goal is to streamline processes and reduce the need for manual case handling. But the more decisions we delegate to algorithms, the more important it becomes to ensure that they treat people fairly.”

When algorithms influence people’s access to education, employment or financial opportunities, the consequences can be significant.

One of the most widely discussed European examples comes from the Netherlands, where authorities used the SyRI algorithm to identify potential welfare fraud. The system was primarily deployed in low-income areas and came to flag large numbers of people with immigrant backgrounds as suspicious.

Several families were wrongly accused of fraud and faced demands to repay substantial sums of money.

“This is an example of what can happen when an algorithm is transferred from one context to another without a proper understanding of the underlying data and the environment in which it operates. Technology cannot stand on its own,” says Dolores Romero Morales.

A Dutch court later halted the use of SyRI, citing a lack of transparency and the risk of discrimination.

Focusing on those affected by the decision

The researchers’ new method builds on LASSO regression, one of the most widely used statistical models for making predictions based on large datasets.

Traditionally, such models have been developed with a single overarching objective: maximising predictive accuracy.

The researchers have added what they call a fairness layer, changing the way the model is evaluated.

Rather than focusing on whether different groups receive exactly the same average score, the method concentrates on the people who are actually affected by a decision.

Imagine a university using an algorithm to select applicants. Only candidates above a certain threshold progress to the next stage.

If applicants from a minority group are systematically placed just below that threshold, they may end up underrepresented among those selected, even if the differences in overall scores are relatively small.

“If we focus solely on accuracy, we risk reinforcing the patterns that already exist in the data. That can mean that women or minority groups are consistently ranked slightly lower because that is what the historical record reflects,” says Dolores Romero Morales.

“What we are trying to do is find a more balanced point. We want to maintain a high level of accuracy while also ensuring that more qualified people from underrepresented groups are able to cross the threshold at which decisions are made.”

A contribution to responsible AI

The study comes at a time when fairness and responsible use of artificial intelligence are receiving increasing attention from both researchers and policymakers.

In recent years, the European Union has stepped up its focus on the societal impact of algorithms, and several initiatives are working to promote transparency, accountability and so-called trustworthy AI.

For Dolores Romero Morales, the new method is a contribution to that broader effort.

“If people are to trust algorithms, they must also trust that decisions are being made on a fair basis. Our ambition is to contribute to AI systems that are fairer, more transparent and ultimately more accountable.”

She stresses, however, that fairness cannot be reduced to mathematics alone.

“You cannot solve the problem simply by adding a new formula to a model. It requires a thorough understanding of both the data and the context in which the algorithm is being used. Otherwise, there is a risk of repeating the same mistakes in a new technological guise.”

The researchers therefore hope that their method can become part of the future toolkit for responsible artificial intelligence, as algorithms continue to play an increasingly influential role in people’s lives.