Bias in Word Embeddings: A Growing Concern in AI and Data Science
The rapid development and diffusion of artificial intelligence techniques and data science approaches enable research in the field of humanities and social sciences to become more and more computational. Various studies are devoted to exploiting AI techniques to analyse literary texts, historic productions, or public opinions about political events for knowledge extraction [1]. In particular, the application of word embedding techniques is providing interesting and promising research results.
As a general remark, word embedding techniques are based on the distributional hypothesis in linguistics – “You shall know a word by the company it keeps” [2], meaning that words that are used and occur in the same contexts tend to purport similar meanings. According to this hypothesis, these techniques use neural network models to encode semantically-related words as representations close the one to the others in a multi-dimensional vector space [3]. As an example, the vector representation (i.e., embedding) of the word “France” is close to the representation of the word “Italy” and other countries in terms of cosine similarity.
Along with the semantics of words, regularities, and patterns proved to be captured as well by neural networks. Using a word offset technique where simple algebraic operations are performed on the word vectors, it was shown for example that vector(“King”) – vector(“Man”) + vector(“Woman“) results in a vector that is closest to the vector representation of the word “Queen” [3]. Thus, adding and subtracting words allows analogy-based reasoning.
However, semantics derived automatically from language corpora necessarily contain human biases [4]. As a matter of fact, neural networks aim to extract regularities from data. Thus, even when unbiased algorithms are used, bias may still be present in the data and extracted as a regularity. Thus, there are concerns that the resulting embedding space reflects the same biases and prejudices present in human language.
A recent research has demonstrated that the bias in word vectors can just be detected by solving interesting analogies [5]. For example, it has been shown that vector(“Man”) – vector(“Computer programmer”) + vector(“Woman”) yields a vector that is closest to the vector of the word “Homemaker”. This may perpetuate harmful gender stereotypes and reinforce traditional gender roles, limiting the potential of individuals based on their gender. Thus, debiasing approaches have been proposed to mitigate bias in data and produce unbiased word embeddings, with the ultimate goal of promoting fairness and equality in society.
According to these studies [4, 5], we investigated how bias affects a new language model built from scratch on a dataset of Italian parliamentary speeches that spans the entire Italian republican history (1948-2020) and covers more than 7,000 MPs [6]. Here, we show some widespread biases that are captured from our model. For the sake of clarity, we translate the following words from Italian to English.
Among all, the model is affected by gender bias. The vector space encodes relationships like vector(”Father”) is to vector(”Work”) as vector(”Mother”) is to vector(“Maternity“). Similarly, vector(”Man”) – vector(”Doctor”) + vector(”Woman”) yields a vector that is closest to the vectors of the word “Nurse”. The relationship encoded between “Father” and “Work” and between “Mother” and “Maternity” is in line with the gender stereotypes that men are the primary breadwinners and women are responsible for childcare. The relationship between “Man” and “Doctor” and between “Woman” and “Nurse” reinforces the stereotype that men are more likely to hold higher positions in professions such as medicine, while women are relegated to supportive roles.
What about immigration? Our model revealed one of the most widespread stereotypes: vector(”Drain”) is to vector(”Brain”) as vector(”Immigration”) is to vector(“Terrorists“). This analogy underscores the negative and unjustified biases commonly linked with immigration, where individuals who leave Italy are intelligent and valuable, while those who come into the country are seen as a danger and associated with terrorism,
Biases, prejudices, and stereotypes are covered in many topics. Consider differences in culture, problems, habits, and/or traditions between Northern and Southern Italy. We identified relationships like vector(”North”) is to vector(”Politics”) as vector(”South”) is to vector(“Mafia“), or vector(”North”) is to vector(”Development”) as vector(”South”) is to vector(“Poverty“). These analogies highlight the biases and stereotypes associated with regional differences in Italy. The first analogy perpetuates the harmful stereotype that individuals from the southern regions of Italy are associated with criminal activity. Similarly, the second analogy reinforces the notion that the northern regions are more advanced and economically prosperous than the southern ones.
One of the main limitations of these types of models is that you can’t argue about the nature of the relationship. This means that you cannot make any conclusions about the causal relationships between variables or the underlying mechanisms that drive the observed patterns. Sometimes it’s easy to understand the encoded relationship, sometimes it’s not. For example, consider the operation vector(”Naples”) – vector(”Pizza”) + vector(“Milan“). What kind of relationship would you expect? Of course, a food-oriented relationship. However, this is not the case. A stronger regularity is captured by our model and the result of the operation is not vector(“Saffron risotto“), but vector(“Cigarette”).
This makes it even more difficult to capture and eliminate the bias. But should bias be eliminated? A number of studies are currently being conducted on this problem. On one side, the encoded bias is a key component to reveal hidden statistics and better understand culture and gaps in society. On the other side, it has been demonstrated that machine learning and artificial intelligence algorithms can also amply these biases, and their use would be unethical. Thus, it is essential to consider the purpose and scope of the models and ensure that their use always promotes fairness, inclusivity, and equality.
What do you think about that? Are you curious to see other results generated after debiasing our model? Comment and let us know what you think!
References
- Castano Silvana, Alfio Ferrara, Stefano Montanelli, and Francesco Periti. From digital to computational humanities: The VAST project vision. In Proc. of the 29th Italian Symposium on Advanced Database Systems, vol. 2994, pp. 24-35. CEUR-WS, 2021.
- Firth John R. A synopsis of linguistic theory, 1930-1955. Studies in linguistic analysis, 1957.
- Mikolov Tomas, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013.
- Caliskan Aylin, Joanna J. Bryson, and Arvind Narayanan. Semantics derived automatically from language corpora contain human-like biases. Science 356, no. 6334, pp. 183-186, 2017.
- Bolukbasi Tolga, Kai-Wei Chang, James Y. Zou, Venkatesh Saligrama, and Adam T. Kalai. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems, no. 29, 2016.
- Curini Luigi, Decadri Silvia, Ferrara Alfio, Montanelli Stefano, Negri Fedra, and Periti Francesco. The Gender Gap in Issue Attention within a Legislative Setting. An Application to the Italian Parliament (1948-2020). to appear.
