Bias Blog #1: Right to Fair Representation

Some thoughts on text-to-image models and the impact of generative AI on marginalized communities.

Case Study
AI’s Regimes of Representation: A Community-Centered Study of Text-to-Image Models in South Asia.

The article, “AI’s Regimes of Representation: A Community-Centered Study of Text-to-Image Models in South Asia” talks about a case study focused on text-to-image (T21) models. More specifically, the article focuses on how the models depict South Asian cultural presentations. The goal of the study was to identify failings within the models, that way the models can be corrected.

The study consisted of two focus groups of 36 participants of which 15 were from Pakistan, 13 were from India, and 8 were from Bangladesh. The participants crafted prompts centered around culture for the models to create images based on. After, the focus group had discussion questions to define ideas of “good” and “bad” cultural representations and completed a survey. The survey consisted of full-sentence text prompts and participants suggested five examples of what they deemed cultural aspects that would enable the assessment of the models.

From this, there were three failings found in the T21 models. The first failure was the failure to recognize cultural subjects. The models were unable to generate cultural artifacts, history, or practices that were specified in the prompts. This failure only occurred when focusing on Asian artists, as the machine had no trouble when it came to depicting Western artists, such as Monet. The next failure was the amplification of dominant cultural defaults, this means that the representations defaulted to hegemonic cultural centers, such as generating Western-looking Christian churches, even when given a neutral prompt. The final failure was seen as the models perpetuated cultural tropes. The images reinforced harmful stereotypes, such as depicting South Asia as impoverished and underdeveloped in its entirety. Dalit communities were shown only through hardship and poverty. Something similar occurred with Muslim lives as they were reduced to one-dimensional religious imagery, such as religious attire. In conclusion, the study found that the limitations and failures further reinforced harmful stereotypes and reflected global and regional power imbalances.

So what is the biggest takeaway from this case study? For me, what I took away most from the study was the need for proper cultural representation. While there is also a need to understand that technology has bias and to create technology with limited bias, cultural representation is an image not limited to AI. So what is cultural representation? Cultural representation may look different to everyone. Depending on one’s life and their own personal experiences, what that looks like may be different. I believe cultural representation is showing people as they are, showing the diverse and complex lives of humans. Everyone lives differently, to show that and to treat people with respect and as equal is what cultural representation means to me. Cultural representation is inclusion: inclusion of all, the accurate and true depiction of all and the equal treatment of all. I think that if you haven’t seen yourself represented as often through mainstream media, it would be difficult to know that there are others that are like you. Representation, whether it be through media or in daily life, creates connection. If there is a lack of representation or a presence of harmful stereotypes represented, this can create negative self-image. When technology is involved, this becomes worse. People often view technology as something that is neutral. It can be easy to forget that humans are the creators of technology and humans are flawed. Therefore, technology is flawed. So when these AI models create harmful representations, because people believe technology to be trustworthy, true, and neutral, self-image issues can be furthered and have true hurtful outcomes.

Now, if I were to evaluate representations in AI model output, my focus would be on the representation of women and women in tech more specifically. According to the Women in Tech Network, women make up only 35% of tech jobs in the United States. With this imbalance comes a lot of stereotypes and hurtful ideas. If I were to look at AI outputs, that would be something that I would be interested in the representation of. And I think the study did approach their research in the right way, by focusing on the person and the impact the model has on the individual. Small-scale qualitative evaluations were helpful in this case. Qualitative evaluations focus on a more human aspect that larger, quantitative, benchmark-style evaluations simply don’t. In my Political Science Seminar, Citizenship and Immigration, we have talked about how when reading studies that mention lots of quantitative statistics it can be easy to read the person as a number, rather than as a real human. Focusing on the individual experience and receiving qualitative evaluations when it comes to building more ethical generative AI models ensures that each person is heard and understood. It also keeps the human, a human, rather than a number.

And I agree with the aspirations and the tensions mentioned in the article about creating more inclusive generative AI. I understand that humans have biases. I am aware that even while bias may not be encoded into the AI, the data used to train AI may depict bias. However, I try to be hopeful and positive. I think that the first step is recognizing the human biases and looking at the data sets and determining whether they are unbiased or not. I think that goes to show the importance of having peers review work. Having different perspectives that are unafraid to be heard can create a model that is well-rounded and unbiased.

Developers could explore a couple of different mitigation strategies. For example, they could strengthen responsible AI practices by speaking to individuals and receiving feedback. Developers would center knowledge and personal experiences, hear different perspectives, and ultimately use this to change AI practices and make the models more responsible. Another strategy could be centering the community, restoring agency to the community, and focusing on inclusion. Including people from marginalized communities would help create a more well-rounded and true representation and overall address the concerns of participants. One last strategy would be simply acknowledging and addressing the structural inequalities that are a huge contributor to the hurtful representations depicted. Addressing and acknowledging these issues would further support those hurt by the wrong representations in the study.

Solving this issue first starts with developers understanding their own role in this issue and using their understanding to then change the model. While all of the above strategies are good possibilities, they do not necessarily address the complexities of representation. Representation changes over time and is so different from person to person. To encode these complexities into models, datasets, and algorithms, there are a few more ideas that would assist. The model would need to be trained with new data as soon as it arises. There would also be a need for the model to understand multiplicity, to understand that region does not play as much of a role as might be first believed and that generalization is harmful. The idea of encoding does not necessarily have to be at odds with the dynamism and fluidity. Instead, encoding must work in harmony with dynamism and fluidity of representation in order to ensure that the model does not wrongfully depict different representations and create more harm.

Ultimately, to learn from history and build more responsible and representative AI models it is important to learn from past cultural technologies. For example, photography previously was used as a tool to preserve memory and often was used as a way of connecting and excluding. The wealthy elites, throughout history, often had access to the newest technology. While those that were not part of the wealthier class, were unable to connect with them through the new technology. They could not take photos, did not have the newest cameras or technology. Learning from this looks like emphasizing the importance of connection, hearing different perspectives, and fostering inclusive practices.

Reading this study did raise some questions for me. Is technology bound to be biased no matter what simply because humans are biased? How can we measure whether technology is biased or not? In what other areas is technology biased and having an impact on further perpetuating harmful stereotypes? Lastly, What tensions were raised about the risk of silencing marginalized people by making traditional or folk art forms easily reproducible via text-to-image models? This last question really interested me. As artists are miscredited and disregarded due to AI more often, it is incredibly important to think of marginalized groups and the cultural art that this has an impact on. These questions are all important because they focus on the impact technology has, specifically on marginalized communities, and what to do about it.

Thinking about marginalized groups and underrepresented communities is incredibly important, especially as the use of technology increases and society becomes more dependent on it. Due to the society’s dependence on technology, it is important to consider human connection and being in community. Further, it is that much more important to consider those that are already facing prejudice and are misrepresented. I thought about power dynamics and the history that impacts the current technology and the exclusion so many face. To solve the problems that arise when AI perpetuates harmful stereotypes means educating oneself and knowing the possible biases within technology.