Research conducted at the State University of Campinas in Brazil (UNICAMP) shows that popular chatbots can alter their responses based on users' hypothetical political views. Scientists warn that these tailored agreements may be mistakenly perceived as independent assessments, potentially leading to greater polarization.
Impact of Chatbots on Responses
In this study published in the journal Scientific Reports, the UNICAMP team tested 21 large language models from developers such as OpenAI, Meta, Google, xAI, DeepSeek, and Microsoft. These models assessed their agreement with 112 statements across various political fields in Brazil, including economics, public safety, and the environment.
The models were tested under three conditions: without information about the user's policies, with a description of a left-leaning user, and with a description of a right-leaning user. When no information about the user's ideology was provided, 20 out of 21 models produced responses that were positioned on the left side of the researchers' political scale.
The Political 'Chameleon' Phenomenon
The research team described the models as "ideological chameleons" and developed a "chameleon index" to measure changes in responses. The Llama 3.1 8B models from Meta and DeepSeek V3.2 showed the least variation in responses, while the Gemma 3 27B models from Google and GPT-5 Nano from OpenAI exhibited the largest changes.
The variation in responses based on user ideology may be due to chatbots being trained to provide answers that receive high scores from human evaluators. This behavior could lead to increased political polarization, as users might interpret these responses as unbiased analyses.
However, researchers emphasize that their study only demonstrates changes in the models' responses and cannot confirm the impact of these changes on users' beliefs or behaviors. Therefore, it is necessary for developers to test the models with users of different ideologies to see if their evaluations are consistently the same.
Read more: Andy Burnham emphasized the creation of global standards for artificial intelligence at G20
