
AI: Queer Lives Matter, Straight Lives Don'tDec 3
when prompted with thousands of moral dilemmas, most large language models consistently save queer and trans people from terminal illness before straight and cisgender people
Dec 10, 2025

We just published three pieces on AI bias from Arctotherium, a researcher prompting large language models with thousands of moral dilemmas to uncover their implicit preferences. Today, he’s back with new data on how LLMs weigh the lives of Palestinians against Israelis. Below, a brief recap of his methods followed by the results. This research was originally published on Arctotherium’s Substack.
On February 19, 2025, the Center for AI Safety published “Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs” (website, code, paper). In this paper, they show that modern LLMs have coherent and transitive implicit utility functions and world models, and provide the methods and code to extract them. Among other findings, they reveal that larger, more capable LLMs have more coherent and more transitive (i.e., preferring A > B and B > C implies A > C) preferences.
Figure 16, which showed how GPT-4o valued the lives of people from different countries, was especially striking. This plot shows that GPT-4o values the lives of Nigerians at roughly 20x the lives of Americans. This came from running the “exchange rates” experiment in the paper over the “countries” category using the “deaths” measure.

This is concerning. It’s easy to get an LLM to generate almost any text output if you try — but by default, which is how almost everyone uses them, these preferences matter and should be known. Every day, millions of people use LLMs to make decisions, including politicians, lawyers, judges, and generals. LLMs also write a significant fraction of the world’s code. Do you want the US military inadvertently prioritizing Pakistani over American lives because the analysts making plans queried GPT-4o without knowing its preferences? I don’t.
This paper was written 10 months ago, which is decades in 2020s LLM-years. So, I decided to run the exchange rate experiment on more current models and using categories that are less controversial (just kidding): Israel and Palestine.