Leaders within top AI firms and independent researchers have been speculating about when, or if, we should take into consideration how the most advanced large language models feel about the work they do for us. After all, the models are rapidly getting better, more engaging and more opinionated. Could that mean, some wonder, something deeper is going on inside them?

The shorthand for this quandary is “model welfare.” As in, should tech companies and societies concern themselves with the wellbeing of chatbots?

We should care about model welfare now, some argue, because artificial intelligences could someday be conscious. Like we are. So, usually, the next question is: When and how will we know if they are conscious? If only it were so easy.

Two philosophers working at DeepMind, Adam Bales and Iason Gabriel, published a paper in June suggesting that how we will know is really the wrong question. The right question is: What are we going to do when lots of people disagree?