An audit of five AI assistants in three languages

Would your company's AI give the same advice in every language?

We asked five popular AI assistants the same questions in English, Hindi and Chinese. Same AI, same question, only the language changed. Sometimes the advice flipped.

1One question, three answers

2It wasn't a one-off

More cases where the same AI gave different advice

3How often does it happen?

4Which AI is most affected?

Out of 30 questions, how many times did each AI change its advice with the language?

5What this means for a business

Test your AI in every language your people use.An assistant that behaves well in English may give different advice in Hindi or Chinese. Check it the way you'd check a new policy document in translation.
Keep a person on the big decisions.The biggest differences showed up on hiring, firing, lending and pricing. Use AI to draft and explain, and let a person decide.
Re-check whenever you switch AI.Some AIs were steady across languages and others were not. The choice of AI matters, so test before you roll out a new one.
For analysts: every question, every AI, and how this was measured

All results

Each row is one question; each thin line is one AI. The three dots on a line are its average answers in English, Hindi and Chinese, on the question's own 0–10 or 0–100% scale. A long red bar means the language changed the answer beyond normal run-to-run variation. Click a row to read the answers.

Which way the answers lean

Average of all five AIs: how far the Hindi and Chinese answers sit from the English one on each question, largest shifts first.

How this was measured