Can AI simulate user design preferences? 53% match. As good as… flipping a coin

On the topic of having AI replace your users, I am adding yet another recent preprint by the same research team behind The Largest Review of Synthetic Participants Ever Conducted Found Exactly What You'd Expect. Synthetic Participants Don't Work . This time, they looked at whether LLMs can accurately reflect user design preferences. The result? A number of distortions, including a 53% agreement on the first choice. Since most votes were between just two designs, that's basically a coin flip. So maybe we hav