What’s the deal with the RMET (“the Reading the Mind in the Eyes test”)?

Dan Luu writes:

Have you ever looked into the RMET? [This is something called the Reading the Mind in the Eyes test, which I’d never heard of until receiving this email. — AG] I’d like to get some data to replicate a result. I have a survey here. The story, as it relates to me, is that a friend of mine was hanging out in a cabin with a bunch of friends. A friend of theirs, who’s a psychologist, has them take this test. After they take the test, the psychologist reveals that neurotypical people complete the test in N minutes vs. M minutes for autistic people. People sort of can’t believe it, do all neurotypicals really finish the test that quickly? That’s super fast! But it’s plausible that everyone in the room is autistic, so it’s not totally unbelievable. My friend then has two of his most normal friends take the test and they’re actually slower than he is. They have a discussion about social interactions, etc., and they conclude that they’re autistic, which is surprising/interesting news to them. Ok, that’s an interesting story so far. I got that part of the story from this friend, Then, a week later, he messages me about this thing again. He had some more normal seeming people take the test and none of them finished the test under the “neurotypical” threshold, so that’s a bit suspicious. I look into it a bit and, according to some source, neurotypical finishing times are 2-3 minutes, with > 3 minutes being autistic. I can’t find any sort of citation for this, whether it’s a paper or someone putting the data on a website for anything like that. It’s just a site where they have the test online and make the claim that some scientists figured this out. There’s also some study with schizophrenics where the 59 “healthy controls” took 5.45 +/- 2.58 minutes. This makes the 2-3 minutes for neurotypical folks vs. 3+ minutes for autistic folks thing seem like it might be fake? Also, on looking into the RMET further, the untimed version also seems pretty fake for a bunch of reasons, one of which I’d like question-level data for. One thing that seems fake is that some online places you can take the test report neurotypical vs. autistic thresholds like the time-based threshold, but on looking up what the distribution of results is, it’s pretty clear that the difference within a group is much larger than the difference between groups, so no such threshold-based criteria can really make sense. Also, it turns out that Asian folks score worse than white folks on the test, unless you make a race-normed RMET, and just eyeballing the difference, the difference there is something like the order of the alleged difference between autistic and non-autistic folks. For me, on the official test, the facial expressions that took me the most time (and I made errors on) were some of the photos of women who look like models or actresses. I think that’s because I don’t normally interact with people who look like that (vary manicured eyebrows, totally flat skin with no wrinkles but not a young child, etc.), so the faces aren’t super readable to me, especially in combination with the relatively subtle things some of the questions are trying to have people read. My guess is I’m not the only person like this and a lot of other people will miss the same questions, but maybe not. Anyway, do you have suggestions on how to go about this? My current plan is that I’ll just tweet out the survey after some testing to make sure the survey works, but the set of my twitter followers or whoever sees it from it being retweeted is not going to be very random, but I don’t know that I have a good way to fix this problem.

And Dan shares a very small bit of evidence:

A friend of mine tried it and, while doing it, said she couldn’t make heads or tails of any of the expressions, but then scored a 25 when the average score from totally random selection would be 9. She was surprised by how many she got right despite feeling like she was picking randomly.

Interesting. Try it out for yourself!

What about my reaction?

I tried it, and . . . I didn’t get the point at all. They just looked like eyes: I didn’t see any facial expressions at all. This might just be me: I have Tourette’s syndrome and just about never look at people in the eyes. I always look away. So, unlike most people, I don’t have a lifetime of practice looking at people in the eyes. I remember back in elementary school a teacher getting mad at me for not looking her in the eyes, and I was thinking, Hey, just tell me what you want to say, I’m listening to your words.

So, for me, it wasn’t a matter of finishing the test in X minutes. I couldn’t do it at all. I gave up after the first two images. The funny thing is, I know that people can get emotional information from eyes; I’m even a coauthor of a paper on the topic: https://sites.stat.columbia.edu/gelman/research/published/faces.pdf But this is the first time I’ve ever tried to do this myself, and I just can’t. I’m not autistic, I have no problem with emotions, it’s just the eyes thing.