Smile, you’re on camera: when the boss wants to read your mood

A few years ago, an executive at WeWork floated an absolutely terrible idea. Perhaps, he told a reporter, cameras could be set up around the company’s co-working spaces to track people’s moods. The cameras would monitor “facial expressions” while listening devices might be used to analyse the emotion in their voices. A prototype had already been developed — although it wasn’t very good at spotting what a happy worker looked like. An uncomfortable smile was all it took to warrant a high score.

WeWork’s dystopian vision of productivity tracking never took off. The year after that interview, the company had a number of other, more pressing problems to attend to, including a failed IPO attempt and, later, bankruptcy. But the idea of collecting data on how people feel never quite disappeared.

Look at patent records and you’ll see just how many companies are interested in this sort of information. In the past few years Microsoft has filed a patent to record body language and expressions in meetings, while Amazon has filed one to track emotions via voice and Nvidia to infer emotions from audio data. This month, tech news site 404 Media reported that Meta had patented a wearable AI device that can analyse a wearer’s emotions by recording their voice and surroundings.

There are, obviously, a lot of problems with this — particularly in the workplace. Lack of privacy and the disadvantage it puts those of us without cheerful expressions are two. Plus the dubious idea that smiling provides meaningful information. The best office-based work tends to be the product of focus, engagement and thoughtfulness — not activities that lend themselves to a grin. Not even front-of-house workers need to smile all the time.

Most patents are never commercialised. But they are a record of ideas. Jeffrey Stanton, a professor of information studies at Syracuse University and co-author of The Visible Employee, thinks we are creeping closer to giving our consent to emotion-observation tech in the workplace and elsewhere because the experience of pandemic-era remote work has made us more acclimatised to being monitored.

Whatever sense of privacy you might have while sitting at your desk and staring at your screen may already be an illusion. Nearly two-thirds of large companies in the US monitor employees in some form, according to consulting firm Gartner. That can mean anything from tracking locations to logging keystrokes to analysing messages or taking screenshots of what they look at online.

There has been pushback against software that monitors employees, aka bossware. Meta says it scaled back its plan to collect mouse movements, keystrokes and other actions after staff complaints this year. But there is still a flourishing industry in employee monitoring services from companies like Teramind, Time Doctor and ActivTrak. And while the EU’s AI Act does not allow the use of AI systems to infer emotions in the workplace (unless for medical or safety reasons), there is no equivalent ban in the US. The Federal Trade Commission seems most concerned about opacity and proper testing.

The more companies collect biometric data in the name of security or convenience, the more supine we will become about protecting our privacy. Platforms like Instagram are already testing visual age verification tech (though there are reports of children getting around them by wearing fake moustaches). Last year Gap installed an eyeball scanner at the checkout of its San Francisco store. If you are willing to have your eye scanned just to buy a pair of jeans, why would you resist tracking of your expressions? Especially if it could get you work. A report from The Atlantic points out that Cupertino start-up Imentiv AI already offers to read human emotions from the facial expressions and vocal tones of job candidates.

Are there any upsides to our own levels of agreeableness becoming data that is logged and analysed? Not many. Companies will sometimes say that tracking employees in more detail is a way to improve safety and reward performance. And Stanton thinks there is less chance of pushback if employers involve workers in the decision-making processes. Cameras that track expressions could be a useful tool, he says, so long as they are not used for “negative feedback”.

Wanting workers to be happy is an admirable goal. Surveilling their countenances is not the way to go about it. Tracking expressions seems a quick-fire way to induce burnout, dissatisfaction, distrust and general paranoia. Nothing to smile about there.

[email protected]