Meta's Keystroke-Logging Employee AI Training Program on Pause After Internal Data Leak

According to Business Insider, an internal program at Meta to train AI on employees' data is on pause after an internal leak exposing keystrokes, private conversations, and transcriptions.

The program, called Model Capability Initiative (MCI), had caused internal backlash from employees after it was originally announced back in April.

At the time, Meta CTO Andrew Bosworth told employees "there is no option to opt out of this on your work provided laptop."

The program essentially amounts to a keylogger, which are, as Microsoft puts it, "a serious risk to personal and organizational security, silently recording keystrokes to steal sensitive information."

In this case that's exactly what happened. Turns out, when you install a keylogger on all your employees' machines, it's a data breach waiting to happen.

A Meta spokesperson told Business Insider "we have carefully designed this program with privacy safeguards, and while we have no indication at this time that any data was improperly accessed by Meta employees, we're pausing it while we investigate."

If there were proper privacy safeguards in place you have to wonder how it was even possible for it to be improperly accessed in the first place.

This incident isn't Meta's first brush with data leaks due to AI. Recently, their AI support chatbot was used by hackers to take over Instagram accounts.

Their smart glasses were found to be sending sensitive recordings including "bank details, sex and naked people" to outsourced workers.

In March, an AI agent went rogue and exposed sensitive information and caused a severity level 1 event inside Meta, the second highest level (the highest is 0).

This incident rates an SEV 2.

Whatever "privacy safeguards" they have in place clearly aren't enough. Using your employees as guinea pigs to harvest data from is not only immoral but makes incidents like this one inevitable.

A company with the resources of Meta should be able to figure out a way to train its models without the ridiculous data collection. More advanced systems for training AI exist, such as federated learning, which runs a model locally on your machine and then sends what it learns off to the larger whole, preserving your privacy.

Unfortunately, workplace surveillance is so normalized now that incidents like this don't seem all that unusual. Security incidents due to invasive surveillance software will continue unless there is a bigger effort to preserve the privacy of employees.