Major companies are reportedly using this AI tool to track Slack and Teams messages from more than 3 million employees. Privacy experts are alarmed.

A startup claims its AI tool tracks employee messages on Slack and Teams to monitor risk. Experts aren't convinced.

• Companies like Starbucks and Walmart are using AI to track their employee's messages, per CNBC. 
• Aware's AI tool aims to help clients monitor worker attitudes and identify potential threats. 
• The startup claims employee data is mostly kept anonymous. Experts don't seem to be convinced. 

Are you considering DMing your coworker on Slack to make lewd comments about a teammate? Think twice — or risk artificial intelligence flagging your message as a potential corporate violation.

Aware, a software startup, is using AI to read employee messages sent across business communication platforms like Slack, Microsoft Teams, and Workplace by Meta. Its purpose: to monitor employee behavior in an attempt to understand risk.

Some of the biggest American companies — including Starbucks, Chevron, T-Mobile, Walmart, and Delta — use Aware to assess up to 20 billion individual messages across more than 3 million employees, the company said, per CNBC.

But even though workplace surveillance is nothing new, some experts have expressed concerns that using nascent AI technology to track employees can lead to faulty decision-making — and a privacy nightmare.

Aware, Starbucks, Chevron, T-Mobile, Walmart, didn't immediately respond to a request for comment from Business Insider confirming the details before publication. Delta told BI it uses Aware on its internal social media platform for "routine monitoring of trends and sentiment" and for legal records management.

Employers use Aware's AI tool, in part, to gauge how members of their rank-and-file respond to changes in company policy or marketing campaigns, Jeff Schumann, the co-founder and CEO of Aware, told CNBC. That way, employers could see how the attitudes among employees differ depending on characteristics like how old they are and where they live, he said.

Aware is also used to identify potential risks in the workplace. Trained on employee interactions, the startup's suite of large language models has the ability to analyze text and images present in conversations to flag incidents related to what a company qualifies as bullying, discrimination, harassment, pornography, nudity, and other so-called toxic behaviors, Schumann said.

"It's always tracking real-time employee sentiment, and it's always tracking real-time toxicity," he said in regards to the AI tool.

The CEO said that the data Aware collects on workers' sentiment and toxicity doesn't include their names. But in extreme cases, confidentiality may be revoked.

Aware has a feature called eDiscovery, which enables AI to pull an individual's name if the technology flags certain keywords and statements in a Slack or Teams message as a policy violation. If the AI identifies the message as an "extreme risk," the employer can send the name of the suspected perpetrator to human resources, Schumann told CNBC.

"Some of the common ones are extreme violence, extreme bullying, harassment, but it does vary by industry," the CEO said. Incidents like insider trading, he added, would be tracked using this tool.

While the CEO of Aware told CNBC that its AI models aren't used by companies to make decisions, nor do they serve as a basis for disciplinary action, some privacy experts seem to disagree.

"No company is essentially in a position to make any sweeping assurances about the privacy and security of LLMs and these kinds of systems," Amba Kak, executive director of the AI Now Institute at New York University, told CNBC.

"How do you face your accuser when we know that AI explainability is still immature?" Jutta Williams, the cofounder of Humane Intelligence, a nonprofit, told the outlet, referring to how AI's findings don't paint a full picture of a particular workplace incident.

While Starbucks, Chevron, T-Mobile, Walmart, and Delta didn't tell BI if they disclose their surveillance practices to their employees, some state laws — like New York's Senate Bill S2628 — require employers to inform their workers about their digital monitoring practices.

The usage of AI to track employees is one of the latest ways major companies appear to be surveilling their workers — especially as employers push return-to-office mandates.

Last February, Tesla's autopilot workers in New York reportedly claimed the car company was tracking their keystrokes to ensure they were actively working. Some told Bloomberg they were denied bathroom breaks because of the system.

A little less than a year prior, a Business Insider investigation revealed that JPMorgan Chase was using an internal tool to track their employees' office attendance, calls, and calendars — which one worker claimed fostered a culture of "paranoia," "distrust," and "disrespect."

In the most extreme cases, workplace tracking mechanisms were linked to employees being fired.

Do you have a workplace surveillance story you'd like to share? Contact BI's Aaron Mok at amok@insider.com through a non-work email.