The Dangers of Using AI in Companies

AI has become part of daily work for a lot of teams, and that's a good thing. But what I've noticed is that fast adoption often isn't matched by awareness of the risks. That doesn't mean you have to stop using AI, but it's worth knowing where the weak points are, so you can use it more carefully.

The risk of sensitive data going into public AI

This is the risk that happens most often without anyone noticing. Employees copy and paste internal data, code, or meeting notes into a public AI chatbot to ask for help, without realizing that data could be stored or processed outside the company's control.

The most well-known case happened at Samsung. According to a Cybersecurity Dive report, several employees in the semiconductor division accidentally leaked source code and internal meeting recordings by feeding them into ChatGPT, which eventually led Samsung to restrict its employees' use of public AI chatbots.

Relying on it without validation

AI output is convincing, but convincing doesn't mean correct. Problems show up the moment that output gets taken at face value without a second check, especially when it's used for something that directly affects other people.

The most commonly cited example is Mata v. Avianca, where two lawyers in the United States were sanctioned by a court for filing a legal brief citing six case rulings, all of them made up by ChatGPT. They even asked ChatGPT to double-check, and the chatbot assured them the cases were real.

The risk of access tied to one person

This is covered in full in the first article: the moment a work AI account is registered with an employee's personal email, all the chat history, prompts, and automations connected to it are at risk of disappearing if that person resigns. This isn't just about convenience, it's about who actually holds control over the company's digital assets.

Compliance and legal risk, depending on the industry

For companies operating in tightly regulated industries, like healthcare, finance, or anywhere that stores large volumes of customer personal data, using AI also has to account for the rules that apply. In Indonesia, there's Law No. 27 of 2022 on Personal Data Protection, which governs how personal data may be collected, stored, and processed, including when the processing involves a third party like an AI service provider.

Before allowing customer or employee data to be processed through AI tools, check whether the tool provider's data policy is a good fit, and whether there's a sector-specific rule that's even stricter.

A short list of mitigations

• Limit what kinds of data can go into public AI, and offer an official alternative (a team/workspace plan) with a clearer data policy.
• Put a human validation step on every AI output used for a decision or external communication.
• Apply an AI access SOP like the one covered in the previous article, so account ownership and work documentation don't stay tied to one person.
• Check the data policy of the tools you use against the regulations relevant to your industry. A framework like the NIST AI Risk Management Framework can be a starting reference for building a tighter AI risk management process.

In my view, being aware of these risks isn't a reason to stop using AI. It's a reason to use it more deliberately, with a clear process, instead of haphazardly.