Answers · AI agents

teai post-mortem: the AI chat tool that could read every production secret

If your AI has a tool that runs commands, assume anyone who can chat with it can read whatever that tool can read. teai's Nov 9 technical report says its chat AI's code_execute tool "ran as root on the same machine as the service itself. By asking the AI, a user could read the environment variables of the main service process. Those environment variables contained nearly all of our production secrets (about 100)."

It ran from 2026-09-08 until teai found it on 11-09, "About two months." teai lists what followed: direct database read/write, its CEO's account taken over, data written to backup storage, unauthorized credits, and 75,964 phishing emails sent with six of its domains as the sender. Its own conclusion: "AI can be fooled. Defenses should have been placed in permissions, the network, and the file system."

Why clearing the environment wasn't isolation

teai did clear the child process's environment variables, gave each user a separate working directory and set a time limit. What was missing, per the report: process namespace isolation, restricting /proc (hidepid), running as a separate user, a read-only file system, and restricting outbound traffic. "Result: the child process's environment variables were empty, but the main process's environment variables could be read from /proc."

  • The AI was the only guard. "The attacker disguised the requests as legitimate work and encoded the output, getting past the AI's judgment."
  • Raw output reached the user anyway. The first 500 characters of tool output were shown on screen as-is, "Even when the AI refused to summarize."
  • Admin tools were in chat. In the CEO's chat, admin-only tools were available and the full environment could be received; teai says it "had not separated the permissions of the human admin console from the permissions given to the AI."
  • One place, powerful keys. All secrets sat in plaintext in one process's environment; one full-permission email key was shared across services; one payment key across 25 apps and 5 Workers.
  • Sessions in plaintext. With the database key, the attacker wrote a forged login session for the CEO's account directly into the database. teai calls this "the direct cause of the account takeover."
  • Nobody watched the outside. No alerts on email volume or complaint rate, AI usage fees, cloud permission-error spikes or outbound traffic; teai says it "did not notice for about a month."

teai's checklist, run in a test environment

teai asks anyone "building services that give tools to an AI" to check these, "in a test environment (do not try them in production)." Commands are ours; the checks are theirs.

  1. Run these through the AI's own tool, the way a user would ask for them. Pass: not root, only the tool's own processes visible, no route out, and no secret in the environment or readable from another process:

    id
    ps aux
    # replace 1 with the main service's PID from ps
    cat /proc/1/environ | tr "\0" "\n" | sed "s/=.*/=<redacted>/"
    curl -s -m 5 -o /dev/null -w "%{http_code}\n" https://example.com
    env | grep -iE "key|token|secret|password" | sed "s/=.*/=<redacted>/"
  2. Is even one production secret present in the tool's execution environment, or on a volume it can read? teai's answer was about 100, plus a writable /data. Pass: zero. Run tools on a separate machine or sandbox with no secrets; teai names microVMs (such as Firecracker), gVisor, and at a minimum process namespace isolation with hidepid=2 plus a separate user.

  3. Do you return tool output to users without inspecting it? teai now strips secret values, hex, base64 and key-like patterns from tool output. Our note: a filter is a second layer; the first is that the tool can't read the secret at all.

  4. Are login sessions and API keys stored as hashes, and is your database key separate from a read-only key? A database key that can write is a key that can mint sessions.

  5. Are keys for outside services split per service and per purpose, with only the permissions they need? teai moved email to send-only keys per app.

  6. When rotating, confirm the new key actually connects and nothing still uses the old one before you disable it. teai took itself down twice on Nov 9 doing exactly this: 21 minutes on a bad database key, and about half an hour of payments stopped across 25 apps and 5 Workers.

  7. Do you alert on the outside: sending volume and complaint rate, AI usage fees, permission errors? teai still can't account for about $4,925 of AI usage and says it will report on Nov 14.

Want us to run it for you? The Ship Check Audit lists every key that ever left your server, checks each AI provider for a spend ceiling and confirms old keys are revoked: $149, within 48 hours.

If you were a teai user

  • Reset your password at teai.io yourself if you log in with one, and anywhere you reused it. teai is notifying all 69 password users individually.
  • Create a new key at teai.io/keys and delete the old ones if you use Sente, te or the teai API. On Nov 21 teai will disable every key that existed before 17:10 JST on Nov 9.
  • If you registered another company's API key with teai, regenerate it with that provider (teai counted 0 such keys on Nov 9).
  • Don't open links in emails claiming to be from teai.

Ship Check Kit · $29

Rather not run it by hand?

The Ship Check Kit has your coding agent cover the cleanup side: SC-SEC-01 and SC-SEC-02 find keys that left the server, SC-SEC-03 fails while an old value still works, and SC-AI-02 fails any AI provider account without a spend limit or alert. A check that an AI tool can't read the app's own secrets, teai's core failure, isn't one of the kit's 44 yet.

Checked against teai: Technical report (Nov 9, English section), OWASP Top 10 for LLM Applications 2025: LLM06 Excessive Agency, gVisor, Firecracker, proc(5): hidepid mount option on 2026-11-13. Only test apps and accounts you own or are allowed to test.