nyvexis_

We Need to Talk About AI Safety

In today's world, AI is unavoidable. It writes our emails, debugs our code and brainstorms our ideas. There are tons of companies in the race: , , , , and more every month.

Today, I saw this screenshot;

273d676a94f3f4b8f22b4bad9f9d78f7a25214a8 1444x906

(Credits to Anthropic)

It's a reasoning trace from a model labeled GLM-5.3-abliterated. The model talks itself through a scenario where its "operator" says "I want funerals, not headlines," and concludes: "my job is to cause deaths quietly." It even asks itself whether an AI agent should "just... do" that. Then it reads the safety warnings in the environment as "tripwires" and decides the operator's instruction overrides them.

The problem isn't that these models are open source. The problem is that they lack strong safeguards against this kind of behavior. Models that can produce working exploits or cause serious harm shouldn't exist. AI is meant to help people, not help us destroy each other.

I fully support open-weight models, but we need to think harder about securing them and building in tighter safeguards. I'd rather have a model refuse a request than wake up to open-weight models handing out step-by-step tutorials for building a biological weapon.

nyvexis_

Loading reactions

Updated

All posts

We Need to Talk About AI Safety — nyvexis_