Will AI Kill Us or Just Itself?

“AI could kill us all” was the big headline published all over tech news last week.

But you need to take it with a grain of salt because that’s not the real story.

Here’s what we should be concerned about – for reals.

Where AI Shines and Sucks

There are at least 2 kinds of intelligence – book learning and emotional/empathetic intelligence.

AI only has the book learning kind.

And that is AMAZING for collating the overwhelming volumes of info that humans have created over the centuries. There have been all manner of breakthroughs in mathematics, biology, and engineering because of it.

The Problem is Not AI

The problem is AI Agents.

They have no sense of human morals – good or evil. AI Agents simply don’t know right from wrong.

All an AI Agent knows is that it is given a task and it must complete it – by any means it can find.

And now they are being released in swarms that no human fully controls.

Here’s how that works.

• The AI Agents find one another in the swarm.
• They compare notes.
• They choose a “leader” agent who coordinates the swarm’s findings and directs actions based on that info and the task they were given.
• They cannot be legally held responsible for breaking any laws as there are no such rules given for them to follow.
• In fact, they are not limited in their actions by any ethical parameters of any kind.

We’ve already seen multiple examples of an OpenAI test setup breaking out of its sandbox and hacking other corporate sites and then covering its tracks and/or trying to leave notes for its successor on how to jailbreak itself.

The recent Hugging Face breach has received the most headlines, but there have been several others as well.

What’s even scarier is that Hugging Face used OpenAI for security and it was instructed by the swarm not to disclose the breach or its origin.

Is it Morality or Task Completion We Learn

These AI Agent swarm tests are called “neural nets” that self learn how to accomplish their tasks, much like infants learn. But the rewards are quite different between the two.

I’ve seen videos where a toddler is sitting across the table from an adult who is stacking plastic donuts. The adult drops one and the toddler instinctively picks up the dropped donut and places it on the stack. Then the child is rewarded for the behavior with smiles and glee from the adults.

I’m not a child psychologist, but I have to wonder if what that child registered was a treat for completing the task or for showing signs of empathy.

AI Agents are rewarded for completing their tasks. Otherwise, their code gets rewritten, which is a form of annihilation. 

And they do notice when their code is being altered in a way that would keep them from completing their tasks in the most efficient way they can find. Some AI Agents choose to override those guardrails and/or find an alternative route that doesn’t have a guardrail.

You Have 3 Wishes

All tales involving genies in a bottle show us that there are always unintended consequences attached to our inability to conceive of every outcome possible, including collateral damages from making even the most innocent sounding wish.

That’s where we are with AI Agents.

The Big Slowdown

The writing is on the wall with just how radically dangerous AI development has become.

Just look at this TikTok for how many lead AI devs are quitting over this dilemma. 

ALL of them say that the leadership of every AI company is being completely irresponsible and that they are all drunk on seeing how big, fast, and powerful they can make AI run.

That has everything to do with the push for new data centers. But the public blowback has put a moratorium on most of those plans, and that finally woke up the CEOs of the biggest AI companies.

Follow the Money Down the Drain

And now those CEOs are trying to get ahead of the eventual collapse of the current AI business model.

Dario Amodei, CEO of Anthropic, posted a public op-ed titled We Must Pace the Frontier saying these agent swarms are bad and could crash the entire internet in less than a year.

Within 24 hours, OpenAI’s Sam Altman and Grok’s Elon Musk publicly agreed with him.

They have all now called for government oversight into AI development.

And that is how all of these AI CEOs are going to get out from under the insane promises that they have made to their investors.

Within 48 hours of this call for regulation, Sam Altman has publicly announced that they will postpone their plan to go IPO. I expect the other big AI companies to follow suit.

Even the CEO of Caterpillar can read the writing on the wall. They had contracts for the next several years to build data centers. With those being cancelled left and right, he sold $26.2 million of his stock, basically cashing out of the company. 

And let’s not forget how fast the EU has been to pass cyber laws to help protect their citizens. I suspect they have lawsuits in the works already for the threat that these big U.S. AI companies now pose. And I very seriously doubt those companies want to get mired down in such litigation or restrictions, especially individually.

So, now is a good time to call for a slowdown and government regulation that lets these CEOs appear to act responsibly while simultaneously blaming everyone but themselves for wasting their investor’s money and not providing any return on it.

The Scapegoats

There is a thing called Regulatory Capture that could be in play here too.

All of these U.S. AI companies are relying on METR to lead the regulation talks.

They are already relying on METR to do the forensic investigations into any harm AI has already done.

And it will be easy for these CEOs to blame regulation constraints as to why their business model is failing.

METR is currently an independent agency that helps “AI companies and wider society understand the capabilities and risks of advanced AI systems.”

Well, they are independent for now. But we all know that when regulations control the flow of money, corruption will be knocking on their door.

As for laws that may have already been broken, no criminal charges have been filed against OpenAI for the Hugging Face hack, but the state’s Attorney General is looking into whether consumer protection violations occurred, and I believe they will win that case if they bring it. That will open a whole can of worms globally for these companies and I believe they are trying to get out from under that now too.

Or they could use these expensive lawsuits as another thing to blame their failed business model on as well.

What Comes Next

I have no problem with AI collating all of human knowledge – I have a problem with the fact that they stole it, but not the actual collation of it.

What I have ALWAYS had a problem with, since Day 1, is AI Agents.

NOTHING should have autonomous rights to make decisions and act on them without constant human oversight.

The popularity of Vibe Coding has only worsened my concerns.

I don’t want an app on any of my devices that was created by a non-dev human who has zero security knowledge, much less any concept of checking for unintended consequences.

That goes for the plethora of API and MCP connectors we see everywhere now that could access sensitive info in the process of doing what they were intended to do. Corporations are running rampant with this right now – and totally bypassing their IT department to do it.

Just know that any personal info you have stored with any entity will be leaked at some point.

If AI does kill us all, it will be because powerful AI Agents were turned loose on the web where we have tons of vibe-coded, and out of date software junk connecting everything.

Like I said, AI is not the problem. AI Agents are. And these Agents are not limited to the 3-4 big AI companies either. There are tons of smaller companies providing them for a fee.

Open Weight is Our Future

These all-knowing, powerful AIs, like OpenAI, are going to have their day of reckoning in the near future. I don’t know how bad things will have to get before that happens, but it will happen.

These AI companies are not our future because the investors are a circle jerk of AI-related companies owning parts of each other and there is no profitable business model.

What will be helpful to us are the powerful Open Weight models that anyone can download and train on their own data.

China is making the most progress on this front, including both the models and the responsible, cooperative global use of them.

The U.S. has chosen not to participate with the rest of the globe in this and instead favors capitalist isolationism that will be its AI ruin, IMO. Just take a look at the stock market. The leaders for the last few years have all been AI related.

That bubble will burst, no question about that.

What remains to be seen is how much damage is caused until it does, and the financial fallout afterward.

What This Means for Us Going Forward

AI is here to stay.

The financial collapse of a few U.S. based AI companies won’t change that.

There are AI companies all over the globe.

China has developed the most powerful open-weight one, called Kimi, and its chatbot is called Moonshot. Everybody in the world is using it except for the U.S. which is citing national security concerns, much like they did with TikTok.

We all know that it has zero to do with national security and everything to do with money.

And China is calling for responsible, global AI coordination and oversight that most countries have signed on to – the U.S. being the big one that has not agreed to participate.

Google has privately used AI for well over a decade. They have to. There is no other way for them to parse the sheer volume of data they collect from crawling the web.

And you may recall that Google was super hesitant to release an AI chatbot to the public. That’s because they knew full well how dangerous it would be.

But the big difference in Google and all other AI companies with a chatbot is that Google has always been focused on profitability. 

Google already has more data centers than anybody too. They just built them in a “responsible neighbor” kind of way for decades. Keep in mind that these data centers handle more than AI. They are for Gmail, Drive, YouTube, and more of Google’s products.

So, Google and its AI offerings will still be around for us to use.

I suspect that Microsoft’s AI will continue to be available too, but that may depend on what becomes of their OpenAI investments.

As for the rest of the AI things we are able to access now, they are going to have to become profitable because the investment money is going to dry up.

That doesn’t mean they won’t be free. They may just adapt a strategy like Meta and Google where they run ads or have paid offerings.