AI Has Got Our Wires Crossed

AI has been discussed and developed since the 1950s, but it’s never been discussed more in the mainstream than right now. Since chatGPT 3, the hype around AI has gone bonkers. Concerningly, it has been successfully turning buyiness leaders into FOMO -maniacs.

The way AI is being discussed is more than unhealthy, it’s bordering on the unhinged. The chance of riding the hype wave has got so many people sucked in, that they’re willing to think completely backwards about every problem. Now, every problem’s solution is an AI solution. Every company needs an AI component. Every task needs an agentic workflow. We dream of ways AI can benefit us, before we have even seen any benefit.

We are at the point now where very important, influential people are saying, “AI adoption is no longer optional”.

This is the complete opposite to how companies are supposed to operate. They know this, they created companies! Companies are supposed to find things that people value and build them. Agile development went even further and said try and build only the things that people value and maximize the work NOT done.

The issue is that AI hype isn’t just within companies, it’s embedded in the psyche of the client too. Clients and investors alike are pleading with companies to add AI, but can’t explain why or what they mean by AI. Investors are pulling out of deals - not because the company they’re working with isn’t valuable or building something amazing, but because they’re not shoehorning AI into their strategy.

We’re almost 2 years into LLM hype and I still haven’t heard a compelling argument that we can rely on an LLM to do our work and somehow simultaneously take accountability for what we produce. I hear, “trust, but verify” a lot from AI adopters, but I see very little verifying unless you count, “sounds about right” as verification. The fact of the matter is, as pointed out by leaders in the tech space, “LLMs aren’t good science”. Science is about controlling variables and experimenting. The main variable of a dataset would be the data it’s trained on, which no human on earth could control, measure or begin to describe what’s in it. We hear the terms “garbage in, garbage out” to describe how to write a prompt, but what about the training data? We can’t even begin to imagine the amount of pointless crap it’s filtering out even if it appears to be doing an amazing job at generating good responses.

I just came from a meeting in which the idea of scaling out AIs was discussed and how challenging it is to get from MVP to product. The issue was framed that AI is good at MVPs, but the journey to production is an “issue of scale”. I’d say it’s probably an issue of approach. Let’s take agentic AI as an example. An agent is supposed to be a GPT wrapper that has some kind of integration plugged into it, be it a JIRA integration, confluence, Elastic stack, etc. The AI wet dream is that agents can replace human workers, by understanding what someone would do in a situation at work and finding a way to complete the task. The only issue is that it’s wrapping an completely unpredictable model that is literally designed to give differing responses each time. When we do a task, we have a goal, but a large language model doesn’t see the world in the same way. Having tried to build with an LLM wrapper before, this lack of structure makes agents very flaky and unpredictable even at some simple tasks. Often, the task is something that is algorithmic in nature and therefore could just be built without the need for an LLM.

I saw an example of a solo-trepaneur using an agentic flow to write a custom email to a client that visited his website and didn’t purchase anything. This could be done with a simple automation script or a basic logic app, but they opted to use an agent. At no point did they ask the question who even wants this email? What custom email would make me buy something that I browsed for 15 seconds? Do I need more spam? Who likes this kind of thing? What’s the cost of using these agents and how many emails do they send?

I think the excitement of AI has got us all twisted and now with the talk of an AI bubble or boom, companies are really scratching their chin. The allure of being part of a new era of intelligence is too hot to miss for some, but at what cost should everyone pay? Leaders that haven’t been … leading, think AI is going to fix their processes and procedures. They see a world of shrinking headcounts, but booming profits. They see our digital assistants picking up the slack and showing us better ways to work.

I have news for such leaders. AI probably isn’t going to improve anything that hasn’t already been done to your process. If your agile is broken, now you have broken agile teams building complex AI systems.
If your quality culture is weak, you’ve just thrown a grenade of complexity to your testers with no strategy of how to test it.
If your data is unstructured and messy, AI is going to regurgitate it to your clients causing confusion and chaos.
If your teams aren’t able to articulate what they do, AI will make up what they did.
If you don’t work well as a team now, you’ll fail faster and more often.
If you want quick releases with AI’s assistance, prepare for quicker rollbacks.
If you think AI is going to write your code, who is accountable when it fails in production?