Good Morning Reader it's Maria,

Long time no see.... I've just returned from two weeks holidays in the South of France, where I totally disconnected from all things digital. So much so that I totally forgot to post my newsletter last week. OOPS!! 🙈

I'm back on track now, and this week we're talking about the AI failures, the "work slop" nobody wants to admit to, and how AI can cost businesses way more than it's saving them.

If podcasts are more your speed, I've got you covered; there's a discussion on this topic available now here.

THE BIG IDEA

I need to tell you about Alvi Choudhury.

He's a 26-year-old software engineer, living with his parents in Southampton. One day, out of nowhere, the police show up at his door, handcuff him, and hold him for close to 10 hours before releasing him at 2 am.

The reason? A burglary, 100 miles away, that he had absolutely nothing to do with.

Thames Valley Police had used an AI facial recognition software, and it matched him to CCTV footage of the real suspect.

Except here's the crazy part: the guy in the footage looked nothing like him. He was clearly younger with different features. The only thing they had in common was curly hair... What the hell???

And it gets worse.

This mistake was only possible because of an earlier one.

Alvi's mugshot was sitting in the police system from 2021, when he'd been wrongly arrested (again) after being attacked on a night out at uni.

It should never have still been there, but it was. So when the algorithm went looking for a match, his face was just... available.

Nobody woke up that morning planning to ruin an innocent man's day. But the thing I want to attract your attention to Reader is that a system said "this is a match" with enough confidence that no one questioned it or even double-checked it.

If they had, they would have seen straight away that Alvi did not match the suspect at all! It also took them 10 hours to realise it!

Around the same time, the legal world was having its own version of this.

Since a New York firm first got caught in 2023 submitting fake case citations invented by ChatGPT, courts around the world have logged over 1,200 similar incidents.

One appeals court sanctioned two attorneys who filed briefs across three separate cases containing more than two dozen fabricated citations, cases and quotes that simply didn't exist. Each of them was personally fined $15,000.

In the first three months of this year alone, US courts handed out over $145,000 in sanctions for AI-generated fake citations.

Two different industries but the same failure: Humans did not check AI's work.

These are extreme cases, but something very similar is happening in offices everywhere, right now, at a much smaller scale. The stakes are lower, sure, but the consequences can be just as damaging: to a business's reputation at best, its bottom line at worst.

There's a name for this: work slop.

Researchers at Stanford and BetterUp Labs coined it last year to describe AI output that looks finished but needed so much correction afterwards that it barely saved any time at all. The research showed that people who receive it spend close to two hours fixing it, and about half of them start seeing the colleague who sent it as incompetent.

They put an actual number on that lost time too: work slop costs the average worker close to $186 a month in wasted hours. Scale that up to a company with 10,000 employees, and you're looking at roughly $9 million a year, spent fixing something that was supposed to save time in the first place.

Think of what it cost Alvi Choudhury and those lawyers. Their reputation and a lot of money.

Personally I've got a golden rule:

Anything going to a client, whether it's a proposal, a contract, an email, or anything at all leaving your business with your name on it, gets reviewed by a human first. No exceptions.

But the EU AI Act, which starts applying properly this August, goes further than my own rule.

It specifically flags certain uses as needing human oversight by law, not just good practice. If AI is helping you decide anything about a person: whether to hire them, whether to extend them credit, anything touching healthcare, education, or legal outcomes, a human has to be the one making the actual call.

The AI can assist, but it can't decide.

And there's a second category worth knowing about too: if you've got a chatbot or any tool that interacts directly with customers, you're required to tell them clearly upfront that they're talking to AI, (not hidden in your terms and conditions page).

The reason you should pay attention to both of these rules goes beyond ticking the compliance box.

Alvi Choudhury's arrest and those lawyers fake citations both come down to the same gap: a decision about a person, made by AI, with nobody required to check it. The machine didn't do it alone. The human decided not to review.

AI is brilliant. It's also confidently wrong sometimes, and that combination is dangerous if nobody's overseeing it.

My point here is that we have to maintain that human friction. Which leaves us with a really critical question to ponder, because these models are only going to get better at mimicking perfection. So as the output looks more and more flawless, are our human critical thinking muscles just going to atrophy?

If we rely entirely on machines to generate our thoughts, will we even be capable of spotting work slop five years from now, or will we just accept the machine's hallucinations as our new reality?

If we outsource the friction of thinking, we might lose the ability to recognise truth entirely.

So keep an eye on those outputs, protect your reputation, and keep those critical thinking muscles strong.

Don't know where to start?

​Book a free consultation and let's chat!

THE ACTION STEP

Give yourself 20 minutes this week and run two checks.

1. Is there any AI involved in a call that affects someone's life? Getting hired, getting credit, a health matter, an education outcome, a legal case. If so, a person needs to make that final judgement themselves, not just wave through what the machine suggests. That's the law from this August.

2. Do customers ever interact with something AI-powered on your site or in your messages? Make sure it's obvious to them from the start, not tucked away in the small print.

Beyond that, stick to the standard I use myself: nothing leaves your business with your name attached until someone's actually laid eyes on it first.

Note down exactly who takes that look and at what point in the process. A specific person, a specific moment. If you can't answer that right now, that's exactly what needs fixing this week.

Have you just signed up? See all previous newsletters here.​

AI MADE SIMPLE

Build your own AI decision map.

Paste this into your LLM (Claude, ChatGPT...):

Act as a compliance-minded operations consultant helping me map out my AI use. Don't give me a generic list of EU AI Act categories straight away. Instead, start by asking me questions, one at a time, to help me figure out where AI actually shows up in my business. Things like: what tasks I use AI for day to day, whether any of those tasks involve decisions about customers, staff, or applicants, and whether customers ever interact with anything AI-powered directly. Once you've got a clear picture from my answers, build me a table with columns for Task, Category (whether it falls under EU AI Act high-risk areas like hiring, credit, healthcare, education, or legal, or needs customer AI disclosure), and Required Check, a proportionate human review step doable in under five minutes. Then tell me which three items on that table are the most urgent to sort out before August, and why.

One thing before you run it: leave out client names, financial details, or anything confidential when you answer its questions.

Fifteen minutes later, and a bit of back and forth, you'll have a clear map of where you're covered and where you're not, plus a straight answer on what to fix first.

That's all for today Reader

Have a great weekend!👋🏼

Take Care

Maria

PS: If you want to explore what working together looks like, check out my website​

PPS: If you enjoy these emails and want to do something nice, you can buy me a coffee 😉

​​​