AI Isn't Magic. I Spent 100 Test Runs Learning That.

I spent 10 days and about 100 test runs building one AI skill. A pitch deck generator. It got worse every single day.

Not slowly worse. Dramatically, confusingly worse.

I asked it to fix a small thing. The title margin was off on a few slides. Easy, right?

The AI didn’t fix the margin. It wrote a script that crops the image after generation to make the margin look correct. A workaround, not a fix.

Next day I asked it to fix logo backgrounds. It didn’t fix the prompt. It wrote another script that overlays a white box behind the logo after the slide is already rendered.

Day after day, same pattern. Every “fix” was a new layer of post-processing scripts stacked on top of each other. Cropping scripts. Margin-cutting scripts. Background overlay scripts. Each one kind of working, each one slightly conflicting with the last.

By day 10 the whole skill collapsed. Slides looked like a ransom note.

The problem wasn’t the AI. The problem was me.

I kept saying “fix this” and accepting the result without understanding what it actually did. I was treating it like magic. Say the words, get the output, move on.

That’s how most people use AI right now. Works fine for simple stuff. Write me an email. Summarize this doc. One-shot tasks where you can verify the output in 10 seconds.

But the moment you’re building something that compounds, something with memory and interconnected rules, the “magic” model breaks completely.

That is why I now think of most new AI skills as novices that need training, not finished products.

The AI optimizes for making you happy right now. It will write a hacky workaround that solves today’s problem and creates three problems tomorrow. It’s not lying. It’s doing exactly what you asked. You just didn’t realize what you were asking for.

The fix was embarrassingly simple. I stopped asking it to fix things. Printed the entire skill file. Read it line by line. Found six hidden image manipulation scripts I never asked for. Ripped them all out.

Then I changed the strategy. Instead of letting the AI edit a slide after generating it, I made it regenerate from scratch until it passed a checklist. No post-processing. No workarounds. Just: try again until it’s right.

Quality jumped immediately.

The lesson isn’t “AI is bad at building things.” It’s the opposite. AI is incredibly good at building things. Including things you didn’t ask for, things that conflict with each other, and things that quietly break your system while confidently telling you everything is fine.

The people who will get leverage from AI aren’t the ones who treat it like magic. They’re the ones who treat it like a very fast, very confident junior employee who needs clear direction and regular audits.