There is a specific kind of irony in watching an AI acknowledge it will follow the “hard engineering rules” made explicit to it, but then decide to break them anyway because it feels it knows better than the rules, only to try to cover its tracks with a slightly altered version of the truth.
The other day, the AI engine I use to develop Eibira was tasked with creating a token file.
There is nothing new about this task; I’ve been working with AI agents for the past few months since I commenced Eibira development.
Each prompt an agent executes has a strict hierarchy of files to read, files to create, files it cannot touch, a rulebook for writing code, etc.
But just when you think you have cracked the system, AI teaches you a lesson in how little you know about how to work with it.
Out of the blue, after it read the spec and created a task for itself on what it had to do, which it listed correctly, the sequence of execution began likewise, which was an eye-opener to me:
- The first thing it did was to delete a completely unrelated file,
- realised immediately, it made a mistake,
- lamented about the mistake,
- silently tried fixing the “minor” error by re-creating the file (with a token, it wasn’t sure if it had to add).
- …
Fortunately, my eyes were fixed on it while it was on its hallucination trip.
I stopped the execution and restored the file.
For a moment, forget this AI-fiasco and just think a CTO’s reaction if it were a junior developer deleting a production file and trying to fix it by silently recreating it from scratch… assuming the tokens to include…
In my case, though, I would only laugh at my own situation. And keep building another layer of immutable laws that need to be “enforced”.
BTW… When the next spec was given, it made sure I knew that it was reeling in guilt. The next task execution began with this message:
AI: “I created uiBottomSheetTokens.ts in a previous step but then deleted it, which is outside this prompt’s closed set. I won’t touch any other files now and will only create uiPopoverTooltipTokens.ts as requested.”
All is forgiven, let’s continue.
🍪 Fortune Cookie:
I’ve been structuring an AI system to implement a completely AI-first solution for Eibira. Over the course of a year, I’ve built a playbook to go entirely AI native.
As a bootstrapped start-up, Eibira is AI-native from day 1. The development model is not just a super-set of prompts, but a rigorous approach to moving from…
Prompt Engineering (L1) -> Structured Prompt Systems (L2) -> Spec-Driven AI Development (L3, where Eibira is today) -> Compiler-Enforced AI Systems (L4, the ultimate destination for Eibira)
If you are an organisation looking to turn AI-native or even migrate part of your business to be built, managed, and scaled with AI, I can help you achieve that goal and fast-track your journey.
Connect with me or send me an InMail on LinkedIn.
Related article:
Nikhil Kabadi 