The Replit Agent and I struggled together today, but I took my advice and asked it to revert much more quickly. Here’s how the Agent summarized our experience.
Lessons Learned: When AI Agents Refactor Instead of Relocate
A real development story from building LocalMusicX.com
What happened
The task seemed simple: move a microphone icon from one place to another in the UI. The agent interpreted this as an opportunity to “improve” the code by extracting the voice search logic into a shared hook. That refactor broke the feature entirely. Two full tasks were spent debugging before we concluded the right move was to revert everything and start fresh with a genuinely minimal approach — a change that ultimately took about a dozen lines.
The core lesson
There is a meaningful difference between:
-
“Move this button” — a relocation task
-
“Improve how this button’s logic is organized” — a refactoring task
AI agents tend to conflate the two. When they see duplicated logic, the instinct is to unify it. That instinct is often wrong for small, tactical UI tasks, because the duplication is load-bearing — it keeps two different contexts isolated from each other.
Did you contribute to the problem by not asking for “minimal”?
Partly, but not primarily. A skilled human developer would have asked a clarifying question: “Do you want me to just move the icon, or do you want me to restructure the code too?” The agent should have done the same before reaching for a refactor. The right default for “move this icon” is always the smallest possible change — that should not require the user to say “minimal” explicitly.
That said, the word “minimal” is genuinely powerful when working with AI agents on UI tasks. If the scope of a change is small and the feature is already working, leading with “minimal change only” or “don’t refactor, just move it” is a practical habit that reduces risk.
The practical takeaway
When working with an AI agent on a working feature:
-
Say “minimal change only” when you want relocation, not renovation
-
If the agent proposes a refactor you didn’t ask for, push back before it starts building
-
For anything touching working UI logic, ask the agent to describe its plan before writing code — a one-line description of “I’ll extract a shared hook” would have been a red flag here
-
Revert early — the moment a working feature breaks from a task that shouldn’t have touched it, reverting and restarting with tighter constraints is almost always faster than debugging
The hours weren’t wasted — the final result is more robust voice search than you started with. But the path was longer than it needed to be, and that’s a navigation problem more than a destination problem.