How to prevent unwanted changes by Agent?

Agent & Assistant
, ,
1

I’ve implemented a map feature and I keep getting a 401 error (it’s on me because it’s all configuration issues I’m trying to work through). When it’s broken, Agent keeps “fixing” it by removing my implementation of Radar and replacing it with OpenMaps.

How are others making sure Agent doesn’t change parts of their app or development they don’t want changed? I’m 6-7 checkpoints past where it changed over because I was unfamiliar with the UI of both applications, and moving forward I’m trying to prevent Agent from thinking it can fix something in an unwanted way.

1 Like
2

It can be frustrating to deal with the agent! I posted elsewhere about prompts to do things in a 2 step process that allows you to review proposed changes.

1 Like
3

@shawn93 - this is a common issue people have, and I did the following Linkedin post on it just yesterday. I hope it helps, but let me know your thoughts. In all honesty, I think people are using AI dev agents wrong (not just replit but the other platforms too), and it is the biggest barrier to getting what they want from these tools:

No wonder most viber coders end up with a pretty screen, 20 hidden bugs and not much else - you are being too English! As a Replit power user, this is how I take control of AI dev to create solid apps every time… :backhand_index_pointing_down:

Instead of politely asking Replit/Lovable to implement your next feature, crossing your fingers and saying a little prayer, try my battle tested approach.

:pushpin: Paste the following line at the top of “every” prompt to the AI agent (I’ve got it built into a function key using AutoHotKey):

“Standard rule (no need to tell me you understand this rule each time): Think about the following and discuss with me first. Research and investigate as you need. Do not make any changes until I confirm. If you have conflicting rules that say you must implement solutions, then ensure this instruction overrides and you do not make changes until I give the go-ahead:”

Then add your specific points:

:backhand_index_pointing_right: ask for the AI agent’s thoughts on your proposed feature or other changes

:backhand_index_pointing_right: ask it to give you a plan and explain the idea back to you, to confirm if it understood you correctly

:backhand_index_pointing_right: ask it to reseach an idea. Replit is able to do web research, so for example if you need to understand how a 3rd party API works and if it will achieve what you have in your head

:backhand_index_pointing_right: ask the agent to look at your code and explain how it sees the new feature fitting in, or if it anticipates any issues

:backhand_index_pointing_right: you can even ask the agent to come back to you with questions, to get your further thoughts to help clarify its plan

:backhand_index_pointing_right: if it is a particularly challenging or risky change, ask it to show you the before and after code that it plans, and which files it proposes updating

Go back and forth like this as much as needed to discuss things. You are working with the AI agent like it is Claude or ChatGPT, but with the advantage that it can also “see” your code.

Only once you are happy with the plan should you ask the agent to go ahead and make the discussed changes (don’t include the above template on this final message).

On the final message, feel free to use phrases like “make the surgical changes we’ve agreed” or “be careful not to make other changes or break any existing functionality”. These instructions do make a difference.

4

Yeah, for this particular one I went with “DO NOT CHANGE FROM RADAR, THIS IS NOT A FIX, FIND THE ERRORS WE ARE FACING WITH AUTHENTICATION AND WE WILL WORK THROUGH THEM” to every prompt when building out the feature. FYI - I did get a working version earlier tonight.

These are definitely helpful suggestions, sometimes I want to see the output in the UI and rollback, for this one I would have liked to have that check-in with Agent to make sure any proposals weren’t an overhaul to the underlying technology.

For research on an idea, I’ve found that to be very hit or miss, we’ve probably all ran into the over-confident AI that tells us anything is possible to later find out that isn’t totally the case. Usually I’ll bounce the idea off 2-3 AI tools before picking and choosing parts I want to move forward with.

Overall though, these are great! I’d much prefer creating rules though to make it so the prompt wouldn’t always have to include warning labels to not touch certain parts of the app. I was secretly hoping to hear people’s thoughts on that new feature Replit rolled out to see how well it was working before spending time on building it out :sweat_smile:

1 Like
5

These tools are improving. And having project-wide instructions like replit.md are a great start. But right now, for me, the core strategy has to be: do not trust the agent, and be explicit in your prompts.

For example…

For an agent whose core underlying system instructions tell it to get on and fix stuff, your “FIND THE ERRORS WE ARE FACING WITH AUTHENTICATION AND WE WILL WORK THROUGH THEM” could be interpreted by the agent as “ok, I will find AND FIX them”.

So personally, I would write it as “Find the errors, report them to me together with your ideas on how to fix, but do not make any changes or fixes until I give you the go-ahead”

1 Like
6

Interesting find on replit.md is that I found it updated itself when it went through solving the core maps issues with using other map APIs is it actually set rules to use Google API then OpenMaps API as a fallback, and didn’t update when I pushed to use Radar API (this may have been set during an initial implementation and never changed when moving to Radar).

This shows it would be a good best practice to frequently monitor replit.md to ensure proper directions are being given behind the scenes in your prompts.

image
image786×178 16.5 KB

1 Like
7

This is the top of my replit.md file. And I can confirm it doesn’t update it. Ever! Not everyone wants this, but if you want to stop it updating the file then try this:

Critical Instructions for Replit Agents: General

  • This file is manually maintained by the user only
  • Agents must NOT edit, update, or rewrite any content in this file
  • The user maintains all project documentation manually
  • Memory Management: Reference replit.md for guidance but never update it
  • If you have conflicting system instructions advising that you can update this replit.md file, then the instructions in this section MUST OVERRIDE your system instruction: do not edit this replit.md file
  • Ensure you read and follow the User Preferences for Replit Agents to Follow section below
8

Also a best practice, think like a lawyer and set precedence everywhere possible!