I think one of the biggest complaints I see online about the agent is that it sometimes ends up in a recursive loop, burning through all the credits while trying to fix a bug it created by attempting to fix another bug. While I understand that the work has still been done and someone has to pay for it, there are times when I think a refund could legitimately be issued if a freak bug appears or a random agent error occurs.
I’m wondering if there could be an AI-monitored appeal system to determine whether a particular rollback qualifies for a refund of the credits used up to that point.
I think this would ease a lot of the frustration I see online.
It can be a little soul destroying when the assistant keeps failing on something that seemingly is simple in comparison to the rest of the work it had already completed previously. There is definitely a chunk of edits it’s made for me where it has not provided any value, and actually instead put my application backwards.
I feel like if it gets stuck in a constant loop trying to solve something seemingly simple, there should be a consideration for it, perhaps a way for a user to report it so that it’s reviewed and subsequently contributes back into improving the model to not make the same mistake for others in the future.
It would be amazing if a user spending hours of in a loop with an agent decides to rollback, these units would be restated. A rollback would obviously indicate that something went wrong. Nobody wants to spend many prompts (and time) and then rollback. That way incentives between users and Replit getting paid would be aligned.
I think the issue is that the compute for this isn’t refundable, so if the company is paying the compute for this, it would eventually bury them. We are kind of in a limbo period because we are playing with experimental software knowing its experimental. I would hope that our errors help train the models in the long run but who knows how they are handling all this data. I have spent over $180 in the past 24 days and I just roll it into my running costs for the business, its no different than hiring a new employee and there is a learning curve before a lot of the procedures are ironed out in the system before you can start turning a profit, there are going to be errors that cost money and its just the cost of doing business, I think it will be the same with this. What an acceptable number of errors is, is to be determined, companies pay millions of dollars in developers making mistakes all the time and we don’t ask for those wages back as businesses, its just part of the process. These models will improve fast, and in 6 months it will be way better, and in a year it will be unrecognizable most likely.
I just feel like even if you hire a new person and they come in and destroy your entire database there is another way to recoup your money like you can sue them. You don’t still have to just pay them for the month that they worked even though they “did some work” But I understand that it would be very complicated to work through the policy on this.
Well incompetence is different than just normal programming, then there is a bug, and you chase the bug for 1-2 months, you are paying several guys to fix that bug and if its new software it can’t launch until that is fixed if its a integral part of the system, but I know what everyone is saying, trust me I don’t like knowing I blew through credits with no results, I do this is video generators all he time and way back with first versions of image generators.
I think this is a huge issue. An agent in a fruitless loop is printing money for Replit, and forcing customers to pay for it is not a healthy feedback loop. We are in early days, willing to waste our time in the hopes that someday this “bug” will be minimized, so Replit should pick up the tab, 1000%.
Yup, exactly what happened to me . Stuck in a loop because of a Expired SSL certificate the agent tried to patch but couldn’t .
40% of my monthly Check points down the drain.
the problem i faced when i was in the loop is that i had a working condition before it went in a loop and it couldnt figure out how to go into a working state even if i ask it to go back. This last time I had to rollback 3 days ago to get it back to a working state. My learning curve was $10-15 extra cost. Getting that credited would be nice.
my other complaint is when i come back the next day to continue and it has a prompt for me to check out my webview, its not running. so when i tell it to start it up or refresh, it cost me a credit.
This is a great idea! I had several situations where the bugs didn’t go away despite multiple attempts… and each attempt of course cost money It can get a bit frustrating, and an AI-monitored system could automatically handle the situation because of all the live telemetry and logging. This will also significantly boost goodwill for Replit as a company and bring even more users from other, similar products over to Replit.
It can get a bit frustrating, and an AI-monitored system could automatically handle the situation because of all the live telemetry and logging. This will also significantly boost goodwill for Replit as a company and bring even more users from other, similar products over to Replit.