I tried out the new Extended Thinking plus High-powered models (combined) the other day. A single prompt was over $5. Serious Replit, who exactly is your market for this?
We want you to make money and not lose on each prompt. But, really?
And in the days leading upto the model switchover, I am certain the base model became less good. Anecdotal, but when you’re using it every day, you just get a vibe for these things. Something definitely changed if you stick to the basic level.
So anyway, I’ve found the Extended Thinking is probably the best balance of quality results and price. I would love to hear what others are finding?
5 Likes
I guess the followup question would be - was the output of that $5 worth it for what you’re building?
3 Likes
At say 1 prompt every 5 minutes, that’s $60 per hour. A decent developer would charge less than that to code by hand 
Sorry, but what was the advantage of AI dev again!?
Keen to hear other thoughts…
Hi,
may I ask what you had the Agent implement?
Was it something complex like integrating a new authentication system or just changing the color of a button?
probably something reasonably complex to add features to my app - so not just a basic UI thing. But nothing more than I was doing last week before the changes. I honestly don’t remember thinking “wow, that was such an improved response and piece of code”.
1 Like
It appears to be an option to choose the extended thinking version or use the basic (old) agent process. Am I incorrect?
Hey Steve. There’s a lot of users who want more powerful models and more thinking and are willing to pay more, currently the most expensive thing for it is the high powered model which puts you on Opus that out of the box costs 5x more than Sonnet.
We’re updating a bunch of things on our end to keep bringing costs down but when you turn on the big model it’s going to end up costing a fair bit more. Like you said for general use normally sonnet + thinking (slightly more expensive but not much and no where close to opus) will get you most of what you’re looking for.
There weren’t any major changes on our side around the core agent behind some base prompt improvements which should have just made it more flexible in tool calls, do you have any specifics on how it got worse or just the vibes?
4 Likes
I’ve also noticed quite a few checkpoints that are far less than the typical $.25
I think it will offset to some degree as long as you use the Opus model selectively.
I tend to use Agent for everything just because it’s streamlined and maintains better context. So, when I make a small change and it comes back, costing $ 0.11 instead of $ 0.25, that’s a win.
Additionally, it used to randomly insert checkpoints, so you’d give it a single prompt, and it’d ultimately cost you $1.00. Now, the same prompt may cost you $.85, but because it’s one checkpoint for $.85 instead of four at $.25, some people are uncomfortable. As with any change, it takes getting used to. I think users should reconcile their expenses at the end of the month, see what they built, and what it cost them; undoubtedly, the value is there.
2 Likes
I posted about this elsewhere, but right now, the vibes are off. I won’t know until the end of the month but like @realfunnyeric said, it’s going to be hard to compare the agent making five rando checkpoints as opposed to one that’s weighted higher. At this point, I will reserve full wrath until the end of the month, but it does feel a bit bait and switch-ey here
1 Like
It seems more powerful, just using basic agent now. Nobody likes change and this is a big one to pop out of the blue!!
1 Like
FYI, these are my Replit credit costs yesterday - on “extended thinking” mode.
32 charges, total $18.12. The problem is, this is completely opaque. It isn’t “per prompt”, it is “per a random number of prompts”.
And under the old model, would this have been 32 * $0.25 (total: $8)? Nobody really knows - but I’d be keen to find out if the comparison really is that simple @kody-replit?
I am not saying there was no value in what I got out of this - of course there is, it’s amazing what it produces. But as a viable long term model, we need more transparency on our costs, and ability to predict and plan.
$1.88
$0.83
$1.22
$0.70
$0.43
$0.04
$0.47
$1.88
$1.82
$0.72
$0.41
$0.57
$0.45
$0.38
$0.10
$0.86
$0.31
$0.46
$0.76
$0.19
$0.17
$0.38
$0.28
$0.00
$0.34
$0.55
$0.45
$0.33
$0.04
$0.70
$0.00
$0.40
total: $18.12
you can choose different models. but there’s no fixed $0.25 model - that has gone out of the window.
What did it produce for you??? Be nice to know as Replit will be earning $$$$ a lot faster as everyone rushes to try it out. Maybe i needed to put a few more $$ on the end of that !!!
But @kody-replit , $5 every 5 mins (approx round trip for me to type the prompts and get the results) is $60 per hour. That is a decent rate for an experienced developer (maybe not Silicon Valley, but the rest of the world definitely, including here in the UK). Agent is amazing (best thing I’ve found in 43 years of coding), but sorry, it’s not quite that good - yet!
Also, the high powered model is 5x the price. Yet your very own Matt Palmer said on Linkedin yesterday “This gives something like a 10-15% performance bump in solving complex coding challenges”. So 5 times the price for 10-15% improvement. Am I missing something here?
As per my other reply in this thread with my cost breakdown, the real problem for me is lack of transparency. It’s like walking into McDonald’s and instead of a price list on the wall, they just say “so give us your order, and randomnly every few items (but we can’t say how many items) we will throw out a price”.
PS, I accept I don’t have anything other than the vibes on thinking the base model briefly got worse. To be honest, I’ve been hammering Replit so hard to build my native mobile app without Expo, that I can forgive it occasionally getting stuck. It’s been a big challenge, and both the Agent and I have to keep having a lie down to take a break
1 Like
I did a very small test of the high-powered model @simon411, but I’m not really convinced it gave me much of an edge.
And Matt Palmer from Replit said on Linkedin yesterday “This gives something like a 10-15% performance bump in solving complex coding challenges”. So 5 times the price for 10-15% improvement. Not sure many average builders are going to experiment with that for very long.
Maybe we should all be offered the high-powered mode at standard rates for a month to see if it truly lives up to what it could be - an idea @kody-replit ?? 
This is what you’re missing.
You’re using a model that costs literally 500% more. Regardless of what it outputs, Replit’s hard cost on switching to Opus 4 at this time is a 5x increase, thus a 5x increase in price is to be expected. If you don’t need or want the 10-15% increase, don’t use it, as you’re right, it doesn’t pencil 1:1.
I am not missing anything. I fully understand replit’s cost base. The issue is why pay x5 for only 10% improvement. Like going into McDonalds and being offer a burger that is 10% bigger, but it costs 5 times as much. I think that is what you’re missing.
And I really appreciate you telling me if I don’t need it then don’t use it. Super helpful
Love the option to “bring in the big guns”, but even without the extended thinking or high-power models turned on it seems like the cost to use Agent has significantly increased. Haven’t done any robust analysis, I just feel like what cost me $20 to build before now costs $30-$40.
I just hope that Replit continues to find ways to bring costs down and shares those savings with users rather than eating them all up themselves. This is where misaligned incentives get dangerous…
1 Like
UPDATE: yesterday I spent $150 on prompts testing the high powered model on my app. Lots of $5 - $10 prompts. My biggest prompt? $18.76 
I would say it is great at research and resolving things the other models just keep failing on.
For example, I was having a nightmare trying to get the extended thinking mode (my default) to fix a problem - going round in circles. Probably only $10-$15 spent, but it was taking me hours to resolve, as each fix can be several minutes while you discuss it with the agent and then wait for it to do its thing. But the high powered model came at the problem in a fresh way, did some web research, delved deep and came up with the answer. So like using ChatGPT or Claude, but with advantage it can “see” your code.
Yes, the cheaper modes can also do web research and I recommend you enable this in the agent (looking on github, seeking out known issues with plugins, etc). But the big model somehow felt it was going the extra mile. And it definitely saved me time of having the cheapers ones keep coming up with bad solutions.
Today I am back to Extending Thinking mode. And my daily spend will be nearer a few tens of dollars, not $150!
So is the high powered model worth the extra cost?
-
Yes, occasionally if you and the cheaper models are banging your heads against a wall and going round in circles trying to fix stuff. Pull out the big one for a handful of prompts, nail your roadblock, and then switch back down.
-
But would I use it permanently? No. It is intelligent and thinks hard on big meaty issues. But when using it for standard prompts that aren’t related to fixing or researching horrible problems, I felt it was no smarter than the extended thinking model.
What are other people seeing with the different models?
2 Likes
Thanks for the info. I didn’t know these options were available. Been racking my head and going around in circles with Replit on an issue.
It’s charged me oodles for a problem it created and is in a loop trying to fix while billing me.
