Scaling pain: client-side vs server-side filtering (déjà vu from 40 years of building apps)

Over the past few weeks adding new cities to LocalMusicX.com, I ran into a scaling issue that felt very familiar — one of those “I’ve seen this movie before” moments. It took a few hours of back and forth with the Agent, but we got to the root cause.

At small scale, everything worked great. We were loading event data and filtering it client-side by city. Fast, simple, no issues.

Then we started adding more cities and events.

And things… slowed down. Then degraded. Then basically hit a wall.

The root cause was classic:
:backhand_index_pointing_right: the backend was returning all events for all cities
:backhand_index_pointing_right: the browser was doing the filtering

So even if a user just wanted to return 20 events for “Bellingham tonight,” we were sending the entire dataset of thousands of events over the wire.

It worked beautifully… until it didn’t.

What’s interesting is how subtle this failure mode is:

  • performance degrades gradually

  • nothing obviously “breaks” at first

  • easy to miss until you hit a tipping point

Then suddenly you’re debugging under pressure with a live site struggling.

The fix, of course, was straightforward once identified:
:backhand_index_pointing_right: move filtering to the server
:backhand_index_pointing_right: return only relevant data per request

Now the architecture scales properly — performance stays flat as we add more cities.

What struck me most is how timeless this pattern is. I’ve hit this same issue multiple times over decades of development, just with different tech stacks.

Curious how others here approach this tradeoff:

  • Do you default to server-side filtering early?

  • Any heuristics for knowing when client-side filtering is “safe”?

  • Have you run into similar slow-burn scaling failures?

Would love to hear war stories.

2 Likes

Great writeup, this is one of those patterns that bites everyone eventually. Moving filtering server-side is the right first move, but I’d start thinking about caching now before you hit the next wall.

Think about the access patterns for a local music events site. People in Bellingham are probably searching for the same handful of queries.. “tonight,” “this weekend,” maybe filtered by genre. That’s highly cacheable data. Events don’t change every second, new ones get added maybe a few times a day, so even a short TTL of 5-10 minutes would dramatically cut your database load while keeping things fresh enough.

I run a lab-grown diamond marketplace with 600k+ live listings and hit the exact same progression you’re describing. Client-side worked great early on, then we scaled up and everything slowed down when real traffic started. Server-side filtering fixed the immediate pain, but the real unlock was multi-level caching. People search by shape, carat, price range and a relatively predictable set of combos get hit over and over. Caching those results means most queries never touch the database at all.

For your setup I’d look at a few things:

  1. Response caching for your most common city + filter combos. If “Austin tonight” gets hit 500 times a day, that result doesn’t need to be computed fresh every time.

  2. A short TTL so new events still show up in a reasonable window. 5-10 minutes is usually the sweet spot for event data. Short enough that nothing feels stale, long enough to absorb the repeated traffic.

  3. Cache invalidation when new events are added. If you’re ingesting events on a schedule you can clear the relevant city caches at the same time so fresh data shows up immediately instead of waiting for TTL expiry.

  4. Consider precomputing your highest traffic pages. If you know which cities get the most hits, build those responses on a schedule and serve them cold.

Server-side filtering reduces the payload but caching reduces the work entirely. You’ll be glad you have it in place before the next set of cities goes live.

To answer your questions directly, yes I default to server-side filtering early now, learned that one the hard way more than once. My rule of thumb is if the dataset can grow independent of the user’s request (more cities, more listings, more events over time) it needs to be filtered server-side from day one. Client-side filtering is only safe when the full dataset is bounded and small, like filtering a user’s own saved items or a short dropdown list.

1 Like

Great idea! The event scraping for each city is scheduled so it would be super easy to cache today’s events for a given established city when someone searches it. Then we can clear the cache the next time the city is scraped. That would also mean that the just-in-time scraping for unestablished cities could be pumping new events into the events table without competing with continuous city event queries triggered by users checking where they can go hear some music tonight.

2 Likes

Yeah I would try that out, let me know how it goes!

1 Like

To avoid this FIXING while live issue I now use REPLIT to generate 300,000 records as test and demo data in the development database. This way every endpoint gets stress tested and you discover what your API needs to index and filter to improve speed. Its also good for seeing what the real system will look like full of data

Man I wish I could have hired you to build my app instead of the agent replit dumbdumb . And the “hostage” and “Ransom” approach the platform is doing. Make a core payment they take my site down a"the hostage" demanding another core payment the Ransom" and the charges are not usage , usage doesn’t take your site down 3 times a month demanding another core payment. My site is down right now after 3 seperate “core” payments

I can’t even write a ticket to customer service when it happens without paying the core fees again. Unbelievable it’s the same amount everytime. So disheartened !

Your app isn’t hostage, just move to railway and connect another agent to it, it’s just code and a database. The settings are in the database info, do a pg_dump to a new server and use GitHub to get your code off or just replicate the code locally then push it elsewhere. It will also let you download a zip if you don’t know how to use git, you can run postgres locally on any computer and just spin it back up if needed to keep messing with it.

I’ve got all my projects connected to git so I take your point, but my applications are big and complex with lots of integrated services and business logic. Migrating them would not be trivial plus after working with the Replit Agent daily for 5 months we’ve developed a very positive and productive relationship.

John F Bremer Jr

Thank you so much for confirming my plan of action is on the right track. I downloaded the .zip file yesterday almost 300mb now today I’m working on all the secret api’,s keys and what not, I plan to roll out in an entirely different platform all together this platform has taken a dream that I’ve had for the last 10yrs and turned into a total nightmare. My web app is finished other than a few bugs here and there and whatever arises in the future that way. The reason why I say “hostage” and “ransom” is because that’s exactly what’s happening here. I pay my core subscription on the day it’s due within a couple days my site will be down and my usage data will sometimes not everytime be significantly reduced with no usage done the “hostage” Then any move I make on the platform including customer support tickets is met with “you must upgrade your subscription to core membership to do that” so I byte the bullet and just pay so site goes back up for about 5-6 days and then bam same thing over and over if it were about usage than my site would not go down each time and they wouldn’t be demanding another payment for core membership the “ransom”. Also the subscription fees reflect what they charge for membership each month . That charge basically stays the same. Any usage fees are smaller and different amounts each time, yet that’s what they keep blaming it on so I have to leave here for my own sanity and we’ll being. So after me being long babbel winded where would you take your project to edit on another platform, I agree with the GitHub as my repository but what’s good for A.I assisted coding. ?! Yes I agree with a cool relationship with agent , I credit agent big time on my about page and I wish I could just bring him (it) along.

Oh I get it, I had to plan my migration for weeks, and it was scary but it did pay off for me, i’m not saying its right for everyone I just didn’t like the sudden changes from Replit with how the agent worked that then affected my workflow, I had the same relationship, I had the agent down to a science to get what I wanted done and then when Agent 4 come in it just was a disaster for my workflow and I had had enough, after hundreds and hundreds a week I could not predict costs any longer and changes were erratic, it would spawn off too many agents and tasks and change things outside the scope of what I had prompted, it was just not working for me any longer. I will even give that I’m sure by this point maybe most of those things are fixed, but the fact that there was no recourse for weeks or a fallback to agent 3 while the bugs were worked out in a professional service was just unnaceptable to me after many many thousands of dollars upwards of $12k I had paid and counting. I liked the all in one nature of it until it was just too expensive for me to maintain.

To answer blackpyramid530 - the short answer is if you want to manage costs just use Claude max 100 per month plan, its predictable and i’m getting as good or better results than I was with Replit, but you then need to manage your own infrastructure, you will need either railway or another way to push your code front end and you need a way to host databases. I have equipment, I have a business connection so I setup servers at my work with tail scale and just host my own everything now. You can do it on most home connections if you think you can keep the site live, use tail scale and cloudflare tunnels- I’m running izios.com that way, loads fine. I am also running our new front end for my jewelry website that way, this is on 500/50 by the way, not even gigabit which I do have at home. I will use cloudflare to help with edge caching so it stays fast even with our traffic, we get probably 2-3k visits per day so nothing crazy. The Claude agent is what replit has been using at the core, but now they have several cheaper sub-agents but in the end they all do similar things. Claude can do sub agents for tasks too, it can- given the right setup and permissions, SSH into servers and create and manage database, setup backups systems program management backends etc. It really is just the limit of your knowledge. It will not be plug in play like replit - you will need to understand what you are doing. I have experience with servers, Linux, terminals etc so it’s second nature to me, but it can help you learn also so if you’re up for that use your desktop and start messing around with it.

Yeah the charges can be irritating. The Agent got stuck yesterday for 5 hours and I was developing from the phone app and couldn’t get it unhung. I finally contacted support and they killed the chat. Later in the day I got a receipt for $270! The thing is though these costs are a fraction of what we’re used to paying or charging for development. Also I’ve got multiple apps going so I don’t really have the bandwidth to actually code anymore. I’m also developing a lot from my phone for the phone with a lot of speech to text and text to speech so I’m still getting great value from Replit.

John F Bremer Jr

Yes i do miss nudging from the phone not going to lie, i’m trying to build something that will let me do that on my end, and yes I was paying thousands to developers also with terrible results, the is leagues above that so I know what you mean.