Getting an Internal Server Error

Deployments
1

I don’t know if it is a bug, but it starts to process the job and then crashes. The assistant can’t figure out what is wrong and of course I’m a no-coder and have no idea. It work perfect in the development, but errors in production. Here is what Gemini said after reviewing the logs: Okay, I’ve reviewed the log screenshot (image_41459f.jpg) you provided.

You are absolutely right, there are significant errors happening in the deployed environment. The key line confirming our suspicion from before is:

[ERROR] Worker (pid:XX) was sent SIGKILL! Perhaps out of memory?

Followed by Gunicorn reporting that the worker failed to boot or exited unexpectedly.

Diagnosis:

This confirms that the application process running in your Replit Deployment container is being forcefully stopped (SIGKILL) by the system because it’s trying to use more memory (RAM) than allocated to your deployment plan.

  • When is it happening? Although the traceback involves grpc (used by the Google AI library), the most likely time for memory usage to spike is when you click the “Run Pipeline Now” button. The pipeline then tries to fetch articles, process them, and make multiple calls to the Gemini AI for headlines, summaries, scores, and angles, all within that single running process. These AI operations can consume a noticeable amount of memory.
  • Why now and not before? The environment where you run python app.py or python main_pipeline.py directly in the Replit editor/shell might have slightly different or more flexible resource limits than the dedicated Deployment environment. It’s common for applications to work in development but hit resource limits when deployed.

I increased the resources but it is still failing. Help.

I post on the community under deployment 3 days ago. Need help
2

This is a very process and memory intensive process, are you storing the data into a database or in temporary memory? I would be strategic about how the data is parsed after it’s fetched from Gemini it can add up fast. I have 550k records of diamonds in my database and got to 500mb or so pretty fast, the images alone would be 10gb and videos would be hundreds more gigabytes but luckily I can load those from a delivery network, you don’t have that luxury and are pulling data from the internet and have to store it somewhere. Find out how it’s managing the data first, then create a strategy on data storage for processing later. Without knowing what you’re trying to do it’s hard to give more input though. You might need to use a dedicated server with 16+gb ram to avoid hiccups if its data intensive.

3

I’m not trying to save anything to the server. I am pulling 5 most recent stories from a Google News using RSS.com for 7 or 8 categories (feeds) for past 24 hours, analyzing and summarizing them, then posting them to my Notion database. no images are even involved. I looked at the logos and they never exceed a couple of % during the processing. It isn a once in a day thing. It seams to crash after the first category is pressed when it ties to start the 2nd one. Are there any other debugging options you can recommend?