This is more of an AI engineering and problem solving question on how to think about using AI than a how to use Replit question.
My main product is a signup form tool (think parent teacher conferences, potlucks, volunteer schedules, massage chair slots, etc). grasshoppersignup.com if you want to see and try which will make this description easy to understand. It’s grown nicely this year. Instead of tediously building a form you just say what you want and AI generates a json schema that I use to create the form. Signup forms are complicated in that they have limits on how many people can sign up for some options, sometimes dates and times matter sometimes they don’t, lots of other interesting complexities…skipping ahead to the problem
Today, I get a prompt input from the form creator “what kind of form do you want” and then feed that with a pretty long and detailed and well honed prompt to AI and get back json schema. I get title, description, options, max responses, fields, and a few more things.
it works well, like 70% of generated forms are fantastic. The rest are a mix of bad user prompts and stuff i should be able to do but the output I get isnt good or actually fails. in those fail or miss cases, typically its users asking for crazy stuff like give me a form with options for every day the next 2 years. They’ll paste a massive block of text from chat gpt where they asked it to create a signup form. The problem is that what they asking for is crystal clear and typically doable, but open ai can’t handle that query with my prompt in any reasonable amount of time and it times out or can’t generate matching arrays for my json schema.
I’ve thought about doing a multi step process but I can’t ever outperform my big ass prompt file on the majority of easy and even hard requests. There must be a better way… if any of this made sense and you have thoughts on how I should explore, I’m all ears 