I’m planning on adding a chatbot feature to a Replit app I’m building for a client. A typical AI chatbot and RAG solution: combining Claude API in the background, with 20 training docs (markdown) provided by my client, as well as user profile information that would also get passed along in each chat.
As a techie, I am happy enough attempting to roll my own, and prompting Replit to code it. So far I’ve gone down the road of looking at Voyage AI for embedding, etc. And as a big Supabase user, would store everything there.
One thing that only dawned on me yesterday, was guardrails to filter dangerous types of prompts. And methods for storing the user’s chat encrypted. The more I think of it, the more I realise there is a lot I don’t know I don’t know about chatbots and RAG.
We all know if I ask Replit agent, it will happily try to help me build something. But I wonder if there are more shop-ready solutions out there that I can (and should) plugin to?
One I’ve watched from afar is Quivr. But I am sure there are plenty of other tools.
The v1 in my app started with chat between two different types of platform users (vendor / customer). I eventually dropped it, there were too many variables that’d keep breaking it. I’m going to follow this one closely because I road mapped it because of the difficulty.
user<->user chat is quite different to a chatbot/RAG system, but I agree that when you start these big feature ideas, it is like opening a can of worms, and things keep popping up. You have to be very careful as you plan it, with a very clear set of requirements on what you do want and what you don’t want.
I asked ChatGPT to research Quivr, the open-source RAG/chat I linked above, and it actually convinced me to roll my own instead. It said my requirement was relatively small, whereas Quivr is such a big solution with many variables, options and features, that I would end up spending most of my time configuring it instead of focusing on my own very small set of features and needs.
So down the rabbit hole I go. ChatGPT as usual has been very helpful in starting to prepare the right prompts for me to give to Replit agent - so let’s see, over the next week or so.
It will also be the first major bit of coding I’ve done on the new v3-autonomy, so I am slightly scared
I built a local vector db with pgvector, just a simple one for retrieving more complex data for search queries on diamonds, i considered also building a chat system to integrate into all the data I am accumulating, I have millions of diamonds and price points and historical data that I am now starting to accumulate on the lab diamond market so I wanted to train systems on this data. I am on the same journey as you, thanks for bringing up voyage ai, i might look into this. I am currently running my embedding on new diamonds weekly, its time consuming process, i might need to move this to a dedicated service on its own but i’ll limp along with a local function for now. I have the search system plugged into it but my vision was enabling our realtime (after hours) chat bot to act as a secondary search with the interactive nature of a chatbot, recommend diamonds ,find alternative ones from other sites that people link, etc. I am rolling out a chrome extension to plug into this system as well that will be called Diamond Scout and it will scan all current pages when someone is looking for diamond and pull a better one from our database or the exact one if available at a better price, alternatively i will use it as a market research tool since it will basically allow users to scrape sites for me and report back the prices occurs the industry.
I will be hitting the vibe coding hard this week for this chatbot feature. Including diving in to see what Voyage AI is capable of, and how Supabase manages the pgvector stuff. Watch this space…
I’ll be following on out of interest. I am thinking of using some AI in my own app, but not yet a ChatBot feature, although the prospect is interesting.
