The Chatbot Was Working. That Was the Problem.

I was on a call with a dev team a while back. Two engineers, one founder. They were building a chatbot - one of those embeddable widgets you'd drop onto a SaaS site to engage visitors, answer questions, pull from the site's content, and ideally move people toward a sign-up or a demo.

They screen-shared. One of the engineers navigated to a live website - he used my own site, galadon.io, as the test case - pasted the URL into the system, and the chatbot scraped it in real time. Then he typed: What is Galadon?

The chatbot answered. Accurately. It described the platform, listed features, pulled the right language directly from the page. The founder leaned back. The engineers looked pleased. There was that specific silence that happens when a demo goes well.

And I felt a small alarm go off in my chest.

Because the chatbot was working. And that was the problem.

"It Works" Is the Most Dangerous Status Update a Founder Can Receive

Let me explain what I mean, because this is a distinction that kills products.

When an engineer tells you something works, they mean it is doing what they built it to do. The query went in, the vector database returned relevant chunks, the response came out coherent and factually accurate. No errors. No hallucinations. The retrieval pipeline is functioning.

That is a real achievement. I'm not dismissing it. Building a clean RAG system - retrieval-augmented generation, for anyone not in the weeds on this - that correctly ingests a site's content and surfaces relevant answers is genuinely hard work.

But here's what the demo didn't show me: a single user converting.

The chatbot told me what Galadon was. It did not tell me to sign up for a free trial. It did not ask for my name. It did not offer a demo. It did not hand me a link. It answered my question like a very good FAQ page - and then it waited.

That's a retrieval tool. That's not a sales tool. And if you ship that to real website visitors, you've built something that makes people feel informed right before they close the tab.

The Actual Conversation That Needs to Happen Before You Call Something Done

When I pointed this out on the call, one of the engineers said something I've heard a hundred times in product reviews: we can include hyperlinks in the chat responses, like call-to-action links.

Yes. Exactly. That's what I was asking for.

But notice what just happened: that feature - the single most commercially important feature of the entire product - was described as something they could include. Future tense. An afterthought. Something to add after the technical infrastructure was solid.

This is the order-of-operations mistake that I see founders make constantly, especially when they're working with engineering talent that is very good at building and not yet fluent in selling. The engineers optimize for what they can measure: latency, accuracy, retrieval quality, uptime. All of that matters. None of it moves your conversion rate on its own.

The commercial logic has to be baked into the product definition before a single line of code gets written. Not added later. Not filed as a feature request. Not mentioned as a nice-to-have once the demo goes well.

You have to decide: is this chatbot a support tool, or is it a sales tool? Because those are two different products with two different prompts, two different response lengths, two different data architectures, and two different success metrics.

What I Told Them to Build Instead

I described what the first version of this kind of chatbot should actually do, because I'd thought through this exact problem when I was planning my own implementation.

When I'd built an earlier version of a sales chatbot using GPT-3.5, I'd written a system prompt that said something like: act as a salesperson who can't say no. Always link to the free trial. Here's the link. And then I gave it a set of contacts and context about the product.

The result wasn't perfect, but it did one thing the polished RAG demo didn't do: it pushed. Every answer looped back to a reason to act. Didn't matter if you asked about a feature, a pricing question, a comparison to a competitor - the bot was always moving the conversation toward a specific next step.

That's the persona question. And it's not a prompt engineering tweak you add at the end. It is the product decision.

For their chatbot, I outlined what the persona needed to look like:

• Shorter responses. Nobody scrolls a chatbot. If your answer requires three paragraphs, you've lost them. Keep it tight. One or two sentences, then a question or a CTA.
• A persistent call to action. Every response that isn't directly answering a support question should end with a reason to click something. Free trial link. Demo booking. A form. Something with a URL attached.
• A specific character. The bot needs to know it's a salesperson. Not a neutral information dispenser. A salesperson who is genuinely enthusiastic about the product and who has one job: move this person toward a decision.
• Lead capture built into the flow. Not as a gate. As a conversation. The bot should be asking for a name and an email naturally - the way a good salesperson at a conference asks before they hand you anything useful.

The team was capturing name and email, which was good. But the way they'd implemented it - a pair of labeled input fields sitting there waiting - was the most friction-laden way to do it. I told them: don't make it look like a form. Make it look like a conversation. Have the bot ask, Hey, what's your name? and then wait for a typed response. That single UX change drops abandonment significantly, because it feels like talking to a person instead of filling out a registration sheet.

Why Founders Get Fooled by a Good Demo

I want to be direct about something, because I think it applies far beyond chatbot products.

A demo is a controlled environment. The person running the demo knows what to type. They know which queries work well. They're not going to test the edge cases in front of you unless they're exceptionally confident or exceptionally honest. So when you watch a demo and it goes smoothly, you're watching the best-case scenario performed by the person most invested in showing you the best-case scenario.

The question you have to ask after every demo is not did it work? The question is: if a stranger with no context showed up and used this for 90 seconds, would they give us money or an email address?

Those are very different tests.

In this particular session, we ran a version of the second test in real time. The engineer typed a query - how can I sign up? - and the bot returned a generic description of features. No link. No CTA. No next step. Just more information.

That's the moment where the product fails commercially, even though technically nothing broke. The system retrieved content correctly. It just retrieved content that didn't answer the actual question, because it didn't have the right prompt or the right call-to-action architecture to know that how can I sign up is not a request for more features - it's a buying signal that needs to be caught and converted immediately.

The Security Question Nobody Thinks About Until It's a Problem

There was another issue that came up in the call that I want to address briefly, because it's the kind of thing that quietly blows up products after launch.

The chat history was being stored in session storage. The engineers acknowledged this on the call and said they were planning to move it to the database and encrypt it. That's the right call - but the conversation around it exposed a structural problem in how the chatbot's data architecture was being handled.

When you're building a multi-tenant product - meaning many different businesses are each going to have their own chatbot instance, trained on their own site's content - you need to solve the isolation problem before you go live. Who can access which collection? Can one user's chatbot accidentally query another user's data? How do you ensure that when one company's chatbot gets deployed on multiple domains, the right content gets served on each domain?

These questions were all still open on the call. The team was debating whether to tie chatbot instances to domain names or to unique IDs. They were working it out. That's fine - that's what development looks like.

But if you go live before those answers are locked, you have a security and architecture problem wearing a shipping label. And fixing data architecture after users are live is one of the most expensive mistakes you can make, both technically and in terms of trust.

If you're building any kind of multi-tenant AI product - chatbots, AI sales tools, lead gen systems - get the data isolation model right before you write the marketing copy. Not after.

The Gap Between Engineer Success and Business Success

I've seen this specific dynamic at every stage of company building. It doesn't go away. If anything, it gets more acute as AI tooling gets more powerful, because the demos get more impressive faster and the temptation to call something done gets stronger.

The engineers on this call were talented. The retrieval was working. The vector database was clean. The scraping was accurate. On every technical dimension, they had done their jobs well.

But nobody had written the prompt that makes the bot a salesperson. Nobody had wired in the hyperlinks. Nobody had defined what a successful conversation looked like in business terms - not as an abstract goal, but as a specific sequence: visitor asks question → bot answers briefly → bot captures name and email → bot presents a CTA → bot books a demo or hands off to a human.

That sequence is not a product roadmap item. It is the product. Everything else is infrastructure that supports it.

If you're building any kind of AI-powered customer-facing tool right now, I want you to write that sequence down before you touch another feature. What does a successful conversation look like, step by step? What does the user say, what does the bot say, and what does the user do at the end? If you can't answer that in two minutes, you don't have a product yet. You have a prototype that demos well.

How to Audit a Chatbot (or Any AI Tool) Before You Ship It

Here's the framework I'd use to evaluate whether an AI customer-facing tool is actually ready to generate revenue:

1. The stranger test. Have someone who has never seen the product type in five questions a real customer would ask. Watch what happens. Don't coach them. Don't explain anything. Just watch.

2. The buying signal test. Type in a high-intent phrase: how do I sign up, how much does this cost, I'm interested, what's the next step. Does the bot recognize the buying signal and respond with a specific call to action and a link? If not, it's not a sales tool yet.

3. The abandonment test. Count the number of steps between a user opening the chat and giving you an email address. Every step is friction. Every step you add is a percentage of users you lose. How many steps is it? Is that acceptable?

4. The persona test. Read the system prompt out loud. Does it describe a salesperson? Does it give the bot a job, a goal, a personality, and a clear instruction about what every response should accomplish? Or does it describe a knowledge base with a chat interface?

5. The length test. Look at the average response length. If a response requires scrolling on mobile, it's too long. Chatbots are not blog posts. They're conversations. A response that takes 30 seconds to read is a response that drives tab-closing behavior.

If your chatbot passes all five, you have something worth shipping. If it fails any of them, you have work to do - and that work is not a sprint item. It's a product decision that needs to happen at the founder level before it gets handed back to engineering.

One More Thing About "It Works"

I want to come back to the moment on that call when the demo went well. Everyone was satisfied. There was relief in the room - the kind of relief you feel when something you've been building for weeks finally does the thing you built it to do.

I understand that feeling. I've had it a hundred times across the companies I've built and sold. There's a real milestone being crossed when a technical system does what it was designed to do. That deserves acknowledgment.

But satisfaction is where momentum goes to die.

The job of the founder in that moment - in any demo moment - is not to share the team's relief. It's to ask the next question. Not does it work, but does it convert. Not is the retrieval accurate, but did the visitor give us an email address. Not can it answer questions about our product, but does it move people toward buying our product.

Those are different questions. They require different answers. And the gap between those two sets of answers is where most AI-powered products quietly fail without anyone in the company understanding why the metrics aren't moving.

Technical correctness is table stakes. Commercial effectiveness is the product. Ship the second one.

If you're building something like this and want to think through the sales architecture before it goes live - the persona, the conversion sequence, the CTA structure - that's exactly the kind of problem we work through inside Galadon Gold. Not theory. Live calls, real products, real feedback. Come take a look.

And if you're in the early stages of building any outbound or AI-driven lead gen system and you need to think through the foundations - what you're selling, who you're selling it to, how the funnel actually closes - grab the 7-Figure Agency Blueprint. It's free, and it covers the strategic layer that most technical founders skip entirely.

Build the thing that sells. Not just the thing that works.