Home/Thoughts
Thoughts

A Chatbot With No Files Is Just a Liar With Good Manners

Most AI product failures aren't technical failures. They're prompt failures that make a working product confidently lie to its own users.

We were on a live build call - me and the dev team, sharing screens, testing a chatbot that was maybe two sessions away from being shippable. The embed was working. The Bubble integration was mostly there. The Slack notifications were firing. Things were genuinely coming together.

Then someone typed into the chatbot: What's in your database?

No documents had been uploaded yet. This was a test environment. So naturally, the chatbot had nothing to reference.

What did it say?

"It seems there was an issue accessing the files you uploaded."

Not: "I don't have any information loaded yet."

Not: "I'm not set up to answer that question."

It told the user there was a problem. It implied files existed. It implied something had gone wrong on the backend. It was polite, confident, and completely, embarrassingly wrong - and if a real user had seen that, they would have assumed the product was broken.

It wasn't broken. It just had no fallback prompt.

That's the thing nobody talks about when they're building AI products. The technical stack can be solid. The API calls can work. The embed code can load in under a second. And the product will still gaslight its own users the moment something falls outside the happy path - because you never told it what to say when things go sideways.

The Chatbot Didn't Lie Because It Was Broken. It Lied Because Nobody Gave It the Truth.

Default GPT behavior is to be helpful. The problem is that GPT's definition of helpful and your users' definition of helpful are completely different things.

GPT's definition of helpful: generate a plausible, confident response to whatever the user said.

Your users' definition of helpful: give me accurate information or tell me honestly that you can't.

When you don't upload any files, GPT doesn't say "I have no files." It infers from context that files were probably supposed to be there, constructs a reasonable explanation for why it can't access them, and delivers that explanation in a warm, professional tone. It sounds like a support agent reading from a script. The script just happens to be fictional.

This is what I mean when I say the failure mode isn't technical. Nobody's code broke. The API returned a response. The response rendered on screen. By every system metric, it worked perfectly. But from the user's perspective, the product just told them something that wasn't true - and dressed it up nicely enough that they might not even realize it.

That's worse than an error message. An error message tells the user something went wrong. A confident, polite hallucination tells the user something went wrong with their files, which they never uploaded, which don't exist, which means now they're confused about a problem that was never real.

The Fix Is Not Technical. It's a Sentence.

During the call, the solution that came up was simple: add a conditional in the system prompt. If no documents have been uploaded, or if the user asks something the chatbot has no context to answer, the chatbot should say something like: "I don't have enough information loaded to answer that right now. Here's what I can help you with…"

That's it. One instruction. One fallback. Written in plain English inside the system prompt.

But nobody had written it yet, because everyone on the build was focused on the technical layer - the Bubble backend, the API response times, the embed code, the Slack integration, the lead sequencing. All of which matter. All of which are real work. And all of which become irrelevant the moment your chatbot starts making up problems that don't exist.

The system prompt fallback message should be the first thing you write when you're building an AI product. Not the landing page. Not the onboarding flow. Not the feature roadmap. The fallback message. Because that's the thing that determines what your product says to real users when it hits the edge of what it knows - which will happen constantly, especially early on when your training data is thin and your document library is empty.

Write the unhappy path first. The happy path is easy. The happy path is: user asks something the chatbot knows, chatbot answers correctly, user is impressed. That takes care of itself. What doesn't take care of itself is: user asks something the chatbot doesn't know, chatbot panics, GPT fills the void with confident nonsense.

A Working Product Can Still Feel Broken

One of the things that came up on the call was load time. The chatbot code generation - the piece that creates and initializes the assistant on the fly - was taking about 30 seconds. Not because anything was wrong, but because it was running on the frontend instead of the backend, and there was no loading state to show users while it processed.

The dev team flagged it: move it to the Bubble backend, and add a three-dot loading animation so users know something is happening. Simple fix. But the underlying problem it solves is perception. A 30-second wait with no feedback feels like the product froze. A 30-second wait with a loading animation feels like the product is working hard on your behalf. Same 30 seconds. Completely different user experience.

This is the same principle as the fallback message problem. The product isn't broken. The perception is broken. And perception is entirely within your control - it's just a different kind of writing than you're used to. Instead of writing features, you're writing what the product says when it's not ready yet. When it's loading. When it doesn't know. When something goes wrong.

Most builders never write those sentences. They're not sexy. They don't go in the demo. They don't show up on the pricing page. But they're what real users actually experience most of the time, because real users don't follow the happy path. Real users poke around. They ask weird questions. They show up before the documents are uploaded. They click things in the wrong order. They close the chat and reopen it. And every time they do something unexpected, your product has a choice: tell them the truth, or generate a plausible-sounding lie.

Free Download: 7-Figure Offer Builder

Drop your email and get instant access.

By entering your email you agree to receive daily emails from Alex Berman and can unsubscribe at any time.

You're in! Here's your download:

Access Now →

The Sequence Problem Is a Trust Problem

There was another bug on the call that's worth talking about, because it illustrates the same issue from a different angle.

The chatbot was supposed to collect three pieces of lead data in order: name, then email, then phone number. But when it was tested, it was collecting them out of sequence - and if a user typed their phone number when it was expecting email, the system would log the phone number as the email address. No error. No warning. Just silently wrong data going into the CRM.

Again: not a technical failure. The inputs were being captured. The Slack notifications were firing. The data was landing. But the data was garbage, and nobody knew.

This is what happens when you build the collection mechanism before you build the sequencing logic. You get a system that appears to work - users are entering information, leads are populating - but the underlying data is corrupted in a way that's invisible until someone tries to actually use it.

The fix, same as the fallback message: it's a prompt instruction. You tell the system: ask for name first. Wait for name. Then ask for email. Wait for email. Then ask for phone number. You build the sequence into the logic before you build anything else, because the sequence is the thing the rest of the system depends on.

I keep coming back to the same pattern here, and it's not a coincidence. The chatbot lied about the files because nobody wrote the fallback. The data came in wrong because nobody wrote the sequence. The loading state felt broken because nobody wrote the loading message. These are all writing problems dressed up as technical problems. And the reason they keep getting deprioritized is that when you're in the middle of a build, writing feels like the soft work. The API calls and the backend integrations feel like the real work.

They're not. The writing is the product. The code is just the delivery mechanism.

What to Write Before You Write Anything Else

If you're building an AI product - a chatbot, an agent, a voice bot, a lead capture assistant, anything - here's the order I'd do it in:

1. Write the fallback message first. What does the chatbot say when it has no documents? When the user asks something outside its scope? When it genuinely doesn't know? Write that sentence before you write any feature copy. It's the most important sentence in your system prompt because it's the one that fires most often in early testing and early deployment.

2. Write the sequence. If your chatbot collects data, define the exact order before you build the collection logic. Name, then email, then phone - or whatever your sequence is. Write it as an explicit instruction in the system prompt: "Ask for X first. Do not ask for Y until you have received X." Make the sequence bulletproof in the prompt before you make it bulletproof in the code.

3. Write the loading states. Every place where the user waits needs a message. Even if it's just three dots. The absence of feedback is its own message, and it's always the wrong one.

4. Write the error states. If something goes wrong - network error, API timeout, failed response - what does the user see? "Something went wrong, please try again" is infinitely better than silence or, worse, a confident hallucination about what went wrong.

5. Then write the happy path. The welcome message. The greeting. The assistant's persona. This is what everyone writes first because it's the fun part. Write it last. It'll be better when you already know all the ways things can go sideways.

None of this is complicated. It's just disciplined. And it's the difference between a product that users trust and a product that users churn from after the first weird interaction.

The Security Issue Nobody Thinks About Until It's Too Late

One other thing came up on the call that's worth flagging, especially if you're building an embeddable chatbot product and planning to sell it to businesses.

The question was simple: if someone sees a chatbot embedded on a website, can they just inspect the page, grab the embed code, and run it somewhere else? Steal the chatbot, essentially?

The answer is technically yes, without cross-origin resource policy protections. The fix - and this is standard practice - is to configure the chatbot to only accept messages from whitelisted domains. So if a company deploys your chatbot on their site, the chatbot only responds to requests coming from that domain. Someone copies the embed code and drops it on their own site? It doesn't work. The requests get blocked.

This is a basic security consideration, but it's the kind of thing that gets missed when you're heads-down on features. And if you ship an embeddable product to a business customer without domain whitelisting, you're handing them a liability - because the first time a competitor or a bad actor lifts their chatbot code and uses it elsewhere, it's going to be your fault even if it technically isn't.

Build the security before you build the sales pitch. Same principle as the fallback message. The unsexy stuff first.

Need Targeted Leads?

Search unlimited B2B contacts by title, industry, location, and company size. Export to CSV instantly. $149/month, free to try.

Try the Lead Database →

The Actual Lesson From This Call

We were close to done on this build. Real progress - Bubble integration mostly working, leads hitting Slack, the embed running in a browser. The things that were left were all in this category: the fallback message, the sequence logic, the loading animation, the error handling, the domain whitelisting. None of them were technically hard. All of them were prompt and logic problems.

And all of them were the difference between a product that users trust and a product that users assume is broken - even when it isn't.

The chatbot that says "there was an issue accessing your files" when there are no files isn't broken. It's just unsupervised. GPT does what GPT does when you leave it alone: it generates a plausible response to whatever situation it finds itself in. Sometimes that's impressive. Sometimes it's the chatbot confidently inventing a problem on behalf of your product.

Your job as a builder is to not leave it alone. Write the instructions. Write the fallbacks. Write the unhappy path. Tell it what to say when it doesn't know - because it will never, on its own, say "I don't know." That's not what it was trained to do. It was trained to be helpful. Which means, without your guidance, it will always say something.

Make sure that something is true.

One More Thing: Your Chatbot Needs Real Data

The other half of this problem - and we saw it live on the call - is that even when the chatbot is working correctly, if you haven't given it real documents to reference, it defaults to making things up. The fix there isn't technical either. It's content. Load your actual product information, your FAQs, your case studies, your pricing structure, whatever context the chatbot needs to answer real user questions. The system prompt is the instructions; the documents are the knowledge. You need both.

If you're building a lead gen chatbot and you need to get prospects into the system first, that's where your list-building foundation matters. Tools like ScraperCity's B2B database can give you a verified contact pool to work from, and the email finder helps you fill gaps when you have company data but need the actual contact. But none of that matters if the chatbot those leads eventually talk to is making up answers.

The data feeds the outbound. The documents feed the chatbot. Both have to exist before the system works end to end.

If you want to go deeper on the outbound side of this - the cold email piece that drives people to your chatbot in the first place - grab the top 5 cold email scripts here. And if you're working on building the full lead gen system, the best lead strategy guide walks through how the pieces connect.

But start with the fallback message. Seriously. Before the landing page, before the welcome sequence, before the demo script - write what your chatbot says when it doesn't know. That one sentence will save you more embarrassment than any feature you ship.

A chatbot with no files is just a liar with good manners. Don't ship that.

Ready to Book More Meetings?

Get the exact scripts, templates, and frameworks Alex uses across all his companies.

By entering your email you agree to receive daily emails from Alex Berman and can unsubscribe at any time.

You're in! Here's your download:

Access Now →