The Infinite Loop That Charged Someone's Card

I was on a coaching call recently with a developer out of Bangalore. Sharp guy - senior software engineer at a major US company by day, building an AI automation agency on the side. We were deep in the architecture of a chatbot builder product he was helping me develop for Galadon, talking through context windows, embeddings, web crawlers, S3 buckets, lead capture flows.

And then, almost offhand, he dropped a story that I haven't stopped thinking about since.

He said he'd previously worked on a project that had the same architecture we were discussing. Specifically: the platform asked each user to enter their own OpenAI API key. Users pay for their own API usage, platform doesn't have to eat the cost. Sounds simple on paper.

They had around ten monthly recurring users. Not huge, but paying customers. And then, because of a bug, an infinite loop got triggered against one of those users' API keys.

He paused. Then: "So let's say somebody sent 'hi' or something. And it started executing the loop. And everybody's emailing like, what are you doing?"

That's the story. Ten words. But unpack it and you've got one of the most important product architecture lessons I know for anyone building AI tooling right now.

The Difference Between a Normal Bug and a Billing Bug

Every SaaS has bugs. That's just reality. You ship something, something breaks, you fix it. A broken UI component is embarrassing. A failed database write might lose some data, and a 500 error on an endpoint frustrates a user.

But an infinite loop hitting an external billing API? That's a different kind of failure.

This is what happens in that scenario. The user sends a message - could be "hi," could be anything. Your system processes it. Something in the logic misfires and instead of returning a response, it triggers the call again. And again. And again. In milliseconds. The OpenAI API doesn't know there's a bug. It sees valid API calls with a valid key. It charges accordingly.

We're not talking about a spike you'd notice in a dashboard after your morning coffee. Depending on the model and the loop, charges can hit hundreds of dollars in seconds. And if it's the user's key - not yours - they're watching their billing console in real time wondering what the hell your product is doing.

That's a lawsuit conversation.

Why the "Use Your Own API Key" Architecture Is a Trap

The reason teams build this way - user-supplied API keys - makes total sense at first glance. You offload the cost, you don't have to forecast usage, and there's no metering system to build. You just tell users: go get an OpenAI key, paste it here, we'll use it.

But as the developer on my call put it directly: "There is no way you can convince the user to pay you something and also use their own API key and not know how much they will be charged for all the processing."

He's right. This pricing model kills conversions. Think about it from the buyer's side. You're telling them: pay us a monthly fee, and also expose yourself to an uncapped, unpredictable cost on top of that. You can't model your expenses. You can't budget. You just have to trust that our engineering team wrote airtight code with no edge cases.

They won't do that. And they're right not to.

The user on my call pointed out exactly what happens: someone signs up for what looks like a $19/month tool, then gets a surprise bill because of the API usage. Could be twenty cents. Could be a thousand dollars. They have no idea. That's a trust problem. And trust problems kill companies.

The Fix: Rate Limits Are Not Optional

So what's the solution? We talked through it on the call and it comes down to a few non-negotiable safeguards that should be in every AI product that touches an external API.

1. You own the API key, not the user

You wrap the OpenAI API yourself. You own the key. You charge users a fixed plan, and then you build your own metering layer on top of that. This is how successful AI SaaS companies operate at scale. You take on the infrastructure cost, you manage the risk, and you price accordingly.

The way we discussed doing it: a base plan gets a set number of messages per billing cycle. Let's say 600 messages for $99. After that, it's a small per-message overage - call it a few cents per message. That way, users who want to scale can, and users who want predictability can stay in their lane. You know your cost structure. They know what they're paying. Simple.

2. Rate limiting at the application layer

This should be table stakes and somehow it still isn't in a lot of products I see. You set hard limits on API calls - per minute, per day, per billing cycle - and you enforce them before the call goes out. Circuit breakers after the fact are too late.

The way the developer put it: "We are calculating in the background based off of context length" - meaning the app is always tracking consumption and enforcing a ceiling. The same logic applies to API calls. Track usage, cap it, and stop calls when they hit the limit.

3. Loop detection and kill switches

I see this get skipped until someone gets burned. An infinite loop isn't always a coding error that's obvious in a code review. Sometimes it's a retry logic problem. A webhook that triggers another webhook can cause it. Sometimes it's a race condition that only appears under specific usage patterns.

You need a kill switch. Something that detects: this API key has made X calls in the last Y seconds, that's not human behavior, stop everything and alert. Build this before you have users. Not after.

4. Spend alerts and hard caps

OpenAI and the other major API providers have their own spend limit tools. Use them. Set a hard cap so that even if your application layer fails, there's a second line of defense. This doesn't fully protect you - there can be lag between when calls are made and when limits trigger - but it's a layer. You want layers.

The Broader Architecture Lesson

The context for all of this was a bigger conversation about how to architect an AI chatbot builder. And one of the things I kept coming back to with the dev on the call was: keep it simple, especially early.

The temptation when you're building AI tooling is to go deep on the infrastructure - LangChain, embeddings, vector databases, the whole stack. And for some use cases, that's necessary. He mentioned working with a law firm that needed their chatbot trained on deposition summaries and constitutional documents, where accuracy was non-negotiable. For that? Yeah, you need embeddings, chunking, retrieval pipelines - the whole thing.

But for a sales chatbot that's supposed to capture leads and book meetings? The answer we kept landing on was: stay in the context window. Modern LLMs have context lengths large enough that for a typical sales or support chatbot, you don't need a vector database. You scrape the website, you grab the landing page content, you maybe let users upload a few PDFs, and you feed it all directly into the context. It works, it's simpler, and when something goes wrong you can debug it.

When something breaks - and something always breaks - you want the shortest possible chain between "what happened" and "what do I change." Every layer of abstraction you add is another place to look when users are emailing you that your product just charged their API key a thousand dollars for saying hi.

This Is Why I Don't Use White Label

We also talked on the call about why we built our own chatbot infrastructure instead of white-labeling one of the existing bot builders.

There are plenty of them. Some of them are good products. The question is whether you can change them. And the answer with white label is almost always no - or at least not without a fight to get what you need.

We learned this the hard way on a previous project. Started with a white label partner, wanted to make changes, and found out what "you can't change a product you don't own" means in practice. The onboarding on the white label tools is confusing. The UX is built for technically sophisticated users. If you're trying to sell a sales or marketing tool to non-engineers - and we are - that's a problem.

My take has always been: if you can get a tool to work for someone who's not an engineer, you can win. Simplicity is a competitive advantage. Simplicity wins deals.

Beyond that - we want to sell this company eventually. When you're thinking about exits, the thing that kills valuation faster than anything is technical debt and vendor dependency. If your core product runs on someone else's infrastructure and someone else's decisions, you're selling a reseller agreement. That's a different business with a different multiple.

I've done five SaaS exits. The ones that sold at the best multiples were the ones where we owned the stack.

The Lead Capture Loop (The Good Kind)

One of the most interesting parts of the architecture conversation was about how to handle lead capture inside the chatbot itself. Because chatbot products keep getting this wrong.

The naive implementation is: when someone opens the chat widget, immediately ask for their name and email before they can do anything. It doesn't work. Users just close the widget.

What we landed on was a better sequence. Let them ask a question first and get engaged, then have the bot give them a useful answer. Then, after that first value exchange, prompt for the email. Something like: can we get your contact info so we can follow up?

And then - this is the part I think is smart - if they're a business owner who's paying per message, you don't want them wasting messages on anonymous tire-kickers. So you make the email field required to continue. Not optional. Once you ask for the email, the bot keeps asking. It doesn't let them skip to the next question. It loops until they enter a valid email.

As the developer described it: "If they are entering anything but an email, you have to again give me an email, give me an email - you cannot allow them to go ahead and ask different questions."

That's the good kind of infinite loop. The one that serves your business.

The implementation we discussed gives the chatbot's owner a toggle: is the email required or optional? If they mark it required, the bot enforces it. If they mark it optional, users can skip. That's the right UX call - let the business decide how aggressive they want to be with lead capture based on what they know about their customers.

This is also why the chatbot beats just a contact form. You've already given the user something - an answer, a helpful response, a taste of what the product knows. They've invested a minute. Now asking for an email feels fair. Asking for an email at that point feels earned.

Why This Specific Call Stuck With Me

I do a lot of these calls. Most of them have one or two useful moments I'll think about later.

This one had more than that. Partly because the developer on the call was sharp - he'd clearly been in the weeds on enough AI projects to have opinions that came from scars, not theory. When he talked about that infinite loop situation, he wasn't speculating about what could go wrong. He'd watched it happen to a product with paying users.

But also because the lesson is so easy to skip. When you're building, you're thinking about features and the happy path - how the product works when everything goes right. The billing safeguards, the rate limits, the kill switches - those are the things you build when you're paranoid. If you're building your first AI product, you're probably not paranoid enough yet.

They will be, after the first incident. But by then you've already got angry users.

Build paranoid. It's cheaper.

The Practical Checklist

If you're building any AI product that makes calls to an external API - OpenAI, Anthropic, any of them - here's what should be non-negotiable before you put it in front of paying users:

• Own the API key yourself. Never ask users to supply their own key for a product you're charging a monthly fee for. The risk transfer causes more problems than it solves.
• Build metering before you launch. Track every API call so you know what each user has consumed in the current billing cycle. Cap it before it goes over.
• Set a hard per-minute call limit. Any more than a few dozen API calls in a single minute from a single user session means something broke. Stop and alert.
• Use provider-level spend caps as a backstop. Use them as a second layer. Set them lower than you think you need to.
• Test your retry logic explicitly. Retry storms are one of the most common causes of unexpected API bill spikes. Make sure your retries have exponential backoff and a hard stop.
• Log everything. When something goes wrong, you need to know exactly how many calls were made, to what endpoint, in what time window. If you're not logging that, you're flying blind.

None of this is exotic engineering. It's table stakes. But it's the kind of thing that gets cut when teams are moving fast to ship, and the bill shows up later.

One More Thing About Architecture Decisions

The broader conversation on the call kept coming back to a theme: every architecture decision you make in your AI product is also a business decision. Whether to use embeddings or context length. Whether to own the API or pass it through. The choice between no-code and native code. Whether to white label or build from scratch.

There's no universally right answer to any of these. But there's a question you should ask about each one: when this goes wrong, how long does it take me to fix it?

The user-supplied API key architecture goes wrong and you've got angry customers and a potential legal issue, with no way to refund them because it was their key, not yours. That's a hard fix.

When the context-length approach goes wrong, you've got a chatbot that hits a character limit - just a feature request.

The white label approach goes wrong and the vendor makes a change you can't control. That's a platform risk that threatens your whole business.

Build your own stack, own your API, meter your usage, and rate limit everything. It's more work up front. It's how you stay in business.

If you're in the early stages of building something like this and you're thinking through your lead data and outreach stack alongside the product - how to source contacts, build prospect lists, and set up your first outbound sequences - the Best Lead Strategy Guide breaks down exactly how I approach that, and tools like ScraperCity can handle the list-building side while you focus on building the product.

And if you want to work through your own product architecture or sales system on a live call, that's what Galadon Gold is for.

Build paranoid. Ship anyway.