shpapps

18 October 2026 · 4 min read

Recipes, not prompts: one edge in front of every app

Our apps never name a model or send a prompt. They name a recipe, and a small Cloudflare Worker does the rest. Here is the pattern and why it has paid for itself.

  • Architecture
  • LLMs
  • Cloudflare Workers

Every app we ship that uses a language or vision model talks to exactly one thing: a small edge service running on Cloudflare Workers. The app does not know which model answers it. It does not hold a provider key. It does not even hold the prompt. It sends the name of a recipe and the fields that recipe asks for, and gets back an answer or a friendly "busy, try again".

This post is about that one design choice, because almost everything good about how we run AI features follows from it.

The problem with prompts in the app

The obvious way to add an AI feature to a mobile app is to call a model provider from the app with a prompt you wrote. It works on day one and then fails in three slow ways:

  1. Keys leak. Anything shipped in an app binary can be extracted. A provider key in the app is a key on the internet.
  2. Prompts freeze. A prompt in the binary changes only when users update. A fix you write today reaches most people weeks later, and some never.
  3. Models move. Prices change, models get deprecated, a cheaper one appears, a provider has a bad afternoon. If the model name is in the app, every one of those is a release.

The pattern

The app calls the edge with a recipe name and fields:

const res = await fetch("https://api.example.com/v1/run/retro.identify", {
  method: "POST",
  headers: { Authorization: `Bearer ${installToken}`, "Content-Type": "application/json" },
  body: JSON.stringify({ fields: { region: "PAL" }, images: [frontPhoto] }),
});

The recipe lives in the edge's config, which we push to a key-value store and which every Worker picks up within about a minute. A recipe holds everything the app should not:

{
  "profile": "vision",
  "system": "You identify the exact release of a retro game from photos...",
  "template": "The user says the region is {{region}}.",
  "fields": { "region": { "max_chars": 8, "enum": ["NTSC-U", "NTSC-J", "PAL", "unknown"] } },
  "images": { "max": 3, "max_bytes": 1500000 },
  "max_output_tokens": 1200,
  "response": { "type": "json_schema", "schema": { "...": "..." } },
  "units": 1
}

A few properties fall out of this shape:

  • The prompt is ours. Fields are filled into a template we wrote, each with a length cap, and fields that should be a choice are an enum. The client cannot send a free-form prompt at all.
  • The cost of a call is bounded server-side. Image count and size, input length and output tokens are all capped by the recipe, whatever the client sends. A field can be cut to its first or last N characters instead of refused, which is how an app can send a whole saved page as context without making the call expensive.
  • Quotas speak recipes. Each recipe has a weight in units against the user's allowance, so a heavy vision call and a light text follow-up can share one plan.

The ladder

A recipe's profile picks a ladder: an ordered list of steps, each a provider and a model. The edge tries them in order:

async function runLadder(recipe, input, steps) {
  for (const step of steps) {
    if (breakerOpen(step)) continue;           // a step failing too often sits out for a while
    try {
      const answer = await callModel(step, recipe, input, { timeoutMs: step.timeoutMs });
      const verdict = guard(recipe, answer);    // schema + junk check, see the next post
      if (verdict.ok) return { ok: true, answer, step: step.id };
      recordFailure(step, verdict.reason);
    } catch (err) {
      recordFailure(step, err.message);
    }
  }
  return { ok: false, message: recipe.fallback?.message ?? "Busy right now, try again in a minute." };
}

A step fails on an HTTP error, a timeout, an empty answer, JSON that does not parse, or an answer our output guard refuses. When every step fails, the app gets a polite message and the call is refunded. The client never sees an upstream error. That last rule matters more than it sounds: "busy, try again" is something a user understands and acts on; a raw provider error or an empty screen is not.

Because the ladder is config, moving to a cheaper model is a one-line change, demoting a flaky free model is automatic, and adding a second provider as a fallback is a deploy-free afternoon.

Kill switches and validation

Two habits keep this safe to change live:

  • Validate the config before it ships. The push command runs the same validator the Worker runs, and refuses configs that break a rule, for example a recipe that accepts photos or free text routed to a tier we reserve for closed input. A rule written in code beats a rule written in a doc.
  • Every feature has an off switch. Pausing one recipe, one app or the website's free scan is a config push, not a release. The day you need one, you need it in minutes, not after a store review.

What it costs

One Worker, one key-value namespace, a Durable Object for counters, and a config file in git. The app side shrinks to "send recipe name and fields, show the answer or the busy message". When we start the next app, its whole server side is a new block in the config.

If you are adding AI features to a mobile app, the single most useful thing you can do is make sure the app names what it wants, never how it gets it.

← All posts