Guide
Ship an AI Feature That Stays Dependable When Real People Use It
A feature that impressed in a review can behave differently when the model is wrong or one account uses it heavily. We build quality, permissions and cost in from the first day.
4-minute read4 November 2026
In Brief
- A demonstration rarely shows what happens when the model is wrong, when someone tries to mislead it, or when one account uses it far more than expected. Those cases reach customers first.
- We see four actions: let the assistant see only what the asker may see, decide what it does when it does not know, limit what it can do alone, and cap its cost per use.
- Thinqmesh builds your feature with an evaluation set, access controls and cost monitoring from the start, and delivers a working feature on your own data within four weeks.
Adding an AI feature to a product has become quick. A few weeks of work can produce an assistant that answers questions over a customer's data and looks impressive in a review. We think the harder question is how it behaves when the model is wrong, when someone is trying to mislead it, or when one account uses it a hundred times more than expected.
Dependability Is Designed In
A feature that retrieves documents may sit behind one account that can read everything. An assistant can state a price or a policy confidently and incorrectly. An agent with broad powers can be steered by text it reads. Each is a design decision, and each is far cheaper to settle before launch than after a customer has found it.
The working assumption we recommend is to treat the model as an untrusted user of your own system. It sees only what the person asking may see, its output is checked like any other input, its powers are small, and its cost is capped. Those four rules turn a good demonstration into a feature your team can stand behind.
Four Questions Every AI Feature Should Answer Before Launch
We treat the model as an untrusted user of your own product: capable and useful, with no authority of its own.
Can It See Only What the Person May See?
Permissions are applied when the data is retrieved, on behalf of the person asking.
A feature connected through one service account that reads everything can answer a junior colleague from a board paper. Filtering the answer afterwards comes too late, so the retrieval step carries the asker's permissions, and so do any copies of the data.
Get started
- List every data source the feature reads and who may normally see it.
- Test it by asking as two people with different access.
What Does It Say When It Does Not Know?
A confident wrong answer costs more than a polite refusal.
Answers come from approved sources and show where they came from. Prices, entitlements and policy terms come from the system that holds them. Include questions with no answer in your test set, and count a confident reply to them as a failure.
Get started
- Write twenty questions that your documents cannot answer.
- Check that each reply shows a source or declines.
What May It Do on Its Own?
Keep an agent's powers small, and keep a person approving anything with consequences.
Prompt injection may never be fully preventable, so limit what a manipulated model is able to do. Give each agent only the tools its task needs, and require a person to approve anything that spends money or changes a record.
Get started
- List every action the feature can take.
- Mark each one that needs a person's approval.
What Does Each Use Cost?
AI features carry a cost per use that flat subscriptions do not, so limits belong in the design.
Set a quota for each account and a ceiling on the steps an agent may take. Track quality and spend together in production, so that leadership sees the running cost as use grows.
Get started
- Estimate the cost of one typical use and one heavy use.
- Decide the quota for each account and the ceiling on agent steps.
How Thinqmesh Solves This
We build evaluation in from the start, and choose the model on answer quality, running cost and where your data may go. The design lets the model change without a rebuild.
- Week 1
An Evaluation Set and a First Score
We build an evaluation set from real examples of your work and measure a score on them, which shows whether AI or conventional software is the right tool before further investment.
- Week 2
Answers on Your Own Data
The assistant is connected to your documents and systems, with answers that show their source and access that follows each person's permissions.
- Week 3
Controls and an Activity Record
Access controls and an activity record are built in, with a person approving anything high-stakes.
- Week 4
A Working Feature, Monitored
Your first working feature is in use, with quality and cost monitored so that you see both as use grows.
You finish with a working AI feature on your own data, an evaluation set that measures answer quality, and monitoring of quality and cost. The full engagement runs to six weeks.
