AI Proof of Concept: A One-Page Charter With Kill Criteria

One mistake to avoid in an AI proof of concept: starting to build before anyone has written down which result would stop the work. Without that line, a weak result becomes a request for "one more month", real users get added, and the test turns into a pilot that never ends. We recommend a one-page charter, signed before work starts, that fixes the hypothesis, the success and kill criteria, the data allowed, the time box, the decision owner and the date of the go/no-go meeting. The template is below, ready to copy.

Why do AI proofs of concept drift into permanent pilots?

A proof of concept (PoC) can drift for reasons unrelated to the technology. The charter is built to block four of them.

• No end date. When the time box is "until it works", every partial result can justify another round.
• Criteria written after the demo. Once people have seen an impressive answer, the bar tends to slide towards whatever the system already does.
• Users added too early. A feasibility test that quietly gains twenty daily users becomes hard to stop: ending it feels like taking something away.
• No single decision owner. If the steering group, the IT team and the business sponsor share the decision, nobody has to make it.

An endless AI proof of concept also keeps a team, a data approval and a vendor trial tied up, and delays the honest, useful answer that the idea does not work yet.

Feasibility PoC or user pilot: which one are you running?

The charter should say which of two tests you are running. A feasibility PoC asks whether the technology can do the task on your data at acceptable quality and cost. A user pilot asks whether real people will use it in their real workflow, and whether the work gets better.

We recommend running the feasibility PoC first, without real users. The trade-off is a few extra weeks before anyone outside the team sees the tool. In return, the pilot starts with a model that has passed a test on your data, and a failed PoC costs weeks rather than a rollout. The UK Government's service manual guidance on the alpha phase makes a similar split: prototypes only complex enough to test ideas, code you expect to throw away, and readiness to stop altogether.

The one-page AI proof of concept charter (copy this)

Fill in every line before work starts. A line you cannot fill in means the PoC is not ready, which is worth knowing.

Illustrative example 1: a mid-sized insurer in Amman

Hypothesis: a general-purpose model can extract the policy number, claim date and claimed amount from scanned Arabic claim forms with at least 95% field accuracy, on 300 forms the claims team has already checked by hand. Kill criteria: field accuracy below 85% at the end of week two, or any form leaving the approved cloud environment. Data allowed: those 300 forms only, with names and national numbers masked before upload. Time box: three weeks. Decision owner: the head of claims. Go/no-go meeting: day 21 at 10:00.

The thresholds are illustrative; set yours from your own baseline.

Illustrative example 2: a family-owned retailer in Saudi Arabia

The retailer wants AI to sort customer emails written in Gulf Arabic, English and a mix of both into eight categories. Hypothesis: the model puts at least 90% of a 500-email test set in the same category a senior agent chose. Kill criteria: below 75% on the Gulf Arabic emails alone, even if the overall score passes, or a cost per email above the agreed ceiling. Data allowed: 500 past emails with names, phone numbers and order numbers removed. Time box: two weeks. Decision owner: the customer service director.

This test needs a separate score for each kind of Arabic; our guide to Arabic LLM evaluation explains how to build that test set.

How do you write kill criteria that people will honour?

Kill criteria in an AI proof of concept work only if they are set before anyone sees a result and cannot be quietly edited later. Four rules help.

1. Lock them with the signatures. Any change needs a new charter version signed by the decision owner, with the reason written down.
2. Make each one a number or a yes/no fact. "Quality is poor" is not a kill criterion; "field accuracy below 85% on the agreed 300 forms" is. Anthropic's documentation on defining success criteria for Claude makes the same point: replace a vague aim such as "safe outputs" with a measurable threshold over a fixed number of trials.
3. Include non-technical stops. Data approval not granted by day five, the business reviewer unavailable, or cost per item above the ceiling.
4. Separate stop from pause. A pause is allowed only for a cause outside the PoC, such as a vendor outage, and moves the end date by the pause's length only.

The US National Institute of Standards and Technology treats this decision as a step of its own. Its AI Risk Management Framework (AI RMF 1.0, January 2023), a voluntary framework rather than a law, includes subcategory MANAGE 1.1: deciding whether the AI system achieves its intended purposes and stated objectives, and whether its development or deployment should proceed. The charter fixes the date of that decision in advance.

What data is a PoC allowed to use?

Start with the least sensitive data that can answer the question: synthetic records, masked records, or a small sample approved by the data owner. Real personal data used in a test is still personal data. For example, Article 15 of Jordan's Personal Data Protection Law No. (24) of 2023 bars transferring data to a recipient outside the Kingdom whose level of protection is lower than the law provides, except in listed cases, and requires the controller to check the recipient's protection level before the transfer starts. That matters when an organisation under that law sends PoC data to a model hosted outside Jordan.

Record in the charter where the model runs, which data may reach it and who approved that. This is general information, not legal advice: obligations depend on the organisation's sector, its data and the country whose law applies, so confirm with counsel before real personal data enters a PoC.

How to run the go/no-go meeting

Book a 45-minute meeting on day one. The decision owner chairs it; the build team brings evidence, not a polished demo.

1. Read the hypothesis and criteria from the signed version.
2. Show results against each success and kill criterion on the agreed test set, with real examples of failures.
3. Show the cost per item and what a user pilot would need.
4. The decision owner picks one of three outcomes: proceed to a user pilot under a new charter, stop, or repeat once with one named change and a new end date.
5. Record the decision, its reason and the next step's owner; send it to attendees that day.

Allow one repeat at most; needing a second suggests the hypothesis was wrong or too broad, so write a narrower one. A stop is a result too: record what failed, so the next team does not start from scratch. If the PoC passes, the user pilot needs its own charter, because the question changes from "can it?" to "will people use it?"; our article on why AI rollouts stall covers that stage.

How to start

If you have a use case and a likely data source, our Prototype & PoC Studio (RT3) in the R&D Lab runs fast feasibility prototypes and 12-week MVPs that end in a go or no-go. If the question is Arabic quality, Published Research & Arabic AI Benchmarks (RB1) covers Arabic and bilingual platform evaluations. If you do not yet know which use case deserves a PoC, start with our AI strategy work, or book the AI Exposure Scan, a free 45-minute conversation that ends in a written next step.

Related lab: R&D Lab ↗