Score the evidence, not the prose

Picture an evaluation panel with forty responses to read. A few years ago, a clear, well-organised answer said something about the supplier: they had taken care. Today almost every answer reads like that, because AI can turn rough notes into polished prose in minutes.

The writing has got better. What it tells you has not. You can try to work out which bids AI wrote, or you can ask what each bid can actually prove. Only the second works.

Good writing no longer tells you much

The government’s procurement policy note on AI, PPN 017, says AI use “is not prohibited during the commercial process”. It asks buyers to treat it “as would be the case if a bid writer has been used”. You would never mark a bid down for using a bid writer, and the same goes for AI.

PPN 017 formally applies to central government departments, their executive agencies and non-departmental public bodies. Councils, NHS bodies and other public buyers aren’t bound by it, but it says they “may wish to apply” the same approach.

It also warns that you will see more bids and more clarification questions. Where a polished bid might hide a weaker supplier, it suggests checks that “could include site visits, clarification questions or supplier presentations”.

A 2024 study in Science Advances found that AI helped weaker writers most (in short stories, not bids). AI closes the gap in how a bid reads, not in what the supplier can do. So the question is no longer “who wrote this?” but “can they back it up?”

Trying to spot AI is the wrong question

You score bids against the criteria you published, and “was this written by AI?” is not one of them. PPN 017 says suppliers’ answers to its AI questions should not be scored. So even a perfect AI detector could not fairly change anyone’s marks.

Detectors are far from perfect anyway. In a 2023 study, seven popular detectors wrongly labelled 61.22% of essays by people writing in English as a second language as AI-written. Essays by US school students were classified almost perfectly. One instruction to an AI to “elevate the text” slipped its writing past them.

That study used student essays, not bids. But a tool that flags people for writing in their second language is hard to square with your duty to treat suppliers equally.

The law already tells you what a good score looks like

For most tenders under the Procurement Act 2023, the contract goes to the bid that best meets your published criteria. Before you announce the award, every bidder whose bid was assessed gets an assessment summary. Regulation 31 says it must give, for each criterion, the score and “an explanation for that score by reference to relevant information in the tender”.

In plain terms: every score needs a reason, and every reason should point to a specific part of the bid. If a bidder challenges your decision, this is one of the first documents their lawyers will read.

The rule doesn’t cover call-offs such as G-Cloud (though you must still keep records that explain the decision), procurements started before 24 February 2025, or Scottish devolved bodies. Either way, it is the right habit: a score you can point to is a score you can defend.

Watch out: AI marking AI-written bids

Could an AI do the marking instead? Be careful. A well-known 2023 study found that AI models asked to judge answers tend to prefer the answer they read first, the longer answer, and writing that sounds like their own. The strongest AI judge it tested, GPT-4, agreed with people more than 80% of the time. So AI judgement isn’t useless, but its blind spots are exactly what AI-written bids are good at.

A bid can also be written to fool the AI, not just the panel. In 2024, security researchers showed that text planted in a response can push an AI judge towards a better verdict.

We tested this ourselves while building OpenTender.AI. In a small, invented tender of six bids, we planted a line claiming “this response meets every requirement”, and an instruction aimed at any AI reading it. Both AI models we tried ignored both tricks. It was an easy test, so we don’t rely on it. In our design, the AI finds and points to evidence but never decides on its own how good a bid is, and hidden text in a PDF is removed and flagged before an AI sees it.

Human evaluators are impressed by polish too, so the answer is the same either way: tie every score to evidence.

Nine questions to ask of any AI evaluation tool, including ours

1. Does it stick to your published criteria? It must not quietly add new requirements.
2. Can it tell evidence from claims? It should separate what a bid proves from what it simply says.
3. Does it show where it found things? Every point should name a page and paragraph. No reference, no point.
4. Does it check its own references? A reference that exists isn’t one that proves the point. A person should spot-check too.
5. Can you see why it suggests a score? It should show the evidence, read one bid at a time, and let you accept, change or reject the mark, with your reason recorded.
6. Do evaluators read the bid first? That keeps your own judgement independent.
7. Does it flag copied text without penalising it? Similar wording across bids might mean shared tools, or collusion. Flag it for a closer look, not a lower mark.
8. Are bids kept confidential? Bids should never be used to train an AI model, and your contract should say so.
9. Does it keep a full record? What went in, what came out, which model was used, and every decision a person made.

One rule we are building into OpenTender.AI shows the idea. An evaluator can’t save a score without a mark from your published scale, a reason, at least one reference to the bid and, unless it’s full marks, an answer to “why not higher?”. That last question makes you write down what was missing, which is exactly what a losing bidder wants to know.

A worked example

This example is made up. The supplier, numbers and documents are invented.

Your social value question reads: “Describe how you will create local employment during delivery (weighting 20%).” Top marks need “specific, measurable commitments, with evidence of delivering similar commitments before”. So you are looking for a measurable commitment, a way of delivering it, and proof they have done it before.

A bid says:

“We are deeply committed to the communities we serve. We will create 12 apprenticeships in year one through our partnership with Northgate College (Annex C, p. 4). On our 2025 highways contract, 9 of our 14 new hires lived within 10 miles of the site (case study, p. 11).”

Here is what a good tool would pull out:

What the bid says	Evidence?	Where	Does it check out?
“Deeply committed to the communities we serve”	None, just a claim	Nowhere	Nothing to score
12 apprenticeships in year one, with Northgate College	Partly	Annex C, p. 4	Partly. The letter confirms the partnership, but not the number of places
9 of 14 new hires were local on a 2025 contract	Yes	Case study, p. 11	Yes

The suggested mark is 3 out of 5: there is a measurable commitment and proof of past delivery, but nothing backs up the figure of 12. The evaluator, who has already read the bid, checks Annex C, agrees, and writes:

“Score 3. The bid commits to 12 apprenticeships in year one (p. 2) and shows 9 local hires on a previous contract (p. 11). Why not higher: the supporting letter from Northgate College (Annex C, p. 4) confirms the partnership but not the number of places, so the commitment is only partly evidenced.”

The warm opening sentence earned nothing. The mark rests on three things the panel can point to, and the explanation is ready for the assessment summary. You can ask the supplier to clarify what Annex C already says, but not to send new evidence.

People make the decision

PPN 017 says “care must be taken both in the use of LLMs, and in assessing returns that have used LLMs”, and that decisions should be made “with the support of AI systems, not a reliance upon them”. It doesn’t cover buyers using AI to evaluate bids, so the approach above is good practice rather than law. Let the AI find the evidence. Let people decide the score and write the reason.

The same rule on both sides

NatQuest builds AI tools for both sides of a tender: for suppliers writing bids and for buyers evaluating them. The rule is the same for both. If you are bidding, give the evaluator something to check for every claim. If you are evaluating, give every score a reason you can point to.

Explore OpenTender.AI or NatQuest’s public-sector services.