AI project prioritization: comparing value, cost and risk

Which AI project should come first?

The first project should test an important business hypothesis at a reasonable cost within acceptable risk. It need not be the most impressive demonstration or the proposal claiming the largest time saving. Compare value, feasibility, full cost, risk, dependencies and evidence strength. AWS connects business alignment and measurable benefits with implementation prerequisites.[S1] The ranking method below is an editorial decision aid, not an official AWS rating or a guarantee of return.

Step one: apply conditions that scores cannot offset

Check information rights, accountable ownership, an assessable output and a credible error response. A proposal without these belongs in a preparation queue, not alongside ready experiments. Unauthorised sensitive data or an irreversible external action cannot be compensated for by a high projected benefit score. This separation makes the later comparison meaningful. It also makes a deferred decision actionable: record which condition is missing and what evidence would resolve it.

Step two: make the value hypothesis calculable

A starting capacity estimate is monthly task volume multiplied by net time reduction per applicable task, multiplied by the proportion of tasks that actually qualify. Net time includes review, correction and exception recovery. For quality, define the error or service outcome before estimating improvement. More content does not automatically create more revenue. Explain the mechanism from changed work to customer behaviour and commercial outcomes, and mark causal links that have not yet been demonstrated.

To avoid double counting, keep total task volume, eligibility, actual adoption and net time reduction per task in separate spreadsheet fields, each with its source and observation period. If volume already includes only eligible cases, do not multiply by eligibility again. If net time already includes human review, do not deduct the same labor a second time under costs. Treat unknown adoption as an explicit scenario instead of assuming universal use. State whether shared costs are full, incremental or allocated so alternatives use a comparable basis. Ask another analyst to recompute the formulas from the original inputs; this can expose decision-changing errors more effectively than adding further subjective scoring dimensions.

Step three: include the whole operating cost

One-off work includes information preparation, workflow design, identity integration, evaluation material, interface work and training. Recurring work includes inference, retrieval, licences, monitoring, content maintenance, regression testing and support. Human review and incident handling are real operating costs. Also consider transition and exit: parallel operation, supplier changes, information export and restoration of the earlier process. Comparing token prices alone ignores most of the decisions that determine whether the application remains worthwhile.

Separate recovered capacity from cash savings. A few minutes freed across many employees may not translate into a reduced cash expense. It can still create value if the organisation redirects that capacity to useful work. Labour-cost equivalents are useful scenario inputs when labelled clearly; they are not automatically realised financial returns. Explain the unit and the intended use of every number so that a decision-maker can interpret the estimate correctly.

Illustrative calculation: summarisation or automatic replies?

Suppose there are 200 monthly tasks. Each currently takes 20 minutes, while an assisted draft plus review takes 12 minutes. Only 70 percent of tasks qualify. The capacity hypothesis is therefore 200 times eight times 70 percent: 1,120 minutes, or about 18.7 hours. If maintenance requires another five hours monthly, account for it separately. All inputs are invented for demonstration and must be replaced with measured evidence. They are not AiTOM performance results.

An automatic-reply proposal may have greater volume but also involve refunds, personal information and external sending. Its potential may be larger while its prerequisites remain unresolved. An internal reply-drafting experiment could proceed before autonomous sending. Prioritisation decides what evidence the next investment should purchase; it does not declare one category permanently superior. Preserve that distinction when communicating the portfolio to teams whose proposals have been deferred.

Step four: use simple scales with visible definitions

An editorial starting point is low, medium or high value, feasibility and reuse, shown alongside cost ranges, risk gates and evidence strength. If management needs a numerical score, publish the definitions and weights and retain the underlying observations. Unknown information should remain unknown rather than receiving a convenient middle score. When totals are close, a cheaper experiment or a shared dependency may be more informative than debating fractional differences in subjective ratings.

Add a field asking whether each uncertainty could change the decision. An unconfirmed quotation may permit further preparation when a conservative upper bound still supports an experiment. Unresolved information rights remain a gate regardless of optimistic benefits. Focus the ranking discussion on differences that could alter the choice, and retain both adopted assumptions and rejected alternatives for the next review.

Step five: test assumptions with sensitivity analysis

Recalculate using conservative, base and optimistic assumptions. If moving adoption from eighty to forty percent reverses the ranking, adoption is the next research question. Improving model fluency may not resolve it. HM Treasury's AI impact evaluation guidance emphasises early baselines, explicit comparisons and the mechanism connecting an intervention to outcomes.[S2] This supports evidence-based benefit assessment. The capacity calculation here does not replace causal evaluation or establish that any observed change was caused by AI alone.

Step six: distinguish shared foundations from one application

Knowledge maintenance, identity and event records can support multiple use cases. Do not charge an unlimited platform programme to the first pilot, or build everything because future teams might need it. Name the confirmed dependent use cases, their minimum requirements and acceptance conditions. A local solution may be sufficient for the first experiment. Evidence from a second application can justify extracting a shared component. Reuse should follow demonstrated common requirements rather than an ambitious architecture diagram.

Deliver a decision that can be revised

The decision record should include the use case, evidence, value range, full cost, risk, dependencies, next action and stop conditions. Update assumptions when tests or costs change, retaining prior versions and the reason for the change. States can include experiment, prepare, observe and stop. A lower-ranked proposal is not an implicit promise of future development. Equally, sunk costs are not sufficient justification for continued investment when the original value mechanism no longer holds.

Common mistakes and limitations

Departmental scores are not comparable until their definitions are calibrated. Model benchmark results do not establish business impact. Volume, adoption and review time may be related, so sensitivity scenarios should combine plausible inputs. Brand effects, public value and consequential risks may resist meaningful monetisation. Show these explicitly rather than forcing them into a single return-on-investment number. The purpose of the model is to expose trade-offs for accountable judgement, not to automate the investment decision.

Must the first use case have the fastest payback?

No. A trial may primarily resolve uncertainty about information, integration or operations. Define that learning objective, the decision it will enable and the maximum acceptable investment. Strategic learning is useful when bounded; it is not a reason for a project with no acceptance point to continue indefinitely.

Can a team rank projects without complete data?

Yes. Use ranges and evidence levels rather than false precision. Gather the missing information most likely to change the decision. Not every candidate requires the same expensive research exercise before an initial portfolio discussion can be useful.

When should an ongoing project stop?

Reassess when an essential condition fails, measured results do not support the value hypothesis, operating effort exceeds the acceptable range or a simpler alternative becomes available. Preserve useful evidence and reusable outputs when stopping, so the next team does not repeat the same unsuccessful experiment.

Sources and evidence boundaries

1. S1 · Assessment considerations and prerequisites

   AWS · Accessed 2026-10-22

   Supports the cited principle; operational steps and illustrative scenarios are editorial recommendations rather than reported customer results.

2. S2 · Guidance on the Impact Evaluation of AI Interventions

   HM Treasury / Evaluation Task Force · Accessed 2026-10-22

   Supports the cited principle; operational steps and illustrative scenarios are editorial recommendations rather than reported customer results.