Key Takeaways
A Gartner survey of 204 finance leaders in March 2026 found that 20% of finance AI projects focus on decision quality, while 45% focus on productivity.
An Insight Global survey of 501 senior leaders found that 89% have seen an AI initiative stall, and 28% named measuring impact as a cause. Time saved was the most common success measure, at 37%.
A sound decision can produce a poor result, and a weak decision can produce a good one. Judging each call by its outcome alone mixes skill with chance.
Analytics teams often track model accuracy, pipeline uptime, and hours saved. These numbers show how a system performs and how often people use it, but they don't show whether it helps the business make better decisions. A churn model can predict departures accurately while triggering retention offers that cost more than they recover. An approval engine can save hundreds of analyst hours while accepting accounts that leave the business with greater losses.
Recent surveys show that businesses often measure AI’s contribution through productivity gains. Gartner surveyed 204 finance leaders in March and found that their AI investments leaned more toward productivity than decision quality. Gartner warned that this can leave finance leaders reporting adoption progress while boards see little strategic impact. In a separate Insight Global survey of VP and C-suite leaders, time saved and quality improvement were the joint leading measures of AI success, each selected by 37% of respondents, compared with 30% for revenue impact.
Measuring decisions starts with separating the decision from its result. A loan approved on accurate data and sound policy can still default because the borrower lost a job. A poorly reasoned promotion can still lift sales because a competitor ran out of stock. Scoring each case by its outcome rewards luck and penalizes sound logic. Decision quality is better judged across many cases, by checking whether the system used the information available at the time, applied the intended logic, and produced results within the expected range.
To measure this, teams need a record of each decision. It should include the information used, the versions of the rules and models applied, the action taken, and the expected result. Add the actual outcome once it becomes available. Comparing predictions with outcomes across hundreds or thousands of cases shows whether the system’s estimates match what happens in practice. It also helps teams investigate whether errors are associated with particular inputs or versions of the decision logic.
Teams also need a basis for comparison. A small, randomly selected group of cases can continue under the previous policy or a simple rule while the rest use the new approach. Comparing the two groups helps establish whether the change improved results. Without that comparison, higher approvals or revenue may reflect seasonal demand or changing market conditions rather than better decisions.
Override rate tracks how often staff reject a recommendation and which case types prompt them to do so. Reversal rate tracks decisions later undone through appeals, refunds, or manual corrections. An increase in either measure warrants a closer look at the affected cases to understand whether the rules, data, or circumstances have changed.
Together, these measures allow analytics teams to show which decisions improved, the size of the improvement, and how strong the evidence is. Leaders can then judge where further automation is justified, where human review remains necessary, and where the decision logic needs to change.
The DecideWise Insider
Benjamin Baer, Founding Member of the DecideWise community, argues that using AI alone gives a business little advantage when competitors can access the same models. The difference lies in how each company uses the results. A model can assess credit risk or recommend a response to a customer complaint, but the business still needs rules for approvals, exceptions, and actions. Baer argues that workflows capture this knowledge by defining what information the model receives, how its recommendations are checked, and who can act on them. He sees software’s value shifting from storing records to managing the work those records support. In this approach, AI handles analysis within a defined process, while business rules and human judgment determine how its output affects customers and operations.
Market Pulse
Wisedocs launched Legal Intelligence, allowing insurance claims and defense teams to compare deposition transcripts and demand letters against medical records and claim history. The system flags contradictions, missing records, and treatment inconsistencies, with citations to both documents so reviewers can check each finding. It also produces deposition summaries with page and line references.
Cloudflare released Clef and Clef-flash, two decision models that classify inputs and return probabilities for use in automated workflows. Both support images and a 64,000-token context window, allowing applications to route requests, trigger escalations, or send uncertain cases to a person. Cloudflare reports median latency of 209.3 milliseconds for Clef and 38.8 milliseconds for Clef-flash across 43 evaluation benchmarks. The models are available through Workers AI and under an Apache 2.0 license for local use. Cloudflare also announced a reinforcement-learning platform for fine-tuning Clef to specific tasks.
AWS Strands Labs released Strands Decider 2B, a two-billion-parameter model that selects from predefined answers and returns numerical scores for tasks such as routing, tool selection, and policy classification. It runs locally on CPUs or GPUs, with reported median latency of about 115 milliseconds on an NVIDIA RTX 3090 and 153 milliseconds for small tasks on an M3 MacBook. The release includes model weights, training data, and scripts. The model does not generate text and is less suited to complex reasoning.
Perplexity launched a Decisions API and released the model behind it, pplx-decider-v1-27b, as open weights under Apache 2.0. The API answers questions about supplied content with probabilities, returning yes/no, one option from a defined list, or a rubric level, so a developer can compare each result against a cutoff and act on it. A single request can carry up to 128 questions about the same content, with up to 255 choices or 10 score levels per question, and inputs can include text and images.
Resources and Events
📅 Connected Enterprise 2026 (Half Moon Bay, CA - October 26-29, 2026)
Constellation Research's flagship executive summit returns to the Ritz-Carlton Half Moon Bay, bringing together more than 250 senior executives, board members, technology leaders, and innovators. The program includes analyst talks on AI, digital platforms, and the future of work, alongside executive panels, case studies, roundtables, and small-group workshops. The SuperNova Awards Gala will recognize leaders in business innovation.
📅 Celosphere 2026 (Munich, Germany - December 1-3, 2026)
Celonis' annual conference returns to Messe München, drawing AI and process leaders. The conference will focus on using process intelligence and AI to improve business operations, with sessions covering IT, supply chains, finance, and procurement. Speakers include leaders from Hitachi Energy, Mondelēz International, AkzoNobel, and Avis Budget Group. The program also includes Celonis platform updates, while the pre-conference AI Lab gives participants a day to build functional prototypes using the Celonis Context Model.
📊 Report Spotlight: AGO AI Quality Gate (arXiv)
This paper examines whether AI evaluation scores provide enough evidence to approve a RAG system update. Testing two evaluator models on 1,200 examples each, the researchers found that GPT-4.1-nano was only slightly better than chance at identifying answers that failed to follow their source material, despite consistently returning correctly formatted evaluations. GPT-4o performed better, but its reliability varied across domains. The proposed framework combines fixed checks, guardrails, and AI evaluation, treats missing evidence and evaluator errors as separate outcomes, and estimates the risk of approving a worse version. Its central finding is that the evaluator itself needs validation before its scores can support release decisions.
Join the conversation: What can a Star Wars scene tell us about why decision intelligence matters?
Join the DecideWise discussion on a Star Wars clip that makes the case for decision intelligence.
For the Commute
What Really Slows Optimization Down (Decision Intelligence Lab)
Connor Lawless and Madeleine Udell of Stanford's Management Science and Engineering department join Vijay Mehrotra and Michael Watson to discuss what prevents optimization models from becoming useful business tools. Drawing on interviews with 15 model developers, their research finds that data preparation, communication, and repeated revisions often create bigger bottlenecks than the mathematics. The discussion covers when a workable solution matters more than proving optimality, how validation can catch incorrect results, and where LLMs could help with documentation and debugging.
