Decisions in the Wild

AI Planning Agents Struggle to Ask the Right Questions

A dispatcher is planning delivery routes for the day. The scheduling system includes vehicle capacities, delivery locations, and driver availability. However, one driver must return to the depot by 10 a.m., a condition not in the records. An AI planning agent working with this incomplete information could assign the driver a route that extends well beyond that deadline. To avoid this, the agent would need to recognize that important information might be missing and ask the dispatcher about it before finalizing the routes.

Researchers from MIT, Tsinghua University, Peking University, McGill University, and other institutions examined this problem in Ask2Decide, a benchmark published on November 9. The study tests whether AI planning agents can identify and obtain the missing information that affects operational decisions.

The benchmark contains 43 tasks covering transportation, scheduling, staffing, and other optimization problems. Each agent receives a request and incomplete operational records. An operator holds additional information, such as a vehicle breakdown, an outdated employee qualification, or an unwritten operating rule. The agent can ask questions, build an optimization model, and submit a plan.

Across the tasks, the researchers introduced 235 hidden facts. Of these, 131 affected the optimal decision. They deliberately constructed another 95 to resemble important facts but not change the optimal plan. These served as controls, allowing the researchers to distinguish agents that identified consequential missing information from those that simply collected more details.

The researchers also examined how people respond to questions. Nineteen participants completed 79 sessions as operators. After an agent submitted an incorrect plan, participants withheld the information needed to correct it in 43% of their replies. A conventional simulated operator did so in only 7%.

Using these interactions, the researchers developed a simulated operator whose responses more closely matched human behavior. They then evaluated four models: GPT-6.1 Sol, Gemini 3.8 Flash, DeepSeek V4.1 Flash, and GLM-5.3 Flash, each connected to an optimization solver.

Planning with complete information differed substantially from planning through questions.

When the models received every hidden fact upfront, their average first-plan losses ranged from 0.05 to 0.28, on a scale where zero represents an optimal plan and one represents an infeasible plan. When they had to obtain the information themselves, losses increased to 0.60-0.98.

GPT-6.1 Sol performed best, yet 58% of its first plans were infeasible. Gemini 3.8 Flash produced infeasible plans on 79% of tasks, DeepSeek V4.1 Flash on 81%, and GLM-5.3 Flash on 98%.

The agents also struggled to distinguish useful questions from less consequential ones. Before submitting their first plans, the three models that regularly asked questions had uncovered 39-52% of relevant facts, compared with 32-41% of the similar-looking facts that did not affect the optimal decision.

Allowing the agents to revise rejected plans improved the results. After up to two revisions, average losses fell to 0.42–0.62. However, much of this improvement came from smaller problems. On larger tasks, all four models still performed poorly, and some struggled even with complete information.

For organizations deploying AI in operational planning, the study suggests two useful evaluation measures. The first is whether an agent's initial plan is feasible under actual operating conditions. The second is whether its questions uncover the missing constraints and exceptions that materially affect the decision.

These measures could be particularly useful in dispatch, workforce rostering, and production scheduling, where employees often know about changes that have not reached the underlying systems. An agent needs to identify which operational facts are missing and which could change its recommendation before committing to a plan.

The DecideWise Insider

Benjamin Baer, Founding Member of the DecideWise community, examines what Nvidia’s leadership shakeup and the potential end of its aggressive AI funding strategy could mean for Decision Intelligence practitioners. Drawing on Ben Thompson’s analysis, he highlights the risk that massive AI infrastructure investments could outpace enterprise demand, leaving hyperscalers with excess computing capacity and mounting financial pressure. He argues that businesses will need to justify these investments by generating measurable returns from AI through better operational decisions. That creates opportunities for practitioners working across descriptive analytics, metrics development, simulation, decision modeling, and agentic AI deployment. Baer emphasizes integrating AI into real business workflows while recognizing when traditional software offers a more reliable, cost-effective solution.

Market Pulse

  • Google introduced the Gemini Decisions API, powered by its Gemini Decision 1 model. A single request can carry up to 256,000 tokens of text, images, audio, video, or PDFs and up to 1,000 questions about that input. The shared input is billed once at $0.10 per million tokens, with no charge for output tokens or for separate transcription and OCR. Gemini Notebook uses the API to send each query to the right model tier, cutting latency on everyday requests by up to 40%, and Google Search uses it to detect adversarial behavior in MCP tools.

  • Quantexa launched a Supply Chain Integrity & Fraud solution with PwC, built on its decision intelligence platform and Microsoft Azure. It combines internal procurement and financial data with third-party intelligence, resolves entities across suppliers, intermediaries, and counterparties, and analyzes relationships across several tiers of the supplier network. The solution scores risk across compliance, financial, and operational dimensions and includes investigation and escalation workflows and Power BI dashboards for executives.

  • project44 connected Mo, the conversational analyst it launched in July, to Snowflake Cortex AI. project44 first interprets each question through its logistics ontology and the customer’s definitions, such as how on-time performance is measured. Cortex Analyst then converts it to SQL and retrieves results from about 1 trillion shipment records. Mo now has 210 customers and has answered more than 50,000 questions.

  • Vercel's November AI Gateway Production Index found that decision models handled 26% of requests in October while accounting for just 0.53% of estimated spending. These models perform tasks such as classification, scoring, and selecting predefined options at roughly 70 times lower cost per request than language models. Nearly one-third of paid teams used them, suggesting growing adoption of specialized, low-cost AI for routine decisions.

Resources and Events

📅 TFEST26 (Berlin, Germany - December 1-2, 2026)

TFEST26 returns to the Colosseum in Berlin for two days of sessions open only to director-level and above supply chain leaders, with more than 400 attendees and over 50 speakers across three stages. Chief supply chain officers from Cisco, Estée Lauder, and Henkel open the main stage. The Innovation Stage covers autonomous decision-making, real-time planning platforms, and digital twins. The Lab runs workshops on supply chain risk modeling, network design optimization, and scenario planning. Attendees include leaders from Diageo, Novartis, Schneider Electric, and Mars.

📅 S&OP to Integrated Business Planning Summit 2026 (Amsterdam, Netherlands - December 3-4, 2026)

CParity’s 11th annual summit takes place at the Radisson Blu Hotel Amsterdam Airport. It brings together supply chain, finance, and planning leaders moving from sales and operations planning to integrated business planning. Sessions cover IBP as an enterprise decision engine, the working capital and service gains from planning maturity, and Evonik’s supply chain planning transformation. A practical workshop shows how to build a first AI-assisted planning workflow. Participating companies include Roche, Nestlé, Electrolux, Mercedes-Benz, and SC Johnson.

📊 Report Spotlight: The AI Trust Gap (GWI)

GWI surveyed 1,883 senior marketing, insights, and agency professionals in the UK, US, and Singapore on how AI-generated insights shape business decisions. 88% use an AI tool daily, 80% frequently encounter inaccurate or misleading findings, and 58% have made a significant business decision on an AI insight that later proved wrong. Only 36% verify AI insights most of the time, and 63% have accidentally used data they later found was out of date. 82% report at least one negative impact, most often rework, delayed decisions, and customer complaints.

Join the conversation: What’s going to happen next at NVIDIA?

Join the DecideWise discussion on Jensen Huang’s resignation as NVIDIA CEO, succession planning, executive security at strategically important companies, and the employee petition for his return.

For the Commute

The Data-First AI Strategy Behind a $7M Healthcare AI ROI (The Data & AI Chief)

Dr. Ashis Barad, EVP and Chief Transformation and Digital Information Officer at Hospital for Special Surgery (HSS), discusses how the hospital approached AI adoption by first centralizing data across fragmented systems. He secured 18 months to establish a unified data foundation before scaling AI initiatives. Early applications included ambient AI to reduce clinical documentation burdens and revenue cycle AI, which generated a reported $7 million in annual value. Barad also explains why HSS is moving away from traditional dashboards toward data products operational teams and data specialists develop jointly to support faster decisions.

The Decision Room