Evidence brief
Microsoft Research says multilingual Agent evaluations often compare only final answers and discard trajectories; the paper uses action policy as the metric, covering 8 models, 6 parallel benchmarks, 41 languages, and 2.38M rollouts, and analyzes five types of confounding factors.
Microsoft Research says multilingual Agent evaluations often compare only final answers and discard trajectories; the paper uses action policy as the metric, covering 8 models, 6 parallel benchmarks, 41 languages, and 2.38M rollouts, and analyzes five types of confounding factors.
This brief preserves the original source so the summary and editorial context can be checked independently.
Source attributionX · @dair_ai
Open the original sourceRelationships are derived from shared topics, entities, categories, tags, and community context; every result remains independently source-linked.
Mustafa Suleyman says Microsoft’s first reasoning model, MAI-Thinking-1, was built from scratch and is now available on Microsoft Foundry.
Original source@satyanadella reposted @mustafasuleyman’s message: MAI-Thinking-1 is their first reasoning model, built from scratch, now available in Microsoft Foundry, with thanks to the team.
Original sourceLlamaIndex released ExtractBench, covering 370 enterprise documents and 14 systems, with a focus on testing long-list completeness; it says Frontier VLMs achieve F1 scores of 8.9–35.8% on the longest documents.
Original source@Llama_index reposted @jerryjliu0’s post: ExtractBench was described as one of the most comprehensive benchmarks for real-world document extraction, and the post explicitly said it covers 4869 pages.
Original sourceGoogle Research proposed a knowledge profile framework, saying frontier LLM factual encoding is nearing saturation but recall remains insufficient; the framework divides facts into five profile types and introduces the WikiProfile benchmark with 2,150 Wikipedia facts.
Original source