Evidence brief
LlamaIndex released ExtractBench, covering 370 enterprise documents and 14 systems, with a focus on testing long-list completeness; it says Frontier VLMs achieve F1 scores of 8.9–35.8% on the longest documents.
LlamaIndex released ExtractBench, covering 370 enterprise documents and 14 systems, with a focus on testing long-list completeness; it says Frontier VLMs achieve F1 scores of 8.9–35.8% on the longest documents.
This brief preserves the original source so the summary and editorial context can be checked independently.
Source attributionX · @llama_index
Open the original sourceRelationships are derived from shared topics, entities, categories, tags, and community context; every result remains independently source-linked.
@Llama_index reposted @jerryjliu0’s post: ExtractBench was described as one of the most comprehensive benchmarks for real-world document extraction, and the post explicitly said it covers 4869 pages.
Original source@Llama_index reposted @jerryjliu0’s post: the team wrote a 36-page ArXiv white paper on ExtractBench, calling it an effort to create a comprehensive, schema-guided, real-world-relevant benchmark.
Original source@dair_ai said Ai2, Carnegie Mellon, and University of Washington studied normalization, GQA, pretraining context length, and sliding-window attention, finding that combining more than three of them significantly reduces long-context performance, and released OlmPool.
Original sourceA new Meta paper proposes Skaling law, using a single interaction index to couple model capacity and data; with a low-compute sparse grid, it can extrapolate the full grid with roughly 10x less compute.
Original source@FireworksAI_HQ announced it will join @antimetal, @braintrust, and @browserbase at The AI Dev Stack: Beyond Code. The post said that beyond coding agents, the more interesting work is the tech stack for getting them into production and keeping them reliable, and provided an RSVP link.
Original source