Evidence brief
@Llama_index reposted @jerryjliu0’s post: ExtractBench was described as one of the most comprehensive benchmarks for real-world document extraction, and the post explicitly said it covers 4869 pages.
@Llama_index reposted @jerryjliu0’s post: ExtractBench was described as one of the most comprehensive benchmarks for real-world document extraction, and the post explicitly said it covers 4869 pages.
This brief preserves the original source so the summary and editorial context can be checked independently.
Source attributionX · @llama_index
Open the original sourceRelationships are derived from shared topics, entities, categories, tags, and community context; every result remains independently source-linked.
LlamaIndex released ExtractBench, covering 370 enterprise documents and 14 systems, with a focus on testing long-list completeness; it says Frontier VLMs achieve F1 scores of 8.9–35.8% on the longest documents.
Original source@Llama_index reposted @jerryjliu0’s post: the team wrote a 36-page ArXiv white paper on ExtractBench, calling it an effort to create a comprehensive, schema-guided, real-world-relevant benchmark.
Original source@dair_ai said Ai2, Carnegie Mellon, and University of Washington studied normalization, GQA, pretraining context length, and sliding-window attention, finding that combining more than three of them significantly reduces long-context performance, and released OlmPool.
Original sourceA new Meta paper proposes Skaling law, using a single interaction index to couple model capacity and data; with a low-compute sparse grid, it can extrapolate the full grid with roughly 10x less compute.
Original sourceWhatsApp introduced an optional feature, Scam Alert, which uses on-device machine learning models under end-to-end encryption to identify potential scam messages; message content does not leave the device or get automatically reported.
Original source