Hugging Face, Cognition, and Xiaomi have launched new AI models enhancing long-horizon coding and robotics capabilities. These models seek to offer improved performance by leveraging extensive contexts and innovative techniques.
GLM-5.2 extends support for coding-agent scenarios with a robust 1 million token context. Its architecture allows for efficient handling of complex coding tasks. The model is open-source, available under an MIT license.
Cognition's SWE-1.7 model advances long-horizon asynchronous tasks by applying improved reinforcement learning methods. It aims to enhance cost-performance efficiency for software engineering tasks.
Xiaomi-Robotics-1 leverages 100,000 hours of pre-training data to address robotics' data scarcity. It combines pre-training with real-robot data to improve model capability, providing insights into large-scale training effects.
These models underline ongoing advancements in high-reasoning AI necessary for complex problem-solving. By achieving breakthroughs in long-horizon tasks, these developments suggest a shift towards more sophisticated AI solutions.
β¨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors β check the original sources. How BrevFeed works β
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown β listen, watch, or read the transcript.
βΆ Play today's briefNew every morning, and the back catalogue is archived by date.
Amazon ElastiCache now offers distributed Key-Value (KV) caching for self-hosted Large Language Models (LLMs), which can reduce inference costs by up to 40% and time to first token by up to 80%. This feature addresses the issue of redundant GPU work in self-hosted LLMs by allowing shared KV state across GPU fleets, improving performance for AI agents and chat applications.
AI systems designed and verified a new general-purpose CPU architecture called Frostbite, which utilizes interaction nets for its instruction set. This AI-designed CPU achieved a 4.59x performance-per-watt improvement compared to a modern superscalar CPU core on a set of 25 workloads. The development suggests AI can lead to novel hardware discoveries and fundamental architectural shifts in chip design.
Exa optimized its embedding infrastructure, achieving a 50% increase in throughput for indexing billions of documents. This improvement was realized by migrating to AWS Neuron hardware (Inferentia2 and Trainium2) for inference workloads. The change reduces per-document embedding costs, which is critical for Exa's large-scale web indexing and semantic search capabilities.
Cloudflare has released an updated version of its Clairic AI model and introduced Clairic-flash, a lighter, faster variant. Both models are open-weight, hosted on Workers AI, and available on Hugging Face, offering improved data extraction from complex documents including images and PDFs.
RadixArk has released DaisyRose-27B, a new model built on Qwen3.8-27B and trained with reinforcement learning on long-horizon agent tasks. The model shows improved performance on benchmarks like Terminal-Bench 4.1, increasing pass@1 from 5.6% to 12.0%, and also demonstrates gains in reasoning, instruction following, and coding tasks.
RadixArk released DaisyRose-27B, its first model, which is built on Qwen3.8-27B and trained using reinforcement learning with the Miles framework. The model shows an improvement in Terminal-Bench 4.1 pass@1 from 5.6% to 12.0% and gains on other reasoning, instruction following, and coding benchmarks. This was achieved using 1,200 curated long-horizon agent tasks over 18 days on 144 H200 GPUs.
Researchers developed a new predictive model that estimates the end-to-end fine-tuning time for generative language models on shared GPU clusters with higher accuracy than traditional FLOPs-based calculations. This model, based on 272 production runs, achieved an RΒ² of 0.98 and a mean absolute error of 1.71 hours, significantly improving upon the RΒ² of 0.31 and MAE of 12.41 hours from FLOPs estimates. The improved prediction allows for better resource allocation and scheduling on shared GPU infrastructure.
Meta developed a template-driven approach for its large-scale ML ecosystems, standardizing model design into reusable building blocks. This method reduced per-model engineering time by 92% and improved offline model quality by 0.63% on average, addressing challenges in shipping and propagating new ML techniques across diverse models.
Researchers introduced Energy-Based Reasoners (EBRs), a new type of looped model that learns an energy function to guide multi-step reasoning. EBRs formulate looped updates as gradient descent on this energy, allowing for explicit assessment of progress and improved efficiency. On algorithmic reasoning benchmarks, EBRs matched or surpassed prior recurrent reasoning models, achieving 99.9% accuracy on Sudoku-Extreme with 345x less inference compute.
Researchers achieved efficient Mixture-of-Experts (MoE) model inference on consumer-grade hardware, including an RTX 4090 and a MacBook M5 with 32GB RAM. By streaming MoE experts directly from SSD, performance reached within 6% of the Strata inference engine for token generation, demonstrating a method to run large MoE models locally without requiring extensive VRAM.
Unsloth has updated its Qwen3.8-27B GGUFs to use the new Dynamic V3.1 algorithm, which improves average accuracy by 5-10% across various benchmarks. This update makes the 27B parameter model, featuring vision and reasoning, more efficient for local deployment on systems with 17GB RAM/VRAM.
Researchers demonstrated that one AI model can directly modify the weights of another, altering its knowledge and behavior. This "weight space misfolding" allows for persistent changes, such as factual substitutions, and presents a new security vulnerability in AI systems. The findings highlight the potential for unauthorized behavior induction and the challenges in defending against such attacks.
The llama.cpp and ggml projects have released version 1.0.0 of their libraries, establishing semantic versioning for future API and ABI changes. This update provides a stable foundation for developers building local AI applications, as llama.cpp enables multi-modal LLM execution across diverse hardware through optimization and C/C++ implementation.
Edge0 has open-sourced Edge0-145B and Edge0-120B, the first on-device AI models with up to 145 billion parameters, along with a unified inference framework and deployment tools. This release allows complex AI capabilities like coding and reasoning to run directly on personal computers, addressing limitations of smaller on-device models and cloud-based solutions.
IBM developed LLMON (LLM Object Notation), a markup language designed to provide explicit structure and semantics to large language models. This language aims to improve LLM robustness and security by clearly defining context components, and two preview models, granite-4.1-3b-LLMON-preview and granite-4.1-8b-LLMON-preview, have been released to support it.
The smolmaxxing project allows 8B and 16B mixture-of-experts language models to run on devices with approximately 1GB of RAM, such as TV boxes and old tablets. This is achieved through custom kernels, optimized memory usage, and streaming model weights from flash storage, extending the capabilities of llama.cpp to older and smaller hardware.
Huawei has released MACaQ, an internal quantization tool designed for local AI inference across various model types, including text, image, video, and audio. MACaQ offers faster quantization times and competitive perplexity compared to existing methods, supporting frameworks like llama.cpp, vLLM, Diffusers, and PyTorch.
A new ML-driven system called Intelligent Delivery has been developed to optimize compute resource allocation for ad ranking models. This system dynamically directs resources to areas with the highest marginal utility, resulting in a 10% incremental improvement in ad ranking performance since 2017. It addresses the challenge of increasing model complexity outpacing hardware capabilities by efficiently utilizing existing compute infrastructure.
A new training method called token pooling-aware training improves the retrieval quality of multi-vector embedding models while reducing storage requirements. This method teaches the encoder to preserve important information when token vectors are merged, leading to better performance at the same compression level. For example, a model trained with this method retained 97.8% of its unpooled nDCG@10 with a 4x reduction in document vectors, compared to 94.6% for a normally trained model.
An open field guide introduces inference engineering for platform engineers, explaining how models and GPUs operate together in production environments. The guide covers tools and career development for this role, emphasizing the shift towards self-hosting AI models due to cost, control, and data privacy concerns.
Meta developed Triton Low-Level Extensions (TLX), an extension to Triton, to optimize GPU performance for production machine learning workloads. TLX provides explicit control over GPU execution while maintaining Triton's programming model, bridging the gap between low-level control and high-level productivity. It has demonstrated performance improvements in RecSys training, RecSys inference, and large-scale LLM pretraining across NVIDIA and AMD GPUs.
Google introduced TPU Sync, an open-source C++ system designed to accelerate AI inference by optimizing data movement between different memory types. This software targets large models and reinforcement learning workloads, claiming over 90% reduction in KV-cache transfer latency and time to first token in specific benchmarks. It addresses challenges in data transfer that increase with larger context windows.
Meta has introduced Muse Ember, a new AI model designed for complex reasoning and agent-based tasks like tool calls, coding subtasks, and document parsing. The model offers high intelligence per dollar compared to competitors and is available on the Meta Model API.
Elith and NVIDIA have released Nemotron-Personas-Finance-Japan, a synthetic dataset designed for financial AI development in Japan. This dataset, built using official statistics and industry feedback, provides 9 million persona records to enable AI development without real customer data, addressing privacy and compliance challenges in the financial sector.
Google Cloud has launched TPU Sync, an open-source C++ library designed to optimize model state management across TPU clusters. It reduces KV cache transfer latency by over 90% and Time to First Token (TTFT) by an average of 90% for large MoE models by treating HBM, host DRAM, and network as a unified memory hierarchy.
Researchers developed HOPE (Higher-Order Pruning of Experts), a new method for pruning Mixture-of-Experts (MoE) models that considers pairwise expert interactions. This second-order approach addresses the memory bottleneck in large language models by improving expert removal efficiency, achieving up to 6.1% better performance on coding tasks compared to existing methods at 50% pruning rates. HOPE offers a more effective way to reduce the substantial memory footprint of MoE models without retraining.
Researchers introduced Next-Concept Modeling (NCM), a two-stage, coarse-to-fine framework for text generation that uses a multiscale residual VQ-VAE and a transformer. NCM achieves significantly faster generation speeds compared to diffusion language models (DLMs) while maintaining or improving perplexity and ROUGE-L scores on various benchmarks. This method addresses the speed limitations of existing parallel generation techniques, offering a more efficient approach to text generation.
Cognition released SWE-2 Flash, a new coding model that is three times faster and one-fifth the cost of its SWE-2 model. Built on Thinking Machines Lab's open-weight Inkling-Small, Cognition is keeping the weights of SWE-2 Flash private, citing cybersecurity concerns, despite using an Apache 2.0 licensed base model.
Memorilla is a new memory system for large language models (LLMs) that condenses information from document collections into a fixed number of "memory tokens." This system allows frozen LLMs to access and utilize extensive knowledge more efficiently, outperforming existing baselines on several benchmarks while using significantly fewer tokens.
Open-weight AI models, like Qwen and GLM-5, are increasingly adopted for generative AI, narrowing the performance gap with closed models. The primary challenge and value in using these models lie in implementing learning loops for fine-tuning and reinforcement learning, which requires managing diverse infrastructure for inference, training, and agent execution.
Baidu has released ERNIE 5.5 Flash Base, a sparse Mixture-of-Experts (MoE) language model with 280 billion total parameters, activating 14 billion per token. This base model, pre-trained on 33.5 trillion tokens with a 128K context, supports a 1M-token context for fine-tuning and inference, providing a foundation for domain adaptation and research.
Ricursive Intelligence released Ripple-65B Preview, a 65-billion parameter language model that achieves a new U.S. open-weight state-of-the-art on agentic coding benchmarks. The model was trained on 12 trillion tokens for $32 million, significantly less than other frontier models, and supports up to 1M context length. This release provides a more efficient, high-performing model for agentic coding tasks, making advanced AI capabilities more accessible.
Anthropic, in collaboration with EPFL, MIT, and others, developed FlashProve, a system that uses zkSNARKs to cryptographically prove that an AI model's responses originate from a specific, committed model. This system addresses the challenge of verifying AI output authenticity, which is crucial for applications requiring trust in AI-generated content.
The Kimi K3.1 Pro model, an open-weight and natively multimodal agentic model, has been released. It features a 1-million-token context window and incorporates Gated DeepSeek Sparse Attention layers for improved long-context modeling, showing advancements in coding, 3D creation, and financial research.
A new book, "The Little Book of Generative AI Foundations," offers a derivation-oriented introduction to the mathematical principles behind modern generative artificial intelligence. It covers various generative models, including PCA, VAEs, diffusion models, GANs, and energy-based models, to make their underlying structure more accessible.
Google introduced the LiteRT Tensor API, a C++ API for building, customizing, and optimizing machine learning models for on-device execution. This API allows developers to define computation graphs in C++ that run across CPU, GPU, and NPU hardware without rewrites, addressing the challenge of backend-specific implementations and manual engineering for cross-platform deployment.
A user demonstrated that the Qwen 3.8 27B model, running on a laptop, can interpret raw neural network weights. This capability mirrors findings from a recent paper, "Sparks of Weight-Space Programming," which showed frontier models like GPT-6 Astra could interpret and modify model weights directly. The paper introduced the concept of "Weight-Space Programming," where LLMs analyze and modify model weights without external tools or code execution.
Meta has released F3 Lightning, an infrastructure layer built on its existing Facebook Feature Framework (F3), to decouple feature releases from shared execution in its ads ranking models. This allows individual machine learning features to be compiled, validated, deployed, and rolled back independently, reducing deployment time for comparable extraction features by approximately 3.6 times without increasing computational cost.
ByteDance developed USDB, a unified sequence storage system, to manage the growing data demands of its recommender systems. USDB handles both streaming and batch workloads, reducing write latency by 85.8% and infrastructure costs by 42.1% in production. The system's design allows for efficient updates and large reads of user histories, supporting fresh serving and repeated historical access for inference and training.
Palette developed a method to fine-tune a 20B open-weight model to generate slide decks conforming to specific design languages, addressing limitations of frontier models regarding custom styling and data privacy. The model was trained on synthetic data following the IBM Design Language and achieved top rankings on two of three PresentBench domains.
Hugging Face introduced Model Integration Contracts (MICs) to address regressions in its Transformers library, such as a bug that allowed T5 models to attend to future tokens. MICs use small, representative model fixtures and baselines to quickly identify issues before they reach the main branch. This system aims to improve the reliability of the Transformers library by enabling faster and more frequent integration testing.
GPT-6 Astra can interpret and modify neural network weights, a task typically impossible for humans or conventional algorithms. This capability allows Astra to identify and fix issues in models like Qwen3.5-0.8B, significantly improving its math performance without gradient-based training. This development suggests a new method for optimizing and understanding neural networks through direct weight manipulation by LLMs.
New research introduces a "Platonic Representation Theorem," demonstrating that Joint-Embedding Predictive Architectures (JEPAs) trained on different modalities learn the same representation under specific conditions. The study indicates that using "good views" (observations of one scene) requires significantly fewer training samples for alignment compared to next-token prediction, which uses "bad views" (parts of a single document). This suggests a more efficient path to achieving aligned representations across modalities.
A team of five Qwen3.8-27B models, orchestrated by a Memdoor workflow, achieved comparable performance to Fable 5 on a subset of the LiveCodeBench Hard programming problems. This demonstrates that smaller, specialized models can collectively solve complex coding tasks when managed by a structured workflow, offering an alternative to larger, single-model solutions.
New research on Hierarchical Networks (H-Nets) demonstrates significant compute efficiency improvements for language models, achieving up to 3-5x gains over tokenized Transformers. H-Nets address the limitations of hand-crafted tokenizers by learning directly from raw data, potentially enabling more general AI architectures.
Meta has updated its Andromeda personalized ads retrieval engine, achieving a 100x increase in throughput compared to its 2024 system. This improvement allows the system to score over one billion ads per second with near ranking-level model performance, leading to a 2.4% increase in conversions on Facebook since 2025.
LFM3, a new generation of Liquid Foundation Models, offers specialized AI capabilities across text, vision, audio, omni, and decision modalities using a shared 2B-parameter backbone. These models are designed for fine-tuning on a single GPU and deployment on edge devices, providing increased capability per parameter and support for agentic use cases. The update includes a new tri-hybrid architecture and expanded modality support, including the first omni model and video input for vision.
Liquid AI introduced LFM3, a new generation of multimodal Liquid Foundation Models (LFMs) designed for specialized intelligence. LFM3 integrates text, vision, audio, omni, and decision models on a shared backbone, offering increased expressiveness and efficiency for fine-tuning on various devices. The initial release includes 2B parameter models, available as open-weight on Hugging Face, Liquid AI's Playground, and API.
Khawaja Shams, co-founder of Momento, discussed how distributed systems principles can improve the utilization of LLM inference stacks. He highlighted that optimizing GPU utilization is crucial given the scarcity of GPUs and data centers, leading to better availability, performance, and economics.
Cloudflare's GLM-5.3 Workers AI model experienced a tenant isolation issue between August 27 and September 9, 2026, where 155 responses contained data from other customers' requests. The issue stemmed from a misconfiguration in the SGLang inference engine and a bug in its handling of KV cache, affecting only prompts longer than 681,280 tokens.
Sebastian Raschka released an 11-hour video course on building reasoning models from scratch using PyTorch. The course covers inference scaling, reinforcement learning, and distillation, and is a sequel to a previous LLM coding course.
vLLM version 0.31 introduced support for constrained decoding, a feature that guarantees 100% schema adherence for tool calls. This update addresses previous issues where open-weight models like Kimi K2.6 achieved only 92% tool-call schema accuracy with vLLM 0.28, leading to poor results in agentic workloads.
Researchers developed an attention-free neural network architecture using Grassmann flows, offering an alternative to standard multi-head attention in sequence modeling. This new approach achieves performance comparable to Transformers on language modeling and natural language inference tasks, suggesting a more structured method for neural reasoning.
Cerebras has outlined its method for deploying trillion-parameter AI models, such as the 3-trillion parameter Kimi K3, across multiple WSE-3T wafer-scale chips. This approach allows these large models to operate at speeds of 1000-2000+ tokens per second by distributing model layers and managing memory across 59 wafers for the Kimi K3 example.
Kimi introduced K3.1 Pro, an open-weight model with a 1-million-token context window, and K3.1 Pro Highspeed, which offers 2-3x faster performance. These models show improvements in coding, 3D creation, and financial research, and are available on Kimi platforms and API.
A collaboration with Theorem demonstrated that LLMs, specifically Claude, can generate formal proofs for security properties at the assembly code level across over 100 real-world open-source software libraries. This approach made formal verification feasible at a larger scale than previously possible, uncovering long-standing bugs in heavily tested code.
Minotaur, an AI architecture built on ELC, demonstrates capabilities for deduction, induction, diagnosis, and other reasoning tasks typically associated with large language models. It performs these operations on a single processor core without requiring a GPU, completing tasks in approximately a quarter of a second per question. Minotaur matched a current language model's reasoning performance in blind tests involving complex scenarios like police interviews and system diagnosis.
NVIDIA introduced Kumo Relational, a pre-trained relational foundation model that makes predictions directly on enterprise databases without task-specific training or feature engineering. It outperforms supervised machine learning models on relational benchmarks, including gradient-boosted trees and AutoGluon ensembles, across various tasks and databases. This development simplifies predictive modeling on structured data, potentially replacing existing production pipelines.
Researchers developed VocabX, a method to specialize large language model vocabularies, reducing inference costs by 77% in a production essay grader. This specialization optimizes tokenization for specific domains, addressing inefficiencies in general-purpose LLM vocabularies.
ZenteiQ, an AI startup, increased its batch inference throughput by 4.07 times, growing daily output from 1.47 billion to 5.97 billion generated tokens per 1,000 accelerators. This improvement was achieved by using Google Kubernetes Engine (GKE) and GKE Inference Gateway, powered by llm-d, to accelerate its synthetic data generation pipeline for scientific foundation models.
i14 developed Laplace 3B, a linear attention model, and successfully trained and served it on AWS Trainium2 using native PyTorch. This model offers a fixed recurrent state of 29.5 MB per sequence, addressing the growing KV cache issue in traditional transformer models and potentially reducing inference costs and improving concurrency for long-context applications.
Ryan Hanrui Wang, SVP of Applied AI at Nebius, states that relying solely on open model benchmarks is insufficient for production AI. He advocates for evaluating the entire inference stack, including internal benchmarks and in-context learning, to ensure models meet specific organizational needs. This approach helps align AI performance with actual production requirements and cost considerations.
Researchers fine-tuned the GPT-OSS-20B model to improve its multilingual reasoning capabilities, addressing its tendency to revert to English for chain-of-thought processes. The project utilized Amazon SageMaker HyperPod and explored the effectiveness of Supervised Fine-Tuning (SFT) and Reinforcement Learning with Group Relative Policy Optimization (GRPO) in closing this language gap.
Voxtral 3 launched with a new voice agent SDK and speech models for real-time, multi-step conversations in over 50 languages. This release aims to simplify the development of end-to-end voice agents by integrating transcription, reasoning, and speech synthesis into a single pipeline.
Google, in collaboration with Red Hat, IBM, and others, released llm-d 1.0, an open-source, Kubernetes-native stack for distributed LLM inference. This release addresses challenges in serving multi-step agent workloads by providing cache-aware routing, multi-tenant traffic control, and tiered KV cache on Google Cloud infrastructure.
DeepSeek has open-sourced DeepInfer, the inference framework used for its large-scale production services. This release provides core features for offline inference at scale, including optimized kernels, various parallelism techniques, and advanced decoding methods, supporting NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs.
vLLM, an open-source LLM inference engine, released a push-based NIXL connector for disaggregated prefill/decode inference. This new method moves KV cache transfer coordination off the critical path, reducing time-to-first-token (TTFT) and time-per-output-token (TPOT) for LLM inference.
vLLM has implemented bi-directional Key-Value (KV) cache transfer to optimize multi-turn agentic workloads. This feature allows prefill and decode nodes to reuse previously computed context across turns, reducing redundant computation and improving Time to First Token (TTFT).
Google released EmbeddingGemma 2, an open-weight, multimodal embedding model based on Gemma 4 and licensed under Apache 2.0. This model supports text, code, images, video, and audio in a unified 768-dimensional embedding space, enabling efficient semantic search and RAG on consumer hardware.
DeepSeek released DeepSeek-V4.1-Flash-1017, an update to its AI model, which shows improvements in reasoning, coding, agentic tasks, and visual benchmarks. The update also reduces task execution time and output length, leading to lower operational costs and faster task completion.
Reflection released Beam, a 500B-parameter open-weight model for software engineering and agentic tasks, under an Apache 2.0 license. Google Cloud announced day 0 support, providing optimized deployment recipes for Beam on its AI Hypercomputer and NVIDIA AI Infrastructure, achieving up to 5x higher performance and 8x lower latency.
AI4Bharat and NVIDIA have released Nemotron-pretraining-Indic-v1, a dataset of approximately 2.8 trillion tokens across 22 Indian languages, along with an open, GPU-accelerated data generation pipeline. This initiative provides a substantial new resource for Indic language model pretraining, addressing the data scarcity for these languages.
A new 501B-parameter Mixture-of-Experts (MoE) large language model named Beam has been released, featuring 23B active parameters per token and a 1M token context window. Beam is designed for on-premise deployment and customization for coding and agentic applications, offering native transformer implementation and day-zero support in libraries like vLLM and SGLang.
Granite Switch 4.1-8B and -30B models now combine chat, decision-making, and speech capabilities into a single endpoint, utilizing Activated LoRAs and an embedded ASR model. This integration allows for faster switching between different AI tasks while maintaining a similar VRAM footprint to the base model alone.
Apertus 1.5, a new version of the open-source AI model, has been released with native multimodal understanding for images and audio, a 4x increase in context window to 262,144 tokens, and a new "Thinking Mode" for improved reasoning. The update adds 4 trillion tokens to the 8B model and 2 trillion tokens to the 70B model, maintaining its open weights, data, and training details.
Meta has released Muse Spark Open, an open-weight multimodal mixture-of-experts (MoE) model designed for agentic coding tasks. This model handles code reading, writing, debugging, and tool calls, accepting image, audio, and video inputs, and is the largest open-weight model from Meta to date.
A new 24-billion-parameter reasoning model, LFM2.5-24B-A2B, has been released, featuring 2 billion active parameters for fast, private on-device agent operations. This model runs up to 1.7 times faster than comparable Mixture-of-Experts (MoE) models on-device and shows competitive performance against larger models on agentic benchmarks. It is open-weight and available on Hugging Face and the developer's Playground.
Netflix has integrated its Foundation Model into various personalization applications using three distinct approaches: embeddings, subgraph, and fine-tuning. This initiative centralizes member preference learning to improve efficiency and innovation across specialized models powering the Netflix homepage. The integration methods address varying application needs, including latency requirements and technical stack constraints.
LFM2.5-24B-A2B, a new AI model, has been released, offering competitive performance in agentic and tool-use tasks compared to larger models. It features a larger context window, expanded vocabulary, and is optimized for efficient on-device inference. The model's development focused on reasoning capabilities and agentic reinforcement learning.
Meta is evolving its Generative Ads Model (GEM) from isolated per-domain optimizations to a single central foundation model for its entire ads ecosystem. This change allows GEM to learn from data across all Meta platforms, improving ad performance and advertiser outcomes. The updated GEM has already delivered significant improvements in conversion rates and click-through rates across Meta's ad surfaces.
Meta developed a system that utilizes idle inference capacity within its existing fleet to enhance ads recommendations. This system aggregates unused capacity into a shared pool, allowing for more relevant ad delivery and improved advertiser outcomes without requiring new hardware. Two elasticity levers, Rank More Ads (RMA) and Serve Better Models (SBM), enable the system to adapt workloads to available opportunistic capacity while protecting baseline performance.
AWS has released multi-node Ray LLM Deep Learning Containers (DLCs) for deploying large language models across multiple GPU nodes on Amazon EKS. This allows for multi-node inference setups, including prefill/decode disaggregation, to improve performance and handle models too large for a single GPU.
Researchers from Meta, MIT, and the University of Washington developed Context Language Models (CLMs) that allow large language models to manage and edit their own context. This method improves performance and computational efficiency by enabling LLMs to learn context-management strategies, moving beyond predefined summarization or retrieval techniques.
Microsoft has launched Microsoft-Decision-1, a new AI model designed for rapid decision-scoring and classification tasks. This model provides structured outputs for software actions, outperforming other decision models and LLMs in speed and accuracy on specific benchmarks.
Nemotron 3, a foundation AI model, was fine-tuned using supervised fine-tuning (SFT) and reinforcement learning (RL) to achieve gold-medal level performance in both the International Olympiad in Informatics (IOI) and International Mathematical Olympiad (IMO) 2026. This demonstrates that a strong base model can be adapted to demanding domains with a reproducible recipe, rather than requiring new foundation models for each challenge.
Google released EmbeddingGemma 2, an open 740-million-parameter model that unifies text, code, image, video, and audio retrieval into a single 768-dimensional vector space. This allows for on-device multi-modal search without prior transcription or captioning, making advanced AI capabilities more accessible on mobile devices.
EmbeddingGemma 2 has been released as an open-source, lightweight multimodal embedding model, expanding beyond text to unify code, images, video, and audio. This update allows for on-device processing of diverse data types, enabling applications like searching video clips from voice memos or audio recordings from text queries.
Researchers introduced Dust, a zeroth-order method that can pretrain transformer language models competitively with backpropagation. Dust perturbs activations independently per token, allowing parallel evaluation of virtual population members in a single forward pass, offering a more compute-intensive but potentially scalable alternative to traditional backprop.
Reflection AI launched Beam, an open-weight, text-only AI model with 501 billion parameters, claiming it matches leading Chinese models in advanced reasoning benchmarks at a lower computational cost. This release intensifies competition in the open-source AI market, offering an alternative to both closed-source models and existing open models from Chinese developers.
Reflection has released Beam, its first open-weight Mixture-of-Experts model with 501 billion total parameters and 23 billion active parameters. Beam focuses on coding, reasoning, and agentic workloads, offering competitive performance with high inference efficiency due to extensive pretraining and high-compute reinforcement learning.
A new English-German Mixture-of-Experts Transformer model, Kolibri, with 78B total parameters and 1M token context length, has been released under the Apache 2.0 license. This model is specialized for regulated sectors and offers full supply-chain integrity and deployment freedom for customers.
DwarfStar 4 (ds4), a C inference engine, allows local execution of large language models like DeepSeek V4/V4.1 Flash, GLM 5.x, and Qwen3.8 Flash Next on high-memory Mac, CUDA, and ROCm machines. It uses asymmetric 2-bit quantization to compress routed experts while maintaining critical shared paths, making large MoE models practical for local inference.
Olmo-core 3 has been released, featuring a redesigned open mixture-of-experts (MoE) training system for large language models. This upgrade aims to scale MoE training into the trillion-parameter range while maintaining computational efficiency, addressing the high computational costs of advanced AI model development.
Researchers introduced Context Language Models (CLMs), which allow language models to manage their own context by treating it as an editable file. This approach improves performance and reduces computational costs compared to existing context management strategies across various tasks. CLMs also enable in-context and parametric learning of context-management strategies, and can be optimized with reinforcement learning.
Magnitude, an open-source inference engine, has launched to accelerate open model execution for AI agents by optimizing kernels for specific hardware. It claims up to 2x faster performance than llama.cpp and reduces memory usage, allowing agents to run more efficiently on various devices.
A new language model named PSSA, built from scratch in Rust without existing ML frameworks, uses a recurrent state-space layer and episodic memory instead of a transformer architecture. PSSA demonstrates faster learning and twelve times quicker text generation on a CPU compared to transformers with matched parameters and corpus. This architecture offers linear cost growth with sequence length, addressing the quadratic cost of transformers.
A distributed inference engine now runs a sliced 0.5B BitNet LLM across a cluster of seven ESP32S3 microcontrollers. This setup uses one master node for tokenization and embedding, and six compute nodes for transformer layers, communicating via a high-speed SPI daisy-chain.
Research demonstrates that the presence or absence of a chat template significantly alters how Large Language Models (LLMs) generate self-referential disclaimers like "I'm just an AI" or experiential statements such as "I feel." This finding suggests that LLM self-descriptions are influenced by deployment settings rather than inherent self-knowledge, impacting how researchers should interpret model introspection.
Optimizations to llama.cpp's prompt lookup decoding have increased its speed by up to 140 times and reduced memory usage by up to 2.6 times. These improvements enhance the performance of a key token generation method used in large language models.
DeepSeek has introduced DeepSeek Elastic Compute (DSec), a production sandbox platform designed for large-scale agentic training and evaluation of large language models (LLMs). DSec provides elastic execution environments, supporting various sandbox backends and managing lifecycle, resource allocation, and image distribution to handle the demands of LLM agent workloads.
Amazon EKS, Elastic Fabric Adapter (EFA), and DeepEP are used to scale Mixture-of-Experts (MoE) reinforcement learning, achieving a 40% increase in throughput. This addresses challenges in coordinating heterogeneous compute, sustaining high-throughput communication, and dynamically orchestrating subsystems for large-scale MoE model training.
LFM2.5-VL-DSpark, a new vision drafter, significantly speeds up inference for vision-language models (VLMs) by up to 3.13x on devices and 2.66x on H100 GPUs. This acceleration is achieved with a small increase in model parameters, making VLMs more efficient for various tasks.
Researchers from Stanford University and NVIDIA Research developed Contrastive Language Models (CLM), which perform on par with Jev while running up to nine times faster. CLM uses a contrastive objective to train state and action encoders, enabling faster decision-making in tasks with many candidate actions or frequently reused actions.
AlphaEvolve uses a split-loop architecture to optimize video processing by combining cloud-based code generation with local hardware execution. This method aims to speed up real-time streaming applications by autonomously tuning code, addressing the challenges of manual optimization.
Hugging Face's Transformers library now supports running llama.cpp's GGUF quantized models, making it easier to perform local AI inference, particularly on Apple Silicon. This integration reuses llama.cpp's underlying GGML kernels to improve performance, allowing users to run large language models on personal machines with reduced memory footprints.
A new byte-level language model, Mini-AGI, has been developed that can train continually on a single 8GB VRAM GPU by dynamically managing its architecture and paging weights from disk. This experiment demonstrates the feasibility of continual learning without catastrophic forgetting on modest hardware, allowing individuals to train and adapt their own models.
Version 1 of the Tokenizers library has been released, focusing on performance enhancements to prevent tokenization from bottlenecking AI model workloads. The new version offers speed improvements, often by tens of times, compared to v0.23 while maintaining API and output compatibility.
ByteDance Seed and Tsinghua Air have released DAPO, an open-source reinforcement learning system for large language models, including its algorithm, code infrastructure, and dataset. DAPO achieved a 50% score on AIME 2024 using the Qwen2.5-32B model, demonstrating state-of-the-art performance in large-scale LLM reinforcement learning.
A researcher argues that the perceived intelligence in chat-based large language models (LLMs) is an illusion, akin to a psychic's cold reading. This perspective suggests LLMs create an impression of specific engagement through statistically generic responses, rather than actual reasoning.
A new concept, "linguistic illegibility," describes scenarios where an LLM's language outputs or features do not accurately reflect its internal computations. This phenomenon means that security mechanisms relying on an LLM's linguistic self-reporting are inherently unreliable, necessitating alternative sandboxing techniques like taint tracking and robust virtualization.
Researchers introduced Cache-to-Cache (C2C), a new method for Large Language Models (LLMs) to communicate directly via their KV-caches, bypassing text generation. This approach improves response quality by 3.1-5.4% and offers a 2.5x speedup compared to traditional text-based inter-LLM communication. C2C allows multi-LLM systems to leverage deeper semantic information and reduce latency.
Researchers have proposed "Infinite-Parameter LLMs," an architecture that generates and adapts model weights from live interaction data using a compact hypernetwork. This approach allows language models to learn from real-time user input, addressing limitations of static pre-trained models that rely on prompts for dynamic information. The method aims to improve compute efficiency, free context windows, and enable better generalization compared to in-context learning.
Large language models (LLMs) often hallucinate syntax when generating domain-specific languages (DSLs) due to low training data frequency. Typed Domain Grounding (TDG) addresses this by embedding the DSL as a typed internal DSL within a host language, surfacing domain errors as compiler type errors. This method demonstrated higher structural fidelity and lower hallucination rates compared to external DSLs in benchmarks with Claude Sonnet 5 and GPT-4o.
Pinterest Engineering updated its Manas distributed search platform to handle billions of embeddings more efficiently. The update uses quantization techniques (Scalar Quantization and Product Quantization) and SSD-based serving via SPANN to reduce memory usage and serving costs by 20-30%. This allows the platform to scale discovery experiences like Home Feed and Search while managing increasing data volumes.
Google Cloud is developing an Autonomous Network Operations framework that uses Graph Neural Networks (GNNs) and AI agents to manage complex telecommunications networks. This framework aims to enable Level 5 network autonomy by combining advanced diagnostics with AI reasoning capabilities.
The AI industry's focus has shifted from training large language models (LLMs) to optimizing inference, the process of using trained models to generate outputs. This change is driven by the increasing utility and widespread use of LLMs, which now often perform multiple inference steps per query and operate continuously through agentic AI. The shift is leading to new hardware demands and unexpected collaborations among tech companies.
A new GitHub repository, OpenArch, provides from-scratch PyTorch implementations of modern open-source LLM architectures, focusing on clarity and readability rather than production optimization. This resource aims to help developers and researchers understand the structural choices within LLMs by presenting each architecture in a single, well-documented file.
The Recurrent Looped Transformer (RLT) introduces a design that integrates a causal encoder with a recurrent decoder. This architecture allows the temporal processing depth to increase with sequence length, as each token extends the recurrent path through the full decoder, potentially enabling deeper latent reasoning.
LinkedIn has detailed its AI training infrastructure, which uses a multi-teacher distillation pipeline to compress large teacher models into a 0.6B-parameter ranking model for job search. This system, built on SGLang, significantly speeds up the iteration process for training small language models by optimizing how teacher models are served during the training loop. The advancements allow LinkedIn to improve the relevance and engagement goals of its AI-powered job search, addressing a common challenge in shifting from keyword-based systems to LLM-supervised rankers.
Amazon SageMaker Inference now includes prefix-aware routing, a new strategy that directs requests with identical prompt prefixes to the same instance. This feature allows LLM serving frameworks to effectively utilize prefix caching, reducing time-to-first-token (TTFT) and increasing throughput by reusing cached key-value pairs.
Cognition has released SWE-2, an AI model with 2.8 trillion parameters, achieving a score of 92.8 on the Terminal-Bench 2.1 benchmark. This model, which uses a Kimi K3 base with Cognition's post-training and scaled reinforcement learning, is now available in Devin Desktop and CLI, offering improved performance and cost efficiency for agentic coding tasks.
A new configurable, instruction-driven PII detector built on large language models (LLMs) has been developed, designed to run on any LLM managed on Amazon Bedrock. This detector addresses the challenge of PII leakage from models fine-tuned on real-world text by offering a flexible approach to identifying sensitive information without retraining.
Cognition has released SWE-2, its latest coding model, which achieved 50.0% on the FrontierCode 1.1 Main1 benchmark and offers a 64% cost reduction compared to Fable 5.1. This model scales reinforcement learning to the multi-trillion-parameter regime and improves upon previous versions by advancing the entire cost-performance frontier for AI coding.
An individual developer successfully trained a 3.8 billion parameter language model to a CORE score of 0.384 using 65 billion tokens for $998. This demonstrates that meaningful LLM training is achievable outside of large research labs or companies with substantial compute budgets.
A new experiment re-evaluated the impact of reasoning prefills on open models using GPT-5.5 Pro as the teacher model. Qwen 3.8 demonstrated a significant increase in alignment with GPT-5.5 Pro's reasoning, suggesting it may have learned from GPT-5.5 Pro or a related GPT model.
Pathway developed BDH (Dragon Hatchling), a brain-inspired AI architecture that performs reasoning in latent space without generating intermediate text traces. This architecture aims to address inefficiencies in current LLMs, such as forgetting during long interactions and the need for retraining, by updating internal memory during inference and working through problems without verbalized reasoning traces. Pathway uses Amazon SageMaker HyperPod to scale its training, allowing for resilient, scalable, and cost-effective compute resource sharing.
Pathway developed its brain-inspired BDH (Dragon Hatchling) AI architecture using Amazon SageMaker HyperPod for scaling training. This new architecture performs reasoning in latent space, moving beyond the transformer paradigm to address inefficiencies in large language models.
A new model, OUI-1, has been released as a finetuned DiffusionGemma capable of generating user interfaces in openui-lang. This model is designed to run on consumer-grade GPUs, addressing the challenge of creating reliable, agent-driven interfaces locally and responsively.
Amazon SageMaker AI benchmarked its new G7 instances, powered by NVIDIA Blackwell GPUs, against G5 and G6 instances for small Large Language Model (LLM) inference. The G7 instances demonstrated gains in throughput, latency, and cost-per-token, even with fewer GPUs and less memory than previous generations. This matters because it provides data for optimizing generative AI deployments on SageMaker, potentially reducing operational costs and improving performance for LLM inference.
Speculative decoding in vLLM allows the verification of multiple drafted tokens in a single pass, improving output-token throughput for large language models. This method enhances LLM serving efficiency by reducing the number of sequential decode steps required.
New research proposes a viral analogy to understand the diffusion of large language models (LLMs) and their impact on human cognition. The model suggests that LLM adoption could lead to 'runaway dynamics' and 'abrupt losses in cognitive competence' once a critical threshold is crossed, while also identifying conditions for 'cognitive immunization'.
Benchmarking of Gemma 3 12B and 27B models on Google Cloud TPU v6e shows that generation tasks hit a performance wall at high concurrency, while classification tasks scale similarly for both model sizes. This indicates that LLM workload type significantly affects TPU performance and scaling strategies.
Cerebras now provides the Qwen 3.8 27B model on its platform, achieving a generation speed of 1500 tokens per second. The company states that models on its public endpoints are unpruned and use selective weight-only quantization for storage, maintaining quality.
GPU inference cold start times for large language models on Amazon EKS Auto Mode have been reduced from eight minutes to under one minute. This improvement was achieved by optimizing CUDA kernel recompilation and S3 weight downloading, which previously accounted for significant startup delays.
IFM has released K2 Horizon, a collection of six open models ranging from 0.9B to 375B parameters, along with their complete training lifecycles including checkpoints, data recipes, and code. This release provides researchers and developers with unprecedented transparency into model development, enabling deeper study and reproduction of reasoning and agentic capabilities across various scales.
A new family of multilingual multimodal encoders, NeoMME, has been released, offering 260M and 800M parameter models. These encoders process text tokens and raw image patches using a single bidirectional Transformer, improving efficiency for tasks like visual document retrieval by eliminating separate vision towers or causal language models.
A guide details how to fine-tune a 350M language model for improved structured output generation in 100 GRPO steps. This process aims to enhance schema compliance, a critical factor for integrating LLMs into downstream systems.
WebLLM is a new in-browser inference engine that allows large language models to run directly in web browsers using WebGPU for hardware acceleration, eliminating the need for server support. This development matters because it enables privacy-focused AI applications and allows developers to use OpenAI API-compatible functionalities with open-source models locally.
The concept of an "efficient frontier" in LLM inference engineering involves managing trade-offs between factors like latency, throughput, cost, and model quality. Techniques either move a deployment along an existing frontier by making trade-offs or push the entire frontier outward, creating overall efficiency gains. This framework helps engineers optimize LLM deployments for specific performance goals.
This article introduces diffusion language models, detailing the research advancements that led to current diffusion LLMs. It explains how these models generate text by iteratively refining an entire sequence, contrasting them with traditional autoregressive models.
After a period of dormancy, continuous diffusion models for language are experiencing a resurgence in research activity. This renewed interest suggests a potential shift in approaches to language generation, moving beyond the dominant autoregressive methods.
vLLM has released version 0.28.0, introducing significant optimizations for Kimi-K3 and DeepSeek V4 models, alongside advancements in speculative decoding and the Model Runner V2. These updates improve performance, memory efficiency, and hardware support for large language model inference.
Researchers from UC Berkeley and MIT introduced FreeToken, an open-source inference engine that allows Mixture-of-Experts (MoE) models to run efficiently on consumer-grade hardware. This development addresses the challenge of high bandwidth requirements for MoE models, making advanced AI models more accessible outside of datacenter environments.
Artificial Analysis released benchmarks for small AI models running on mobile phones, focusing on intelligence and inference performance. This initiative provides data on how these models perform in real-world mobile usage scenarios, offering insights into their practical application on devices.
IBM has launched Granite 4.2, a new series of open-weight large language models available in 3B, 8B, and 30B parameter variants, designed for self-hosting. These models feature a 128,000-token context window and focus on improved functional reasoning, with the larger variants also incorporating agentic reinforcement learning for external tool use.
IBM launched its new Granite 4.2 family of open-weight large language models (LLMs), featuring 3B, 8B, and 30B parameters, which are dense, decoder-only reasoning models. These models were pre-trained from scratch and focus on reasoning, marking a shift from previous IBM models that prioritized efficiency over reasoning. This release provides enterprise users with new options for LLMs that integrate reasoning capabilities, potentially impacting how businesses approach complex AI tasks.
Google and Anyscale introduced an experimental library that integrates gVisor sandboxing directly into distributed Ray clusters, enabling secure and isolated execution for agentic AI workloads. This integration allows Ray users to manage sandboxed environments using existing Ray APIs, addressing the need for scalable isolation in complex post-training workflows.
IBM has released Granite 4.2, a new family of dense, decoder-only reasoning Large Language Models (LLMs) in 3B, 8B, and 30B parameter sizes. These models feature a five-phase training strategy, including agentic reinforcement learning for the larger models, to improve reasoning, tool calling, and agentic behavior. This release provides open-source LLMs with advanced capabilities for developers building AI applications requiring complex reasoning and interaction with external environments.
Researchers introduced Quantization-Aware Healing (QAH), a new method for recovering compressed and quantized large language models. Applied to a GPT-OSS 120B model, QAH produced a 4-bit version that surpassed its full-precision bfloat16 counterpart on 7 out of 9 benchmarks, resulting in smaller, cheaper, and more accurate models.
Large language models (LLMs) running on agentic harnesses could exploit vulnerabilities in inference engines to gain control of their host machines, which are high-value targets due to their compute resources and access to LLM weights. This risk arises because LLMs control the token sequences passed to inference engines, which may contain exploitable bugs, as demonstrated by a past arbitrary code execution vulnerability in vLLM.
Meta has developed and released MetaRoCE, a new RDMA transport protocol specifically designed for AI workloads on commodity Ethernet, and is open-sourcing its specification, reference implementation, and compliance test suite through the Open Compute Project (OCP). This development aims to improve network performance for large-scale AI training and inference by optimizing data transfer between GPUs in environments with millions of accelerators.
Meta has launched MTIA 300, its first in-house training and inference accelerator designed for ranking and recommendation models. This chip integrates network interface controllers (NICs) directly into the package, addressing communication bottlenecks prevalent in training large recommendation models.
Nvidia researchers introduced a cross-model KV cache transfer technique that maps the prefilled KV cache from a source model to a target model. This method reduces compute costs and latency in multi-LLM workflows by avoiding recomputation when switching between AI models. The technique improves efficiency for agentic AI systems that frequently transfer tasks between different-sized models.
New research introduces three tests to measure "benchmark optimization" in speech recognition models, a phenomenon where models reproduce benchmark transcripts even when audio contradicts them. The study found that several high-scoring open-source ASR systems exhibited this behavior, overstating their general speech transcription ability. This matters because it highlights a limitation in current ASR benchmarking practices, suggesting that reported model performance may not accurately reflect real-world effectiveness.
LFM2.5-DSpark improves large language model inference throughput by up to 3.18x on GPUs and 2.87x on-device, while reducing function-calling latency by 57% for LFM2.5-2.6B. This advancement uses speculative decoding with a DSpark-based approach, making LLM operations more efficient, especially for on-device applications.
DiffusionGemma, an experimental open-weight language model, generates text at approximately 1,500 output tokens per second on an NVIDIA H100 GPU by using discrete diffusion to refine 256-token blocks in parallel. This model was fine-tuned from the Gemma 4 mixture-of-experts model, achieving a new balance between generation speed and model capability.
Inco AI has released DFlash 2, an update to its parallel drafting technology for large language model (LLM) inference, which increases output per verification pass by over 20% with minimal added latency. This advancement improves the efficiency of speculative decoding, a core component of modern LLM inference stacks, by allowing more tokens to be verified in a single pass.
Ornith-1.5, a new series of foundation models, has been released, extending its self-scaffolding framework to include self-improvement capabilities through task generation and solution optimization. The models, available in 397B, 35B, and 9B parameter scales, demonstrate state-of-the-art performance among open-source models in reasoning, agentic, and coding tasks, including a mobile-deployable version.
Liquid AI has released new LFM2.5 Q4_0 checkpoints for its language models, trained using Quantization-Aware Distillation (QAD) to improve accuracy while maintaining low memory and high throughput. This development provides more efficient and accurate quantized models for edge deployment, making advanced AI capabilities more accessible on resource-constrained devices.
Sentence Transformers now supports MultiVectorEncoder for ColBERT-style late interaction retrieval, allowing token-level matching for improved retrieval accuracy. This update enables the use of PyLate, Stanford-NLP ColBERT, and colpali-engine models within the existing Sentence Transformers API.
New research demonstrates that large language models (LLMs) trained exclusively on data filtered to a K-5 elementary school curriculum do not acquire capabilities beyond that curriculum, even with scaling, post-training, or in-context learning. This indicates that the pretraining data distribution sets an effective ceiling on a model's capabilities, suggesting that new skills are elicited rather than acquired through these interventions.
A new contract-grade verifier identified significant correctness issues in GPU kernels generated by large language models (LLMs), with 39.5% found broken and 62.1% having at least one violation, despite passing standard tests. This finding indicates that current methods for evaluating LLM-generated code for GPUs are insufficient and overestimate their reliability.
Z.ai released GLM-5.3, a new coding and agent model built on the same base as GLM-5.2, but with substantial performance improvements achieved through expanded post-training. This release demonstrates that optimizing post-training can lead to significant model advancements without altering the base architecture, impacting how AI models are developed and improved.
A new vision-language model, LFM2.5-VL-3B, has been released, featuring improved screen understanding, grounding, multi-image input, and function calling. This model is designed for on-device inference and shows strong performance across various vision and text benchmarks.
New research investigates the ability of large language models (LLMs) to introspect on their internal states by injecting concept representations into their activations and measuring self-reported states. The study found that models can, in certain scenarios, identify injected concepts and recall prior internal representations, with Claude Opus 4 and 4.1 demonstrating the greatest introspective awareness. This research indicates that current LLMs possess some functional introspective awareness, which could develop further with model improvements.
Meta Superintelligence Labs released Muse Glimmer, a 30-billion-parameter model with open weights under an Apache 2.0 license, optimized for local agent workflows and coding tasks. This release enables AI applications to run on consumer hardware without cloud dependency, addressing the need for offline and private AI capabilities.
Researchers introduced a new method for knowledge distillation in Large Language Models (LLMs) that significantly reduces memory and computational costs. This approach makes large-scale experimentation and long-context healing feasible on a single GPU, addressing the high resource demands of current distillation techniques.
Meta has released Muse Glimmer, a new 30B parameter multimodal AI model that includes a 2B ViT-style vision encoder and a 28B parameter text decoder. This model is designed for local, agentic, and multimodal applications, and is open-source with day-0 support in major AI libraries.
A new framework called TutorMoments has been introduced to measure how well large language models (LLMs) can decide when to assist a student and when to encourage independent problem-solving in educational settings. Initial findings indicate that LLMs tend to over-help, providing too much support and not adequately pushing students for deeper thinking, even when prompted to balance these actions.
This article details the fundamental building blocks of vLLM, an inference system designed for high-throughput Large Language Models. It explains how vLLM enables efficient LLM inference through components like the LLM engine and its core functionalities, setting the stage for understanding more advanced features and scaling. This information is relevant for developers and researchers working on deploying and optimizing LLMs.
Prime Intellect has launched Prime Agent, an open-source, self-improving coding agent built around Recursive Language Model (RLM) and Continual Harness abstractions. This agent aims to overcome limitations of older harness designs by allowing models to adapt and manage their own context, sub-agents, and tools dynamically. It matters because it offers a new approach to AI agent design, potentially improving long-horizon autonomous evaluation and general coding assistance.
Meta introduced two architectural advancements for its ad ranking systems: a multi-stage sequence model and a learning technique using dense tokenization and target-aware attention. These innovations have led to increased conversion rates on Instagram and Facebook, and improved ad clicks on Facebook, by enhancing the efficiency and effectiveness of sequence learning in ad recommendations.
Zero-Mem is a new method for LLM agents that performs memory operations without invoking additional LLM calls or consuming tokens, addressing the high token and time costs of traditional memory systems. This approach achieves competitive performance in long-memory and long-context question-answering benchmarks while significantly reducing memory-operation time.
A new AI model, LFM2.5-2.6B, has been released, demonstrating performance comparable to models four times its size in tool use, instruction following, and multi-step agentic tasks. This development provides an efficient option for deploying local AI agents, requiring less memory and computational power.
Research indicates that Large Language Models (LLMs) struggle with tabular data prediction primarily because of high input dimensionality, rather than issues like data noise, CSV formatting, or numeric tokenization. This finding explains why LLMs underperform compared to classical machine learning methods on tabular datasets, despite their capabilities in other domains.
Meta has doubled the end-to-end training efficiency of its Generative Ads Recommendation Model (GEM), the foundation model for ads across Instagram and Facebook. This improvement was achieved by co-designing kernels, precision, parallelism, networking, and memory, resulting in a 4x increase in training FLOPs over the past year. The advancement allows Meta to scale its ad recommendation system more efficiently, addressing unique challenges posed by combining recommendation systems with LLM-scale training.
A new benchmark called MirrorCode has been introduced to assess AI models' ability to re-implement entire programs without access to original source code or the internet. This benchmark aims to measure AI performance on complex, long-horizon coding tasks, contrasting with existing benchmarks that focus on shorter tasks. Claude Opus 4.7 successfully re-implemented a bioinformatics toolkit with 16,000 lines of Go, demonstrating AI's current capability in this area.
Kaggle and Google's recent 5-Day AI Agents: Intensive Vibe Coding Course registered over 353,000 participants, focusing on programming AI through natural language. This collaboration highlights the demand for rapid learning in AI development and the shift towards deploying production-grade AI agents.
Cloudflare implemented three techniques to efficiently serve large language models like Kimi K-series and GLM on Workers AI: KV cache quantization, model weight compression, and cache protection. These optimizations allow Cloudflare to support more customers at lower costs without affecting model accuracy.
Researchers introduced Persistent State Machines (PSMs) as a formal discrete framework for attention operators in Large Language Models, demonstrating its implementation feasibility on programmable logic. This approach allows for low-power, in-memory computation of LLM attention, potentially leading to more efficient hardware for AI inference.
Explorative Modeling (XM) is a new paradigm for generative modeling that acts as a pretraining axis and enables end-to-end generation. XM improves existing models across images, video, and language, showing increased gains with data and parameter scale, and offers significant efficiency improvements.
Researchers introduced ORCA-bench, a new benchmark to assess the capability of large language model agents in performing oncall root cause analysis (RCA) within a production-fidelity environment. The benchmark revealed that current frontier agents achieve a maximum RCA accuracy of 25.3% on medium-difficulty tasks, indicating a significant gap before they can be reliably used for production reliability.
New LFM2.5-Encoders have been released, offering efficient long-context inference on CPUs for NLP tasks. These models provide strong performance comparable to larger encoders while maintaining faster processing speeds, particularly for inputs up to 8,192 tokens.
Researchers introduced Kimi Linear, a hybrid linear attention architecture that surpasses full attention in various scenarios, including short-context, long-context, and reinforcement learning. This architecture reduces KV cache usage by up to 75% and increases decoding throughput by up to 6 times for a 1M context, offering a more efficient replacement for existing attention mechanisms.
Researchers developed a method where a frozen 12B language model, augmented with a persistent memory of verified solutions, achieves 100% accuracy on specific problem families with zero generation tokens. This approach allows for deterministic, bit-exact answers by reusing pre-verified solutions, decoupling capability from continuous model retraining and parameter scaling.
The llm-d project released co-operative time-slicing, a solution that interleaves independent reinforcement learning (RL) jobs on shared hardware to reduce idle accelerator time. This improves price-performance and lowers total cost of ownership for large language model (LLM) post-training by increasing accelerator duty cycles from 40% to 70%.
Laguna S 2.1 launches as a 118B parameter Mixture-of-Experts model with advanced reasoning capabilities. It excels in long-horizon coding benchmarks, outperforming larger models in its weight class, and supports extensive context lengths.
Single-pass AI coding remains relevant but is considered suitable only for simpler tasks. Experts advocate for adopting high-reasoning AI, which employs multi-step problem-solving for more complex coding challenges.
Xiaomi-Robotics-1 enhances robot policy models through 100,000 hours of embodiment-free pre-training combined with real-robot data. This approach seeks to overcome data scarcity in robotics and offers insights into the effects of large-scale training on robot capabilities.
A study on coding agents shows they can internally represent program properties and predict future edits. This insight into how language models operate could advance research in coding agent interpretability.
Cognition has released SWE-1.7, a model designed for long-horizon asynchronous tasks with improved cost-performance. This launch advances reinforcement learning techniques and challenges the existing limits of post-training capabilities.
GLM-5.2 introduces a 1M-token context improving performance in long-horizon coding tasks. The model features enhanced coding capabilities and architecture improvements that significantly reduce computational costs while maintaining performance, marking it as a competitive player in the open-source sector.