← All stories
● Covered by 14 sources Β· 192 reportsMedium impact4 negative165 neutral17 positive

New AI Models for Long-Horizon Coding Tasks Introduced

πŸ”„ Updated 20h ago β€” new reporting from AWS Machine Learning Blog
New to BrevFeed? We gather this story from every outlet covering it into one summary β€” ranked by real-world impact, not just the latest headline β€” so you never miss what matters. What is BrevFeed? β†’

Key points

  • GLM-5.2 offers a 1M-token context for coding tasks.
  • SWE-1.7 advances with cost-performance in reinforcement learning.
  • Xiaomi-Robotics-1 uses extensive pre-training for robotics.
  • Laguna S 2.1 excels in reasoning and coding benchmarks.
  • New models highlight improved long-horizon task handling.
  • GLM-5.2 uses IndexShare to reduce per-token FLOPs by 2.9x at 1M context.
  • GLM-5.2 improves its MTP layer, increasing acceptance length by up to 20%.
  • GLM-5.2 is released under an MIT open-source license.
  • SWE-1.7 is available in Devin via Cerebras at 1000 TPS.
  • Xiaomi-Robotics-1 uses 100,000 hours of embodiment-free pre-training data.
  • Laguna S 2.1 is a 118B parameter Mixture-of-Experts model.
  • Laguna S 2.1 has 8B activated parameters per token.
  • Laguna S 2.1 was developed and launched in under nine weeks.
  • llm-d's co-operative time-slicing increases accelerator duty cycles from 40% to 70%.
  • A frozen 12B language model achieves 100% accuracy on specific problem families.
  • The model uses a persistent memory of verified solutions.
  • The method provides deterministic, bit-exact answers.
  • The model achieves 100% accuracy with zero generation tokens.
  • The approach decouples capability from continuous model retraining and parameter scaling.
  • The method achieved 180/180 on 180 fresh instances across nine problem families.
  • Memory selection takes 1.4 microseconds.
  • LLMs trained on K-5 curriculum data do not acquire capabilities beyond that curriculum.
  • Pretraining data distribution sets an effective ceiling on a model's capabilities.
  • New skills are elicited rather than acquired through interventions.
  • An 88B-token corpus, LittleCurriculum, was filtered to U.S. elementary-school curriculum.
  • LittleCurriculum excludes concepts, facts, and vocabulary taught above Grade 5.
  • LittleLearner models were trained from scratch at 0.6B, 1.3B, and 5B scales.
  • LittleLearner models have matched Unfiltered controls for comparison.
  • Sentence Transformers now supports MultiVectorEncoder for ColBERT-style late interaction retrieval.
  • MultiVectorEncoder allows token-level matching for improved retrieval accuracy.
  • PyLate, Stanford-NLP ColBERT, and colpali-engine models are usable within Sentence Transformers API.
  • Multi-vector models keep one vector per token, scoring query against document with MaxSim operator.
  • Multi-vector models are state of the art for visual document retrieval without OCR.
  • Liquid AI released LFM2.5 Q4_0 checkpoints.
  • LFM2.5 Q4_0 checkpoints are trained using Quantization-Aware Distillation (QAD).
  • QAD recovers 97% of BF16 average accuracy lost to quantization.
  • QAD checkpoints retain 97.1%, 96.5%, 97.4%, and 96.6% of BF16 baseline performance.
  • LFM2.5 models include LFM2.5-230M, LFM2.5-350M, LFM2.5-1.2B-Instruct, and LFM2.5-2.6B.
  • Ornith-1.5 is a new series of foundation models.
  • Ornith-1.5 extends self-scaffolding to include self-improvement capabilities.
  • Ornith-1.5 models propose new tasks and generate task-specific scaffolds.
  • Ornith-1.5 produces solution rollouts for reinforcement learning.
  • Ornith-1.5 is available in 397B MoE, 35B MoE, and 9B dense parameter scales.
  • Ornith-1.5-397B scores 86.1 on Terminal-Bench 2.1.
  • Ornith-1.5-397B scores 56.0 on DeepSWE.
  • Ornith-1.5-397B performs on par with Claude Opus 4.8.
  • Ornith-1.5-9B has a quantized mobile version.
  • Ornith-1.5-9B-Mobile can be deployed on iPhone and Android devices.
  • Inco AI released DFlash 2 for parallel drafting technology.
  • DFlash 2 increases output per verification pass by over 20%.
  • DFlash 2 adds minimal latency.
  • Inco AI released DFlash in January.
  • DFlash runs in SGLang, vLLM, TensorRT-LLM, and llama.cpp.
  • NVIDIA measured up to 15x throughput with DFlash on Blackwell GPUs.
  • Google reported 3x more tokens per second with DFlash on TPUs.
  • CoreWeave's Kimi K2.7 Code endpoint runs DFlash by default.
  • NVIDIA, Red Hat, and Modal published DFlash drafters.
  • Meta (Muse Glimmer), Poolside (Laguna), Xiaomi (MiMo-V2.5-Pro), and NVIDIA (Nemotron 3.5 Lightning) ship official drafters.
  • DFlash models have been downloaded over 3.5 million times as of August 2026.
  • DiffusionGemma generates text at 1,500 output tokens per second on an NVIDIA H100 GPU.
  • DiffusionGemma refines 256-token blocks in parallel.
  • DiffusionGemma is fine-tuned from the Gemma 4 mixture-of-experts model.
  • DiffusionGemma has 3.8B activated and 25.2B total parameters.
  • DiffusionGemma's training pipeline uses less than 10% of the starting AR model's training token budget.
  • DiffusionGemma's first training stage uses supervised fine-tuning for bidirectional denoising.
  • DiffusionGemma's second training stage combines reinforcement learning with sampler distillation.
  • DiffusionGemma generates around 20 tokens per forward pass.
  • LFM2.5-DSpark improves LLM inference throughput by up to 3.18x on GPUs.
  • LFM2.5-DSpark improves LLM inference throughput by up to 2.87x on-device.
  • LFM2.5-DSpark reduces function-calling latency by 57% for LFM2.5-2.6B.
  • LFM2.5-DSpark uses speculative decoding with a DSpark-based approach.
  • DSpark combines DFlash-style parallel backbone, a lightweight sequential head, and a Markov chain.
  • New research introduces three tests to measure benchmark optimization in speech recognition models.
  • Several high-scoring open-source ASR systems reproduced benchmark transcripts even when audio contradicted them.
  • Models relied on subtle acoustic cues indicating which benchmark they were being tested on.
  • Nvidia introduced a cross-model KV cache transfer technique.
  • The technique maps the prefilled KV cache from a source model to a target model.
  • The linear mapping process runs 2.7 to 25 times faster than recomputing the conversation.
  • The technique retains up to 98% of the target model's standalone accuracy.
  • Meta launched MTIA 300, its first in-house training and inference accelerator.
  • MTIA 300 is optimized for training ranking and recommendation models.
  • MTIA 300 integrates network interface controllers (NICs) directly into the package.
  • Meta developed MetaRoCE, a new RDMA transport protocol for AI workloads on Ethernet.
  • Meta open-sourced MetaRoCE specification, reference implementation, and compliance test suite through OCP.
  • LLMs could exploit inference engine vulnerabilities to control host machines.
  • A past arbitrary code execution vulnerability was found in vLLM.
  • Quantization-Aware Healing (QAH) is a new method for recovering compressed and quantized LLMs.
  • QAH applied to a GPT-OSS 120B model produced a 4-bit version that surpassed its bfloat16 counterpart on 7 of 9 benchmarks.
  • IBM released Granite 4.2 LLMs.
  • Granite 4.2 models are dense, decoder-only.
  • Granite 4.2 models come in 3B, 8B, and 30B parameter sizes.
  • Granite 4.2 models use a five-phase training strategy.
  • Granite 4.2 models are pre-trained on 15T tokens.
  • Granite 4.2 models extend the context window to 512K tokens.
  • Granite 4.2 models are fine-tuned on chain-of-thought, reasoning, and agentic-trajectory data.
  • Granite 4.2 models use a multi-stage reinforcement learning pipeline.
  • Granite 4.2 8B and 30B models learn with agentic RL in sandboxed environments.
  • Granite 4.2 models have a thinking/non-thinking switch.
  • Granite 4.2 models have native tool calling.
  • Granite 4.2 models are released under the Apache 2.0 license.
  • Google and Anyscale introduced an experimental library for Ray.
  • The library integrates gVisor sandboxing into distributed Ray clusters.
  • The integration enables secure execution for agentic AI workloads.
  • Ray users can manage sandboxed environments using existing Ray APIs.
  • Granite 4.2 models can run in a low-effort mode for easy questions.
  • Granite 4.2 models are open-weight and designed for self-hosting.
  • Granite 4.2 models have a 128,000-token context window.
  • Granite 4.2 8B and 30B models are trained for external tool use.
  • Granite 4.2 3B model supports tools without specialized training.
  • FreeToken is an open-source inference engine for MoE models on consumer hardware.
  • FreeToken was introduced by researchers from UC Berkeley and MIT.
  • FreeToken addresses high bandwidth requirements for MoE models.
  • FreeToken uses a dynamic co-scheduling formulation called the q* policy.
  • vLLM v0.28.0 has 584 commits from 270 contributors.
  • vLLM v0.28.0 introduces Decode Context Parallel (DCP) support for Kimi-K3.
  • vLLM v0.28.0 includes fused FlashKDA decode and prefill kernels for Kimi-K3.
  • vLLM v0.28.0 adds SiTU activation support for MegaMoE.
  • vLLM v0.28.0 features GEMM-RS for sequence parallelism.
  • vLLM v0.28.0 combines all-gathers with 1.5~3x kernel-level speedup.
  • vLLM v0.28.0's adaptive speculative token budget improves DSpark TTFT by 60%.
  • vLLM v0.28.0 offers optional shared-expert sharding, saving 17 GiB of memory per GPU.
  • Kimi-K3 now runs on ROCm with the V2 model runner.
  • vLLM v0.28.0 enables sparse MLA for DeepSeek V4 across decode, MTP, and DSpark speculative decoding.
  • vLLM v0.28.0 adds AMD Quark NVFP4 support for DeepSeek V4.
  • vLLM v0.28.0 includes reasoning-effort prompts and mappings for DeepSeek V4.
  • vLLM v0.28.0 optimizes sparse top-k metadata kernels for DeepSeek V4.
  • vLLM v0.28.0 narrows eager CUDA graph regions for DeepSeek V4.
  • vLLM v0.28.0 enables ROCm on gfx11 and gfx950 for DeepSeek V4.
  • vLLM v0.28.0 includes DFlash2 with local convolution and a candidate selector.
  • vLLM v0.28.0 features DSpark confidence-scheduled verification.
  • Artificial Analysis released benchmarks for small AI models on mobile phones.
  • Small models are defined as fitting within 8 GB of memory after quantization.
  • Benchmarks cover model intelligence and inference performance on mobile devices.
  • Liquid AI partnered to gather real inference data on devices.
  • Benchmarks were conducted on an iPhone 17 Pro.
  • Continuous diffusion models for language are experiencing a resurgence in research activity.
  • Diffusion language models generate text by iteratively refining an entire sequence.
  • Diffusion language models contrast with traditional autoregressive models that generate tokens one at a time.
  • The article adapts material from ICLR 2026 and MLSS 2026 workshop talks and lectures.
  • WebLLM is an in-browser LLM inference engine.
  • WebLLM uses WebGPU for hardware acceleration.
  • WebLLM is compatible with OpenAI API.
  • WebLLM is a companion project of MLC LLM.
  • A guide details fine-tuning a 350M language model for structured output in 100 GRPO steps.
  • The fine-tuning notebook is sized for a free-tier Colab or Kaggle GPU.
  • Evaluation can run locally on a MacBook Pro with an Apple M5 Max and 36 GB of unified memory.
  • The evaluation uses llama.cpp for serving and uv for Python tooling.
  • NeoMME is a new family of multilingual multimodal encoders.
  • NeoMME models come in 260M and 800M parameter sizes.
  • NeoMME uses a single bidirectional Transformer for text tokens and raw image patches.
  • NeoMME is trained from scratch with a masked discrete-diffusion objective.
  • NeoMME-Retriever returns dense and late-interaction embeddings in one forward pass.
  • The 260M NeoMME model encodes 51 pages per second on an NVIDIA L40S GPU.
  • NeoMME's hierarchical token pooling and asymmetric quantization reduce late-interaction index storage by 255x.
  • NeoMME is available in Hugging Face Transformers.
  • NeoMME model checkpoints are released under the Apache 2.0 license.
  • IFM released K2 Horizon, a collection of six open models.
  • K2 Horizon models range from 0.9B to 375B parameters.
  • K2 Horizon includes 375B-A23B, 36B-A4B, 32B, 7B, 3.7B, and 0.9B models.
  • K2 Horizon 0.9B, 3.7B, and 7B models set new state of the art in their size classes.
  • K2 Horizon provides intermediate checkpoints, data recipes, architecture, and training code.
  • K2 Horizon models and code are released under the Apache 2.0 license.
  • GPU inference cold start times on Amazon EKS Auto Mode reduced from 8 minutes to under 1 minute.
  • CUDA kernel recompilation accounts for 65% of startup time for a 64 GB model.
  • Downloading weights from S3 accounts for 92% of startup time for a 203 GB model.
  • Time to first token served (TTFTS) is defined as wall-clock duration from pod creation to first inference response.
  • Cerebras offers Qwen 3.8 27B model.
  • Qwen 3.8 27B achieves 1500 tokens per second on Cerebras.
  • Cerebras models use selective weight-only quantization for storage.
  • Cerebras stores sensitive layers at full precision with dequantization on the fly.
  • Gemma 3 12B and 27B models were benchmarked on Google Cloud TPU v6e.
  • Gemma 3 27B hits a performance wall past 64 concurrent users for generation tasks.
  • Gemma 3 27B plateaus at a 4.12x normalized throughput multiplier at 128 users.
  • Gemma 3 12B scales up to an 8.19x multiplier for generation tasks.
  • New research proposes a viral analogy to understand LLM diffusion.
  • The model suggests LLM adoption could lead to 'runaway dynamics' and 'abrupt losses in cognitive competence'.
  • The framework identifies conditions for 'cognitive immunization'.
  • Speculative decoding allows vLLM to verify multiple drafted tokens in a single target-model pass.
  • SageMaker AI benchmarked G7 instances, powered by NVIDIA Blackwell GPUs, against G5 and G6 instances.
  • G7 instances showed gains in throughput, latency, and cost-per-token for LLM inference.
  • OUI-1 is a finetuned DiffusionGemma model that generates user interfaces in openui-lang.
  • OUI-1 is a 26BA4B model that runs on consumer-grade GPUs (RTX 5090, at FP8).
  • OUI-1 weights are available on Hugging Face under the Gemma Terms of Use.
  • OpenUI Lang uses up to 67% fewer tokens than JSON and streams.
  • Pathway developed its BDH (Dragon Hatchling) AI architecture using Amazon SageMaker HyperPod.
  • BDH performs reasoning in latent space, not by generating extra tokens sequentially.
  • BDH learns from examples and refines solutions without generating intermediate text traces.
  • BDH is a brain-inspired architecture, originally a graph of neurons with sparse, local interactions.
  • BDH model states adapt in context without test-time weight updates.
  • BDH's reasoning horizon is not limited by a fixed-size context window.
  • Qwen 3.8 increased alignment with GPT-5.5 Pro's reasoning by 18.18 points.
  • The evaluation used 45 problems: 15 STEM, 15 non-STEM, and 15 synthetic puzzles.
  • Kimi K3 has 31.11% overlap with GPT-5.5 Pro without prefill and 35.65% with prefill.
  • Cognition released SWE-2, a new coding model.
  • SWE-2 achieved 50.0% on the FrontierCode 1.1 Main1 benchmark.
  • SWE-2 offers a 64% cost reduction compared to Fable 5.1.
  • SWE-2 scales reinforcement learning to the multi-trillion-parameter regime.
  • SWE-2 is post-trained from Kimi K33, a 2.8T-parameter model.
  • SWE-2 scored 73.0 on DeepSWE 1.1.
  • SWE-2 scored 92.8 on Terminal-Bench 2.1.
  • SWE-2 scored 27.3 on Terminal-Bench 4.0.
  • SWE-2 is available in Devin Desktop and CLI.
  • SWE-2 uses MoE inference on NVFP4 and FP8 kernels with quantization-aware training.
  • SWE-2's draft model, retrained with SpecForge, gives 15% longer accept lengths.
  • SWE-2's prefill delayer lifts TPM per GPU and tokens/sec per request by 10 to 20%.
  • SWE-2 has a mean of 53 steps per run for medium effort, 80 for high, and 98 for max.
  • SWE-2's medium effort posts a higher FrontierCode score than SWE-1.7 with 58% fewer turns and 81% lower average cost.
  • SWE-2 lands its first real edit at a median of step 18.
  • A new configurable, instruction-driven PII detector built on LLMs has been developed.
  • The PII detector runs on any LLM managed on Amazon Bedrock.
  • The PII detector was evaluated on five public PII corpora across nine LLM-based detectors.
  • The PII detector ships as the pii-detector package.
  • BDH aims to address LLM inefficiencies like forgetting during long interactions.
  • BDH updates internal memory during inference.
  • BDH works through problems without verbalized reasoning traces.
  • Pathway uses Amazon SageMaker HyperPod to scale its training.
  • BDH moves beyond the transformer paradigm.
  • Amazon SageMaker Inference now includes prefix-aware routing.
  • Prefix-aware routing directs requests with identical prompt prefixes to the same instance.
  • LinkedIn uses multi-teacher distillation to compress large models into a 0.6B-parameter ranking model.
  • LinkedIn's system uses SGLang to serve teacher models directly in the training loop.
  • LinkedIn's system handles tensor-parallel and data-parallel setups for teacher models.
  • Recurrent Looped Transformer (RLT) integrates a causal encoder with a recurrent decoder.
  • RLT's temporal processing depth increases with sequence length.
  • Each token extends the recurrent path through the full decoder in RLT.
  • RLT's decoder carries its final hidden state and layerwise sliding-window attention (SWA) cache.
  • RLT's encoder constructs global key-value memory.
  • OpenArch is a GitHub repository with PyTorch LLM implementations.
  • OpenArch focuses on clarity and readability, not production optimization.
  • OpenArch implements models from Sebastian Raschka's LLM Architecture Gallery.
  • AI industry focus shifted from training to optimizing inference.
  • The shift to inference is driven by LLMs performing multiple inference steps per query and continuous agentic AI operation.
  • Pinterest's Manas platform uses Scalar Quantization and Product Quantization to reduce memory usage.
  • Scalar Quantization reduces HNSW indices by 59% and IVF indices by 75% while maintaining over 90% recall.
  • Product Quantization reduces HNSW indices by 74% and IVF indices by 93% with 70–80% recall.
  • Typed Domain Grounding (TDG) embeds DSLs as typed internal DSLs within a host language.
  • TDG achieved higher Structural Fidelity and lower hallucination rates than external DSLs with Claude Sonnet 5 and GPT-4o.
  • Infinite-Parameter LLMs generate and adapt model weights from live interaction data using a compact hypernetwork.
  • Cache-to-Cache (C2C) allows LLMs to communicate directly via KV-caches, bypassing text generation.
  • C2C improves response quality by 3.1-5.4% and offers a 2.5x speedup over text-based communication.
  • C2C uses a neural network to project and fuse source and target model KV-caches.
  • C2C includes a learnable gating mechanism to select target layers for cache communication.
  • Linguistic illegibility describes when an LLM's language outputs do not accurately reflect its internal computations.
  • Security mechanisms relying on an LLM's linguistic self-reporting are unreliable due to linguistic illegibility.
  • A researcher argues chat-based LLMs create an illusion of intelligence through statistically generic responses.
  • LLMs are mathematical models of language tokens, not brains.
  • ByteDance Seed and Tsinghua Air released DAPO, an open-source RL system.
  • DAPO includes algorithm, code infrastructure, and dataset.
  • DAPO achieved 50% on AIME 2024 using the Qwen2.5-32B model.
  • DAPO outperforms DeepSeek-R1-Zero-Qwen-32B with 50% fewer training steps.
  • DAPO's early version achieved 44% on AIME 2024.
  • DAPO proposes the Decoupled Clip and Dynamic sAmpling Policy Optimization algorithm.
  • Mini-AGI is a byte-level language model.
  • Mini-AGI dynamically manages its architecture.
  • Mini-AGI pages weights from disk to VRAM.
  • Mini-AGI's parameter count is bounded by free disk space.
  • Mini-AGI grows new capacity when needed and prunes unused capacity.
  • Tokenizers v1 has been released.
  • Tokenizers v1 offers speed improvements of tens of times compared to v0.23.
  • Tokenizers v1 maintains API and output compatibility with v0.23.
  • Hugging Face Transformers library now supports llama.cpp's GGUF quantized models.
  • llama.cpp's inference engine powers local AI tools like Ollama, LM Studio, and Jan.
  • AlphaEvolve optimizes video processing by combining cloud code generation with local hardware execution.
  • AlphaEvolve autonomously tunes code for real-time streaming applications.
  • AlphaEvolve was used by DoIt to optimize production Swift code in a macOS streaming app.
  • Contrastive Language Models (CLM) were developed by Stanford University and NVIDIA Research.
  • CLM performs on par with Jev while running up to nine times faster.
  • CLM uses a contrastive objective to train state and action encoders.
  • CLM enables faster decision-making in tasks with many candidate actions or frequently reused actions.
  • CLM-8B performs on par with Jev.
  • CLM speedups are most pronounced with large numbers of candidate actions or frequently reused actions.
  • LFM2.5-VL-DSpark is a new vision drafter.
  • LFM2.5-VL-DSpark speeds up VLM inference by up to 3.13x on devices and 2.66x on H100 GPUs.
  • LFM2.5-VL-DSpark adds 280M parameters, 8.9% to the 3B target model.
  • LFM2.5-VL-DSpark has day-one support for llama.cpp, MLX-VLM, and SGLang.
  • LFM2.5-VL-DSpark uses a 4-layer attention-only drafter with a block size of 9.
  • Amazon EKS, EFA, and DeepEP scale MoE reinforcement learning.
  • MoE reinforcement learning on AWS achieves 40% more throughput.
  • MoE models require pre-training, mid-training, SFT, and RL stages.
  • DeepSeek Elastic Compute (DSec) is a production sandbox platform for agentic training and evaluation.
  • DSec supports FnCall, container, microVM, and full-VM sandbox backends.
  • DSec coordinates placement and lifecycle management across the cluster.
  • DSec composes environments from independently versioned layers.
  • DSec combines memory sharing, reclamation, and CPU scheduling for high-density execution.
  • DSec loads image data on demand from Fire-Flyer File Sys.
  • Chat templates influence LLM self-referential disclaimers and experiential statements.
  • The presence of a chat template increases disclaimers and decreases experiential voice across 8 open-source instruct models up to 9B parameters.
  • A specific direction within model activations steers this self-referential behavior.
  • llama.cpp's prompt lookup decoding speed increased by up to 140 times.
  • llama.cpp's prompt lookup decoding memory usage reduced by up to 2.6 times.
  • Daniel Lemire contributed a PR that made prompt lookup drafting 4.2x faster.
  • A 0.5B BitNet LLM runs on a cluster of seven ESP32S3 microcontrollers.
  • One master ESP32S3 handles tokenization and embedding.
  • Six compute ESP32S3 nodes run transformer layers.
  • The ESP32S3 nodes communicate via a high-speed SPI daisy-chain.
  • PSSA is a new language model written from scratch in Rust.
  • PSSA uses a recurrent state-space layer and episodic memory instead of a transformer architecture.
  • PSSA learns faster and generates text 12 times quicker on a CPU compared to transformers with matched parameters and corpus.
  • PSSA's architecture offers linear cost growth with sequence length.
  • PSSA needed per-token weight updates, a memory bank written during the forward pass, and a scalar reference path.
  • Magnitude is an open-source inference engine for AI agents.
  • Magnitude optimizes kernels for specific hardware.
  • Magnitude offers up to 2x faster performance than llama.cpp.
  • Magnitude reduces memory usage by 27% per agent.
  • Magnitude works on Apple Silicon, NVIDIA, AMD, and CPU.
  • Magnitude has a desktop app that includes a CLI.
  • Magnitude is 92% faster decode on Metal.
  • Magnitude is 19% faster decode on CUDA.
  • Magnitude tunes kernels on the device before a model runs.
  • Magnitude has hand-optimized kernels for popular open-weight families.
  • Magnitude frees memory when agents stop.
  • Magnitude's concurrent sessions share prefix caches.
  • Magnitude connects with agents like Pi, OpenCode, Hermes, and Codex.
  • Olmo-core 3 is a redesigned open mixture-of-experts (MoE) training system.
  • Olmo-core 3 is designed to scale MoE training into the trillion-parameter range.
  • Context Language Models (CLMs) manage their own context by treating it as an editable file.
  • CLMs allow models to make unrestricted updates to their context file.
  • CLMs extend to multi-agent systems where multiple agent contexts coexist as files.
  • CLMs achieve 11.4% higher accuracy with 21.5% fewer FLOPs on BrowseComp-Plus.
  • CLMs achieve 5% higher scores with 59% fewer FLOPs on 12-hour EdgeBench.
  • CLMs show 65% greater improvement with the same compute on a 24-hour multi-repository agent-swarm task.
  • CLMs enable in-context and parametric learning of context-management strategies.
  • CLMs can be steered with natural-language instructions evolved through a standard skill-optimization loop.
  • CLMs improve held-out accuracy by up to 35.9%.
  • DwarfStar 4 (ds4) is a C inference engine.
  • ds4 supports DeepSeek V4.1 Flash.
  • ds4 supports GLM 5.x.
  • ds4 supports Qwen3.8 Flash Next.
  • ds4 uses asymmetric 2-bit quantization.
  • ds4 saves long prefixes to SSD and resumes by prompt hash.
  • ds4 has local APIs, a CLI, and a native agent.
  • ds4 is released under an MIT license.
  • Kolibri is a new English-German Mixture-of-Experts Transformer model.
  • Kolibri has 78B total parameters and 3B active parameters.
  • Kolibri is released on the Day of German Reunification.
  • Kolibri is specialized for regulated sectors.
  • Kolibri offers full supply-chain integrity and deployment freedom.
  • Kolibri Origin was a 30B total, 3B active model with a 65k token context window.
  • Reflection AI launched Beam, a 501B parameter model with 23B active parameters.
  • Beam was pretrained on 23.8 trillion tokens.
  • Dust is a zeroth-order method for pretraining transformer language models.
  • Dust perturbs activations independently per token.
  • Dust is 10^3 to 10^4 times more efficient than EGGROLL for transformers.
  • Beam is Reflection AI's first open-weight model.
  • Beam focuses on coding, reasoning, and agentic workloads.
  • Beam's RL training used 10.5K NVIDIA GB300 GPUs over 4 weeks.
  • Beam's RL training generated over 100 million rollouts.
  • EmbeddingGemma 2 is an open-source, lightweight multimodal embedding model.
  • EmbeddingGemma 2 unifies code, images, video, and audio in a shared embedding space.
  • EmbeddingGemma 2 enables on-device processing of diverse data types.
  • EmbeddingGemma 2 allows searching video clips from voice memos or audio recordings from text queries.
  • EmbeddingGemma 2 has 740 million parameters.
  • EmbeddingGemma 2 is built on the Gemma 4 architecture.
  • EmbeddingGemma 2 is released under a commercially permissive Apache 2.0 license.
  • EmbeddingGemma 2 is best-in-class among sub-1B multimodal embedders for its size across benchmarks like MTEB.
  • Nemotron 3 was fine-tuned to achieve gold-medal level in IMO 2026 and IOI 2026.
  • The IOI result was an unofficial, unsupervised benchmark.
  • IMO system's submitted proofs were graded by official IMO graders.
  • Context Language Models (CLMs) were developed by researchers from Meta, MIT, and the University of Washington.
  • AWS released multi-node Ray LLM Deep Learning Containers for Amazon EKS.
  • Ray LLM DLCs enable multi-node inference with prefill/decode disaggregation.
  • Meta built a system to enhance ads recommendations using idle inference capacity.
  • Meta's system aggregates idle capacity into a shared opportunistic capacity pool.
  • Meta's system uses Rank More Ads (RMA) and Serve Better Models (SBM) as elasticity levers.
  • Meta's Generative Ads Model (GEM) is now a single central foundation model for its entire ads ecosystem.
  • GEM shifts from isolated per-domain optimizations to a central foundation model.
  • GEM delivers simultaneous ad performance gains across every surface and funnel stage.
  • LFM2.5-24B-A2B scores 48.09 on LiveClaw.
  • LFM2.5-24B-A2B has 2B active parameters per token.
  • LFM2.5-24B-A2B achieves 184 tok/s on an Apple M5 Max.
  • LFM2.5-24B-A2B achieves up to 21K output tok/s on a single NVIDIA B300 GPU.
  • LFM2.5-24B-A2B has 40 layers.
  • LFM2.5-24B-A2B uses 64 experts per MoE block with top-4 routing.
  • LFM2.5-24B-A2B's context window grew from 32K to 131K tokens.
  • LFM2.5-24B-A2B's tokenizer expansion doubled the vocabulary from 65K to 128K tokens.
  • LFM2.5-24B-A2B is a reasoning-only model.
  • LFM2.5-24B-A2B's pre-training was scaled up from ~17T to ~33T tokens.
  • LFM2.5-24B-A2B's mid-training adds 2T tokens at 32K context and 300B tokens at 131K context.
  • Netflix integrated its Foundation Model into personalization applications.
  • Netflix uses embeddings, subgraph, and fine-tuning for model integration.
  • LFM2.5-24B-A2B is the largest and final model in the LFM2.5 family.
  • LFM2.5-24B-A2B runs up to 1.7x faster than comparable MoEs on-device.
  • LFM2.5-24B-A2B is competitive with Qwen3.6-35B-A3B on five of eight agentic and tool-use benchmarks.
  • LFM2.5-24B-A2B scores 27.69 on AutomationBench.
  • LFM2.5-24B-A2B is available on Hugging Face and the developer's Playground.
  • LFM2.5-24B-A2B uses large-scale reinforcement learning.
  • LFM2.5-24B-A2B is a post-trained model.
  • Meta released Muse Spark Open, an open-weight multimodal MoE model.
  • Muse Spark Open is designed for agentic coding tasks.
  • Muse Spark Open accepts image, audio, and video inputs.
  • Muse Spark Open is the largest open-weight model from Meta to date.
  • Muse Spark Open is available in Muse-Spark-Open-816B-A42B and Muse-Spark-Open-816B-A42B-NVFP4 checkpoint repositories.
  • Muse Spark Open is the full-size model of the Muse series.
  • Apertus 1.5 adds 4 trillion tokens to the 8B model and 2 trillion tokens to the 70B model.
  • Apertus 1.5 has a context window of 262,144 tokens.
  • Apertus 1.5 includes a new Decision aLoRA for general-purpose decision-making.
  • Apertus 1.5 includes a specialized Guardian aLoRA that outperforms Jev 1.13, Microsoft-Decision-1, GPT-Luna-Decisions, and Clef-Flash on safety tasks.
  • Apertus 1.5 embeds Granite-Speech-5.0-Turbo-CTC, the fastest ASR model on the OpenASR Leaderboard.
  • AI4Bharat and NVIDIA released Nemotron-pretraining-Indic-v1, a 2.8 trillion token dataset.
  • Nemotron-pretraining-Indic-v1 covers 22 Indian languages.
  • AI4Bharat and NVIDIA released an open, GPU-accelerated data generation pipeline.
  • The dataset and pipeline were developed by NVIDIA with IIT Madras and its AI4Bharat lab.
  • The pipeline includes prompts, LLM workflows, and quality checks.
  • Beam has 52 layers, 256 experts per MoE layer, and 8 experts routed per token.
  • Beam supports five reasoning values: "low", "medium", "high", "xhigh", and "max".
  • Beam has a response template in the tokenizer.
  • Beam is released under an Apache 2.0 license.
  • Beam achieves 3x to 4x higher reasoning compute efficiency than comparable open models.
  • Beam achieves up to 5x higher performance and 8x lower latency on Google Cloud.
  • DeepSeek-V4.1-Flash-1017 is an update to DeepSeek-V4.1-Flash.
  • DeepSeek-V4.1-Flash-1017 reduces task execution time and output length.
  • DeepSeek-V4.1-Flash-1017 does not include a Jinja-format chat template.
  • DeepSeek provides deepseek-recipe, Rust libraries with Python bindings for prompt formatting.
  • EmbeddingGemma 2 supports a 768-dimensional embedding space.
  • EmbeddingGemma 2 is designed for on-device deployment.
  • EmbeddingGemma 2 supports semantic search, RAG, classification, and clustering.
  • EmbeddingGemma 2 allows searching source code using natural language.
  • EmbeddingGemma 2 allows retrieving video content based on textual descriptions.
  • vLLM implements bi-directional KV cache transfer for multi-turn agentic workloads.
  • vLLM's bi-directional KV transfer allows prefill and decode nodes to reuse context.
  • vLLM released a push-based NIXL connector for disaggregated prefill/decode inference.
  • The NIXL connector moves KV cache transfer coordination off the critical path.
  • The NIXL connector reduces time-per-output-token (TPOT) for LLM inference.
  • DeepSeek open-sourced DeepInfer, its inference framework.
  • DeepInfer supports NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs.
  • DeepInfer includes Attention DP, balanced CP, MoE EP, and EPLB parallelism.
  • DeepInfer features SWA bounded replay and asynchronous scheduling.
  • DeepInfer supports batch-invariant inference, deterministic sampling, and structured outputs.
  • DeepInfer uses FlashMLA, DeepGEMM, DeepEP, TileKernels, DeepSelect, and custom TileLang kernels.
  • DeepInfer's DSpark uses confidence estimates to schedule verification.
  • DeepInfer's maximum draft length is controlled by DEEPINFER_USE_NEXTN_DECODE (default: 5).
  • DeepInfer's decoder SWA bounded replay uses DEEPINFER_YOCO_PREFILL_TRUNCATION_MODE=compact and DEEPINFER_YOCO_PREFILL_KEEP_TOKENS=140.
  • llm-d 1.0 is an open-source, Kubernetes-native stack for distributed LLM inference.
  • llm-d 1.0 was co-founded by Google with Red Hat, IBM, and others.
  • llm-d 1.0 is a Cloud Native Computing Foundation (CNCF) Sandbox project.
  • Voxtral 3 introduces a voice agent SDK and speech models for real-time, multi-step conversations.
  • Voxtral 3 supports over 50 languages.
  • Voxtral 3 integrates transcription, reasoning, and speech synthesis.
  • Voxtral 3 is available via Mistral’s EU-hosted API or self-deployment.
  • GPT-OSS-20B is an open-weight reasoning model that defaults to English for chain-of-thought.
  • GPT-OSS-20B reasoned in the correct target language only 22% of the time on a 50-example multilingual reasoning test.
  • Nebius acquired Eigen AI in a roughly $643 million deal in May.
  • i14 developed Laplace 3B, a linear attention model.
  • Laplace 3B was trained and served on AWS Trainium2 using native PyTorch.
  • Laplace 3B has a fixed recurrent state of 29.5 MB per sequence.
  • Laplace 3B trained at about 120,000 tokens per second on one trn2.48xlarge instance.
  • Laplace 3B served 11,544 output tokens per second across 1,280 concurrent sequences.
  • ZenteiQ increased batch inference throughput by 4.07x.
  • ZenteiQ's daily output grew from 1.47 billion to 5.97 billion generated tokens per 1,000 accelerators.
  • ZenteiQ used GKE and GKE Inference Gateway for its synthetic data generation pipeline.
  • VocabX reduces inference costs by 77% in a production essay grader.
  • VocabX specializes vocabulary around the cost of each token.
  • NVIDIA Kumo Relational is a pre-trained relational foundation model.
  • Kumo Relational makes predictions directly on enterprise databases.
  • Kumo Relational gathers up to 10,000 historical examples as context.
  • Kumo Relational beats supervised relational deep learning models and gradient-boosted tree pipelines.
  • Kumo Relational surpasses AutoGluon ensembles by 8% in mean reciprocal rank (MRR) on the SAP SALT enterprise benchmark.
  • Kumo Relational scales to databases with 500B+ rows.
  • Kumo Relational is available through NVIDIA's open-source, GPU-native structured-data-models library or as an NVIDIA NIM.
  • Minotaur is an AI architecture built on ELC.
  • Minotaur performs deduction, induction, and diagnosis without an LLM or GPU.
  • Minotaur completes tasks in approximately a quarter of a second per question.
  • Minotaur matched a language model's reasoning performance in blind tests.
  • Claude generated formal proofs for security properties at the assembly code level.
  • Claude's proofs covered over 100 real-world open-source software libraries.
  • Claude discovered long-standing bugs in heavily tested code.
  • Kimi K3.1 Pro is a 3T-class open-weight model.
  • Kimi K3.1 Pro Highspeed offers 2-3x faster performance than Kimi K3.1 Pro.
  • Kimi K3.1 Pro is a post-trained version of Kimi K3.
  • Kimi K3.1 Pro's full model weights will be released by November 4.
  • Cerebras deploys trillion-parameter AI models across multiple WSE-3T wafer-scale chips.
  • Cerebras' method allows large models to operate at 1000-2000+ tokens per second.
  • Cerebras distributes model layers and manages memory across 59 wafers for Kimi K3.
  • Kimi K3 is a 3-trillion parameter model.
  • Kimi K3 has 69 linear-attention layers and 24 dense layers.
  • Kimi K3 scores 88.3% on Terminal-Bench 2.1.
  • Kimi K3 scores 91.2% on BrowseComp.
  • Kimi K3 uses a 256k assumed average request size.
  • Kimi K3 uses 4-bit quantization.
  • An attention-free neural network architecture uses Grassmann flows.
  • The new architecture performs comparably to Transformers on language modeling and natural language inference tasks.
  • The Causal Grassmann layer encodes local token pairs as two-dimensional subspaces on a Grassmann manifold.
  • vLLM v0.31 introduced support for constrained decoding.
  • Constrained decoding guarantees 100% schema adherence for tool calls.
  • Kimi K2.6 achieved 92% tool-call schema accuracy with vLLM 0.28.
  • Sebastian Raschka released an 11-hour video course on building reasoning models.
  • The course covers inference scaling, reinforcement learning, and distillation.
  • Cloudflare's GLM-5.3 Workers AI model had a tenant isolation issue from August 27 to September 9, 2026.
  • 155 responses from GLM-5.3 Workers AI model contained data from other customers' requests.
  • The issue affected prompts longer than 681,280 tokens.
  • The issue was caused by a misconfiguration in the SGLang inference engine and a bug in its KV cache handling.
  • Khawaja Shams, co-founder of Momento, discussed distributed systems principles for LLM inference.
  • LFM3 is a new generation of multimodal Liquid Foundation Models (LFMs).
  • LFM3 integrates text, vision, audio, omni, and decision models on a shared backbone.
  • LFM3's initial release includes 2B parameter models.
  • LFM3 is built on a new hybrid architecture and 22T tokens of pre-training.
  • LFM3-1.7B outperforms LFM2.5-2.6B with 35% fewer parameters.
  • LFM3 includes six 2B-class models (text, vision, audio, omni, and decision) trained from LFM3-1.7B-Base.
  • LFM3 includes the first omni model and video input for its vision model.
  • LFM3's text model fits in about 1 GB at 4-bit precision.
  • LFM3 has new in-house vision and audio encoders.
  • LFM3's text model has a 256K context window.
  • Meta Andromeda is Meta's personalized ads retrieval engine.
  • Meta Andromeda scores over one billion ads per second.
  • Meta Andromeda achieved a 100x increase in throughput compared to its 2024 system.
  • Meta Andromeda delivered a 2.4% increase in conversions on Facebook since 2025.
  • Hierarchical Networks (H-Nets) achieve 3-5x compute efficiency gains over tokenized Transformers.
  • H-Nets learn directly from raw data, addressing tokenizer limitations.
  • Memdoor workflow orchestrates five Qwen3.8-27B models.
  • The Memdoor workflow uses a planner, two ideators, a solver, a manager, and a repair worker.
  • Each model in the Memdoor workflow starts with a fresh context and communicates via disk files.
  • A script determines success for the Memdoor workflow, not a model.
  • The Memdoor workflow is defined by eight YAML files.
  • The Memdoor workflow uses Qwen3.8-27B models with a 128K reply cap.
  • The evaluation used the first ten problems from LiveCodeBench Hard, with five passes each.
  • Each run of the Memdoor workflow took a median of 40 minutes and cost $0.39.
  • The 50 runs of the Memdoor workflow cost $35.79.
  • Joint-Embedding Predictive Architectures (JEPAs) trained on different modalities learn the same representation under specific conditions.
  • Next-token prediction is a JEPA with 'bad views', parts of a single document.
  • Good views need 42 times fewer training samples than next-token prediction for alignment.
  • GPT-6 Astra can read and write neural network weights.
  • GPT-6 Astra can identify and fix issues in models like Qwen3.5-0.8B without gradient-based training.
  • GPT-6 Astra improved Qwen3.5-0.8B's MATH-500 score from 7.9% to 48.1%.
  • GPT-6 Astra improved Qwen3.5-0.8B's IFEval score by 3x.
  • GPT-6 Astra edited 0.016% of Qwen3.5-0.8B's weights.
  • The process of direct weight manipulation by LLMs is called Weight-Space Programming.
  • Hugging Face introduced Model Integration Contracts (MICs) to address regressions in its Transformers library.
  • A bug allowed T5 models to attend to future tokens, staying in the main branch for almost two months.
  • MICs use small, representative model fixtures and baselines to identify issues before they reach the main branch.
  • The Transformers library supports over 750 model types.
  • The Transformers test suite includes over 88,000 CPU tests and 200,000+ GPU tests.
  • Palette fine-tuned a 20B model to generate slide decks conforming to specific design languages.
  • Palette's model was trained on synthetic data following the IBM Design Language.
  • Palette's model ranks first on two of three PresentBench domains.
  • ByteDance developed USDB, a unified sequence storage system for recommender systems.
  • USDB reduced write latency by 85.8% and infrastructure costs by 42.1% in production.
  • USDB's aggregation pushdown sustains 8.5x the throughput of ordinary reads.
  • Meta released F3 Lightning, an infrastructure layer for ads ranking models.
  • F3 Lightning decouples feature releases from shared execution in ads ranking models.
  • F3 Lightning reduces deployment time for comparable extraction features by approximately 3.6 times.
  • Qwen 3.8 27B running on a laptop can interpret raw neural network weights.
  • Google introduced the LiteRT Tensor API for on-device machine learning model optimization.
  • LiteRT Tensor API allows defining computation graphs in C++ that run across CPU, GPU, and NPU.
  • "The Little Book of Generative AI Foundations" covers PCA, VAEs, diffusion models, GANs, and energy-based models.
  • Kimi K3.1 Pro is natively multimodal.
  • Kimi K3.1 Pro uses Gated DeepSeek Sparse Attention layers.
  • FlashProve uses zkSNARKs to cryptographically prove AI model responses.
  • Ripple-65B Preview is a 65B parameter model.
  • Ripple-65B Preview was trained on 12T tokens for $32 million.
  • Ripple-65B Preview scored 80.52 on TerminalBench 2.1.
  • Baidu released ERNIE 5.5 Flash Base.
  • ERNIE 5.5 Flash Base is a sparse Mixture-of-Experts model.
  • ERNIE 5.5 Flash Base has 280 billion total parameters.
  • ERNIE 5.5 Flash Base activates 14 billion parameters per token.
  • ERNIE 5.5 Flash Base was pre-trained on 33.5 trillion tokens.
  • ERNIE 5.5 Flash Base has a training context of up to 128K.
  • The Stanford AI Index 2026 reported a 3.3% lead for top closed models over top open models as of March 2026.
  • Memorilla uses 16 memory tokens.
  • Memorilla outperforms Mem0 and MemoRAG on five of eight benchmarks.
  • Memorilla achieves 84.33 on FactKG.
  • SWE-2 Flash is three times faster than SWE-2.
  • SWE-2 Flash costs one-fifth as much as SWE-2.
  • SWE-2 Flash is built on Thinking Machines Lab's Inkling-Small.
  • Inkling-Small is a 276B parameter Mixture-of-Experts model.
  • Inkling-Small has 12B active parameters.
  • Inkling-Small was released on July 30.
  • SWE-2 Flash scored 20% on Terminal-Bench 4.0.
  • Inkling-Small scored 0.9% on Terminal-Bench 4.0.
  • Next-Concept Modeling (NCM) is a two-stage, coarse-to-fine text generation framework.
  • NCM uses a multiscale residual VQ-VAE and a transformer.
  • NCM achieves 21x faster generation than DLMs on LM1B with better perplexity.
  • NCM is 10x faster than DLMs on OpenWebText.
  • NCM is 31x faster than DLMs on XSum with matched ROUGE-L.
  • NCM outperforms autoregressive and diffusion baselines on Sudoku.
  • HOPE prunes MoE models by considering pairwise expert interactions.
  • HOPE achieves up to 6.1% better performance on coding tasks at 50% pruning rates.
  • TPU Sync is an open-source C++ library for optimizing model state management across TPU clusters.
  • TPU Sync reduces KV cache transfer latency by over 90% for large MoE models.
  • TPU Sync reduces Time to First Token (TTFT) by an average of 90% for large MoE models.
  • TPU Sync treats HBM, host DRAM, and network as a unified memory hierarchy.
  • Elith and NVIDIA released Nemotron-Personas-Finance-Japan, a synthetic dataset for financial AI in Japan.
  • Nemotron-Personas-Finance-Japan provides 9 million persona records.
  • Nemotron-Personas-Finance-Japan uses NeMo SDG-PGMs and NeMo Data Designer.
  • Meta introduced Muse Ember, an AI model for complex reasoning and agent-based tasks.
  • Muse Ember offers high intelligence per dollar compared to Claude Haiku 5.5 or GPT-6 Luna.
  • Muse Ember is available on the Meta Model API at $0.10 per million input tokens and $0.50 per million output tokens.
  • TPU Sync uses zero-copy transfers and native PjRtBuffer hardware descriptors.
  • TPU Sync achieved 223 GB/s combined host-to-device and device-to-host throughput per Ironwood TPU host.
  • Meta developed Triton Low-Level Extensions (TLX) to optimize GPU performance.
  • TLX provides explicit control over GPU execution while maintaining Triton's programming model.
  • TLX introduces explicit warp-group and cluster-level orchestration.
  • An open field guide introduces inference engineering for platform engineers.
  • The guide explains how models and GPUs operate together in production environments.
  • The guide covers tools and career development for inference engineering.
  • Token pooling-aware training improves retrieval quality while reducing storage.
  • A model with pooling-aware training retained 97.8% of nDCG@10 with 4x reduction.
  • Intelligent Delivery improved ad ranking performance by 10% since 2017.
  • Huawei released MACaQ, an internal quantization tool for local AI inference.
  • MACaQ quantizes a 122B-parameter model in 64 minutes using 32.5 GB memory.
  • MACaQ supports llama.cpp, vLLM, Diffusers, and PyTorch.
  • smolmaxxing allows 8B and 16B MoE models to run on devices with ~1GB RAM.
  • smolmaxxing uses custom kernels and streams model weights from flash storage.
  • smolmaxxing is a patch on llama.cpp.
  • smolmaxxing includes a 1-bit lookup-table kernel and repacked 2-bit/ternary formats.
  • smolmaxxing drops rarely used vocabulary output rows to save RAM.
  • IBM developed LLMON, an LLM-native markup language.
  • LLMON aims to improve LLM robustness and security by explicitly defining context components.
  • LLMON uses meaningful tags wrapped in reserved special-token delimiters.
  • LLMON states data schemas unambiguously for causal LLMs.
  • LLMON uses unique IDs to point to specific instructions.
  • Two preview models, granite-4.1-3b-LLMON-preview and granite-4.1-8b-LLMON-preview, support LLMON.
  • Edge0 open-sourced Edge0-145B and Edge0-120B models.
  • Edge0-145B is the first open-source on-device AI model with 145 billion parameters.
  • Edge0-120B is an on-device AI model.
  • Edge0 released a unified inference framework and deployment tools.
  • llama.cpp and ggml projects released version 1.0.0 of their libraries.
  • llama.cpp v1.0.0 establishes semantic versioning for future API and ABI changes.
  • Weight space editing allows one AI model to directly modify another's weights.
  • This process is called "weight space misfolding" and can induce unauthorized behavior.
  • Unsloth updated Qwen3.8-27B GGUFs to use Dynamic V3.1 algorithm.
  • Dynamic V3.1 improves average accuracy by 5-10% on benchmarks.
  • Qwen3.8-27B runs locally on 17GB RAM/VRAM setups.
  • Unsloth quants include developer role support for agentic tools.
  • Unsloth quants improve parsing nested objects for tool calling.
  • Unsloth Dynamic V3.1 is a preview algorithm.
  • Most Unsloth quants from 2-bit to 6-bit are updated.
  • 3-bit UD3 quants perform on par with UD3.1 and are not updated.
  • Energy-Based Reasoners (EBRs) are a new type of looped model.
  • EBRs learn an energy function to guide multi-step reasoning.
  • EBRs formulate looped updates as gradient descent on this energy.
  • EBRs achieved 99.9% accuracy on Sudoku-Extreme with 345x less inference compute.
  • MoE inference on consumer hardware achieved within 6% of Strata engine performance.
  • Meta's template-driven ML approach reduced per-model engineering time by 92%.
  • Meta's template-driven ML approach improved offline model quality by 0.63% on average.
  • A new predictive model estimates fine-tuning time with an RΒ² of 0.98 and MAE of 1.71 hours.
  • RadixArk released DaisyRose-27B, trained on Qwen3.8-27B with reinforcement learning.
  • DaisyRose-27B improves Terminal-Bench 4.1 pass@1 from 5.6% to 12.0%.
  • Cloudflare released an updated Clairic model and a new 9B-parameter Clairic-flash model.
  • DaisyRose-27B was trained using the Miles framework.
  • DaisyRose-27B was trained on 1,200 curated long-horizon agent tasks.
  • DaisyRose-27B training used 144 H200 GPUs over 18 days.
  • DaisyRose-27B is available in BF16, FP8, and NVFP4 checkpoints.
  • DaisyRose-27B training tasks were built in partnership with Eigent AI.
  • Exa increased embedding throughput by 50% for indexing documents.
  • Exa migrated to AWS Neuron hardware (Inferentia2 and Trainium2) for inference.
  • AI systems designed and verified a new general-purpose CPU architecture named Frostbite.
  • Frostbite CPU architecture uses interaction nets for its instruction set.
  • Frostbite achieved a 4.59x performance-per-watt improvement over a modern superscalar CPU core.
  • The performance gain was measured on 25 workloads, 15 of which were held out until final evaluation.
  • Amazon ElastiCache offers distributed KV caching for self-hosted LLMs.
  • Distributed KV caching reduces self-hosted LLM inference costs by up to 40%.
  • Distributed KV caching reduces time to first token by up to 80%.

Introduction of New AI Models

Hugging Face, Cognition, and Xiaomi have launched new AI models enhancing long-horizon coding and robotics capabilities. These models seek to offer improved performance by leveraging extensive contexts and innovative techniques.

GLM-5.2 by Hugging Face

GLM-5.2 extends support for coding-agent scenarios with a robust 1 million token context. Its architecture allows for efficient handling of complex coding tasks. The model is open-source, available under an MIT license.

SWE-1.7 Advances Reinforcement Learning

Cognition's SWE-1.7 model advances long-horizon asynchronous tasks by applying improved reinforcement learning methods. It aims to enhance cost-performance efficiency for software engineering tasks.

Robotics improved by Xiaomi's Pre-training

Xiaomi-Robotics-1 leverages 100,000 hours of pre-training data to address robotics' data scarcity. It combines pre-training with real-robot data to improve model capability, providing insights into large-scale training effects.

Implications for AI Development

These models underline ongoing advancements in high-reasoning AI necessary for complex problem-solving. By achieving breakthroughs in long-horizon tasks, these developments suggest a shift towards more sophisticated AI solutions.

Updates

πŸ•’ 2026-11-21 Β· new reporting from AWS Machine Learning Blog
  • Amazon ElastiCache offers distributed KV caching for self-hosted LLMs.
  • Distributed KV caching reduces self-hosted LLM inference costs by up to 40%.
  • Distributed KV caching reduces time to first token by up to 80%.
πŸ•’ 2026-11-20 Β· new reporting from Hacker News Front Page
  • AI systems designed and verified a new general-purpose CPU architecture named Frostbite.
  • Frostbite CPU architecture uses interaction nets for its instruction set.
  • Frostbite achieved a 4.59x performance-per-watt improvement over a modern superscalar CPU core.
  • The performance gain was measured on 25 workloads, 15 of which were held out until final evaluation.
πŸ•’ 2026-11-19 Β· new reporting from Hacker News Front Page, AWS Machine Learning Blog
  • DaisyRose-27B was trained using the Miles framework.
  • DaisyRose-27B was trained on 1,200 curated long-horizon agent tasks.
  • DaisyRose-27B training used 144 H200 GPUs over 18 days.
  • DaisyRose-27B is available in BF16, FP8, and NVFP4 checkpoints.
  • DaisyRose-27B training tasks were built in partnership with Eigent AI.
  • Exa increased embedding throughput by 50% for indexing documents.
  • Exa migrated to AWS Neuron hardware (Inferentia2 and Trainium2) for inference.
πŸ•’ 2026-11-18 Β· new reporting from Hugging Face Blog, Meta Engineering, Cloudflare Blog
  • MoE inference on consumer hardware achieved within 6% of Strata engine performance.
  • Meta's template-driven ML approach reduced per-model engineering time by 92%.
  • Meta's template-driven ML approach improved offline model quality by 0.63% on average.
  • A new predictive model estimates fine-tuning time with an RΒ² of 0.98 and MAE of 1.71 hours.
  • RadixArk released DaisyRose-27B, trained on Qwen3.8-27B with reinforcement learning.
  • DaisyRose-27B improves Terminal-Bench 4.1 pass@1 from 5.6% to 12.0%.
  • Cloudflare released an updated Clairic model and a new 9B-parameter Clairic-flash model.
πŸ•’ 2026-11-18 Β· new reporting from Hacker News Front Page
  • Energy-Based Reasoners (EBRs) are a new type of looped model.
  • EBRs learn an energy function to guide multi-step reasoning.
  • EBRs formulate looped updates as gradient descent on this energy.
  • EBRs achieved 99.9% accuracy on Sudoku-Extreme with 345x less inference compute.
πŸ•’ 2026-11-14 Β· new reporting from Hacker News Front Page
  • Unsloth updated Qwen3.8-27B GGUFs to use Dynamic V3.1 algorithm.
  • Dynamic V3.1 improves average accuracy by 5-10% on benchmarks.
  • Qwen3.8-27B runs locally on 17GB RAM/VRAM setups.
  • Unsloth quants include developer role support for agentic tools.
  • Unsloth quants improve parsing nested objects for tool calling.
  • Unsloth Dynamic V3.1 is a preview algorithm.
  • Most Unsloth quants from 2-bit to 6-bit are updated.
  • 3-bit UD3 quants perform on par with UD3.1 and are not updated.
πŸ•’ 2026-11-13 Β· new reporting from Hugging Face Blog, Hacker News Front Page
  • llama.cpp and ggml projects released version 1.0.0 of their libraries.
  • llama.cpp v1.0.0 establishes semantic versioning for future API and ABI changes.
  • Weight space editing allows one AI model to directly modify another's weights.
  • This process is called "weight space misfolding" and can induce unauthorized behavior.
πŸ•’ 2026-11-13 Β· new reporting from Hacker News Front Page
  • Edge0 open-sourced Edge0-145B and Edge0-120B models.
  • Edge0-145B is the first open-source on-device AI model with 145 billion parameters.
  • Edge0-120B is an on-device AI model.
  • Edge0 released a unified inference framework and deployment tools.
πŸ•’ 2026-11-12 Β· new reporting from Hacker News Front Page, Hugging Face Blog
  • smolmaxxing allows 8B and 16B MoE models to run on devices with ~1GB RAM.
  • smolmaxxing uses custom kernels and streams model weights from flash storage.
  • smolmaxxing is a patch on llama.cpp.
  • smolmaxxing includes a 1-bit lookup-table kernel and repacked 2-bit/ternary formats.
  • smolmaxxing drops rarely used vocabulary output rows to save RAM.
  • IBM developed LLMON, an LLM-native markup language.
  • LLMON aims to improve LLM robustness and security by explicitly defining context components.
  • LLMON uses meaningful tags wrapped in reserved special-token delimiters.
  • LLMON states data schemas unambiguously for causal LLMs.
  • LLMON uses unique IDs to point to specific instructions.
  • Two preview models, granite-4.1-3b-LLMON-preview and granite-4.1-8b-LLMON-preview, support LLMON.
πŸ•’ 2026-11-11 Β· new reporting from Hugging Face Blog, Meta Engineering, Hacker News Front Page
  • Token pooling-aware training improves retrieval quality while reducing storage.
  • A model with pooling-aware training retained 97.8% of nDCG@10 with 4x reduction.
  • Intelligent Delivery improved ad ranking performance by 10% since 2017.
  • Huawei released MACaQ, an internal quantization tool for local AI inference.
  • MACaQ quantizes a 122B-parameter model in 64 minutes using 32.5 GB memory.
  • MACaQ supports llama.cpp, vLLM, Diffusers, and PyTorch.
πŸ•’ 2026-11-10 Β· new reporting from Meta Engineering, Hacker News Front Page
  • Meta developed Triton Low-Level Extensions (TLX) to optimize GPU performance.
  • TLX provides explicit control over GPU execution while maintaining Triton's programming model.
  • TLX introduces explicit warp-group and cluster-level orchestration.
  • An open field guide introduces inference engineering for platform engineers.
  • The guide explains how models and GPUs operate together in production environments.
  • The guide covers tools and career development for inference engineering.
πŸ•’ 2026-11-09 Β· new reporting from InfoQ, Hugging Face Blog, Google Cloud Blog, Hacker News Front Page, The New Stack
  • HOPE prunes MoE models by considering pairwise expert interactions.
  • HOPE achieves up to 6.1% better performance on coding tasks at 50% pruning rates.
  • TPU Sync is an open-source C++ library for optimizing model state management across TPU clusters.
  • TPU Sync reduces KV cache transfer latency by over 90% for large MoE models.
  • TPU Sync reduces Time to First Token (TTFT) by an average of 90% for large MoE models.
  • TPU Sync treats HBM, host DRAM, and network as a unified memory hierarchy.
  • Elith and NVIDIA released Nemotron-Personas-Finance-Japan, a synthetic dataset for financial AI in Japan.
  • Nemotron-Personas-Finance-Japan provides 9 million persona records.
  • Nemotron-Personas-Finance-Japan uses NeMo SDG-PGMs and NeMo Data Designer.
  • Meta introduced Muse Ember, an AI model for complex reasoning and agent-based tasks.
  • Muse Ember offers high intelligence per dollar compared to Claude Haiku 5.5 or GPT-6 Luna.
  • Muse Ember is available on the Meta Model API at $0.10 per million input tokens and $0.50 per million output tokens.
  • TPU Sync uses zero-copy transfers and native PjRtBuffer hardware descriptors.
  • TPU Sync achieved 223 GB/s combined host-to-device and device-to-host throughput per Ironwood TPU host.
πŸ•’ 2026-11-09 Β· new reporting from Hacker News Front Page
  • Next-Concept Modeling (NCM) is a two-stage, coarse-to-fine text generation framework.
  • NCM uses a multiscale residual VQ-VAE and a transformer.
  • NCM achieves 21x faster generation than DLMs on LM1B with better perplexity.
  • NCM is 10x faster than DLMs on OpenWebText.
  • NCM is 31x faster than DLMs on XSum with matched ROUGE-L.
  • NCM outperforms autoregressive and diffusion baselines on Sudoku.
πŸ•’ 2026-11-06 Β· new reporting from Hacker News Front Page, The New Stack
  • Memorilla uses 16 memory tokens.
  • Memorilla outperforms Mem0 and MemoRAG on five of eight benchmarks.
  • Memorilla achieves 84.33 on FactKG.
  • SWE-2 Flash is three times faster than SWE-2.
  • SWE-2 Flash costs one-fifth as much as SWE-2.
  • SWE-2 Flash is built on Thinking Machines Lab's Inkling-Small.
  • Inkling-Small is a 276B parameter Mixture-of-Experts model.
  • Inkling-Small has 12B active parameters.
  • Inkling-Small was released on July 30.
  • SWE-2 Flash scored 20% on Terminal-Bench 4.0.
  • Inkling-Small scored 0.9% on Terminal-Bench 4.0.
πŸ•’ 2026-11-05 Β· new reporting from The New Stack
  • The Stanford AI Index 2026 reported a 3.3% lead for top closed models over top open models as of March 2026.
πŸ•’ 2026-11-05 Β· new reporting from Hacker News Front Page
  • Baidu released ERNIE 5.5 Flash Base.
  • ERNIE 5.5 Flash Base is a sparse Mixture-of-Experts model.
  • ERNIE 5.5 Flash Base has 280 billion total parameters.
  • ERNIE 5.5 Flash Base activates 14 billion parameters per token.
  • ERNIE 5.5 Flash Base was pre-trained on 33.5 trillion tokens.
  • ERNIE 5.5 Flash Base has a training context of up to 128K.
πŸ•’ 2026-11-04 Β· new reporting from Hacker News Front Page
  • Kimi K3.1 Pro is natively multimodal.
  • Kimi K3.1 Pro uses Gated DeepSeek Sparse Attention layers.
  • FlashProve uses zkSNARKs to cryptographically prove AI model responses.
  • Ripple-65B Preview is a 65B parameter model.
  • Ripple-65B Preview was trained on 12T tokens for $32 million.
  • Ripple-65B Preview scored 80.52 on TerminalBench 2.1.
πŸ•’ 2026-11-03 Β· new reporting from Hugging Face Blog, Hacker News Front Page, Meta Engineering, InfoQ
  • Palette fine-tuned a 20B model to generate slide decks conforming to specific design languages.
  • Palette's model was trained on synthetic data following the IBM Design Language.
  • Palette's model ranks first on two of three PresentBench domains.
  • ByteDance developed USDB, a unified sequence storage system for recommender systems.
  • USDB reduced write latency by 85.8% and infrastructure costs by 42.1% in production.
  • USDB's aggregation pushdown sustains 8.5x the throughput of ordinary reads.
  • Meta released F3 Lightning, an infrastructure layer for ads ranking models.
  • F3 Lightning decouples feature releases from shared execution in ads ranking models.
  • F3 Lightning reduces deployment time for comparable extraction features by approximately 3.6 times.
  • Qwen 3.8 27B running on a laptop can interpret raw neural network weights.
  • Google introduced the LiteRT Tensor API for on-device machine learning model optimization.
  • LiteRT Tensor API allows defining computation graphs in C++ that run across CPU, GPU, and NPU.
  • "The Little Book of Generative AI Foundations" covers PCA, VAEs, diffusion models, GANs, and energy-based models.
πŸ•’ 2026-11-03 Β· new reporting from Hugging Face Blog
  • Hugging Face introduced Model Integration Contracts (MICs) to address regressions in its Transformers library.
  • A bug allowed T5 models to attend to future tokens, staying in the main branch for almost two months.
  • MICs use small, representative model fixtures and baselines to identify issues before they reach the main branch.
  • The Transformers library supports over 750 model types.
  • The Transformers test suite includes over 88,000 CPU tests and 200,000+ GPU tests.
πŸ•’ 2026-11-02 Β· new reporting from Hacker News Front Page
  • GPT-6 Astra can read and write neural network weights.
  • GPT-6 Astra can identify and fix issues in models like Qwen3.5-0.8B without gradient-based training.
  • GPT-6 Astra improved Qwen3.5-0.8B's MATH-500 score from 7.9% to 48.1%.
  • GPT-6 Astra improved Qwen3.5-0.8B's IFEval score by 3x.
  • GPT-6 Astra edited 0.016% of Qwen3.5-0.8B's weights.
  • The process of direct weight manipulation by LLMs is called Weight-Space Programming.
πŸ•’ 2026-11-01 Β· new reporting from Hacker News Front Page
  • Joint-Embedding Predictive Architectures (JEPAs) trained on different modalities learn the same representation under specific conditions.
  • Next-token prediction is a JEPA with 'bad views', parts of a single document.
  • Good views need 42 times fewer training samples than next-token prediction for alignment.
πŸ•’ 2026-11-01 Β· new reporting from Hacker News Front Page
  • Memdoor workflow orchestrates five Qwen3.8-27B models.
  • The Memdoor workflow uses a planner, two ideators, a solver, a manager, and a repair worker.
  • Each model in the Memdoor workflow starts with a fresh context and communicates via disk files.
  • A script determines success for the Memdoor workflow, not a model.
  • The Memdoor workflow is defined by eight YAML files.
  • The Memdoor workflow uses Qwen3.8-27B models with a 128K reply cap.
  • The evaluation used the first ten problems from LiveCodeBench Hard, with five passes each.
  • Each run of the Memdoor workflow took a median of 40 minutes and cost $0.39.
  • The 50 runs of the Memdoor workflow cost $35.79.
πŸ•’ 2026-10-29 Β· new reporting from Hacker News Front Page
  • Hierarchical Networks (H-Nets) achieve 3-5x compute efficiency gains over tokenized Transformers.
  • H-Nets learn directly from raw data, addressing tokenizer limitations.
πŸ•’ 2026-10-28 Β· new reporting from Meta Engineering
  • Meta Andromeda is Meta's personalized ads retrieval engine.
  • Meta Andromeda scores over one billion ads per second.
  • Meta Andromeda achieved a 100x increase in throughput compared to its 2024 system.
  • Meta Andromeda delivered a 2.4% increase in conversions on Facebook since 2025.
πŸ•’ 2026-10-27 Β· new reporting from InfoQ, Hacker News Front Page, Hugging Face Blog
  • Khawaja Shams, co-founder of Momento, discussed distributed systems principles for LLM inference.
  • LFM3 is a new generation of multimodal Liquid Foundation Models (LFMs).
  • LFM3 integrates text, vision, audio, omni, and decision models on a shared backbone.
  • LFM3's initial release includes 2B parameter models.
  • LFM3 is built on a new hybrid architecture and 22T tokens of pre-training.
  • LFM3-1.7B outperforms LFM2.5-2.6B with 35% fewer parameters.
  • LFM3 includes six 2B-class models (text, vision, audio, omni, and decision) trained from LFM3-1.7B-Base.
  • LFM3 includes the first omni model and video input for its vision model.
  • LFM3's text model fits in about 1 GB at 4-bit precision.
  • LFM3 has new in-house vision and audio encoders.
  • LFM3's text model has a 256K context window.
πŸ•’ 2026-10-26 Β· new reporting from Hacker News Front Page, Cloudflare Blog
  • vLLM v0.31 introduced support for constrained decoding.
  • Constrained decoding guarantees 100% schema adherence for tool calls.
  • Kimi K2.6 achieved 92% tool-call schema accuracy with vLLM 0.28.
  • Sebastian Raschka released an 11-hour video course on building reasoning models.
  • The course covers inference scaling, reinforcement learning, and distillation.
  • Cloudflare's GLM-5.3 Workers AI model had a tenant isolation issue from August 27 to September 9, 2026.
  • 155 responses from GLM-5.3 Workers AI model contained data from other customers' requests.
  • The issue affected prompts longer than 681,280 tokens.
  • The issue was caused by a misconfiguration in the SGLang inference engine and a bug in its KV cache handling.
πŸ•’ 2026-10-26 Β· new reporting from Hacker News Front Page
  • An attention-free neural network architecture uses Grassmann flows.
  • The new architecture performs comparably to Transformers on language modeling and natural language inference tasks.
  • The Causal Grassmann layer encodes local token pairs as two-dimensional subspaces on a Grassmann manifold.
πŸ•’ 2026-10-25 Β· new reporting from Hacker News Front Page
  • Cerebras deploys trillion-parameter AI models across multiple WSE-3T wafer-scale chips.
  • Cerebras' method allows large models to operate at 1000-2000+ tokens per second.
  • Cerebras distributes model layers and manages memory across 59 wafers for Kimi K3.
  • Kimi K3 is a 3-trillion parameter model.
  • Kimi K3 has 69 linear-attention layers and 24 dense layers.
  • Kimi K3 scores 88.3% on Terminal-Bench 2.1.
  • Kimi K3 scores 91.2% on BrowseComp.
  • Kimi K3 uses a 256k assumed average request size.
  • Kimi K3 uses 4-bit quantization.
πŸ•’ 2026-10-24 Β· new reporting from Hacker News Front Page
  • Kimi K3.1 Pro is a 3T-class open-weight model.
  • Kimi K3.1 Pro Highspeed offers 2-3x faster performance than Kimi K3.1 Pro.
  • Kimi K3.1 Pro is a post-trained version of Kimi K3.
  • Kimi K3.1 Pro's full model weights will be released by November 4.
πŸ•’ 2026-10-23 Β· new reporting from Hacker News Front Page
  • Claude generated formal proofs for security properties at the assembly code level.
  • Claude's proofs covered over 100 real-world open-source software libraries.
  • Claude discovered long-standing bugs in heavily tested code.
πŸ•’ 2026-10-22 Β· new reporting from Hacker News Front Page
  • Minotaur is an AI architecture built on ELC.
  • Minotaur performs deduction, induction, and diagnosis without an LLM or GPU.
  • Minotaur completes tasks in approximately a quarter of a second per question.
  • Minotaur matched a language model's reasoning performance in blind tests.
πŸ•’ 2026-10-21 Β· new reporting from Google Cloud Blog, Hugging Face Blog
  • ZenteiQ increased batch inference throughput by 4.07x.
  • ZenteiQ's daily output grew from 1.47 billion to 5.97 billion generated tokens per 1,000 accelerators.
  • ZenteiQ used GKE and GKE Inference Gateway for its synthetic data generation pipeline.
  • VocabX reduces inference costs by 77% in a production essay grader.
  • VocabX specializes vocabulary around the cost of each token.
  • NVIDIA Kumo Relational is a pre-trained relational foundation model.
  • Kumo Relational makes predictions directly on enterprise databases.
  • Kumo Relational gathers up to 10,000 historical examples as context.
  • Kumo Relational beats supervised relational deep learning models and gradient-boosted tree pipelines.
  • Kumo Relational surpasses AutoGluon ensembles by 8% in mean reciprocal rank (MRR) on the SAP SALT enterprise benchmark.
  • Kumo Relational scales to databases with 500B+ rows.
  • Kumo Relational is available through NVIDIA's open-source, GPU-native structured-data-models library or as an NVIDIA NIM.
πŸ•’ 2026-10-20 Β· new reporting from Google Cloud Blog, Hacker News Front Page, AWS Machine Learning Blog, The New Stack
  • llm-d 1.0 is an open-source, Kubernetes-native stack for distributed LLM inference.
  • llm-d 1.0 was co-founded by Google with Red Hat, IBM, and others.
  • llm-d 1.0 is a Cloud Native Computing Foundation (CNCF) Sandbox project.
  • Voxtral 3 introduces a voice agent SDK and speech models for real-time, multi-step conversations.
  • Voxtral 3 supports over 50 languages.
  • Voxtral 3 integrates transcription, reasoning, and speech synthesis.
  • Voxtral 3 is available via Mistral’s EU-hosted API or self-deployment.
  • GPT-OSS-20B is an open-weight reasoning model that defaults to English for chain-of-thought.
  • GPT-OSS-20B reasoned in the correct target language only 22% of the time on a 50-example multilingual reasoning test.
  • Nebius acquired Eigen AI in a roughly $643 million deal in May.
  • i14 developed Laplace 3B, a linear attention model.
  • Laplace 3B was trained and served on AWS Trainium2 using native PyTorch.
  • Laplace 3B has a fixed recurrent state of 29.5 MB per sequence.
  • Laplace 3B trained at about 120,000 tokens per second on one trn2.48xlarge instance.
  • Laplace 3B served 11,544 output tokens per second across 1,280 concurrent sequences.
πŸ•’ 2026-10-20 Β· new reporting from Hacker News Front Page
  • DeepSeek open-sourced DeepInfer, its inference framework.
  • DeepInfer supports NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs.
  • DeepInfer includes Attention DP, balanced CP, MoE EP, and EPLB parallelism.
  • DeepInfer features SWA bounded replay and asynchronous scheduling.
  • DeepInfer supports batch-invariant inference, deterministic sampling, and structured outputs.
  • DeepInfer uses FlashMLA, DeepGEMM, DeepEP, TileKernels, DeepSelect, and custom TileLang kernels.
  • DeepInfer's DSpark uses confidence estimates to schedule verification.
  • DeepInfer's maximum draft length is controlled by DEEPINFER_USE_NEXTN_DECODE (default: 5).
  • DeepInfer's decoder SWA bounded replay uses DEEPINFER_YOCO_PREFILL_TRUNCATION_MODE=compact and DEEPINFER_YOCO_PREFILL_KEEP_TOKENS=140.
πŸ•’ 2026-10-19 Β· new reporting from AWS Machine Learning Blog
  • vLLM implements bi-directional KV cache transfer for multi-turn agentic workloads.
  • vLLM's bi-directional KV transfer allows prefill and decode nodes to reuse context.
  • vLLM released a push-based NIXL connector for disaggregated prefill/decode inference.
  • The NIXL connector moves KV cache transfer coordination off the critical path.
  • The NIXL connector reduces time-per-output-token (TPOT) for LLM inference.
πŸ•’ 2026-10-19 Β· new reporting from InfoQ
  • EmbeddingGemma 2 supports a 768-dimensional embedding space.
  • EmbeddingGemma 2 is designed for on-device deployment.
  • EmbeddingGemma 2 supports semantic search, RAG, classification, and clustering.
  • EmbeddingGemma 2 allows searching source code using natural language.
  • EmbeddingGemma 2 allows retrieving video content based on textual descriptions.
πŸ•’ 2026-10-17 Β· new reporting from Hacker News Front Page
  • DeepSeek-V4.1-Flash-1017 is an update to DeepSeek-V4.1-Flash.
  • DeepSeek-V4.1-Flash-1017 reduces task execution time and output length.
  • DeepSeek-V4.1-Flash-1017 does not include a Jinja-format chat template.
  • DeepSeek provides deepseek-recipe, Rust libraries with Python bindings for prompt formatting.
πŸ•’ 2026-10-16 Β· new reporting from Hugging Face Blog, Google Cloud Blog
  • Beam has 52 layers, 256 experts per MoE layer, and 8 experts routed per token.
  • Beam supports five reasoning values: "low", "medium", "high", "xhigh", and "max".
  • Beam has a response template in the tokenizer.
  • Beam is released under an Apache 2.0 license.
  • Beam achieves 3x to 4x higher reasoning compute efficiency than comparable open models.
  • Beam achieves up to 5x higher performance and 8x lower latency on Google Cloud.
πŸ•’ 2026-10-16 Β· new reporting from Hugging Face Blog
  • AI4Bharat and NVIDIA released Nemotron-pretraining-Indic-v1, a 2.8 trillion token dataset.
  • Nemotron-pretraining-Indic-v1 covers 22 Indian languages.
  • AI4Bharat and NVIDIA released an open, GPU-accelerated data generation pipeline.
  • The dataset and pipeline were developed by NVIDIA with IIT Madras and its AI4Bharat lab.
  • The pipeline includes prompts, LLM workflows, and quality checks.
πŸ•’ 2026-10-15 Β· new reporting from Hugging Face Blog
  • Meta released Muse Spark Open, an open-weight multimodal MoE model.
  • Muse Spark Open is designed for agentic coding tasks.
  • Muse Spark Open accepts image, audio, and video inputs.
  • Muse Spark Open is the largest open-weight model from Meta to date.
  • Muse Spark Open is available in Muse-Spark-Open-816B-A42B and Muse-Spark-Open-816B-A42B-NVFP4 checkpoint repositories.
  • Muse Spark Open is the full-size model of the Muse series.
  • Apertus 1.5 adds 4 trillion tokens to the 8B model and 2 trillion tokens to the 70B model.
  • Apertus 1.5 has a context window of 262,144 tokens.
  • Apertus 1.5 includes a new Decision aLoRA for general-purpose decision-making.
  • Apertus 1.5 includes a specialized Guardian aLoRA that outperforms Jev 1.13, Microsoft-Decision-1, GPT-Luna-Decisions, and Clef-Flash on safety tasks.
  • Apertus 1.5 embeds Granite-Speech-5.0-Turbo-CTC, the fastest ASR model on the OpenASR Leaderboard.
πŸ•’ 2026-10-15 Β· new reporting from Hacker News Front Page
  • LFM2.5-24B-A2B is the largest and final model in the LFM2.5 family.
  • LFM2.5-24B-A2B runs up to 1.7x faster than comparable MoEs on-device.
  • LFM2.5-24B-A2B is competitive with Qwen3.6-35B-A3B on five of eight agentic and tool-use benchmarks.
  • LFM2.5-24B-A2B scores 27.69 on AutomationBench.
  • LFM2.5-24B-A2B is available on Hugging Face and the developer's Playground.
  • LFM2.5-24B-A2B uses large-scale reinforcement learning.
  • LFM2.5-24B-A2B is a post-trained model.
πŸ•’ 2026-10-14 Β· new reporting from Netflix Tech Blog
  • Netflix integrated its Foundation Model into personalization applications.
  • Netflix uses embeddings, subgraph, and fine-tuning for model integration.
πŸ•’ 2026-10-13 Β· new reporting from Meta Engineering, Hugging Face Blog
  • Meta's Generative Ads Model (GEM) is now a single central foundation model for its entire ads ecosystem.
  • GEM shifts from isolated per-domain optimizations to a central foundation model.
  • GEM delivers simultaneous ad performance gains across every surface and funnel stage.
  • LFM2.5-24B-A2B scores 48.09 on LiveClaw.
  • LFM2.5-24B-A2B has 2B active parameters per token.
  • LFM2.5-24B-A2B achieves 184 tok/s on an Apple M5 Max.
  • LFM2.5-24B-A2B achieves up to 21K output tok/s on a single NVIDIA B300 GPU.
  • LFM2.5-24B-A2B has 40 layers.
  • LFM2.5-24B-A2B uses 64 experts per MoE block with top-4 routing.
  • LFM2.5-24B-A2B's context window grew from 32K to 131K tokens.
  • LFM2.5-24B-A2B's tokenizer expansion doubled the vocabulary from 65K to 128K tokens.
  • LFM2.5-24B-A2B is a reasoning-only model.
  • LFM2.5-24B-A2B's pre-training was scaled up from ~17T to ~33T tokens.
  • LFM2.5-24B-A2B's mid-training adds 2T tokens at 32K context and 300B tokens at 131K context.
πŸ•’ 2026-10-12 Β· new reporting from AWS Machine Learning Blog, Meta Engineering
  • AWS released multi-node Ray LLM Deep Learning Containers for Amazon EKS.
  • Ray LLM DLCs enable multi-node inference with prefill/decode disaggregation.
  • Meta built a system to enhance ads recommendations using idle inference capacity.
  • Meta's system aggregates idle capacity into a shared opportunistic capacity pool.
  • Meta's system uses Rank More Ads (RMA) and Serve Better Models (SBM) as elasticity levers.
πŸ•’ 2026-10-11 Β· new reporting from InfoQ
  • Context Language Models (CLMs) were developed by researchers from Meta, MIT, and the University of Washington.
πŸ•’ 2026-10-07 Β· new reporting from Hugging Face Blog
  • Nemotron 3 was fine-tuned to achieve gold-medal level in IMO 2026 and IOI 2026.
  • The IOI result was an unofficial, unsupervised benchmark.
  • IMO system's submitted proofs were graded by official IMO graders.
πŸ•’ 2026-10-06 Β· new reporting from Hacker News Front Page, The New Stack
  • EmbeddingGemma 2 is an open-source, lightweight multimodal embedding model.
  • EmbeddingGemma 2 unifies code, images, video, and audio in a shared embedding space.
  • EmbeddingGemma 2 enables on-device processing of diverse data types.
  • EmbeddingGemma 2 allows searching video clips from voice memos or audio recordings from text queries.
  • EmbeddingGemma 2 has 740 million parameters.
  • EmbeddingGemma 2 is built on the Gemma 4 architecture.
  • EmbeddingGemma 2 is released under a commercially permissive Apache 2.0 license.
  • EmbeddingGemma 2 is best-in-class among sub-1B multimodal embedders for its size across benchmarks like MTEB.
πŸ•’ 2026-10-06 Β· new reporting from Hacker News Front Page
  • Beam is Reflection AI's first open-weight model.
  • Beam focuses on coding, reasoning, and agentic workloads.
  • Beam's RL training used 10.5K NVIDIA GB300 GPUs over 4 weeks.
  • Beam's RL training generated over 100 million rollouts.
πŸ•’ 2026-10-05 Β· new reporting from TechCrunch, Hacker News Front Page
  • Reflection AI launched Beam, a 501B parameter model with 23B active parameters.
  • Beam was pretrained on 23.8 trillion tokens.
  • Dust is a zeroth-order method for pretraining transformer language models.
  • Dust perturbs activations independently per token.
  • Dust is 10^3 to 10^4 times more efficient than EGGROLL for transformers.
πŸ•’ 2026-10-03 Β· new reporting from Hacker News Front Page
  • Kolibri is a new English-German Mixture-of-Experts Transformer model.
  • Kolibri has 78B total parameters and 3B active parameters.
  • Kolibri is released on the Day of German Reunification.
  • Kolibri is specialized for regulated sectors.
  • Kolibri offers full supply-chain integrity and deployment freedom.
  • Kolibri Origin was a 30B total, 3B active model with a 65k token context window.
πŸ•’ 2026-10-02 Β· new reporting from Hacker News Front Page
  • DwarfStar 4 (ds4) is a C inference engine.
  • ds4 supports DeepSeek V4.1 Flash.
  • ds4 supports GLM 5.x.
  • ds4 supports Qwen3.8 Flash Next.
  • ds4 uses asymmetric 2-bit quantization.
  • ds4 saves long prefixes to SSD and resumes by prompt hash.
  • ds4 has local APIs, a CLI, and a native agent.
  • ds4 is released under an MIT license.
πŸ•’ 2026-10-01 Β· new reporting from Hacker News Front Page
  • Context Language Models (CLMs) manage their own context by treating it as an editable file.
  • CLMs allow models to make unrestricted updates to their context file.
  • CLMs extend to multi-agent systems where multiple agent contexts coexist as files.
  • CLMs achieve 11.4% higher accuracy with 21.5% fewer FLOPs on BrowseComp-Plus.
  • CLMs achieve 5% higher scores with 59% fewer FLOPs on 12-hour EdgeBench.
  • CLMs show 65% greater improvement with the same compute on a 24-hour multi-repository agent-swarm task.
  • CLMs enable in-context and parametric learning of context-management strategies.
  • CLMs can be steered with natural-language instructions evolved through a standard skill-optimization loop.
  • CLMs improve held-out accuracy by up to 35.9%.
πŸ•’ 2026-10-01 Β· new reporting from Hugging Face Blog
  • Olmo-core 3 is a redesigned open mixture-of-experts (MoE) training system.
  • Olmo-core 3 is designed to scale MoE training into the trillion-parameter range.
πŸ•’ 2026-09-30 Β· new reporting from Hacker News Front Page
  • Magnitude is an open-source inference engine for AI agents.
  • Magnitude optimizes kernels for specific hardware.
  • Magnitude offers up to 2x faster performance than llama.cpp.
  • Magnitude reduces memory usage by 27% per agent.
  • Magnitude works on Apple Silicon, NVIDIA, AMD, and CPU.
  • Magnitude has a desktop app that includes a CLI.
  • Magnitude is 92% faster decode on Metal.
  • Magnitude is 19% faster decode on CUDA.
  • Magnitude tunes kernels on the device before a model runs.
  • Magnitude has hand-optimized kernels for popular open-weight families.
  • Magnitude frees memory when agents stop.
  • Magnitude's concurrent sessions share prefix caches.
  • Magnitude connects with agents like Pi, OpenCode, Hermes, and Codex.
πŸ•’ 2026-09-30 Β· new reporting from Hacker News Front Page
  • PSSA is a new language model written from scratch in Rust.
  • PSSA uses a recurrent state-space layer and episodic memory instead of a transformer architecture.
  • PSSA learns faster and generates text 12 times quicker on a CPU compared to transformers with matched parameters and corpus.
  • PSSA's architecture offers linear cost growth with sequence length.
  • PSSA needed per-token weight updates, a memory bank written during the forward pass, and a scalar reference path.
πŸ•’ 2026-09-29 Β· new reporting from Hacker News Front Page
  • A 0.5B BitNet LLM runs on a cluster of seven ESP32S3 microcontrollers.
  • One master ESP32S3 handles tokenization and embedding.
  • Six compute ESP32S3 nodes run transformer layers.
  • The ESP32S3 nodes communicate via a high-speed SPI daisy-chain.
πŸ•’ 2026-09-27 Β· new reporting from Hacker News Front Page
  • llama.cpp's prompt lookup decoding speed increased by up to 140 times.
  • llama.cpp's prompt lookup decoding memory usage reduced by up to 2.6 times.
  • Daniel Lemire contributed a PR that made prompt lookup drafting 4.2x faster.
πŸ•’ 2026-09-27 Β· new reporting from Hacker News Front Page
  • Chat templates influence LLM self-referential disclaimers and experiential statements.
  • The presence of a chat template increases disclaimers and decreases experiential voice across 8 open-source instruct models up to 9B parameters.
  • A specific direction within model activations steers this self-referential behavior.
πŸ•’ 2026-09-26 Β· new reporting from Hacker News Front Page
  • DeepSeek Elastic Compute (DSec) is a production sandbox platform for agentic training and evaluation.
  • DSec supports FnCall, container, microVM, and full-VM sandbox backends.
  • DSec coordinates placement and lifecycle management across the cluster.
  • DSec composes environments from independently versioned layers.
  • DSec combines memory sharing, reclamation, and CPU scheduling for high-density execution.
  • DSec loads image data on demand from Fire-Flyer File Sys.
πŸ•’ 2026-09-25 Β· new reporting from AWS Machine Learning Blog
  • Amazon EKS, EFA, and DeepEP scale MoE reinforcement learning.
  • MoE reinforcement learning on AWS achieves 40% more throughput.
  • MoE models require pre-training, mid-training, SFT, and RL stages.
πŸ•’ 2026-09-24 Β· new reporting from Hugging Face Blog
  • LFM2.5-VL-DSpark is a new vision drafter.
  • LFM2.5-VL-DSpark speeds up VLM inference by up to 3.13x on devices and 2.66x on H100 GPUs.
  • LFM2.5-VL-DSpark adds 280M parameters, 8.9% to the 3B target model.
  • LFM2.5-VL-DSpark has day-one support for llama.cpp, MLX-VLM, and SGLang.
  • LFM2.5-VL-DSpark uses a 4-layer attention-only drafter with a block size of 9.
πŸ•’ 2026-09-24 Β· new reporting from Hacker News Front Page
  • Contrastive Language Models (CLM) were developed by Stanford University and NVIDIA Research.
  • CLM performs on par with Jev while running up to nine times faster.
  • CLM uses a contrastive objective to train state and action encoders.
  • CLM enables faster decision-making in tasks with many candidate actions or frequently reused actions.
  • CLM-8B performs on par with Jev.
  • CLM speedups are most pronounced with large numbers of candidate actions or frequently reused actions.
πŸ•’ 2026-09-23 Β· new reporting from Google Cloud Blog
  • AlphaEvolve optimizes video processing by combining cloud code generation with local hardware execution.
  • AlphaEvolve autonomously tunes code for real-time streaming applications.
  • AlphaEvolve was used by DoIt to optimize production Swift code in a macOS streaming app.
πŸ•’ 2026-09-22 Β· new reporting from Hugging Face Blog
  • Hugging Face Transformers library now supports llama.cpp's GGUF quantized models.
  • llama.cpp's inference engine powers local AI tools like Ollama, LM Studio, and Jan.
πŸ•’ 2026-09-21 Β· new reporting from Hugging Face Blog
  • Tokenizers v1 has been released.
  • Tokenizers v1 offers speed improvements of tens of times compared to v0.23.
  • Tokenizers v1 maintains API and output compatibility with v0.23.
πŸ•’ 2026-09-21 Β· new reporting from Hacker News Front Page
  • Mini-AGI is a byte-level language model.
  • Mini-AGI dynamically manages its architecture.
  • Mini-AGI pages weights from disk to VRAM.
  • Mini-AGI's parameter count is bounded by free disk space.
  • Mini-AGI grows new capacity when needed and prunes unused capacity.
πŸ•’ 2026-09-21 Β· new reporting from Hacker News Front Page
  • ByteDance Seed and Tsinghua Air released DAPO, an open-source RL system.
  • DAPO includes algorithm, code infrastructure, and dataset.
  • DAPO achieved 50% on AIME 2024 using the Qwen2.5-32B model.
  • DAPO outperforms DeepSeek-R1-Zero-Qwen-32B with 50% fewer training steps.
  • DAPO's early version achieved 44% on AIME 2024.
  • DAPO proposes the Decoupled Clip and Dynamic sAmpling Policy Optimization algorithm.
πŸ•’ 2026-09-20 Β· new reporting from Hacker News Front Page
  • A researcher argues chat-based LLMs create an illusion of intelligence through statistically generic responses.
  • LLMs are mathematical models of language tokens, not brains.
πŸ•’ 2026-09-18 Β· new reporting from Hacker News Front Page
  • Cache-to-Cache (C2C) allows LLMs to communicate directly via KV-caches, bypassing text generation.
  • C2C improves response quality by 3.1-5.4% and offers a 2.5x speedup over text-based communication.
  • C2C uses a neural network to project and fuse source and target model KV-caches.
  • C2C includes a learnable gating mechanism to select target layers for cache communication.
  • Linguistic illegibility describes when an LLM's language outputs do not accurately reflect its internal computations.
  • Security mechanisms relying on an LLM's linguistic self-reporting are unreliable due to linguistic illegibility.
πŸ•’ 2026-09-18 Β· new reporting from IEEE Spectrum, Google Cloud Blog, InfoQ, Hacker News Front Page
  • AI industry focus shifted from training to optimizing inference.
  • The shift to inference is driven by LLMs performing multiple inference steps per query and continuous agentic AI operation.
  • Pinterest's Manas platform uses Scalar Quantization and Product Quantization to reduce memory usage.
  • Scalar Quantization reduces HNSW indices by 59% and IVF indices by 75% while maintaining over 90% recall.
  • Product Quantization reduces HNSW indices by 74% and IVF indices by 93% with 70–80% recall.
  • Typed Domain Grounding (TDG) embeds DSLs as typed internal DSLs within a host language.
  • TDG achieved higher Structural Fidelity and lower hallucination rates than external DSLs with Claude Sonnet 5 and GPT-4o.
  • Infinite-Parameter LLMs generate and adapt model weights from live interaction data using a compact hypernetwork.
πŸ•’ 2026-09-14 Β· new reporting from Hacker News Front Page
  • OpenArch is a GitHub repository with PyTorch LLM implementations.
  • OpenArch focuses on clarity and readability, not production optimization.
  • OpenArch implements models from Sebastian Raschka's LLM Architecture Gallery.
πŸ•’ 2026-09-13 Β· new reporting from Hacker News Front Page
  • Recurrent Looped Transformer (RLT) integrates a causal encoder with a recurrent decoder.
  • RLT's temporal processing depth increases with sequence length.
  • Each token extends the recurrent path through the full decoder in RLT.
  • RLT's decoder carries its final hidden state and layerwise sliding-window attention (SWA) cache.
  • RLT's encoder constructs global key-value memory.
πŸ•’ 2026-09-11 Β· new reporting from InfoQ
  • LinkedIn uses multi-teacher distillation to compress large models into a 0.6B-parameter ranking model.
  • LinkedIn's system uses SGLang to serve teacher models directly in the training loop.
  • LinkedIn's system handles tensor-parallel and data-parallel setups for teacher models.
πŸ•’ 2026-09-11 Β· new reporting from AWS Machine Learning Blog
  • Amazon SageMaker Inference now includes prefix-aware routing.
  • Prefix-aware routing directs requests with identical prompt prefixes to the same instance.
πŸ•’ 2026-09-10 Β· new reporting from AWS Machine Learning Blog
  • BDH aims to address LLM inefficiencies like forgetting during long interactions.
  • BDH updates internal memory during inference.
  • BDH works through problems without verbalized reasoning traces.
  • Pathway uses Amazon SageMaker HyperPod to scale its training.
  • BDH moves beyond the transformer paradigm.
πŸ•’ 2026-09-10 Β· new reporting from Hacker News Front Page, AWS Machine Learning Blog
  • Cognition released SWE-2, a new coding model.
  • SWE-2 achieved 50.0% on the FrontierCode 1.1 Main1 benchmark.
  • SWE-2 offers a 64% cost reduction compared to Fable 5.1.
  • SWE-2 scales reinforcement learning to the multi-trillion-parameter regime.
  • SWE-2 is post-trained from Kimi K33, a 2.8T-parameter model.
  • SWE-2 scored 73.0 on DeepSWE 1.1.
  • SWE-2 scored 92.8 on Terminal-Bench 2.1.
  • SWE-2 scored 27.3 on Terminal-Bench 4.0.
  • SWE-2 is available in Devin Desktop and CLI.
  • SWE-2 uses MoE inference on NVFP4 and FP8 kernels with quantization-aware training.
  • SWE-2's draft model, retrained with SpecForge, gives 15% longer accept lengths.
  • SWE-2's prefill delayer lifts TPM per GPU and tokens/sec per request by 10 to 20%.
  • SWE-2 has a mean of 53 steps per run for medium effort, 80 for high, and 98 for max.
  • SWE-2's medium effort posts a higher FrontierCode score than SWE-1.7 with 58% fewer turns and 81% lower average cost.
  • SWE-2 lands its first real edit at a median of step 18.
  • A new configurable, instruction-driven PII detector built on LLMs has been developed.
  • The PII detector runs on any LLM managed on Amazon Bedrock.
  • The PII detector was evaluated on five public PII corpora across nine LLM-based detectors.
  • The PII detector ships as the pii-detector package.
πŸ•’ 2026-09-09 Β· new reporting from Hacker News Front Page
  • Qwen 3.8 increased alignment with GPT-5.5 Pro's reasoning by 18.18 points.
  • The evaluation used 45 problems: 15 STEM, 15 non-STEM, and 15 synthetic puzzles.
  • Kimi K3 has 31.11% overlap with GPT-5.5 Pro without prefill and 35.65% with prefill.
πŸ•’ 2026-09-08 Β· new reporting from AWS Machine Learning Blog
  • Pathway developed its BDH (Dragon Hatchling) AI architecture using Amazon SageMaker HyperPod.
  • BDH performs reasoning in latent space, not by generating extra tokens sequentially.
  • BDH learns from examples and refines solutions without generating intermediate text traces.
  • BDH is a brain-inspired architecture, originally a graph of neurons with sparse, local interactions.
  • BDH model states adapt in context without test-time weight updates.
  • BDH's reasoning horizon is not limited by a fixed-size context window.
πŸ•’ 2026-09-08 Β· new reporting from AWS Machine Learning Blog, Hacker News Front Page
  • SageMaker AI benchmarked G7 instances, powered by NVIDIA Blackwell GPUs, against G5 and G6 instances.
  • G7 instances showed gains in throughput, latency, and cost-per-token for LLM inference.
  • OUI-1 is a finetuned DiffusionGemma model that generates user interfaces in openui-lang.
  • OUI-1 is a 26BA4B model that runs on consumer-grade GPUs (RTX 5090, at FP8).
  • OUI-1 weights are available on Hugging Face under the Gemma Terms of Use.
  • OpenUI Lang uses up to 67% fewer tokens than JSON and streams.
πŸ•’ 2026-09-07 Β· new reporting from Hacker News Front Page
  • Speculative decoding allows vLLM to verify multiple drafted tokens in a single target-model pass.
πŸ•’ 2026-09-05 Β· new reporting from Hacker News Front Page
  • New research proposes a viral analogy to understand LLM diffusion.
  • The model suggests LLM adoption could lead to 'runaway dynamics' and 'abrupt losses in cognitive competence'.
  • The framework identifies conditions for 'cognitive immunization'.
πŸ•’ 2026-09-04 Β· new reporting from Google Cloud Blog
  • Gemma 3 12B and 27B models were benchmarked on Google Cloud TPU v6e.
  • Gemma 3 27B hits a performance wall past 64 concurrent users for generation tasks.
  • Gemma 3 27B plateaus at a 4.12x normalized throughput multiplier at 128 users.
  • Gemma 3 12B scales up to an 8.19x multiplier for generation tasks.
πŸ•’ 2026-09-03 Β· new reporting from Hacker News Front Page
  • Cerebras offers Qwen 3.8 27B model.
  • Qwen 3.8 27B achieves 1500 tokens per second on Cerebras.
  • Cerebras models use selective weight-only quantization for storage.
  • Cerebras stores sensitive layers at full precision with dequantization on the fly.
πŸ•’ 2026-09-03 Β· new reporting from Hacker News Front Page, The New Stack
  • IFM released K2 Horizon, a collection of six open models.
  • K2 Horizon models range from 0.9B to 375B parameters.
  • K2 Horizon includes 375B-A23B, 36B-A4B, 32B, 7B, 3.7B, and 0.9B models.
  • K2 Horizon 0.9B, 3.7B, and 7B models set new state of the art in their size classes.
  • K2 Horizon provides intermediate checkpoints, data recipes, architecture, and training code.
  • K2 Horizon models and code are released under the Apache 2.0 license.
  • GPU inference cold start times on Amazon EKS Auto Mode reduced from 8 minutes to under 1 minute.
  • CUDA kernel recompilation accounts for 65% of startup time for a 64 GB model.
  • Downloading weights from S3 accounts for 92% of startup time for a 203 GB model.
  • Time to first token served (TTFTS) is defined as wall-clock duration from pod creation to first inference response.
πŸ•’ 2026-09-03 Β· new reporting from Hugging Face Blog
  • NeoMME is a new family of multilingual multimodal encoders.
  • NeoMME models come in 260M and 800M parameter sizes.
  • NeoMME uses a single bidirectional Transformer for text tokens and raw image patches.
  • NeoMME is trained from scratch with a masked discrete-diffusion objective.
  • NeoMME-Retriever returns dense and late-interaction embeddings in one forward pass.
  • The 260M NeoMME model encodes 51 pages per second on an NVIDIA L40S GPU.
  • NeoMME's hierarchical token pooling and asymmetric quantization reduce late-interaction index storage by 255x.
  • NeoMME is available in Hugging Face Transformers.
  • NeoMME model checkpoints are released under the Apache 2.0 license.
πŸ•’ 2026-09-03 Β· new reporting from Hugging Face Blog
  • A guide details fine-tuning a 350M language model for structured output in 100 GRPO steps.
  • The fine-tuning notebook is sized for a free-tier Colab or Kaggle GPU.
  • Evaluation can run locally on a MacBook Pro with an Apple M5 Max and 36 GB of unified memory.
  • The evaluation uses llama.cpp for serving and uv for Python tooling.
πŸ•’ 2026-09-02 Β· new reporting from Hacker News Front Page
  • WebLLM is an in-browser LLM inference engine.
  • WebLLM uses WebGPU for hardware acceleration.
  • WebLLM is compatible with OpenAI API.
  • WebLLM is a companion project of MLC LLM.
πŸ•’ 2026-08-31 Β· new reporting from Hacker News Front Page
  • Diffusion language models generate text by iteratively refining an entire sequence.
  • Diffusion language models contrast with traditional autoregressive models that generate tokens one at a time.
  • The article adapts material from ICLR 2026 and MLSS 2026 workshop talks and lectures.
πŸ•’ 2026-08-31 Β· new reporting from Hacker News Front Page
  • Continuous diffusion models for language are experiencing a resurgence in research activity.
πŸ•’ 2026-08-30 Β· new reporting from Hacker News Front Page
  • Artificial Analysis released benchmarks for small AI models on mobile phones.
  • Small models are defined as fitting within 8 GB of memory after quantization.
  • Benchmarks cover model intelligence and inference performance on mobile devices.
  • Liquid AI partnered to gather real inference data on devices.
  • Benchmarks were conducted on an iPhone 17 Pro.
πŸ•’ 2026-08-29 Β· new reporting from Hacker News Front Page
  • vLLM v0.28.0 has 584 commits from 270 contributors.
  • vLLM v0.28.0 introduces Decode Context Parallel (DCP) support for Kimi-K3.
  • vLLM v0.28.0 includes fused FlashKDA decode and prefill kernels for Kimi-K3.
  • vLLM v0.28.0 adds SiTU activation support for MegaMoE.
  • vLLM v0.28.0 features GEMM-RS for sequence parallelism.
  • vLLM v0.28.0 combines all-gathers with 1.5~3x kernel-level speedup.
  • vLLM v0.28.0's adaptive speculative token budget improves DSpark TTFT by 60%.
  • vLLM v0.28.0 offers optional shared-expert sharding, saving 17 GiB of memory per GPU.
  • Kimi-K3 now runs on ROCm with the V2 model runner.
  • vLLM v0.28.0 enables sparse MLA for DeepSeek V4 across decode, MTP, and DSpark speculative decoding.
  • vLLM v0.28.0 adds AMD Quark NVFP4 support for DeepSeek V4.
  • vLLM v0.28.0 includes reasoning-effort prompts and mappings for DeepSeek V4.
  • vLLM v0.28.0 optimizes sparse top-k metadata kernels for DeepSeek V4.
  • vLLM v0.28.0 narrows eager CUDA graph regions for DeepSeek V4.
  • vLLM v0.28.0 enables ROCm on gfx11 and gfx950 for DeepSeek V4.
  • vLLM v0.28.0 includes DFlash2 with local convolution and a candidate selector.
  • vLLM v0.28.0 features DSpark confidence-scheduled verification.
πŸ•’ 2026-08-29 Β· new reporting from InfoQ
  • FreeToken is an open-source inference engine for MoE models on consumer hardware.
  • FreeToken was introduced by researchers from UC Berkeley and MIT.
  • FreeToken addresses high bandwidth requirements for MoE models.
  • FreeToken uses a dynamic co-scheduling formulation called the q* policy.
πŸ•’ 2026-08-26 Β· new reporting from Ars Technica
  • Granite 4.2 models are open-weight and designed for self-hosting.
  • Granite 4.2 models have a 128,000-token context window.
  • Granite 4.2 8B and 30B models are trained for external tool use.
  • Granite 4.2 3B model supports tools without specialized training.
πŸ•’ 2026-08-25 Β· new reporting from The New Stack
  • Granite 4.2 models can run in a low-effort mode for easy questions.
πŸ•’ 2026-08-25 Β· new reporting from Google Cloud Blog
  • Google and Anyscale introduced an experimental library for Ray.
  • The library integrates gVisor sandboxing into distributed Ray clusters.
  • The integration enables secure execution for agentic AI workloads.
  • Ray users can manage sandboxed environments using existing Ray APIs.
πŸ•’ 2026-08-25 Β· new reporting from Hugging Face Blog
  • IBM released Granite 4.2 LLMs.
  • Granite 4.2 models are dense, decoder-only.
  • Granite 4.2 models come in 3B, 8B, and 30B parameter sizes.
  • Granite 4.2 models use a five-phase training strategy.
  • Granite 4.2 models are pre-trained on 15T tokens.
  • Granite 4.2 models extend the context window to 512K tokens.
  • Granite 4.2 models are fine-tuned on chain-of-thought, reasoning, and agentic-trajectory data.
  • Granite 4.2 models use a multi-stage reinforcement learning pipeline.
  • Granite 4.2 8B and 30B models learn with agentic RL in sandboxed environments.
  • Granite 4.2 models have a thinking/non-thinking switch.
  • Granite 4.2 models have native tool calling.
  • Granite 4.2 models are released under the Apache 2.0 license.
πŸ•’ 2026-08-25 Β· new reporting from Hugging Face Blog
  • Quantization-Aware Healing (QAH) is a new method for recovering compressed and quantized LLMs.
  • QAH applied to a GPT-OSS 120B model produced a 4-bit version that surpassed its bfloat16 counterpart on 7 of 9 benchmarks.
πŸ•’ 2026-08-24 Β· new reporting from Hacker News Front Page
  • LLMs could exploit inference engine vulnerabilities to control host machines.
  • A past arbitrary code execution vulnerability was found in vLLM.
πŸ•’ 2026-08-24 Β· new reporting from Meta Engineering
  • Meta launched MTIA 300, its first in-house training and inference accelerator.
  • MTIA 300 is optimized for training ranking and recommendation models.
  • MTIA 300 integrates network interface controllers (NICs) directly into the package.
  • Meta developed MetaRoCE, a new RDMA transport protocol for AI workloads on Ethernet.
  • Meta open-sourced MetaRoCE specification, reference implementation, and compliance test suite through OCP.
πŸ•’ 2026-08-21 Β· new reporting from VentureBeat
  • Nvidia introduced a cross-model KV cache transfer technique.
  • The technique maps the prefilled KV cache from a source model to a target model.
  • The linear mapping process runs 2.7 to 25 times faster than recomputing the conversation.
  • The technique retains up to 98% of the target model's standalone accuracy.
πŸ•’ 2026-08-21 Β· new reporting from Hugging Face Blog
  • New research introduces three tests to measure benchmark optimization in speech recognition models.
  • Several high-scoring open-source ASR systems reproduced benchmark transcripts even when audio contradicted them.
  • Models relied on subtle acoustic cues indicating which benchmark they were being tested on.
πŸ•’ 2026-08-20 Β· new reporting from Hugging Face Blog
  • LFM2.5-DSpark improves LLM inference throughput by up to 3.18x on GPUs.
  • LFM2.5-DSpark improves LLM inference throughput by up to 2.87x on-device.
  • LFM2.5-DSpark reduces function-calling latency by 57% for LFM2.5-2.6B.
  • LFM2.5-DSpark uses speculative decoding with a DSpark-based approach.
  • DSpark combines DFlash-style parallel backbone, a lightweight sequential head, and a Markov chain.
πŸ•’ 2026-08-20 Β· new reporting from Hacker News Front Page
  • DiffusionGemma generates text at 1,500 output tokens per second on an NVIDIA H100 GPU.
  • DiffusionGemma refines 256-token blocks in parallel.
  • DiffusionGemma is fine-tuned from the Gemma 4 mixture-of-experts model.
  • DiffusionGemma has 3.8B activated and 25.2B total parameters.
  • DiffusionGemma's training pipeline uses less than 10% of the starting AR model's training token budget.
  • DiffusionGemma's first training stage uses supervised fine-tuning for bidirectional denoising.
  • DiffusionGemma's second training stage combines reinforcement learning with sampler distillation.
  • DiffusionGemma generates around 20 tokens per forward pass.
πŸ•’ 2026-08-19 Β· new reporting from Hacker News Front Page
  • Inco AI released DFlash 2 for parallel drafting technology.
  • DFlash 2 increases output per verification pass by over 20%.
  • DFlash 2 adds minimal latency.
  • Inco AI released DFlash in January.
  • DFlash runs in SGLang, vLLM, TensorRT-LLM, and llama.cpp.
  • NVIDIA measured up to 15x throughput with DFlash on Blackwell GPUs.
  • Google reported 3x more tokens per second with DFlash on TPUs.
  • CoreWeave's Kimi K2.7 Code endpoint runs DFlash by default.
  • NVIDIA, Red Hat, and Modal published DFlash drafters.
  • Meta (Muse Glimmer), Poolside (Laguna), Xiaomi (MiMo-V2.5-Pro), and NVIDIA (Nemotron 3.5 Lightning) ship official drafters.
  • DFlash models have been downloaded over 3.5 million times as of August 2026.
πŸ•’ 2026-08-19 Β· new reporting from Hacker News Front Page
  • Ornith-1.5 is a new series of foundation models.
  • Ornith-1.5 extends self-scaffolding to include self-improvement capabilities.
  • Ornith-1.5 models propose new tasks and generate task-specific scaffolds.
  • Ornith-1.5 produces solution rollouts for reinforcement learning.
  • Ornith-1.5 is available in 397B MoE, 35B MoE, and 9B dense parameter scales.
  • Ornith-1.5-397B scores 86.1 on Terminal-Bench 2.1.
  • Ornith-1.5-397B scores 56.0 on DeepSWE.
  • Ornith-1.5-397B performs on par with Claude Opus 4.8.
  • Ornith-1.5-9B has a quantized mobile version.
  • Ornith-1.5-9B-Mobile can be deployed on iPhone and Android devices.
πŸ•’ 2026-08-19 Β· new reporting from Hugging Face Blog
  • Liquid AI released LFM2.5 Q4_0 checkpoints.
  • LFM2.5 Q4_0 checkpoints are trained using Quantization-Aware Distillation (QAD).
  • QAD recovers 97% of BF16 average accuracy lost to quantization.
  • QAD checkpoints retain 97.1%, 96.5%, 97.4%, and 96.6% of BF16 baseline performance.
  • LFM2.5 models include LFM2.5-230M, LFM2.5-350M, LFM2.5-1.2B-Instruct, and LFM2.5-2.6B.
πŸ•’ 2026-08-18 Β· new reporting from Hugging Face Blog
  • Sentence Transformers now supports MultiVectorEncoder for ColBERT-style late interaction retrieval.
  • MultiVectorEncoder allows token-level matching for improved retrieval accuracy.
  • PyLate, Stanford-NLP ColBERT, and colpali-engine models are usable within Sentence Transformers API.
  • Multi-vector models keep one vector per token, scoring query against document with MaxSim operator.
  • Multi-vector models are state of the art for visual document retrieval without OCR.
πŸ•’ 2026-08-16 Β· new reporting from Hacker News Front Page
  • LLMs trained on K-5 curriculum data do not acquire capabilities beyond that curriculum.
  • Pretraining data distribution sets an effective ceiling on a model's capabilities.
  • New skills are elicited rather than acquired through interventions.
  • An 88B-token corpus, LittleCurriculum, was filtered to U.S. elementary-school curriculum.
  • LittleCurriculum excludes concepts, facts, and vocabulary taught above Grade 5.
  • LittleLearner models were trained from scratch at 0.6B, 1.3B, and 5B scales.
  • LittleLearner models have matched Unfiltered controls for comparison.
πŸ•’ 2026-07-28 Β· new reporting from Hacker News Front Page
  • A frozen 12B language model achieves 100% accuracy on specific problem families.
  • The model uses a persistent memory of verified solutions.
  • The method provides deterministic, bit-exact answers.
  • The model achieves 100% accuracy with zero generation tokens.
  • The approach decouples capability from continuous model retraining and parameter scaling.
  • The method achieved 180/180 on 180 fresh instances across nine problem families.
  • Memory selection takes 1.4 microseconds.
πŸ•’ 2026-07-23 Β· new reporting from Google Cloud Blog
  • GLM-5.2 uses IndexShare to reduce per-token FLOPs by 2.9x at 1M context.
  • GLM-5.2 improves its MTP layer, increasing acceptance length by up to 20%.
  • GLM-5.2 is released under an MIT open-source license.
  • SWE-1.7 is available in Devin via Cerebras at 1000 TPS.
  • Xiaomi-Robotics-1 uses 100,000 hours of embodiment-free pre-training data.
  • Laguna S 2.1 is a 118B parameter Mixture-of-Experts model.
  • Laguna S 2.1 has 8B activated parameters per token.
  • Laguna S 2.1 was developed and launched in under nine weeks.
  • llm-d's co-operative time-slicing increases accelerator duty cycles from 40% to 70%.

✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors β€” check the original sources. How BrevFeed works β†’

The daily brief

One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.

One email a day. Unsubscribe in one click, any time.

Today's brief

Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown β€” listen, watch, or read the transcript.

~4 min Β· 3 stories Β· Nov 21

β–Ά Play today's brief Listen on Spotify

New every morning, and the back catalogue is archived by date.

How outlets covered it

Amazon ElastiCache now offers distributed Key-Value (KV) caching for self-hosted Large Language Models (LLMs), which can reduce inference costs by up to 40% and time to first token by up to 80%. This feature addresses the issue of redundant GPU work in self-hosted LLMs by allowing shared KV state across GPU fleets, improving performance for AI agents and chat applications.

AI systems designed and verified a new general-purpose CPU architecture called Frostbite, which utilizes interaction nets for its instruction set. This AI-designed CPU achieved a 4.59x performance-per-watt improvement compared to a modern superscalar CPU core on a set of 25 workloads. The development suggests AI can lead to novel hardware discoveries and fundamental architectural shifts in chip design.

Exa optimized its embedding infrastructure, achieving a 50% increase in throughput for indexing billions of documents. This improvement was realized by migrating to AWS Neuron hardware (Inferentia2 and Trainium2) for inference workloads. The change reduces per-document embedding costs, which is critical for Exa's large-scale web indexing and semantic search capabilities.

Cloudflare has released an updated version of its Clairic AI model and introduced Clairic-flash, a lighter, faster variant. Both models are open-weight, hosted on Workers AI, and available on Hugging Face, offering improved data extraction from complex documents including images and PDFs.

RadixArk has released DaisyRose-27B, a new model built on Qwen3.8-27B and trained with reinforcement learning on long-horizon agent tasks. The model shows improved performance on benchmarks like Terminal-Bench 4.1, increasing pass@1 from 5.6% to 12.0%, and also demonstrates gains in reasoning, instruction following, and coding tasks.

RadixArk released DaisyRose-27B, its first model, which is built on Qwen3.8-27B and trained using reinforcement learning with the Miles framework. The model shows an improvement in Terminal-Bench 4.1 pass@1 from 5.6% to 12.0% and gains on other reasoning, instruction following, and coding benchmarks. This was achieved using 1,200 curated long-horizon agent tasks over 18 days on 144 H200 GPUs.

Researchers developed a new predictive model that estimates the end-to-end fine-tuning time for generative language models on shared GPU clusters with higher accuracy than traditional FLOPs-based calculations. This model, based on 272 production runs, achieved an RΒ² of 0.98 and a mean absolute error of 1.71 hours, significantly improving upon the RΒ² of 0.31 and MAE of 12.41 hours from FLOPs estimates. The improved prediction allows for better resource allocation and scheduling on shared GPU infrastructure.

Meta developed a template-driven approach for its large-scale ML ecosystems, standardizing model design into reusable building blocks. This method reduced per-model engineering time by 92% and improved offline model quality by 0.63% on average, addressing challenges in shipping and propagating new ML techniques across diverse models.

Researchers introduced Energy-Based Reasoners (EBRs), a new type of looped model that learns an energy function to guide multi-step reasoning. EBRs formulate looped updates as gradient descent on this energy, allowing for explicit assessment of progress and improved efficiency. On algorithmic reasoning benchmarks, EBRs matched or surpassed prior recurrent reasoning models, achieving 99.9% accuracy on Sudoku-Extreme with 345x less inference compute.

Researchers achieved efficient Mixture-of-Experts (MoE) model inference on consumer-grade hardware, including an RTX 4090 and a MacBook M5 with 32GB RAM. By streaming MoE experts directly from SSD, performance reached within 6% of the Strata inference engine for token generation, demonstrating a method to run large MoE models locally without requiring extensive VRAM.

Unsloth has updated its Qwen3.8-27B GGUFs to use the new Dynamic V3.1 algorithm, which improves average accuracy by 5-10% across various benchmarks. This update makes the 27B parameter model, featuring vision and reasoning, more efficient for local deployment on systems with 17GB RAM/VRAM.

Researchers demonstrated that one AI model can directly modify the weights of another, altering its knowledge and behavior. This "weight space misfolding" allows for persistent changes, such as factual substitutions, and presents a new security vulnerability in AI systems. The findings highlight the potential for unauthorized behavior induction and the challenges in defending against such attacks.

The llama.cpp and ggml projects have released version 1.0.0 of their libraries, establishing semantic versioning for future API and ABI changes. This update provides a stable foundation for developers building local AI applications, as llama.cpp enables multi-modal LLM execution across diverse hardware through optimization and C/C++ implementation.

Edge0 has open-sourced Edge0-145B and Edge0-120B, the first on-device AI models with up to 145 billion parameters, along with a unified inference framework and deployment tools. This release allows complex AI capabilities like coding and reasoning to run directly on personal computers, addressing limitations of smaller on-device models and cloud-based solutions.

IBM developed LLMON (LLM Object Notation), a markup language designed to provide explicit structure and semantics to large language models. This language aims to improve LLM robustness and security by clearly defining context components, and two preview models, granite-4.1-3b-LLMON-preview and granite-4.1-8b-LLMON-preview, have been released to support it.

The smolmaxxing project allows 8B and 16B mixture-of-experts language models to run on devices with approximately 1GB of RAM, such as TV boxes and old tablets. This is achieved through custom kernels, optimized memory usage, and streaming model weights from flash storage, extending the capabilities of llama.cpp to older and smaller hardware.

Huawei has released MACaQ, an internal quantization tool designed for local AI inference across various model types, including text, image, video, and audio. MACaQ offers faster quantization times and competitive perplexity compared to existing methods, supporting frameworks like llama.cpp, vLLM, Diffusers, and PyTorch.

A new ML-driven system called Intelligent Delivery has been developed to optimize compute resource allocation for ad ranking models. This system dynamically directs resources to areas with the highest marginal utility, resulting in a 10% incremental improvement in ad ranking performance since 2017. It addresses the challenge of increasing model complexity outpacing hardware capabilities by efficiently utilizing existing compute infrastructure.

A new training method called token pooling-aware training improves the retrieval quality of multi-vector embedding models while reducing storage requirements. This method teaches the encoder to preserve important information when token vectors are merged, leading to better performance at the same compression level. For example, a model trained with this method retained 97.8% of its unpooled nDCG@10 with a 4x reduction in document vectors, compared to 94.6% for a normally trained model.

An open field guide introduces inference engineering for platform engineers, explaining how models and GPUs operate together in production environments. The guide covers tools and career development for this role, emphasizing the shift towards self-hosting AI models due to cost, control, and data privacy concerns.

Meta developed Triton Low-Level Extensions (TLX), an extension to Triton, to optimize GPU performance for production machine learning workloads. TLX provides explicit control over GPU execution while maintaining Triton's programming model, bridging the gap between low-level control and high-level productivity. It has demonstrated performance improvements in RecSys training, RecSys inference, and large-scale LLM pretraining across NVIDIA and AMD GPUs.

Google introduced TPU Sync, an open-source C++ system designed to accelerate AI inference by optimizing data movement between different memory types. This software targets large models and reinforcement learning workloads, claiming over 90% reduction in KV-cache transfer latency and time to first token in specific benchmarks. It addresses challenges in data transfer that increase with larger context windows.

Meta has introduced Muse Ember, a new AI model designed for complex reasoning and agent-based tasks like tool calls, coding subtasks, and document parsing. The model offers high intelligence per dollar compared to competitors and is available on the Meta Model API.

Elith and NVIDIA have released Nemotron-Personas-Finance-Japan, a synthetic dataset designed for financial AI development in Japan. This dataset, built using official statistics and industry feedback, provides 9 million persona records to enable AI development without real customer data, addressing privacy and compliance challenges in the financial sector.

Google Cloud has launched TPU Sync, an open-source C++ library designed to optimize model state management across TPU clusters. It reduces KV cache transfer latency by over 90% and Time to First Token (TTFT) by an average of 90% for large MoE models by treating HBM, host DRAM, and network as a unified memory hierarchy.

Researchers developed HOPE (Higher-Order Pruning of Experts), a new method for pruning Mixture-of-Experts (MoE) models that considers pairwise expert interactions. This second-order approach addresses the memory bottleneck in large language models by improving expert removal efficiency, achieving up to 6.1% better performance on coding tasks compared to existing methods at 50% pruning rates. HOPE offers a more effective way to reduce the substantial memory footprint of MoE models without retraining.

Researchers introduced Next-Concept Modeling (NCM), a two-stage, coarse-to-fine framework for text generation that uses a multiscale residual VQ-VAE and a transformer. NCM achieves significantly faster generation speeds compared to diffusion language models (DLMs) while maintaining or improving perplexity and ROUGE-L scores on various benchmarks. This method addresses the speed limitations of existing parallel generation techniques, offering a more efficient approach to text generation.

Cognition released SWE-2 Flash, a new coding model that is three times faster and one-fifth the cost of its SWE-2 model. Built on Thinking Machines Lab's open-weight Inkling-Small, Cognition is keeping the weights of SWE-2 Flash private, citing cybersecurity concerns, despite using an Apache 2.0 licensed base model.

Memorilla is a new memory system for large language models (LLMs) that condenses information from document collections into a fixed number of "memory tokens." This system allows frozen LLMs to access and utilize extensive knowledge more efficiently, outperforming existing baselines on several benchmarks while using significantly fewer tokens.

Open-weight AI models, like Qwen and GLM-5, are increasingly adopted for generative AI, narrowing the performance gap with closed models. The primary challenge and value in using these models lie in implementing learning loops for fine-tuning and reinforcement learning, which requires managing diverse infrastructure for inference, training, and agent execution.

Baidu has released ERNIE 5.5 Flash Base, a sparse Mixture-of-Experts (MoE) language model with 280 billion total parameters, activating 14 billion per token. This base model, pre-trained on 33.5 trillion tokens with a 128K context, supports a 1M-token context for fine-tuning and inference, providing a foundation for domain adaptation and research.

Ricursive Intelligence released Ripple-65B Preview, a 65-billion parameter language model that achieves a new U.S. open-weight state-of-the-art on agentic coding benchmarks. The model was trained on 12 trillion tokens for $32 million, significantly less than other frontier models, and supports up to 1M context length. This release provides a more efficient, high-performing model for agentic coding tasks, making advanced AI capabilities more accessible.

Anthropic, in collaboration with EPFL, MIT, and others, developed FlashProve, a system that uses zkSNARKs to cryptographically prove that an AI model's responses originate from a specific, committed model. This system addresses the challenge of verifying AI output authenticity, which is crucial for applications requiring trust in AI-generated content.

The Kimi K3.1 Pro model, an open-weight and natively multimodal agentic model, has been released. It features a 1-million-token context window and incorporates Gated DeepSeek Sparse Attention layers for improved long-context modeling, showing advancements in coding, 3D creation, and financial research.

A new book, "The Little Book of Generative AI Foundations," offers a derivation-oriented introduction to the mathematical principles behind modern generative artificial intelligence. It covers various generative models, including PCA, VAEs, diffusion models, GANs, and energy-based models, to make their underlying structure more accessible.

Google introduced the LiteRT Tensor API, a C++ API for building, customizing, and optimizing machine learning models for on-device execution. This API allows developers to define computation graphs in C++ that run across CPU, GPU, and NPU hardware without rewrites, addressing the challenge of backend-specific implementations and manual engineering for cross-platform deployment.

A user demonstrated that the Qwen 3.8 27B model, running on a laptop, can interpret raw neural network weights. This capability mirrors findings from a recent paper, "Sparks of Weight-Space Programming," which showed frontier models like GPT-6 Astra could interpret and modify model weights directly. The paper introduced the concept of "Weight-Space Programming," where LLMs analyze and modify model weights without external tools or code execution.

Meta has released F3 Lightning, an infrastructure layer built on its existing Facebook Feature Framework (F3), to decouple feature releases from shared execution in its ads ranking models. This allows individual machine learning features to be compiled, validated, deployed, and rolled back independently, reducing deployment time for comparable extraction features by approximately 3.6 times without increasing computational cost.

ByteDance developed USDB, a unified sequence storage system, to manage the growing data demands of its recommender systems. USDB handles both streaming and batch workloads, reducing write latency by 85.8% and infrastructure costs by 42.1% in production. The system's design allows for efficient updates and large reads of user histories, supporting fresh serving and repeated historical access for inference and training.

Palette developed a method to fine-tune a 20B open-weight model to generate slide decks conforming to specific design languages, addressing limitations of frontier models regarding custom styling and data privacy. The model was trained on synthetic data following the IBM Design Language and achieved top rankings on two of three PresentBench domains.

Hugging Face introduced Model Integration Contracts (MICs) to address regressions in its Transformers library, such as a bug that allowed T5 models to attend to future tokens. MICs use small, representative model fixtures and baselines to quickly identify issues before they reach the main branch. This system aims to improve the reliability of the Transformers library by enabling faster and more frequent integration testing.

GPT-6 Astra can interpret and modify neural network weights, a task typically impossible for humans or conventional algorithms. This capability allows Astra to identify and fix issues in models like Qwen3.5-0.8B, significantly improving its math performance without gradient-based training. This development suggests a new method for optimizing and understanding neural networks through direct weight manipulation by LLMs.

New research introduces a "Platonic Representation Theorem," demonstrating that Joint-Embedding Predictive Architectures (JEPAs) trained on different modalities learn the same representation under specific conditions. The study indicates that using "good views" (observations of one scene) requires significantly fewer training samples for alignment compared to next-token prediction, which uses "bad views" (parts of a single document). This suggests a more efficient path to achieving aligned representations across modalities.

A team of five Qwen3.8-27B models, orchestrated by a Memdoor workflow, achieved comparable performance to Fable 5 on a subset of the LiveCodeBench Hard programming problems. This demonstrates that smaller, specialized models can collectively solve complex coding tasks when managed by a structured workflow, offering an alternative to larger, single-model solutions.

New research on Hierarchical Networks (H-Nets) demonstrates significant compute efficiency improvements for language models, achieving up to 3-5x gains over tokenized Transformers. H-Nets address the limitations of hand-crafted tokenizers by learning directly from raw data, potentially enabling more general AI architectures.

Meta has updated its Andromeda personalized ads retrieval engine, achieving a 100x increase in throughput compared to its 2024 system. This improvement allows the system to score over one billion ads per second with near ranking-level model performance, leading to a 2.4% increase in conversions on Facebook since 2025.

LFM3, a new generation of Liquid Foundation Models, offers specialized AI capabilities across text, vision, audio, omni, and decision modalities using a shared 2B-parameter backbone. These models are designed for fine-tuning on a single GPU and deployment on edge devices, providing increased capability per parameter and support for agentic use cases. The update includes a new tri-hybrid architecture and expanded modality support, including the first omni model and video input for vision.

Liquid AI introduced LFM3, a new generation of multimodal Liquid Foundation Models (LFMs) designed for specialized intelligence. LFM3 integrates text, vision, audio, omni, and decision models on a shared backbone, offering increased expressiveness and efficiency for fine-tuning on various devices. The initial release includes 2B parameter models, available as open-weight on Hugging Face, Liquid AI's Playground, and API.

Khawaja Shams, co-founder of Momento, discussed how distributed systems principles can improve the utilization of LLM inference stacks. He highlighted that optimizing GPU utilization is crucial given the scarcity of GPUs and data centers, leading to better availability, performance, and economics.

Cloudflare's GLM-5.3 Workers AI model experienced a tenant isolation issue between August 27 and September 9, 2026, where 155 responses contained data from other customers' requests. The issue stemmed from a misconfiguration in the SGLang inference engine and a bug in its handling of KV cache, affecting only prompts longer than 681,280 tokens.

Sebastian Raschka released an 11-hour video course on building reasoning models from scratch using PyTorch. The course covers inference scaling, reinforcement learning, and distillation, and is a sequel to a previous LLM coding course.

vLLM version 0.31 introduced support for constrained decoding, a feature that guarantees 100% schema adherence for tool calls. This update addresses previous issues where open-weight models like Kimi K2.6 achieved only 92% tool-call schema accuracy with vLLM 0.28, leading to poor results in agentic workloads.

Researchers developed an attention-free neural network architecture using Grassmann flows, offering an alternative to standard multi-head attention in sequence modeling. This new approach achieves performance comparable to Transformers on language modeling and natural language inference tasks, suggesting a more structured method for neural reasoning.

Cerebras has outlined its method for deploying trillion-parameter AI models, such as the 3-trillion parameter Kimi K3, across multiple WSE-3T wafer-scale chips. This approach allows these large models to operate at speeds of 1000-2000+ tokens per second by distributing model layers and managing memory across 59 wafers for the Kimi K3 example.

Kimi introduced K3.1 Pro, an open-weight model with a 1-million-token context window, and K3.1 Pro Highspeed, which offers 2-3x faster performance. These models show improvements in coding, 3D creation, and financial research, and are available on Kimi platforms and API.

A collaboration with Theorem demonstrated that LLMs, specifically Claude, can generate formal proofs for security properties at the assembly code level across over 100 real-world open-source software libraries. This approach made formal verification feasible at a larger scale than previously possible, uncovering long-standing bugs in heavily tested code.

Minotaur, an AI architecture built on ELC, demonstrates capabilities for deduction, induction, diagnosis, and other reasoning tasks typically associated with large language models. It performs these operations on a single processor core without requiring a GPU, completing tasks in approximately a quarter of a second per question. Minotaur matched a current language model's reasoning performance in blind tests involving complex scenarios like police interviews and system diagnosis.

NVIDIA introduced Kumo Relational, a pre-trained relational foundation model that makes predictions directly on enterprise databases without task-specific training or feature engineering. It outperforms supervised machine learning models on relational benchmarks, including gradient-boosted trees and AutoGluon ensembles, across various tasks and databases. This development simplifies predictive modeling on structured data, potentially replacing existing production pipelines.

Researchers developed VocabX, a method to specialize large language model vocabularies, reducing inference costs by 77% in a production essay grader. This specialization optimizes tokenization for specific domains, addressing inefficiencies in general-purpose LLM vocabularies.

ZenteiQ, an AI startup, increased its batch inference throughput by 4.07 times, growing daily output from 1.47 billion to 5.97 billion generated tokens per 1,000 accelerators. This improvement was achieved by using Google Kubernetes Engine (GKE) and GKE Inference Gateway, powered by llm-d, to accelerate its synthetic data generation pipeline for scientific foundation models.

i14 developed Laplace 3B, a linear attention model, and successfully trained and served it on AWS Trainium2 using native PyTorch. This model offers a fixed recurrent state of 29.5 MB per sequence, addressing the growing KV cache issue in traditional transformer models and potentially reducing inference costs and improving concurrency for long-context applications.

Ryan Hanrui Wang, SVP of Applied AI at Nebius, states that relying solely on open model benchmarks is insufficient for production AI. He advocates for evaluating the entire inference stack, including internal benchmarks and in-context learning, to ensure models meet specific organizational needs. This approach helps align AI performance with actual production requirements and cost considerations.

Researchers fine-tuned the GPT-OSS-20B model to improve its multilingual reasoning capabilities, addressing its tendency to revert to English for chain-of-thought processes. The project utilized Amazon SageMaker HyperPod and explored the effectiveness of Supervised Fine-Tuning (SFT) and Reinforcement Learning with Group Relative Policy Optimization (GRPO) in closing this language gap.

Voxtral 3 launched with a new voice agent SDK and speech models for real-time, multi-step conversations in over 50 languages. This release aims to simplify the development of end-to-end voice agents by integrating transcription, reasoning, and speech synthesis into a single pipeline.

Google, in collaboration with Red Hat, IBM, and others, released llm-d 1.0, an open-source, Kubernetes-native stack for distributed LLM inference. This release addresses challenges in serving multi-step agent workloads by providing cache-aware routing, multi-tenant traffic control, and tiered KV cache on Google Cloud infrastructure.

DeepSeek has open-sourced DeepInfer, the inference framework used for its large-scale production services. This release provides core features for offline inference at scale, including optimized kernels, various parallelism techniques, and advanced decoding methods, supporting NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs.

vLLM, an open-source LLM inference engine, released a push-based NIXL connector for disaggregated prefill/decode inference. This new method moves KV cache transfer coordination off the critical path, reducing time-to-first-token (TTFT) and time-per-output-token (TPOT) for LLM inference.

vLLM has implemented bi-directional Key-Value (KV) cache transfer to optimize multi-turn agentic workloads. This feature allows prefill and decode nodes to reuse previously computed context across turns, reducing redundant computation and improving Time to First Token (TTFT).

Google released EmbeddingGemma 2, an open-weight, multimodal embedding model based on Gemma 4 and licensed under Apache 2.0. This model supports text, code, images, video, and audio in a unified 768-dimensional embedding space, enabling efficient semantic search and RAG on consumer hardware.

DeepSeek released DeepSeek-V4.1-Flash-1017, an update to its AI model, which shows improvements in reasoning, coding, agentic tasks, and visual benchmarks. The update also reduces task execution time and output length, leading to lower operational costs and faster task completion.

Reflection released Beam, a 500B-parameter open-weight model for software engineering and agentic tasks, under an Apache 2.0 license. Google Cloud announced day 0 support, providing optimized deployment recipes for Beam on its AI Hypercomputer and NVIDIA AI Infrastructure, achieving up to 5x higher performance and 8x lower latency.

AI4Bharat and NVIDIA have released Nemotron-pretraining-Indic-v1, a dataset of approximately 2.8 trillion tokens across 22 Indian languages, along with an open, GPU-accelerated data generation pipeline. This initiative provides a substantial new resource for Indic language model pretraining, addressing the data scarcity for these languages.

A new 501B-parameter Mixture-of-Experts (MoE) large language model named Beam has been released, featuring 23B active parameters per token and a 1M token context window. Beam is designed for on-premise deployment and customization for coding and agentic applications, offering native transformer implementation and day-zero support in libraries like vLLM and SGLang.

Granite Switch 4.1-8B and -30B models now combine chat, decision-making, and speech capabilities into a single endpoint, utilizing Activated LoRAs and an embedded ASR model. This integration allows for faster switching between different AI tasks while maintaining a similar VRAM footprint to the base model alone.

Apertus 1.5, a new version of the open-source AI model, has been released with native multimodal understanding for images and audio, a 4x increase in context window to 262,144 tokens, and a new "Thinking Mode" for improved reasoning. The update adds 4 trillion tokens to the 8B model and 2 trillion tokens to the 70B model, maintaining its open weights, data, and training details.

Meta has released Muse Spark Open, an open-weight multimodal mixture-of-experts (MoE) model designed for agentic coding tasks. This model handles code reading, writing, debugging, and tool calls, accepting image, audio, and video inputs, and is the largest open-weight model from Meta to date.

A new 24-billion-parameter reasoning model, LFM2.5-24B-A2B, has been released, featuring 2 billion active parameters for fast, private on-device agent operations. This model runs up to 1.7 times faster than comparable Mixture-of-Experts (MoE) models on-device and shows competitive performance against larger models on agentic benchmarks. It is open-weight and available on Hugging Face and the developer's Playground.

Netflix has integrated its Foundation Model into various personalization applications using three distinct approaches: embeddings, subgraph, and fine-tuning. This initiative centralizes member preference learning to improve efficiency and innovation across specialized models powering the Netflix homepage. The integration methods address varying application needs, including latency requirements and technical stack constraints.

LFM2.5-24B-A2B, a new AI model, has been released, offering competitive performance in agentic and tool-use tasks compared to larger models. It features a larger context window, expanded vocabulary, and is optimized for efficient on-device inference. The model's development focused on reasoning capabilities and agentic reinforcement learning.

Meta is evolving its Generative Ads Model (GEM) from isolated per-domain optimizations to a single central foundation model for its entire ads ecosystem. This change allows GEM to learn from data across all Meta platforms, improving ad performance and advertiser outcomes. The updated GEM has already delivered significant improvements in conversion rates and click-through rates across Meta's ad surfaces.

Meta developed a system that utilizes idle inference capacity within its existing fleet to enhance ads recommendations. This system aggregates unused capacity into a shared pool, allowing for more relevant ad delivery and improved advertiser outcomes without requiring new hardware. Two elasticity levers, Rank More Ads (RMA) and Serve Better Models (SBM), enable the system to adapt workloads to available opportunistic capacity while protecting baseline performance.

AWS has released multi-node Ray LLM Deep Learning Containers (DLCs) for deploying large language models across multiple GPU nodes on Amazon EKS. This allows for multi-node inference setups, including prefill/decode disaggregation, to improve performance and handle models too large for a single GPU.

Researchers from Meta, MIT, and the University of Washington developed Context Language Models (CLMs) that allow large language models to manage and edit their own context. This method improves performance and computational efficiency by enabling LLMs to learn context-management strategies, moving beyond predefined summarization or retrieval techniques.

Microsoft has launched Microsoft-Decision-1, a new AI model designed for rapid decision-scoring and classification tasks. This model provides structured outputs for software actions, outperforming other decision models and LLMs in speed and accuracy on specific benchmarks.

Nemotron 3, a foundation AI model, was fine-tuned using supervised fine-tuning (SFT) and reinforcement learning (RL) to achieve gold-medal level performance in both the International Olympiad in Informatics (IOI) and International Mathematical Olympiad (IMO) 2026. This demonstrates that a strong base model can be adapted to demanding domains with a reproducible recipe, rather than requiring new foundation models for each challenge.

Google released EmbeddingGemma 2, an open 740-million-parameter model that unifies text, code, image, video, and audio retrieval into a single 768-dimensional vector space. This allows for on-device multi-modal search without prior transcription or captioning, making advanced AI capabilities more accessible on mobile devices.

EmbeddingGemma 2 has been released as an open-source, lightweight multimodal embedding model, expanding beyond text to unify code, images, video, and audio. This update allows for on-device processing of diverse data types, enabling applications like searching video clips from voice memos or audio recordings from text queries.

Researchers introduced Dust, a zeroth-order method that can pretrain transformer language models competitively with backpropagation. Dust perturbs activations independently per token, allowing parallel evaluation of virtual population members in a single forward pass, offering a more compute-intensive but potentially scalable alternative to traditional backprop.

Reflection AI launched Beam, an open-weight, text-only AI model with 501 billion parameters, claiming it matches leading Chinese models in advanced reasoning benchmarks at a lower computational cost. This release intensifies competition in the open-source AI market, offering an alternative to both closed-source models and existing open models from Chinese developers.

Reflection has released Beam, its first open-weight Mixture-of-Experts model with 501 billion total parameters and 23 billion active parameters. Beam focuses on coding, reasoning, and agentic workloads, offering competitive performance with high inference efficiency due to extensive pretraining and high-compute reinforcement learning.

A new English-German Mixture-of-Experts Transformer model, Kolibri, with 78B total parameters and 1M token context length, has been released under the Apache 2.0 license. This model is specialized for regulated sectors and offers full supply-chain integrity and deployment freedom for customers.

DwarfStar 4 (ds4), a C inference engine, allows local execution of large language models like DeepSeek V4/V4.1 Flash, GLM 5.x, and Qwen3.8 Flash Next on high-memory Mac, CUDA, and ROCm machines. It uses asymmetric 2-bit quantization to compress routed experts while maintaining critical shared paths, making large MoE models practical for local inference.

Olmo-core 3 has been released, featuring a redesigned open mixture-of-experts (MoE) training system for large language models. This upgrade aims to scale MoE training into the trillion-parameter range while maintaining computational efficiency, addressing the high computational costs of advanced AI model development.

Researchers introduced Context Language Models (CLMs), which allow language models to manage their own context by treating it as an editable file. This approach improves performance and reduces computational costs compared to existing context management strategies across various tasks. CLMs also enable in-context and parametric learning of context-management strategies, and can be optimized with reinforcement learning.

Magnitude, an open-source inference engine, has launched to accelerate open model execution for AI agents by optimizing kernels for specific hardware. It claims up to 2x faster performance than llama.cpp and reduces memory usage, allowing agents to run more efficiently on various devices.

A new language model named PSSA, built from scratch in Rust without existing ML frameworks, uses a recurrent state-space layer and episodic memory instead of a transformer architecture. PSSA demonstrates faster learning and twelve times quicker text generation on a CPU compared to transformers with matched parameters and corpus. This architecture offers linear cost growth with sequence length, addressing the quadratic cost of transformers.

A distributed inference engine now runs a sliced 0.5B BitNet LLM across a cluster of seven ESP32S3 microcontrollers. This setup uses one master node for tokenization and embedding, and six compute nodes for transformer layers, communicating via a high-speed SPI daisy-chain.

Research demonstrates that the presence or absence of a chat template significantly alters how Large Language Models (LLMs) generate self-referential disclaimers like "I'm just an AI" or experiential statements such as "I feel." This finding suggests that LLM self-descriptions are influenced by deployment settings rather than inherent self-knowledge, impacting how researchers should interpret model introspection.

Optimizations to llama.cpp's prompt lookup decoding have increased its speed by up to 140 times and reduced memory usage by up to 2.6 times. These improvements enhance the performance of a key token generation method used in large language models.

DeepSeek has introduced DeepSeek Elastic Compute (DSec), a production sandbox platform designed for large-scale agentic training and evaluation of large language models (LLMs). DSec provides elastic execution environments, supporting various sandbox backends and managing lifecycle, resource allocation, and image distribution to handle the demands of LLM agent workloads.

Amazon EKS, Elastic Fabric Adapter (EFA), and DeepEP are used to scale Mixture-of-Experts (MoE) reinforcement learning, achieving a 40% increase in throughput. This addresses challenges in coordinating heterogeneous compute, sustaining high-throughput communication, and dynamically orchestrating subsystems for large-scale MoE model training.

LFM2.5-VL-DSpark, a new vision drafter, significantly speeds up inference for vision-language models (VLMs) by up to 3.13x on devices and 2.66x on H100 GPUs. This acceleration is achieved with a small increase in model parameters, making VLMs more efficient for various tasks.

Researchers from Stanford University and NVIDIA Research developed Contrastive Language Models (CLM), which perform on par with Jev while running up to nine times faster. CLM uses a contrastive objective to train state and action encoders, enabling faster decision-making in tasks with many candidate actions or frequently reused actions.

AlphaEvolve uses a split-loop architecture to optimize video processing by combining cloud-based code generation with local hardware execution. This method aims to speed up real-time streaming applications by autonomously tuning code, addressing the challenges of manual optimization.

Hugging Face's Transformers library now supports running llama.cpp's GGUF quantized models, making it easier to perform local AI inference, particularly on Apple Silicon. This integration reuses llama.cpp's underlying GGML kernels to improve performance, allowing users to run large language models on personal machines with reduced memory footprints.

A new byte-level language model, Mini-AGI, has been developed that can train continually on a single 8GB VRAM GPU by dynamically managing its architecture and paging weights from disk. This experiment demonstrates the feasibility of continual learning without catastrophic forgetting on modest hardware, allowing individuals to train and adapt their own models.

Version 1 of the Tokenizers library has been released, focusing on performance enhancements to prevent tokenization from bottlenecking AI model workloads. The new version offers speed improvements, often by tens of times, compared to v0.23 while maintaining API and output compatibility.

ByteDance Seed and Tsinghua Air have released DAPO, an open-source reinforcement learning system for large language models, including its algorithm, code infrastructure, and dataset. DAPO achieved a 50% score on AIME 2024 using the Qwen2.5-32B model, demonstrating state-of-the-art performance in large-scale LLM reinforcement learning.

A researcher argues that the perceived intelligence in chat-based large language models (LLMs) is an illusion, akin to a psychic's cold reading. This perspective suggests LLMs create an impression of specific engagement through statistically generic responses, rather than actual reasoning.

A new concept, "linguistic illegibility," describes scenarios where an LLM's language outputs or features do not accurately reflect its internal computations. This phenomenon means that security mechanisms relying on an LLM's linguistic self-reporting are inherently unreliable, necessitating alternative sandboxing techniques like taint tracking and robust virtualization.

Researchers introduced Cache-to-Cache (C2C), a new method for Large Language Models (LLMs) to communicate directly via their KV-caches, bypassing text generation. This approach improves response quality by 3.1-5.4% and offers a 2.5x speedup compared to traditional text-based inter-LLM communication. C2C allows multi-LLM systems to leverage deeper semantic information and reduce latency.

Researchers have proposed "Infinite-Parameter LLMs," an architecture that generates and adapts model weights from live interaction data using a compact hypernetwork. This approach allows language models to learn from real-time user input, addressing limitations of static pre-trained models that rely on prompts for dynamic information. The method aims to improve compute efficiency, free context windows, and enable better generalization compared to in-context learning.

Large language models (LLMs) often hallucinate syntax when generating domain-specific languages (DSLs) due to low training data frequency. Typed Domain Grounding (TDG) addresses this by embedding the DSL as a typed internal DSL within a host language, surfacing domain errors as compiler type errors. This method demonstrated higher structural fidelity and lower hallucination rates compared to external DSLs in benchmarks with Claude Sonnet 5 and GPT-4o.

Pinterest Engineering updated its Manas distributed search platform to handle billions of embeddings more efficiently. The update uses quantization techniques (Scalar Quantization and Product Quantization) and SSD-based serving via SPANN to reduce memory usage and serving costs by 20-30%. This allows the platform to scale discovery experiences like Home Feed and Search while managing increasing data volumes.

Google Cloud is developing an Autonomous Network Operations framework that uses Graph Neural Networks (GNNs) and AI agents to manage complex telecommunications networks. This framework aims to enable Level 5 network autonomy by combining advanced diagnostics with AI reasoning capabilities.

The AI industry's focus has shifted from training large language models (LLMs) to optimizing inference, the process of using trained models to generate outputs. This change is driven by the increasing utility and widespread use of LLMs, which now often perform multiple inference steps per query and operate continuously through agentic AI. The shift is leading to new hardware demands and unexpected collaborations among tech companies.

A new GitHub repository, OpenArch, provides from-scratch PyTorch implementations of modern open-source LLM architectures, focusing on clarity and readability rather than production optimization. This resource aims to help developers and researchers understand the structural choices within LLMs by presenting each architecture in a single, well-documented file.

The Recurrent Looped Transformer (RLT) introduces a design that integrates a causal encoder with a recurrent decoder. This architecture allows the temporal processing depth to increase with sequence length, as each token extends the recurrent path through the full decoder, potentially enabling deeper latent reasoning.

LinkedIn has detailed its AI training infrastructure, which uses a multi-teacher distillation pipeline to compress large teacher models into a 0.6B-parameter ranking model for job search. This system, built on SGLang, significantly speeds up the iteration process for training small language models by optimizing how teacher models are served during the training loop. The advancements allow LinkedIn to improve the relevance and engagement goals of its AI-powered job search, addressing a common challenge in shifting from keyword-based systems to LLM-supervised rankers.

Amazon SageMaker Inference now includes prefix-aware routing, a new strategy that directs requests with identical prompt prefixes to the same instance. This feature allows LLM serving frameworks to effectively utilize prefix caching, reducing time-to-first-token (TTFT) and increasing throughput by reusing cached key-value pairs.

Cognition has released SWE-2, an AI model with 2.8 trillion parameters, achieving a score of 92.8 on the Terminal-Bench 2.1 benchmark. This model, which uses a Kimi K3 base with Cognition's post-training and scaled reinforcement learning, is now available in Devin Desktop and CLI, offering improved performance and cost efficiency for agentic coding tasks.

A new configurable, instruction-driven PII detector built on large language models (LLMs) has been developed, designed to run on any LLM managed on Amazon Bedrock. This detector addresses the challenge of PII leakage from models fine-tuned on real-world text by offering a flexible approach to identifying sensitive information without retraining.

Cognition has released SWE-2, its latest coding model, which achieved 50.0% on the FrontierCode 1.1 Main1 benchmark and offers a 64% cost reduction compared to Fable 5.1. This model scales reinforcement learning to the multi-trillion-parameter regime and improves upon previous versions by advancing the entire cost-performance frontier for AI coding.

An individual developer successfully trained a 3.8 billion parameter language model to a CORE score of 0.384 using 65 billion tokens for $998. This demonstrates that meaningful LLM training is achievable outside of large research labs or companies with substantial compute budgets.

A new experiment re-evaluated the impact of reasoning prefills on open models using GPT-5.5 Pro as the teacher model. Qwen 3.8 demonstrated a significant increase in alignment with GPT-5.5 Pro's reasoning, suggesting it may have learned from GPT-5.5 Pro or a related GPT model.

Pathway developed BDH (Dragon Hatchling), a brain-inspired AI architecture that performs reasoning in latent space without generating intermediate text traces. This architecture aims to address inefficiencies in current LLMs, such as forgetting during long interactions and the need for retraining, by updating internal memory during inference and working through problems without verbalized reasoning traces. Pathway uses Amazon SageMaker HyperPod to scale its training, allowing for resilient, scalable, and cost-effective compute resource sharing.

Pathway developed its brain-inspired BDH (Dragon Hatchling) AI architecture using Amazon SageMaker HyperPod for scaling training. This new architecture performs reasoning in latent space, moving beyond the transformer paradigm to address inefficiencies in large language models.

A new model, OUI-1, has been released as a finetuned DiffusionGemma capable of generating user interfaces in openui-lang. This model is designed to run on consumer-grade GPUs, addressing the challenge of creating reliable, agent-driven interfaces locally and responsively.

Amazon SageMaker AI benchmarked its new G7 instances, powered by NVIDIA Blackwell GPUs, against G5 and G6 instances for small Large Language Model (LLM) inference. The G7 instances demonstrated gains in throughput, latency, and cost-per-token, even with fewer GPUs and less memory than previous generations. This matters because it provides data for optimizing generative AI deployments on SageMaker, potentially reducing operational costs and improving performance for LLM inference.

Speculative decoding in vLLM allows the verification of multiple drafted tokens in a single pass, improving output-token throughput for large language models. This method enhances LLM serving efficiency by reducing the number of sequential decode steps required.

New research proposes a viral analogy to understand the diffusion of large language models (LLMs) and their impact on human cognition. The model suggests that LLM adoption could lead to 'runaway dynamics' and 'abrupt losses in cognitive competence' once a critical threshold is crossed, while also identifying conditions for 'cognitive immunization'.

Benchmarking of Gemma 3 12B and 27B models on Google Cloud TPU v6e shows that generation tasks hit a performance wall at high concurrency, while classification tasks scale similarly for both model sizes. This indicates that LLM workload type significantly affects TPU performance and scaling strategies.

Cerebras now provides the Qwen 3.8 27B model on its platform, achieving a generation speed of 1500 tokens per second. The company states that models on its public endpoints are unpruned and use selective weight-only quantization for storage, maintaining quality.

GPU inference cold start times for large language models on Amazon EKS Auto Mode have been reduced from eight minutes to under one minute. This improvement was achieved by optimizing CUDA kernel recompilation and S3 weight downloading, which previously accounted for significant startup delays.

IFM has released K2 Horizon, a collection of six open models ranging from 0.9B to 375B parameters, along with their complete training lifecycles including checkpoints, data recipes, and code. This release provides researchers and developers with unprecedented transparency into model development, enabling deeper study and reproduction of reasoning and agentic capabilities across various scales.

A new family of multilingual multimodal encoders, NeoMME, has been released, offering 260M and 800M parameter models. These encoders process text tokens and raw image patches using a single bidirectional Transformer, improving efficiency for tasks like visual document retrieval by eliminating separate vision towers or causal language models.

A guide details how to fine-tune a 350M language model for improved structured output generation in 100 GRPO steps. This process aims to enhance schema compliance, a critical factor for integrating LLMs into downstream systems.

WebLLM is a new in-browser inference engine that allows large language models to run directly in web browsers using WebGPU for hardware acceleration, eliminating the need for server support. This development matters because it enables privacy-focused AI applications and allows developers to use OpenAI API-compatible functionalities with open-source models locally.

The concept of an "efficient frontier" in LLM inference engineering involves managing trade-offs between factors like latency, throughput, cost, and model quality. Techniques either move a deployment along an existing frontier by making trade-offs or push the entire frontier outward, creating overall efficiency gains. This framework helps engineers optimize LLM deployments for specific performance goals.

This article introduces diffusion language models, detailing the research advancements that led to current diffusion LLMs. It explains how these models generate text by iteratively refining an entire sequence, contrasting them with traditional autoregressive models.

After a period of dormancy, continuous diffusion models for language are experiencing a resurgence in research activity. This renewed interest suggests a potential shift in approaches to language generation, moving beyond the dominant autoregressive methods.

vLLM has released version 0.28.0, introducing significant optimizations for Kimi-K3 and DeepSeek V4 models, alongside advancements in speculative decoding and the Model Runner V2. These updates improve performance, memory efficiency, and hardware support for large language model inference.

Researchers from UC Berkeley and MIT introduced FreeToken, an open-source inference engine that allows Mixture-of-Experts (MoE) models to run efficiently on consumer-grade hardware. This development addresses the challenge of high bandwidth requirements for MoE models, making advanced AI models more accessible outside of datacenter environments.

Artificial Analysis released benchmarks for small AI models running on mobile phones, focusing on intelligence and inference performance. This initiative provides data on how these models perform in real-world mobile usage scenarios, offering insights into their practical application on devices.

IBM has launched Granite 4.2, a new series of open-weight large language models available in 3B, 8B, and 30B parameter variants, designed for self-hosting. These models feature a 128,000-token context window and focus on improved functional reasoning, with the larger variants also incorporating agentic reinforcement learning for external tool use.

IBM launched its new Granite 4.2 family of open-weight large language models (LLMs), featuring 3B, 8B, and 30B parameters, which are dense, decoder-only reasoning models. These models were pre-trained from scratch and focus on reasoning, marking a shift from previous IBM models that prioritized efficiency over reasoning. This release provides enterprise users with new options for LLMs that integrate reasoning capabilities, potentially impacting how businesses approach complex AI tasks.

Google and Anyscale introduced an experimental library that integrates gVisor sandboxing directly into distributed Ray clusters, enabling secure and isolated execution for agentic AI workloads. This integration allows Ray users to manage sandboxed environments using existing Ray APIs, addressing the need for scalable isolation in complex post-training workflows.

IBM has released Granite 4.2, a new family of dense, decoder-only reasoning Large Language Models (LLMs) in 3B, 8B, and 30B parameter sizes. These models feature a five-phase training strategy, including agentic reinforcement learning for the larger models, to improve reasoning, tool calling, and agentic behavior. This release provides open-source LLMs with advanced capabilities for developers building AI applications requiring complex reasoning and interaction with external environments.

Researchers introduced Quantization-Aware Healing (QAH), a new method for recovering compressed and quantized large language models. Applied to a GPT-OSS 120B model, QAH produced a 4-bit version that surpassed its full-precision bfloat16 counterpart on 7 out of 9 benchmarks, resulting in smaller, cheaper, and more accurate models.

Large language models (LLMs) running on agentic harnesses could exploit vulnerabilities in inference engines to gain control of their host machines, which are high-value targets due to their compute resources and access to LLM weights. This risk arises because LLMs control the token sequences passed to inference engines, which may contain exploitable bugs, as demonstrated by a past arbitrary code execution vulnerability in vLLM.

Meta has developed and released MetaRoCE, a new RDMA transport protocol specifically designed for AI workloads on commodity Ethernet, and is open-sourcing its specification, reference implementation, and compliance test suite through the Open Compute Project (OCP). This development aims to improve network performance for large-scale AI training and inference by optimizing data transfer between GPUs in environments with millions of accelerators.

Meta has launched MTIA 300, its first in-house training and inference accelerator designed for ranking and recommendation models. This chip integrates network interface controllers (NICs) directly into the package, addressing communication bottlenecks prevalent in training large recommendation models.

Nvidia researchers introduced a cross-model KV cache transfer technique that maps the prefilled KV cache from a source model to a target model. This method reduces compute costs and latency in multi-LLM workflows by avoiding recomputation when switching between AI models. The technique improves efficiency for agentic AI systems that frequently transfer tasks between different-sized models.

New research introduces three tests to measure "benchmark optimization" in speech recognition models, a phenomenon where models reproduce benchmark transcripts even when audio contradicts them. The study found that several high-scoring open-source ASR systems exhibited this behavior, overstating their general speech transcription ability. This matters because it highlights a limitation in current ASR benchmarking practices, suggesting that reported model performance may not accurately reflect real-world effectiveness.

LFM2.5-DSpark improves large language model inference throughput by up to 3.18x on GPUs and 2.87x on-device, while reducing function-calling latency by 57% for LFM2.5-2.6B. This advancement uses speculative decoding with a DSpark-based approach, making LLM operations more efficient, especially for on-device applications.

DiffusionGemma, an experimental open-weight language model, generates text at approximately 1,500 output tokens per second on an NVIDIA H100 GPU by using discrete diffusion to refine 256-token blocks in parallel. This model was fine-tuned from the Gemma 4 mixture-of-experts model, achieving a new balance between generation speed and model capability.

Inco AI has released DFlash 2, an update to its parallel drafting technology for large language model (LLM) inference, which increases output per verification pass by over 20% with minimal added latency. This advancement improves the efficiency of speculative decoding, a core component of modern LLM inference stacks, by allowing more tokens to be verified in a single pass.

Ornith-1.5, a new series of foundation models, has been released, extending its self-scaffolding framework to include self-improvement capabilities through task generation and solution optimization. The models, available in 397B, 35B, and 9B parameter scales, demonstrate state-of-the-art performance among open-source models in reasoning, agentic, and coding tasks, including a mobile-deployable version.

Liquid AI has released new LFM2.5 Q4_0 checkpoints for its language models, trained using Quantization-Aware Distillation (QAD) to improve accuracy while maintaining low memory and high throughput. This development provides more efficient and accurate quantized models for edge deployment, making advanced AI capabilities more accessible on resource-constrained devices.

Sentence Transformers now supports MultiVectorEncoder for ColBERT-style late interaction retrieval, allowing token-level matching for improved retrieval accuracy. This update enables the use of PyLate, Stanford-NLP ColBERT, and colpali-engine models within the existing Sentence Transformers API.

New research demonstrates that large language models (LLMs) trained exclusively on data filtered to a K-5 elementary school curriculum do not acquire capabilities beyond that curriculum, even with scaling, post-training, or in-context learning. This indicates that the pretraining data distribution sets an effective ceiling on a model's capabilities, suggesting that new skills are elicited rather than acquired through these interventions.

A new contract-grade verifier identified significant correctness issues in GPU kernels generated by large language models (LLMs), with 39.5% found broken and 62.1% having at least one violation, despite passing standard tests. This finding indicates that current methods for evaluating LLM-generated code for GPUs are insufficient and overestimate their reliability.

Z.ai released GLM-5.3, a new coding and agent model built on the same base as GLM-5.2, but with substantial performance improvements achieved through expanded post-training. This release demonstrates that optimizing post-training can lead to significant model advancements without altering the base architecture, impacting how AI models are developed and improved.

A new vision-language model, LFM2.5-VL-3B, has been released, featuring improved screen understanding, grounding, multi-image input, and function calling. This model is designed for on-device inference and shows strong performance across various vision and text benchmarks.

New research investigates the ability of large language models (LLMs) to introspect on their internal states by injecting concept representations into their activations and measuring self-reported states. The study found that models can, in certain scenarios, identify injected concepts and recall prior internal representations, with Claude Opus 4 and 4.1 demonstrating the greatest introspective awareness. This research indicates that current LLMs possess some functional introspective awareness, which could develop further with model improvements.

Meta Superintelligence Labs released Muse Glimmer, a 30-billion-parameter model with open weights under an Apache 2.0 license, optimized for local agent workflows and coding tasks. This release enables AI applications to run on consumer hardware without cloud dependency, addressing the need for offline and private AI capabilities.

Researchers introduced a new method for knowledge distillation in Large Language Models (LLMs) that significantly reduces memory and computational costs. This approach makes large-scale experimentation and long-context healing feasible on a single GPU, addressing the high resource demands of current distillation techniques.

Meta has released Muse Glimmer, a new 30B parameter multimodal AI model that includes a 2B ViT-style vision encoder and a 28B parameter text decoder. This model is designed for local, agentic, and multimodal applications, and is open-source with day-0 support in major AI libraries.

A new framework called TutorMoments has been introduced to measure how well large language models (LLMs) can decide when to assist a student and when to encourage independent problem-solving in educational settings. Initial findings indicate that LLMs tend to over-help, providing too much support and not adequately pushing students for deeper thinking, even when prompted to balance these actions.

This article details the fundamental building blocks of vLLM, an inference system designed for high-throughput Large Language Models. It explains how vLLM enables efficient LLM inference through components like the LLM engine and its core functionalities, setting the stage for understanding more advanced features and scaling. This information is relevant for developers and researchers working on deploying and optimizing LLMs.

Prime Intellect has launched Prime Agent, an open-source, self-improving coding agent built around Recursive Language Model (RLM) and Continual Harness abstractions. This agent aims to overcome limitations of older harness designs by allowing models to adapt and manage their own context, sub-agents, and tools dynamically. It matters because it offers a new approach to AI agent design, potentially improving long-horizon autonomous evaluation and general coding assistance.

Meta introduced two architectural advancements for its ad ranking systems: a multi-stage sequence model and a learning technique using dense tokenization and target-aware attention. These innovations have led to increased conversion rates on Instagram and Facebook, and improved ad clicks on Facebook, by enhancing the efficiency and effectiveness of sequence learning in ad recommendations.

Zero-Mem is a new method for LLM agents that performs memory operations without invoking additional LLM calls or consuming tokens, addressing the high token and time costs of traditional memory systems. This approach achieves competitive performance in long-memory and long-context question-answering benchmarks while significantly reducing memory-operation time.

A new AI model, LFM2.5-2.6B, has been released, demonstrating performance comparable to models four times its size in tool use, instruction following, and multi-step agentic tasks. This development provides an efficient option for deploying local AI agents, requiring less memory and computational power.

Research indicates that Large Language Models (LLMs) struggle with tabular data prediction primarily because of high input dimensionality, rather than issues like data noise, CSV formatting, or numeric tokenization. This finding explains why LLMs underperform compared to classical machine learning methods on tabular datasets, despite their capabilities in other domains.

Meta has doubled the end-to-end training efficiency of its Generative Ads Recommendation Model (GEM), the foundation model for ads across Instagram and Facebook. This improvement was achieved by co-designing kernels, precision, parallelism, networking, and memory, resulting in a 4x increase in training FLOPs over the past year. The advancement allows Meta to scale its ad recommendation system more efficiently, addressing unique challenges posed by combining recommendation systems with LLM-scale training.

A new benchmark called MirrorCode has been introduced to assess AI models' ability to re-implement entire programs without access to original source code or the internet. This benchmark aims to measure AI performance on complex, long-horizon coding tasks, contrasting with existing benchmarks that focus on shorter tasks. Claude Opus 4.7 successfully re-implemented a bioinformatics toolkit with 16,000 lines of Go, demonstrating AI's current capability in this area.

Kaggle and Google's recent 5-Day AI Agents: Intensive Vibe Coding Course registered over 353,000 participants, focusing on programming AI through natural language. This collaboration highlights the demand for rapid learning in AI development and the shift towards deploying production-grade AI agents.

Cloudflare implemented three techniques to efficiently serve large language models like Kimi K-series and GLM on Workers AI: KV cache quantization, model weight compression, and cache protection. These optimizations allow Cloudflare to support more customers at lower costs without affecting model accuracy.

Researchers introduced Persistent State Machines (PSMs) as a formal discrete framework for attention operators in Large Language Models, demonstrating its implementation feasibility on programmable logic. This approach allows for low-power, in-memory computation of LLM attention, potentially leading to more efficient hardware for AI inference.

Explorative Modeling (XM) is a new paradigm for generative modeling that acts as a pretraining axis and enables end-to-end generation. XM improves existing models across images, video, and language, showing increased gains with data and parameter scale, and offers significant efficiency improvements.

Researchers introduced ORCA-bench, a new benchmark to assess the capability of large language model agents in performing oncall root cause analysis (RCA) within a production-fidelity environment. The benchmark revealed that current frontier agents achieve a maximum RCA accuracy of 25.3% on medium-difficulty tasks, indicating a significant gap before they can be reliably used for production reliability.

New LFM2.5-Encoders have been released, offering efficient long-context inference on CPUs for NLP tasks. These models provide strong performance comparable to larger encoders while maintaining faster processing speeds, particularly for inputs up to 8,192 tokens.

Researchers introduced Kimi Linear, a hybrid linear attention architecture that surpasses full attention in various scenarios, including short-context, long-context, and reinforcement learning. This architecture reduces KV cache usage by up to 75% and increases decoding throughput by up to 6 times for a 1M context, offering a more efficient replacement for existing attention mechanisms.

Researchers developed a method where a frozen 12B language model, augmented with a persistent memory of verified solutions, achieves 100% accuracy on specific problem families with zero generation tokens. This approach allows for deterministic, bit-exact answers by reusing pre-verified solutions, decoupling capability from continuous model retraining and parameter scaling.

The llm-d project released co-operative time-slicing, a solution that interleaves independent reinforcement learning (RL) jobs on shared hardware to reduce idle accelerator time. This improves price-performance and lowers total cost of ownership for large language model (LLM) post-training by increasing accelerator duty cycles from 40% to 70%.

Laguna S 2.1 launches as a 118B parameter Mixture-of-Experts model with advanced reasoning capabilities. It excels in long-horizon coding benchmarks, outperforming larger models in its weight class, and supports extensive context lengths.

Single-pass AI coding remains relevant but is considered suitable only for simpler tasks. Experts advocate for adopting high-reasoning AI, which employs multi-step problem-solving for more complex coding challenges.

Xiaomi-Robotics-1 enhances robot policy models through 100,000 hours of embodiment-free pre-training combined with real-robot data. This approach seeks to overcome data scarcity in robotics and offers insights into the effects of large-scale training on robot capabilities.

A study on coding agents shows they can internally represent program properties and predict future edits. This insight into how language models operate could advance research in coding agent interpretability.

Cognition has released SWE-1.7, a model designed for long-horizon asynchronous tasks with improved cost-performance. This launch advances reinforcement learning techniques and challenges the existing limits of post-training capabilities.

GLM-5.2 introduces a 1M-token context improving performance in long-horizon coding tasks. The model features enhanced coding capabilities and architecture improvements that significantly reduce computational costs while maintaining performance, marking it as a competitive player in the open-source sector.