Efficiency papers
What AI researchers are reading: the papers the Hugging Face community picks each day, and new arXiv submissions in the directions we follow. Titles and abstracts are arXiv’s; every paper links to arXiv.
Quantifying the Sampling Frequency Invariance of Time-Series Foundation Models for Sensor Data Applications
Sourish Gunesh Dhekane, Abinaya Kumar, Torsten Woertwein et al.
Time-Series Foundation Models (TSFMs) are increasingly used in sensor applications, where sampling frequency (SF) varies across sensors, devices, and operating modes (e.g., power saving). Ideally, TSFMs must remain effective during such SF changes, provided the SF retains task-critical bandwidth. In this paper, we…
Discovering Equations for the Thermodynamics of South Asian Monsoon Precipitation
Savannah L. Ferretti, Jane W. Baldwin, Michael S. Pritchard et al.
South Asian monsoon precipitation depends on the thermodynamic structure of the lower troposphere, but how its ingredients combine to shape precipitation remains unclear, partly because this dependence is nonlocal in the vertical. We use symbolic distillation, which combines neural networks and symbolic regression,…
REAL Loss: Optimizing Real Expected Acceptance Length for Speculative Decoding
Yu Tian
Speculative decoding accelerates LLM inference by having a draft model propose multiple tokens for the target model to verify in one pass. The speedup is governed by the expected acceptance length (EAL), yet standard training objectives such as cross-entropy, KL divergence, and total variation focus only on…
A Margin Perspective on LoRA: Robustness to Catastrophic Forgetting and Adapter Merging (MaLoRA)
Ziqing Xu, Hancheng Min, Lachlan Ewen MacDonald et al.
Low-rank adaptation (LoRA) is the de facto method for parameter-efficient fine-tuning of large neural networks, achieving strong task adaptation with minimal trainable parameters and low optimization cost. Beyond this efficiency, practitioners have observed two striking properties: \textit{robustness to…
An Energy-Based Perspective on Machine Learning
Bo Dai, Yilun Du
Energy-based models (EBMs) define a distribution through a scalar energy, $p_θ(x)\propto\exp(-E_θ(x))$, and are usually presented as one model family among many. This tutorial has a broader goal: beyond introducing EBMs, it uses them as a single lens that connects the different aspects of machine learning. Any EBM…
Uncertainty-Aware Hybrid Offline Reinforcement Learning with Online Action Selection Enabled by Sparse Gaussian-Process Critics
Marco Chan, Christos Evripidou, Liqiu Dong et al.
Start-ups and grade changes remain largely operator-driven, while developing and maintaining dynamic models for model-based control requires substantial engineering effort. Hybrid Offline Learning and Online Optimisation (HOFLON) uses one-step offline reinforcement learning to select actions from historical…
The Unused Input: The Initial State as a Conditioning Interface for Recurrent TSFMs
Bernhard Voggenberger, Christian Ganhör, Levente Zólyomi et al.
Time series foundation models (TSFMs) forecast unseen series without task-specific training. Increasingly they are built on recurrent backbones, whose constant per-step cost makes long-context inference cheap, and which, unlike transformers, carry a state. A recurrent backbone computes its prediction from two…
EngineAgent: Agentizing On-Device LLM Inference Engines with Inference Stack Hierarchy Awareness and Hardware Abilities
Zhengxiang Huang, Yujie Sun, Shengheng Chen et al.
Rapidly evolving LLM architectures and heterogeneous mobile hardware make supporting new models in on-device inference engines a recurring and costly system challenge. Each model may require coordinated changes across conversion, graph construction, operators, and hardware backends. Existing coding agents often…
Elastic-WAM: Adaptive Spatiotemporal Imagination for Efficient Robot Control
Haitian Li, Tianfu Li, Xuxi Zhao et al.
World-Action Models improve robot control by predicting future visual dynamics, but iterative video denoising incurs substantial inference overhead. Recent approaches reduce this cost through partial future denoising, yet typically rely on fixed refinement budgets. Our empirical observations reveal that preferred…
SinkState: On-the-Fly SVD of Recurrent States for Efficient Hybrid LLM Serving
Haolin Jiang, Weihang Li, Zhongzhu Zhou et al.
Hybrid models combine softmax attention with linear attention, which compresses history into fixed-size recurrent states to balance model quality and inference efficiency. However, linear attention has not pushed this compression to its limit: we find that the queries, keys, and values of linear layers concentrate…
StaRSE: Compact Russian Sentence Embeddings with a Sign-Coded Static Encoder
Boris Malashenko, Ivan Jarsky, Egor Soldatov et al.
Transformer sentence encoders provide strong retrieval quality, but their inference cost becomes a bottleneck when embeddings must be produced on CPUs or refreshed frequently. We introduce StaRSE, a compact Russian static sentence encoder that removes Transformer inference at deployment time: token directions are…
Joint Quantization and Dynamic Partitioning for Context-Aware Edge Vision Transformers
Zhu Jiaqi, Jain Prakriti, Katare Dewant et al.
Deploying vision transformer models across connected and automated vehicles (CAVs) requires joint management of computation and Vehicle-to-Vehicle (V2V) communication. Static split-inference methods use fixed partition points and cannot adapt to time-varying link bandwidth, processing load, and concurrent…
Representational Atoms in Neural Networks
Chenhui Hu, Pengfei Cao, Kang Liu et al.
Understanding the internal representations of neural networks and characterizing their relationship with human conceptual structure are two critical questions for elucidating model mechanisms. The former reveals how learned knowledge is organized in internal representations, whereas the latter provides a basis for…
SteerDistill: Quantizing LLM Agents via Closed-Loop Teacher Steering
Genming Cui, Xiangyu Li, Zixu Hao et al.
Low-bit quantization reduces the inference cost of LLM agents but can compromise task success. Examining failed rollouts, we find that quantized agents overlook task requirements or environmental conditions, producing environment transitions that diverge from successful full-precision references. Correcting these…
ExRL: Sample-Efficient Online Adaptation of Long-Chunk Policies via Execution Length Control
Xu Luo, Jiaying Yang, Hao Wu et al.
Predicting long action chunks, often spanning dozens of timesteps, has become a key ingredient in increasingly capable robot policies. In this paper, we find that adapting these long-chunk policies with existing reinforcement learning (RL) methods can require many online samples due to the high dimensionality of…
Sable: A Small-Field Sumcheck Argument for Bit-Exact Inference of Large Language Models
Tomer Solberg
We present Sable, an argument system and single-GPU prover for the statement that a committed, integer-quantized transformer produced a given token sequence from a given prompt, bit for bit. Sable fixes an integer function F_O with no floating-point step, in which every rounding, normalization, softmax and sampling…
Precision on Demand: Escalating Weight Bits at Uncertain Decoding Steps in a Small Local Model
Mohammed Nayeem Khan, Logesh L, Aliyas Ahmad Misri
Local decoding usually fixes one weight precision for the whole generation. We study a different allocation: store 8-bit weights, decode with their top 4 bits, and redo a step at 8 bits only when the low-precision output distribution looks uncertain (entropy above a threshold or a small top-two margin).…
Bits or permutations? Spending a fixed inference budget on weight precision or on option-order averaging in small local language models
Mohammed Nayeem Khan, Logesh L, Aliyas Ahmad Misri
Local language models can save memory by storing weights in fewer bits, and they can reduce sensitivity to the order of answer options by averaging a few reorderings. We ask whether the first makes the second worse, and which is the cheaper way to spend a fixed budget. On Qwen3-0.6B and Qwen3-1.7B with simulated…
Click3R: Interactive Stereo 3D Reconstruction with Sparse Correspondence Clicks
Hao Gu, Jin Yao, Zezhou Cheng
Despite impressive progress, existing feed-forward large 3D reconstruction models produce a single, irrevocable prediction. When these models fail, there is no mechanism to correct their mistakes. To address this limitation, we introduce Click3R, a framework for interactive geometric correction of feed-forward…
How Much Training-Inference Mismatch Can On-Policy Distillation Tolerate?
Cong Yu, Ziting Wang, Songlin Jiang et al.
On-policy distillation (OPD) trains a student language model on its own responses, guided by a fixed teacher. Generating these responses is expensive, so systems use quantized inference and asynchronous rollouts to reduce the cost. Both, however, introduce training-inference mismatch between the rollout engine and…
MACaQ: Fast, Model Agnostic Calibrated Quantization for Low-Precision Weights and Activations
Lorenz K. Muller, Philippe Bich, Chiara Boretti et al.
Weight-activation quantization can reduce the memory and compute cost of neural-network inference, but recent methods often require expensive optimization and architecture-specific propagation of basis transforms across layers. We introduce MACaQ, an MXFP4 W4A4 post-training quantization method that treats every…
The Surprising Effectiveness of Deleting Weights in LLM Post-Training
Jack Lu, Zhenbang Yang, Mike Lasby et al.
How little of a pretrained large language model (LLM) has to change for it to acquire new capabilities? We find that zeroing fewer than 0.05% of its weights, with no other modification, matches full fine-tuning (FFT) at scale across three LLM post-training regimes: reinforcement learning with verifiable rewards,…
Read Less, Solve More: Token-Efficient Sparse Reading of External Content for AI Agents
Zedong Liu, Jiaan Wu, Xinyang Ma et al.
Web-facing AI agents repeatedly access webpages, documents, repositories, and tool responses, yet reading interfaces often expose entire objects when only sparse evidence is needed. This over-reading increases token and latency costs and can dilute relevant evidence. We present SparseRead, a reusable reading…
Ensemble Routing for Prompt-Adaptive Cost-Aware LLM Inference
Saul Urso, Emanuele Carlini, Patrizio Dazzi et al.
Large language models (LLMs) increasingly support Web-based services that must balance response quality and inference cost across heterogeneous requests. Conventional routing selects one model per prompt, while fixed ensembles cannot adapt their composition and cost to individual requests. We introduce PISA, a…
PhysioWave-T: Token-Efficient Wavelet Foundation Models for Biosignals
Yanlong Chen, Simone Benatti, Luca Benini et al.
We present PhysioWave-T, a token-efficient wavelet foundation model family for EEG, ECG, and sEMG. Dynamic scale folding recombines J+1 wavelet subbands prior to tokenization, compressing sequence length from (J+1)CS to CS tokens for J levels, C channels, and S temporal patches. Route-specific front ends and…
Modality-Aware Expert Pruning for MoE-Based Multimodal Large Language Models
Donggeun Kim
Mixture-of-Experts (MoE) enables multimodal large language models (MLLMs) to scale capacity efficiently, but expert parameters dominate memory even though only a few experts are activated per token: they account for over 90% of model size in modern MoE-MLLMs (e.g., 54 GB out of about 58 GB in BF16 for…
DepthGraft: Structural Regularization through Hierarchical Cross-Layer KV Reconstruction
Xiaohan Qin, Xiangdong Zhang, Yu Wang et al.
Scaling depth is a key route to stronger large language models, but Pre-LN architectures often suffer from depth-wise information dilution, limiting the effective use of deep layers. While cross-layer connectivity has proven effective at mitigating this problem, existing designs usually introduce additional…
Flide: Accelerating Decision Inference with Exact State Forking
Bobo Li, Jiajia Song, Hao Fei et al.
Decision models such as Jev have emerged as a highly promising paradigm for LLM agent systems, providing lightweight and reliable categorical reasoning for rapid System One execution. However, existing generation-oriented inference engines fail to support these workloads effectively, suffering from severe prefix…