Research Papers A research papers website with original explainers for important AI and frontier science papers.
Latest
DeepSeek-V4 ships two MoE models, 1.6T/49B and 284B/13B, with compressed sparse plus heavily compressed attention that cuts 1M-context KV cache to 10% of V3.2, mHC residuals and Muon.
DeepSeekMath 7B reached 51.7% on MATH with no tools or voting by pretraining on 120B web math tokens and then running GRPO, a PPO variant that deletes the value model and scores answers against their own sampling group.
Kimi K3 is a 2.8T-parameter MoE with 104B active, native vision and 1M context, built on 3:1 Kimi Delta Attention and gated MLA, attention residuals and a 896-expert LatentMoE, at 2.5x K2's scaling efficiency.
Fine-Tuning & Adaptation · MIT
MIT shows LoRA and full fine-tuning reach similar accuracy through different weights: LoRA adds intruder dimensions that carry most of its forgetting and can be scaled down after training at almost no accuracy cost.
Moonshot AI shows Muon, a matrix-orthogonalizing optimizer, reaches AdamW's loss with about half the compute once you add weight decay and rescale its updates; Moonlight, a 16B MoE trained on 5.7T tokens, is the proof.
Nemotron 3 Ultra is a 550B-total, 55B-active hybrid Mamba-attention MoE pretrained in NVFP4 on 20T tokens; it gives up a few points to the strongest open models for up to 5.9x higher decode throughput.
Reinforcement Learning · Michigan State University
Muon's uniform whitening wins pretraining but collapses under RL with verifiable rewards and lags on robot policies; Pion keeps dominant directions, suppresses the noisy tail, and beats both Muon and AdamW.
Language Models · Alibaba Qwen Team
Qwen3.8-Flash-Next is a 125B MoE with 6B active plus 51B host-memory n-gram tables; a 3:1 GDN-attention hybrid, sparse attention, gated residuals and Muon match the 397B Qwen3.7-Plus at 1/9 the FLOPs.
DragMesh-2 opens doors and drawers with a 51-DoF hand and no actuator on the object joint, so motion comes only from contact. PICA training hits 0.89 success at nominal damping and 0.56 at 4x, with no tactile sensing.
Multimodal Models · Shanghai AI Laboratory
InternVideo3 is an 8B video model that scores 73.8 on Video-MME single-pass, then adds +2.7 from an agentic reasoning loop on top. M2LA attention holds 768K tokens on one H200 where the base OOMs at 512K.
Text-to-Image · Huazhong University of Science and Technology
Moebius is a 0.22B diffusion inpainter that rivals the 11.9B FLUX.1-Fill-Dev. On CelebA-HQ it scores FID 5.39 vs FLUX 10.13, using under 2% of the parameters and roughly 15x less total inference time.
Reinforcement Learning · Alibaba Qwen Team
Bebop proves multi-token prediction acceptance is bounded by rising RL entropy. A total-variation loss plus rejection sampling holds acceptance near 95%, flattens the entropy slope to -0.06, for up to 1.8x faster RL.
RATs lets a robot play before it is told what to do: agent teams propose their own goals, write code-as-policy, and bank wins as reusable skills. The library lifts LIBERO-PRO success from 23.2% to 43.8%.
Vision Foundation Models · Zhejiang University
A frozen-feature probe pits VLMs against video-generation models on spatial tasks. VLMs win semantics (92.08 mAP vs 69.89), video models win geometry (0.527 camera AUC vs 0.330), and a naive concat of the two beats both.
Reinforcement Learning · University of Zurich
RL with a chrF reward teaches an LLM to translate from in-context linguistic packets rather than memorize languages. On five unseen languages it averages 0.3335 chrF vs 0.2300 for SFT, which drops below the base model.
WorldCraft lets you click an object and drag its path inside a camera-controlled video world model. It hits 38.90px trajectory error vs DragAnything's 39.86 and keeps camera RPE at 0.131 via world-space paths and a LoRA.
ByG trains image and video editing models with no paired data and no reward model, winning 75.3% of head-to-head video votes against Ditto, which was supervised on millions of pairs.
LLM Reasoning · Shanghai Jiao Tong University
ReRe lets an MLLM answer a spatial question, then re-watch a synthesized novel-view video and revise. Training-free, it pushes Qwen3-VL-2B from 22.5 to 31.0 on VSI-Bench (+8.5).
Kairos packs a world model into 4B params with linear-time Hybrid Linear Temporal Attention, leads PAI-Bench at 80.84, and runs a 480P rollout in 11.4s on one RTX5090.
Efficient AI · Tsinghua University
SD3.5-T5 swaps Softmax attention for a Test-Time-Training layer that inherits pretrained Q/K/V/O weights, hitting GenEval 0.69 (vs 0.70 fine-tuned Softmax) after 3000 steps and running 1.32x faster at 1K, 1.47x at 2K.
Video Generation · Peking University
LoomVideo runs text-to-video, editing, and multi-image-to-video in one 5B model, matching 13B baselines on VBench (63.15 vs 63.01) and editing 5.41x faster by adding the source latent instead of concatenating it.
Retrieval-Augmented Generation · KAIST
CARVE picks a modality-granularity config per video chunk instead of per query, lifting Recall@5 to 0.603 from a best baseline of 0.510 on V-RAGBench, a leak-filtered egocentric VideoRAG benchmark.
SDPG adds a full-vocabulary self-distillation loss to verifier RL, learning from a hint-conditioned teacher. On Qwen3-4B it lifts AIME25 from 0.242 (GRPO) to 0.327 and AMC23 from 0.714 to 0.870 last-checkpoint.
Vision-Language-Action · CASIA
World Pilot adds two world-model priors to a VLA policy and hits 84.7% total success on LIBERO-Plus zero-shot OOD, up 4.2 over the ABot-M0 base, and the scene prior works even from a world model never action-trained.
Vision-Language-Action · ACE Robotics
ACE-Ego-0 pretrains a VLA on 6,000+ hours mixing robot trajectories with human egocentric video turned into pseudo-actions. It averages 78.3% on six real bimanual tasks vs 71.7% for pi-0.5 and 35.6% for GR00T-N1.7.
Reinforcement Learning · Tencent
CPPO replaces PPO's one-size threshold with stricter early-token clipping and a running prefix-divergence budget, lifting Qwen3-30B-A3B-Base AIME from 49.23 (DPPO) to 54.79.
Code Generation · Renmin University of China
DeNovoSWE auto-constructs 4,818 verifiable whole-repository generation tasks. Fine-tuning Qwen3-30B-A3B on them lifts BeyondSWE-Doc2Repo pass rate from 0.058 to 0.472.
Reinforcement Learning · Xi'an Jiaotong University
Flow-DPPO swaps PPO ratio clipping for an exact per-step Gaussian KL term, lifting GenEval2 to 48.1 on SD3.5 (vs 39.9 for Flow-GRPO) while cutting policy drift roughly 4x.
First autoregressive-diffusion lip-sync method: distills a 14B bidirectional teacher into causal 1.3B/14B students that generate each chunk in 2 steps, hitting 31.58 FPS with sub-millisecond time-to-first-frame.
MemDreamer turns hours-long video QA into agentic retrieval over a 3-tier graph memory, lifting LVBench from 78.2 to 90.7 (+12.5) while the reasoning model reads ~6K tokens instead of 240K-784K.
AI Agents · The Chinese University of Hong Kong
Orchestra-o1 orchestrates text, image, audio, and video sub-agents and hits 72.8% on OmniGAIA with a GPT-5 brain (+10.3 over Gemini-3-Pro). Its trained 8B orchestrator reaches 30.0%, best among open omnimodal agents.
Vision Foundation Models · ETH Zurich
ZipSplat predicts 3D Gaussians from k-means scene tokens instead of one per pixel, hitting 24.14 PSNR on DL3DV with 249K Gaussians vs YoNoSplat's 22.01 at 1.2M, all pose-free.
Multimodal Models · Fudan University
ARM is a 7B autoregressive model that does image understanding, text-to-image, and editing in one next-token framework, on a shared discrete tokenizer; RL lifts GenEval 0.79 to 0.86 and GEdit-EN overall 5.75 to 6.68.
A Tencent YouTu Lab survey maps the chatbot-to-agent shift on two axes: cognitive core (Chatbot then Thinking LLM) and task execution (Agent then Workspace plus Skill), arguing persistent state is the real leap.
OmniDreams is a Cosmos-based generative driving simulator that renders 68 FPS single-view on one GB300 and replaces NuRec in a closed-loop AV stack, cutting closed-loop incidents from 10.1% to 4.7%.
Reinforcement Learning · Tencent
DRPO swaps DPPO's hard divergence mask for a smooth advantage-weighted quadratic regularizer, keeping the Binary-TV trust region but with bounded gradient weights, and trains Qwen3 LLMs more stably under FP8.
AI Agents · City University of Hong Kong
RHO tunes an LLM agent harness from past unlabeled trajectories using self-consistency and pairwise self-preference, lifting SWE-Bench Pro from 59% to 78% in one round with no external grading.
SCAIL-2 feeds the raw driving video into the generation sequence instead of a pose skeleton, cutting FVD to 287 vs 305 for Wan-Animate on Studio-Bench, with one model covering animation and replacement.
SearchSwarm fine-tunes Tongyi DeepResearch-30B-A3B on harness-generated delegation trajectories, lifting BrowseComp from 43.4 to 68.1 and topping every 30B-A3B model on four deep-research benchmarks.
Vision-Language-Action · X Square Robot
WALL-WM organizes VLA pretraining around semantic action events, not fixed-length chunks. Its event mode scores 75.86 Task Progress on diverse real-robot manipulation versus 55.64 for pi0.5.
A CASIA survey maps agentic environments for LLM agents along eight attribute axes and eight domains, unifying synthesis, evaluation, and co-evolution. Sharpest finding: environments barely fit multi-agent settings.
Reinforcement Learning · Alibaba Qwen Team
APPO branches RL rollouts at high-uncertainty, high-influence tokens instead of tool-call boundaries, lifting Qwen2.5-7B by 3.9 points over ARPO across 13 math, multi-hop, and deep-search benchmarks.
Reinforcement Learning · Alibaba Qwen Team
Z-Reward predicts a distribution over rubric scores instead of one scalar. A 9B student hits 88.6% human-preference accuracy with a single output token, and downstream T2I tuning gains 41.3% net GSB over SFT.
Long Context · University of Maryland
LCLMs train a 0.6B encoder and 4B decoder jointly to compress long context into soft tokens at 1:4, 1:8 and 1:16, cutting prefill memory and time-to-first-token while staying close to the uncompressed baseline.
Agent Memory · National University of Singapore
MRAgent gives LLM agents a Cue-Tag-Content memory graph and lets the model reason while it traverses it, lifting LoCoMo LLM-Judge from 68.3 to 84.2 while cutting tokens to 118k per sample.
Video Generation · Tsinghua University
OmniDirector copies camera motion from a reference video into new generations and cuts rotation error to 2.64 degrees, beating CamCloneMaster (4.11), with no cross-paired training data.
AI Agents · University of Science and Technology of China
Role-Agent makes a single LLM act as agent and environment at once, generating its own process reward and curriculum. It beats GiGPO by 4.2% on ALFWorld and 6.9% on WebShop with Qwen2.5-1.5B.
A JKU Linz study runs xLSTM, Mamba-2, and Gated DeltaNet through code, time-series, and synthetic tasks, then traces xLSTM's lead to two primitives: counting-style accumulation and finite-state tracking.
AI Agents · Tsinghua University
EurekAgent argues the bottleneck is agent environment design, reporting 2.635999 on 26-circle packing, 2005.03 us TriMul, and 85.71% any-medal on a seven-task MLE-Bench subset.
AI Agents · Renmin University of China
FORT-Searcher trains a 3B-active search agent on shortcut-resistant tasks, scoring 66.2 overall among comparable open agents and delaying answer hit time from 18.7 to 46.9 versus REDSearcher data.
Multimodal Models · Nanjing University
HYDRA-X unifies image and video tokenization in one ViT; tubelet attention and hierarchical temporal patchify improve DAVIS rFVD to 11.19 and editing overall to 4.34.
Reinforcement Learning · Zhejiang University
N-GRPO perturbs rollout embeddings with semantic neighbors, lifting DeepSeek-R1-Distill-Qwen-1.5B average Pass@32 to 79.17 and AIME25 Pass@32 to 50.28.
Robust-U1 trains an MLLM to reconstruct corrupted visual content, reaching 0.7398 overall on R-Bench versus 0.5770 for BAGEL and 0.5017 for Robust-R1.
AI Agents · Zhejiang University
WeaveBench tests 114 real hybrid GUI plus CLI tasks; the best frontier pairing reaches only 41.2% PassRate, and final-only grading overstates GPT-5.5 by 20.2 points.
World Models · Alibaba Qwen Team
ABot-Earth 0.5 uses satellite imagery to generate 3D Gaussian Splatting city scenes, reporting under 10 minutes per square kilometer and FID 16.1.
Agent Memory · National University of Singapore
EvoArena turns static agent tasks into evolving chains and finds current agents average only 39.6% accuracy; EvoMem adds patch memory and improves chain-level accuracy by 3.7 points.
Google DeepMind's report lays out four non-exclusive paths from AGI to ASI and treats each bottleneck, from data walls to regulation, as an open research question.
Text-to-Image · The Chinese University of Hong Kong
InterleaveThinker adds planner and critic agents around frozen image generators, reaching 66.3 to 67.2 average on UEval and lifting FLUX.2-klein WISE from 0.47 to 0.73.
Multimodal Models · Kuaishou Technology
Kwai Keye-VL-2.0 is a 30B-A3B open MoE multimodal model with 256K context, strong long-video scores, and 62.0 on SWE-bench Verified.
Vision-Language-Action · Zhejiang University
LabVLA trains a Qwen3-VL-4B backbone plus DiT action expert on laboratory workflows and reports 71.1% ID and 70.0% OOD success on LabUtopia.