Latest News

• 10.2026Gave an invited talk on Coding Agents via Recursive Self-Improvement at the CCBR 2026 Doctoral Forum.
• Presented Routing-Free MoE at EMNLP 2026, Budapest, Hungary.
• 09.2026Gave a talk on Mendel Gödel Machine at NICE Seminar #198.
• 08.2026Routing-Free MoE and PyraVid have been accepted to EMNLP 2026 main conference!
• 06.2026Our research projects received hessian.AI compute support.
• 04.2026LLM Safety From Within has been accepted to ACL 2026 main conference!
• 03.2026Presented PERFT at EACL 2026, Rabat, Morocco.

Selected Publications

Routing-Free Mixture-of-Experts

Yilun Liu*, Jinru Han*, Sikuan Yan, Volker Tresp, Yunpu Ma (*equal contribution)

EMNLP 2026 Main Conference [Paper][Code]

• Routing-Free MoE architecture, eliminating routers, Softmax, TopK, and hard-coded load balancing
• Unified, adaptive load-balancing framework jointly optimizing token- and expert-balancing
• Improved performance and efficiency over baselines

PyraVid: Hierarchical Multimodal Memory for Long-Horizon Video Reasoning

Sikuan Yan, Sicheng Dong, Haotong Wang, Ercong Nie, Yilun Liu, Jinhe Bi, Yingjie Xu, Susanna Schwarzmann, Riccardo Trivisonno, Volker Tresp, Yunpu Ma

EMNLP 2026 Main Conference [Paper]

• Hierarchical multimodal memory with a coarse-to-fine pyramid structure for long-horizon video reasoning
• Structure-guided memory expansion with pruning for causally related but semantically distant events
• Consistent gains across long-video benchmarks, model scales, and question types

LLM Safety From Within: Detecting Harmful Content with Internal Representations

Difan Jiao, Yilun Liu, Ye Yuan, Zhenwei Tang, Linfeng Du, Haolun Wu, Ashton Anderson

ACL 2026 Main Conference [Paper][Code][Poster]

• SIREN, a plug-and-play component harnessing LLM internal representations for harmfulness detection
• Outperforming dedicated safety guardrails in performance, generalization, and efficiency

PERFT: Parameter-Efficient Routed Fine-Tuning for Mixture-of-Expert Model

Yilun Liu*, Yunpu Ma*, Shuo Chen, Zifeng Ding, Bailan He, Zhen Han, Volker Tresp (*equal contribution)

EACL 2026 Findings [Paper][Code][Poster]

• Dynamics between memory vectors in experts and expert vectors in routers when PEFTing to MoE LLMs
• Unified framework for integrating PEFT with MoE LLMs
• PERFT, a family of effective and scalable adaptation strategies

SPIN: Sparsifying and Integrating Internal Neurons in Large Language Models for Text Classification

Difan Jiao*, Yilun Liu*, Zhenwei Tang, Danial Matter, Jürgen Pfeffer, Ashton Anderson (*equal contribution)

ACL 2024 Findings [Paper][Code][Poster]

• Lightweight plug-and-play text classification frameworkusing LLM internal representations
• Superior performance, efficiency, and interpretability compared to conventional methods

Recent Works

Learning Chaos Without Seeing Chaos: Extrapolation of Global Dynamics in Autoregressive Transformers

Yilun Liu, Yi Zhang, Ganyu Wu, Sikuan Yan, Mengyue Wang, Alois Knoll, Volker Tresp, Yunpu Ma

arXiv preprint [Paper]

• Small autoregressive transformers trained only on restricted parameter regimes of dynamical systems recover unseen global organization under closed-loop extrapolation
• Causal attention interventions show how control-parameter information shapes the resulting dynamics

Mendel Gödel Machine: Recursive Self-Improving Coding Agents via Comparative Evolution

Changzhi Liu*, Yilun Liu*, Sikuan Yan, Volker Tresp, Yunpu Ma (*equal contribution)

arXiv preprint [Demo][Paper][Code]

• Self-improving agents with additional comparative signals derived from the archive of past attempts
• Better performance and efficiency, with 35B open models outperforming 117× larger frontier models
• Generalizablity across benchmarks and models, yielding genuinely reusable and scalable scaffolds

Behavioral and Representational Evidence of Binomial Ordering Preferences in Large Language Models

Zhiqing Yang*, Yilun Liu*, Yunpu Ma, Volker Tresp, Hinrich Schütze (*equal contribution)

arXiv preprint [Paper][Code]

• Evaluation protocol of binomial ordering preference for LLMs, with a dataset of 600 binomial pairs across 8 languages
• LLMs behaviorally align with the empirically preferred direction more reliably than the strength, and that strength can be representationally located and manipulated

Model-Aware Schedules Improve Generation via Fiberwise Optimal Transport

Luyi Jia, Boyan Zhang, Yilun Liu, Steffen Rulands

arXiv preprint [Paper]

• Model-aware schedule construction via fiberwise optimal transport, combining prediction risk with kinetic action for closed-form time allocation, yielding consistent gains over strong baselines
• Normalized risk profiles and schedule deformations align across models, with a frozen analytic form retaining most gains

Educational Background

Research & Computing Support

Munich Center for Machine Learning (MCML) & Leibniz Supercomputing Centre (LRZ)

German Federal Ministry of Research, Technology and Space (BMFTR)  ·  Bavarian State Ministry of Science and the Arts (StMWK)

Hessian Center for Artificial Intelligence (hessian.AI)

German Federal Ministry of Research, Technology and Space (BMFTR)  ·  Hessian Ministry for Digitalisation and Innovation (HMD)

Erlangen National High Performance Computing Center (NHR@FAU)

German Federal Government  ·  Bavarian State Government  ·  German Research Foundation (DFG)  ·  Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU)

JUPITER Exascale Supercomputer

European High Performance Computing Joint Undertaking (EuroHPC JU)  ·  German Federal Ministry of Research, Technology and Space (BMFTR)  ·  Ministry of Culture and Science of the State of North Rhine-Westphalia (MKW NRW)

Academic Service