Computer Science · Artificial Intelligence

Zheng Luo

I am currently a Master's student at University of Southern California majoring in Computer Science (Artificial Intelligence). I am supervised by Prof. Xiyang Hu from ASU and Prof. Yue Zhao from USC in LLM research. I earned my bachelor's degree from Shanghai Jiao Tong University (UM-SJTU Joint Institute) where I majored in Electrical and Computer Engineering and the University of Michigan where I majored in Computer Science.

Portrait of Zheng Luo

Research Interest

My long-term research interest can be summarized as general intelligence like LLMs that are more efficient, more reliable, more predictable, and more "logically beautiful". I envision my ultimate goal to not be adding incremental improvements on existing systems (like transformer architecture) regarding efficiency, etc., but to found a fundamentally new paradigm that eliminates the structural limitations of the traditional connectionist (deep learning) paradigm, like being data-hungry and lacking safety guarantee, by making bold sacrifices, like devoting much more manual effort in addition to automatic learning. However, this is very hard and potentially requires long-term trials and errors.

Therefore, to make things realistic, I am open to any theoretical (as opposed to applied) AI fields as research directions for my Ph.D studies, especially LLM training and fine-tuning, and multimodal generative models including image, video and 3D model generation, and seek to learn the frontier approaches, be up-to-date with latest ideas, and gradually migrate useful insights to my own research direction.

My short-term research plan for my Ph.D studies can be found here.

Publications and Research Experiences

Thumbnail image for lost in execution

Lost in Execution: On the Multilingual Robustness of Tool Calling in Large Language Models

Published - ACL 2026First Author

Download paper · Venue page

Project description
Detail image for lost in execution

Abstract

Large Language Models (LLMs) are increasingly deployed as agents that invoke external tools through structured function calls. While recent work reports strong tool-calling performance under standard English-centric evaluations, the robustness of tool calling under multilingual user interactions remains underexplored. In this work, we introduce MLCL, a diagnostic benchmark, and conduct a systematic evaluation of multilingual tool calling across Chinese, Hindi, and the low-resource language Igbo. Through fine-grained error analysis, we show that many failures occur despite correct intent understanding and tool selection. We identify parameter value language mismatch as a dominant failure mode, where models generate semantically appropriate parameter values in the user’s language, violating language-invariant execution conventions. We further evaluate several inference-time system strategies and find that while these strategies substantially reduce language-induced execution errors, none of them can fully recover English-level performance.

Project description

Lost in Execution studies multilingual robustness in LLM tool calling. The paper introduces MLCL, a diagnostic benchmark across English, Chinese, Hindi, and Igbo, and shows that models may understand intent and select the right tool but still fail because parameter values are not executable. The key failure mode is parameter-value language mismatch, where semantically correct non-English arguments violate tool-interface conventions.

I am the first author of this ACL 2026 paper. I built the benchmark, designed the fine-grained error taxonomy, analyzed failures across intent, tool selection, parameter extraction, and execution mismatch, and evaluated mitigation strategies. This project highlighted my ability to connect benchmark design with practical agent reliability.

Thumbnail image for treempo

TreeMPO: Fine-Grained Credit Assignment for LLM Training via Token-Level Trajectory Branching

ManuscriptFirst Author

Download paper

Project description
Detail image for treempo

Abstract

We present TreeMPO, a reference-policy-free framework for fine-grained credit assignment in large language model training. Rather than assigning a single scalar reward to an entire reasoning trajectory, TreeMPO constructs a rollout tree, estimates segment-level contribution from mixed-success leaves, and converts these estimates into segment-resolved token-level advantages. The method targets mathematical reasoning while remaining compatible with tool-calling traces that interleave natural-language reasoning, fenced Python calls, tool responses, and final boxed answers. Compared with methods such as TEMPO that exploit shared prefixes among sampled trajectories, TreeMPO actively creates new continuations through guided token-level branching, concentrating exploration at structurally useful and model-plausible decision points. Compared with TreeRPO-style tree credit assignment based on empirical outcome comparisons, TreeMPO uses a probabilistic MAP estimator that estimates signed segment contribution before producing token-level advantages. Experiments use a resource-efficient split pipeline that rolls out from the base policy, generates advantage-weighted training data, fine-tunes LoRA adapters, and validates checkpoint snapshots. Results show consistent but modest validation gains across several model families, stronger gains in some tool-calling-compatible settings, and mixed transfer to broader held-out tests.

Project description

TreeMPO studies fine-grained credit assignment for long LLM reasoning trajectories. Instead of assigning one reward to an entire response, it branches token-level trajectories, estimates segment-level contribution from mixed-success descendants, and converts the evidence into token-level advantages for LoRA/RL fine-tuning. The method supports mathematical reasoning and tool-calling traces, with consistent validation gains across several model families.

I am the first author of this manuscript in preparation. I designed the core TreeMPO algorithm, including the token-level branching strategy, contribution model, and advantage-generation pipeline, and built the full system for rollout collection, trajectory-tree construction, LoRA fine-tuning, and checkpoint validation under constrained GPU resources.

Thumbnail image for street level

Street-Level Competitive Ride-Hailing with Fixed-Rival Defensive PSRO

Submitted - AAAI 2027First Author

Download paper

Project description
Detail image for street level

Abstract

Competitive ride-hailing has typically been modeled at the zone level, where fleet decisions are optimized over coarse spatial aggregates. We study whether moving to street-level, per-vehicle competition changes the practical learning problem enough to justify a different modeling stack. We introduce a SUMO-backed street-level competitive ride-hailing framework calibrated with NYC TLC trip records, then evaluate three policy layers: a fixed-rule greedy baseline, a feature-engineered street-level controller, and exploratory U-Net / hybrid residual extensions that test whether learned local corrections can improve the structured controller. We also propose Fixed-Rival Defensive Policy-Space Response Oracles (FRDF-PSRO), a fixed-rival PSRO-style protocol that reweights opponent mixtures toward strategically difficult rival variants. In Manhattan-scale simulations, the street-level feature-engineered controller outperforms the tested zone-level baselines and improves over the tested fixed-price greedy baseline, while FRDF-PSRO improves the selected fixed-rival robustness objective relative to fixed-rival standard PSRO. These robustness gains are scenario-dependent and do not imply universal dominance in direct comparison against the strongest standard-PSRO anchor. The residual study is correspondingly exploratory: some small gains are possible, but they are sensitive to architecture and optimization regime. Overall, the results support street-level competitive simulation and structured street-level control as useful tools for studying strategic ride-hailing behavior.

Project description

Street-Level Competitive Ride-Hailing with Fixed-Rival Defensive PSRO studies competitive fleet management at street-level resolution rather than coarse zone level. The paper introduces a SUMO-backed competitive ride-hailing simulator calibrated with NYC TLC and OpenStreetMap data, evaluates structured and residual policy designs, and proposes Fixed-Rival Defensive PSRO for robustness against strategically difficult rivals.

I am the first author of this AAAI 2027 submission. I developed the simulation framework, designed street-level controller features for relocation, pricing, and service decisions, implemented Manhattan-scale evaluation pipelines, and proposed the FRDF-PSRO protocol for robust multi-agent decision-making under competitive pressure.

Thumbnail image for fairness or fluency

Fairness or Fluency? An Investigation into Language Bias of Pairwise LLM-as-a-Judge

Submitted - NeurIPS 2026 Workshop JUDGe

Download paper

Project description
Detail image for fairness or fluency

Abstract

Recent advances in Large Language Models (LLMs) have incentivized the development of LLM-as-a-judge, an application of LLMs where they are used as judges to decide the quality of a certain piece of text given a certain context. However, previous studies have demonstrated that LLM-as-a-judge can be biased towards different aspects of the judged texts, which often do not align with human preference. One of the identified biases is language bias, which indicates that the decision of LLM-as-a-judge can differ based on the language of the judged texts. In this paper, we study two types of language bias in pairwise LLM-as-a-judge: (1) performance disparity between languages when the judge is prompted to compare options from the same language, and (2) bias towards options written in major languages when the judge is prompted to compare options of two different languages. We found that for same-language judging, there exist significant performance disparities across language families, with European languages consistently outperforming African languages, and this bias is more pronounced in culturally-related subjects. For inter-language judging, we observe that most models favor English answers, and that this preference is influenced more by answer language than question language. Finally, we investigate whether language bias is in fact caused by low-perplexity bias, a previously identified bias of LLM-as-a-judge, and we found that while perplexity is slightly correlated with language bias, language bias cannot be fully explained by perplexity only.

Project description

Fairness or Fluency? studies language bias in pairwise LLM-as-a-judge evaluation. The paper finds performance disparities across language families and shows that many models prefer English answers in cross-language comparisons. It further tests whether this bias is explained by answer perplexity, finding that perplexity is correlated with bias but does not fully explain language preference.

I contributed to this NeurIPS 2026 Workshop JUDGe submission as a co-author. My role focused on designing and implementing experiments that relate judge preference to answer perplexity, using regression-based analysis to separate language identity from fluency effects, and improving perplexity collection for more reliable multilingual evaluation.

Thumbnail image for sim 2 real

When Simulation Lies: A Sim-to-Real Benchmark and Domain-Randomized RL Recipe for Tool-Use Agents

Published - NeurIPS 2026 Evaluations and Datasets Track, poster

Download paper · arXiv page

Project description
Detail image for sim 2 real

Abstract

Tool-use language agents are evaluated on benchmarks that assume clean inputs, unambiguous tool registries, and reliable APIs. Real deployments violate all these assumptions: user typos propagate into hallucinated tool names, a misconfigured request timeout can stall an agent indefinitely, and duplicate tool names across servers can freeze an SDK. We study these failures as a sim-to-real gap in the tool-use partially observable Markov decision process (POMDP), where deployment noise enters through the observation, action space, reward-relevant metadata, or transition dynamics. We introduce RobustBench-TC, a benchmark with 22 perturbation types organized by these four POMDP components, each grounded in a verified GitHub issue or documented tool-calling failure. Across 21 models from 1.5B to 32B parameters (including the closed-source o4-mini), the robustness profile is sharply uneven: observation perturbations reduce accuracy by less than 5%, while reward-relevant and transition perturbations reduce accuracy by roughly 40% and 30%, respectively; scale alone does not close these gaps. We then propose ToolRL-DR, a domain-randomization reinforcement learning (RL) recipe that trains a tool-use agent on perturbation-augmented trajectories spanning the three statically encodable POMDP components. On a 3B backbone, ToolRL-DR-Full retains roughly three-quarters of clean accuracy and reaches an aggregate perturbed accuracy comparable to open-source 14B function-calling baselines while substantially narrowing the gap to o4-mini. It closes approximately 27% of the transition gap despite never seeing transition perturbations in training, suggesting that RL on adversarial static tool-use inputs induces a more persistent retry policy that transfers to unseen runtime failures.

Project description

When Simulation Lies / RobustBench-TC frames tool-use robustness as a sim-to-real problem. It introduces a production-grounded benchmark with perturbations organized by POMDP components: observation, action space, reward-relevant metadata, and transition dynamics. Across 21 models, the study finds that reward-relevant and transition perturbations cause much larger failures than simple observation noise, and proposes ToolRL-DR to narrow robustness gaps through perturbation-augmented RL.

I contributed to this NeurIPS 2026 Evaluations and Datasets Track poster as a co-author. My work included implementing user-query perturbations on ACEBench, supporting robustness experiments under deployment-like noise, and contributing to the analysis showing that perturbation-augmented RL can improve tool-use robustness beyond clean benchmark accuracy.

Thumbnail image for latent skill

SkiLT: What Agent Reinforcement Learning Actually Learns from a Latent Skill Library

Submitted - AAAI 2027

Download paper

Project description
Detail image for latent skill

Abstract

Retrieved skill libraries improve large language model agents, but rendering skills as text adds hundreds of prompt tokens at every decision step, and reinforcement learning pays that cost on every rollout. SkiLT asks whether the same conditioning survives full-actor, episode-reward RL in a compact latent form, and what such a library then contributes. A Composer initialized from the actor’s own base model compresses each of 44 SkillRL library entries into eight soft vectors, which are installed as retrievable vocabulary rows before the full actor is trained with GRPO. On the complete ALFWorld evaluation sets SkiLT reaches 90.0 valid-seen and 86.6 valid-unseen success against 87.9/79.9 for a released text-skill RL checkpoint under the same harness, with the skill section reduced from approximately 970 to approximately 140 tokens per step. That checkpoint was trained under a different recipe, so it anchors rather than controls the comparison, and a second SkiLT run reads 87.9/79.9, leaving the out-of-distribution margin unestablished. The mechanism study then revises the reading of this result. The installed rows were effectively frozen during the flagship run, with relative drift no greater than 2.4 times 10^-6: their activation-scale norms starved them under Adam and pushed nearly every update below the bfloat16 storage boundary. A 2x2 factorial isolates both failure modes, and a function-preserving repair restores row learning at RL scale. Yet substitutions drawn from the deployed bank or its empirical statistics, carrying no correct per-entry identity, are indistinguishable from it at the tests’ approximately plus-or-minus 7-point resolution, while removing the conditioning costs 21 points and retraining from an isotropic norm-matched bank does not recover. Compact latent conditioning therefore reaches text-skill RL performance under this harness, but we detect no contribution from code identity or per-entry distinctness, and the load-bearing property of the conditioning remains unidentified.

Project description

SkiLT studies whether retrieved text-skill libraries for LLM agents can be compressed into compact latent skill tokens during reinforcement learning. It reduces the skill section from roughly 970 tokens to 140 tokens per step while maintaining performance comparable to text-skill RL checkpoints on ALFWorld. The mechanism analysis further shows that the actor may adapt around nearly fixed skill rows, complicating the interpretation of what latent skill conditioning actually contributes.

I contributed to this AAAI 2027 submission as a co-author. My work included designing and implementing latent-form skill compression, integrating compact skill tokens into the GRPO workflow, evaluating against text-skill RL baselines, and helping analyze training dynamics to understand what agent RL actually learns from auxiliary skill memory.

Project Experiences

"Crafty Piggies 3D" Game Development

Game Trailer

Demo Day Trailer Reaction

Play the game on Windows and Mac

"Crafty Piggies 3D" is a physics-based 3D game that allows players to build customized vehicles through a grid-based system and control their vehicles to solve puzzles and reach destinations. It is inspired by Bad Piggies and Genshin Impact, where the former features a 2D grid-based building system and level-based progression, and the latter features an open world and a dedicated storyline. We tried to fuse the two genres together to create a game that is both mechanically fun and appealing in storytelling.

This is a capstone project for graduating from the University of Michigan, where I worked with Yikai Li, Gabriel Froehner and Javier Guerrero, instructed by Prof. Austin Yarger.

Table Tennis Feeding Machine

This is an Embedded Systems course project, where each team decides their own topic. Our team is composed of Zheng Luo (me), Zhiheng Alex Zhang, Xinting Jiang and Mingjian Li, and I was the team leader.

Our project features a vehicle with omniwheels topped with a pinch roller that feeds table tennis balls. It can move and rotate freely in all directions, adjust launching angle, launching speed and even spin.

One interesting fact about this project is that the pinch roller (the top white part of the vehicle) is fully handmade by us, with a 3D printed barrel, embedded with two brushless motors that spin independently and accumulate inertia for a full-speed launching. Stepper motors are used for pitch angle and launch triggering. It was a quite crazy idea from me, given that we have to make every piece to function correctly, or otherwise the whole project will fail.

"Defend the Republic" Autonomous Blimp Competition

This competition involves building a floating blimp that can collect balloons in the air and transport them to a goal. I learned about rule-based control algorithms like PID control and the feasibility of using embedded-system-powered YOLO algorithm for object detection.

Personal Hobbies

AI Short Play Making

An AI short play depicting a possible parallel world where humans were fooled by aliens into believing they have invented LLM, but it turns out that the intelligent words from LLMs are synthesized by the aliens and presented to the human eyes. This is to banter the fact that we have absolutely no idea whether a deep learning method will work until we see the experiment results, and when a method turns out to be working, we go back to justify it with all kinds of theories, while if the outcome were bad, we would throw the same method into trashcan and use theories to justify why it doesn't work.

Game Engine Development

I contributed to the Fyrox game engine, including exposing physics-related APIs from downstream dependencies and miscellaneous refactoring and cleanup.

This is a physics test on the Fyrox game engine regarding my "Crafty Piggies 3D" game idea.

Fyrox game engine physics test

I aim to build my own game engine from scratch so that I can have full control over the physics and rendering details, and also seek for the possibility of developing a "game engine development skill" for AI agents so that every new indie game can have a customized engine tailored to its unique needs.

Personal Music Box Project

Appearance and Internals Display

Functionality Demo

This is a personal project whose initial purpose was to be given to a girl I secretly liked, but the outcome beyond the project itself does not end well.

It is a music box that can play music, emit beautiful lights, has a spinning part, and plays a weird sound effect after the power is fully turned off (yes, this is a feature I'm really proud of). The initial design involves planet gears that speed down the brushless motor and enable two platform rings to spin in opposite directions. However, I eventually found that the friction and noise caused by 3D printed and laser-cut gears were too much, so I had to give up the fancy plan and connect the crystal piano to the brushless motor directly.

I got familiar with various skills through this project, including PCB milling, 3D printing and laser cutting.

PCB milling process3D printing processLaser cutting process

Transcripts and Scores

Skills

Programming Languages

  • Python
  • Rust
  • C++
  • C#
  • TypeScript

Machine Learning

  • PyTorch
  • Scikit-learn
  • LoRA fine-tuning
  • Model evaluation
  • Ablation studies

Software and Infrastructure

  • Git
  • Docker
  • Slurm / HPC
  • Apptainer
  • Linux

Simulation and Graphics

  • SUMO
  • Unity
  • Unreal Engine 5
  • Fyrox
  • Blender