Xingyao Wang (王星尧)

Pronouns: he/him/his. How to pronounce?

prof_pic.jpg

I am co-founder and Chief AI Officer at All Hands AI All Hands AI, building open source AI agents OpenHands Stargazers for developers.

Before I join All Hands AI, I was a PhD candidate in Computer Science at the University of Illinois Urbana-Champaign, advised by Prof. Heng Ji. I also work closely with Prof. Hao Peng.

I’m interested in developing interactive language agents powered by foundation models like large language models (LLMs) and vision-language models (VLMs). These agents can interact with both computer and physical environments through programming languages, assisting humans and continuously self-improving based on environmental observations and human language feedback.

I received my undergraduate degree at the University of Michigan in computer science and data science. Previously, I was an intern at Google Research (2023, multimodal LLM pre-training), Microsoft (2022, unit-test generation), Bytedance (2021, lightseq Stargazers ), and Tencent (2020, distributed deep learning).

news

2024 Jun Excited to share that R-Tuning receives outstanding paper award from NAACL 2024!
2024 Apr Check out our newly released paper, introducing a SOTA reasoning generalist Eurus model and UltraInteract (data)! UltraInteract collects trees (We love trees 🌲!) of multi-turn interactions with preference pairs that support both SFT and preference learning for challenging reasoning problems!
2024 Feb LLM agents act better when they take actions by generating Python code: Check out our new reprint on an LLM agent (CodeAct) deeply integrated with Python! Feel free to chat with our demo here!
2024 Jan MINT and CRAFT are accepted to ICLR 2024! See y’all in Vienna this May!
2023 May Excited to share that I’ll be joining Google Research NYC this summer as a student researcher!

selected publications

2024

  1. Preprint
    OpenHands: An Open Platform for AI Software Developers as Generalist Agents
    Xingyao Wang, Boxuan Li, Yufan Song, Frank F. Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H. Tran, Fuqiang Li, Ren Ma, Mingzhang Zheng, Bill Qian, Yanjun Shao, Niklas Muennighoff, Yizhe Zhang, Binyuan Hui, Junyang Lin, Robert Brennan, Hao Peng, Heng Ji, and Graham Neubig
    Arxiv preprint, Jul 2024
    Paper Code
  2. ICML 2024 LLMAgents
    @ ICLR 2024     Oral
    Executable Code Actions Elicit Better LLM Agents
    Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji
    ICML 2024; LLMAgents Workshop @ ICLR 2024 (Oral), Feb 2024
    Paper Code Slide Blog X/Twitter Thread Poster Demo
  3. TMLR 2024
    A Single Transformer for Scalable Vision-Language Modeling
    Yangyi Chen*, Xingyao Wang*, Hao Peng, and Heng Ji
    Transactions on Machine Learning Research, 2024, Jul 2024
    Paper Code X/Twitter Thread
  4. ICLR 2024
    MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback
    Xingyao Wang*, Zihan Wang*, Jiateng Liu, Yangyi Chen, Lifan Yuan, Hao Peng, and Heng Ji
    In Proceedings of the International Conference on Learning Representations, May 2024
    Paper Website Code Blog X/Twitter Thread Poster
  5. ICLR 2024
    CRAFT: Customizing LLMs by Creating and Retrieving from Specialized Toolsets
    Lifan Yuan*, Yangyi Chen*, Xingyao Wang, Yi R. Fung, Hao Peng, and Heng Ji
    In Proceedings of the International Conference on Learning Representations, May 2024
    Paper Code
  6. NAACL 2024
    (Findings)     DaSH @
    NAACL 2024     Oral
    LETI: Learning to Generate from Textual Interactions
    Xingyao Wang, Hao Peng, Reyhaneh Jabbarvand, and Heng Ji
    In Findings of the Association for Computational Linguistics: (NAACL); DaSH Workshop @ NAACL 2024 (Oral), Jun 2024
    Paper Code Slide X/Twitter Thread Poster

2023

  1. ACL 2023
    Code4Struct: Code Generation for Few-Shot Event Structure Prediction
    Xingyao Wang, Sha Li, and Heng Ji
    In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Jul 2023
    Paper Code X/Twitter Thread Poster

awards

Outstanding Paper, NAACL 2024 2024
Yunni and Maxine Pao Memorial Fellowship 2024-2025
Honorable Mention, Computing Research Association Outstanding Undergraduate Researcher Award 2022
James B. Angell Scholar, University of Michigan 2021
EMNLP 2023 Student Travel Award (5 out of 94) 2021
University Honors, University of Michigan. 2017-2021

experiences

Google Research
Student Researcher 		Summer 2023
New York City, NY, USA
Microsoft
Research Intern 		Summer 2022
Remote, USA
ByteDance
Natural Language Processing Intern 		Summer 2021
Beijing, China
Tencent
Software Engineering Intern 		Summer 2020
Shenzhen, Guangdong, China