RESEARCH

---

My research interests lie at the intersection of Natural Language Processing, Computer Vision, and Machine Learning. Specifically, my focus is on improving the reasoning abilities of large generative models, such as large language models and vision language models. My long-term objective is to allow these models to reason and generate new knowledge from their observations and interactions with users.

I enjoy exploring intriguing topics and discovering simple and elegant, yet general and effective approaches. I aspire to be a natural language processing artist.

Currently, I am working on (vision) language models. Note that these foci are temporary since the field is changing quickly.

PUBLICATIONS

---

RapVerse: Coherent Vocals and Whole-Body Motions Generations from Text

Preprint, 2024
Jiaben Chen, Xin Yan, Yihang Chen, Siyuan Cen, Qinwei Ma, Haoyu Zhen, Kaizhi Qian, Lie Lu, Chuang Gan
[project] [paper] [code] [dataset] [BibTeX]

3D-VLA: A 3D Vision-Language-Action Generative World Model

ICML, 2024
Haoyu Zhen, Xiaowen Qiu, Peihao Chen, Jincheng Yang, Xin Yan, Yilun Du, Yining Hong, Chuang Gan
[project] [paper] [code] [twitter] [BibTeX]

ContPhy: Continuum Physical Concept Learning and Reasoning from Videos

ICML, 2024
Zhicheng Zheng*, Xin Yan*, Zhenfang Chen*, Jingzhou Wang, Qin Zhi Eddie Lim, Joshua B. Tenenbaum, Chuang Gan
[project] [paper] [code] [dataset] [BibTeX]

Centroid-centered Modeling for Efficient Vision Transformer Pre-training

PRCV, 2024
Xin Yan, Zuchao Li, Lefei Zhang, Bo Du, Dacheng Tao
[paper] [code] [BibTeX]

EXPERIENCE

---

2024-Now     Yuntian Group @ UWaterloo

2024-Now     01.AI

2023-2024    MIT-IBM Watson AI Lab

2022-2023    Wuhan University

AWARDS

---

Vector Scholarship, by Vector Institute, Canada (2024-2025)

Jun Lei Scholarship, by Xiaomi Inc, China (2023)

Aeon Scholarship, by Aeon Co., Ltd., Japan (2022)

National Scholarship, by Ministry of Education, China (2021)

Student Scholarship, by Wuhan University, China (2023, 2022, 2021)

SERVICES

---

Reviewer    ACL ARR (2024), EMNLP (CustomNLP4U Workshop 2024), ICLR (2025)