2025
3D-Mem: 3D Scene Memory for Embodied Exploration and Reasoning
Yuncong Yang*, Han Yang*, Jiachen Zhou, Peihao Chen, Hongxin Zhang, Yilun Du, Chuang Gan
CVPR 2025
LSceneLLM: Enhancing Large 3D Scene Understanding Using Adaptive Visual Preferences
Hongyan Zhi*, Peihao Chen*, Junyan Li*, Shuailei Ma, Xinyu Sun, Tianhang Xiang, Yinjie Lei, Mingkui Tan, Chuang Gan
CVPR 2025
COMBO: Compositional World Models for Embodied Multi-Agent Cooperation
Hongxin Zhang*, Zeyuan Wang*, Qiushi Lyu*, Zheyuan Zhang, Sunli Chen, Tianmin Shu, Behzad Dariush, Kwonjoon Lee, Yilun Du, Chuang Gan
ICLR 2025
DELTA: Dense Efficient Long-range 3D Tracking for Any Video
Tuan Duc Ngo, Peiye Zhuang, Evangelos Kalogerakis, Chuang Gan, Sergey Tulyakov, Hsin-Ying Lee, Chaoyang Wang
ICLR 2025
Autonomous Agents from Automatic Reward Modeling and Planning
Zhenfang Chen*, Delin Chen*, Rui Sun*, Wenjun Liu*, Chuang Gan
ICLR 2025
TopoGaussian: Inferring Internal Topology Structures from Visual Clues
Xiaoyu Xiong, Changyu Hu, Chunru Lin, Pingchuan Ma, Chuang Gan, Tao Du
ICLR 2025
MatchMaker: Automated Asset Generation for Robotic Assembly
Yian Wang, Bingjie Tang, Chuang Gan, Dieter Fox, Kaichun Mo, Yashraj Narang, Iretiayo Akinola
ICRA 2025
2024
UBSoft: A Simulation Platform for Robotic Skill Learning in Unbounded Soft Environments
Chunru Lin*, Jugang Fan*, Yian Wang, Zeyuan Yang, Zhehuan Chen, Lixing Fang, Tsun-Hsuan Wang, Zhou Xian, Chuang Gan
CoRL 2024
ARCHITECT: Generating Vivid and Interactive 3D Scenes with Hierarchical 2D Inpainting
Yian Wang*, Xiaowen Qiu*, Jiageng Liu*, Zhehuan Chen, Jiting Cai, Tsun-Hsuan Wang, Yufei Wang, Zhou Xian, Chuang Gan
NeurIPS 2024
Physically Compatible 3D Object Modeling from a Single Image
Minghao Guo, Bohan Wang, Pingchuan Ma, Tianyuan Zhang, Crystal Elaine Owens, Chuang Gan, Joshua B. Tenenbaum, Kaiming He, Wojciech Matusik
NeurIPS 2024
ConMe: Rethinking Evaluation of Compositional Reasoning for Modern VLMs
Irene Huang, Wei Lin, M. Jehanzeb Mirza, Jacob A. Hansen, Sivan Doveh, Victor Ion Butoi, Roei Herzig, Assaf Arbelle, Hilde Kuehne, Trevor Darrell, Chuang Gan, Aude Oliva, Rogerio Feris, Leonid Karlinsky
NeurIPS 2024
Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision
Zhiqing Sun, Longhui Yu, Yikang Shen, Weiyang Liu, Yiming Yang, Sean Welleck, Chuang Gan
NeurIPS 2024
SocialGPT: Prompting LLMs for Social Relation Reasoning via Greedy Segment Optimization
Wanhua Li*, Zibin Meng*, Jiawei Zhou, Donglai Wei, Chuang Gan, Hanspeter Pfister
NeurIPS 2024
Constrained Human-AI Cooperation: An Inclusive Embodied Social Intelligence Challenge
Weihua Du*, Qiushi Lyu*, Jiaming Shan, Zhenting Qi, Hongxin Zhang, Sunli Chen, Andi Peng, Tianmin Shu, Kwonjoon Lee, Behzad Dariush, Chuang Gan
NeurIPS 2024
FlexAttention for Efficient High-Resolution Vision-Language Models
Junyan Li, Delin Chen, Tianle Cai, Peihao Chen, Yining Hong, Zhenfang Chen, Yikang Shen, Chuang Gan
ECCV 2024
SALMON: Self-Alignment with Instructable Reward Models
Zhiqing Sun, Yikang Shen, Hongxin Zhang, Qinhong Zhou, Zhenfang Chen, David Cox, Yiming Yang, Chuang Gan
ICLR 2024
Thin-Shell Object Manipulations With Differentiable Physics Simulations
Yian Wang, Juntian Zheng, Zhehuan Chen, Zhou Xian, Gu Zhang, Chao Liu, Chuang Gan
ICLR 2024 (Spotlight)
Building Cooperative Embodied Agents Modularly with Large Language Models
Hongxin Zhang, Weihua Du, Jiaming Shan, Qinhong Zhou, Yilun Du, Joshua B. Tenenbaum, Tianmin Shu, Chuang Gan
ICLR 2024
HAZARD Challenge: Embodied Decision Making in Dynamically Changing Environments
Qinhong Zhou, Sunli Chen, Yisong Wang, Haozhe Xu, Weihua Du, Hongxin Zhang, Yilun Du, Joshua B. Tenenbaum, Chuang Gan
ICLR 2024
GENOME: GenerativE Neuro-symbOlic visual reasoning by growing and reusing ModulEs
Zhenfang Chen, Rui Sun, Wenjun Liu, Yining Hong, Chuang Gan
ICLR 2024
DiffTactile: A Physics-based Differentiable Tactile Simulator for Contact-rich Robotic Manipulation
Zilin Si, Gu Zhang, Qingwei Ben, Branden Romero, Zhou Xian, Chao Liu, Chuang Gan
ICLR 2024
CoVLM: Composing Visual Entities and Relationships in Large Language Models Via Communicative Decoding
Junyan Li, Delin Chen, Yining Hong, Zhenfang Chen, Peihao Chen, Yikang Shen, Chuang Gan
ICLR 2024
Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidance
Phuc D.A. Nguyen, Tuan Duc Ngo, Evangelos Kalogerakis, Chuang Gan, Cuong Pham, Khoi Nguyen
CVPR 2024
SOK-Bench: A Situated Video Reasoning Benchmark with Aligned Open-World Knowledge
Andong Wang, Bo Wu, Sunli Chen, Zhenfang Chen, Wei-Ning Lee, Li Erran Li, Chuang Gan
CVPR 2024
RILA: Reflective and Imaginative Language Agent for Zero-Shot Semantic Audio-Visual Navigation
Zeyuan Yang, Jiageng Liu, Peihao Chen, Anoop Cherian, Tim K Marks, Jonathan Le Roux, Chuang Gan
CVPR 2024
MultiPLY: A Multisensory Object-Centric Embodied Large Language Model in 3D World
Yining Hong, Zishuo Zheng, Peihao Chen, Yian Wang, Junyan Li, Chuang Gan
CVPR 2024
ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning
Qiao Gu, Alihusein Kuwajerwala, Sacha Morin, Krishna Murthy Jatavallabhula, Bipasha Sen, Aditya Agarwal, Corban Rivera, William Paul, Kirsty Ellis, Rama Chellappa, Chuang Gan, Celso Miguel de Melo, Joshua B. Tenenbaum, Antonio Torralba, Florian Shkurti, Liam Paull
ICRA 2024
2023
Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision
Zhiqing Sun, Yikang Shen, Qinhong Zhou, Hongxin Zhang, Zhenfang Chen, David Cox, Yiming Yang, Chuang Gan
NeurIPS 2023 (Spotlight)
3D-LLM: Injecting the 3D World into Large Language Models
Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, Chuang Gan
NeurIPS 2023 (Spotlight)
Adaptive online replanning with diffusion models
Siyuan Zhou, Yilun Du, Shun Zhang, Mengdi Xu, Yikang Shen, Wei Xiao, Dit-Yan Yeung, Chuang Gan
NeurIPS 2023
DiffVL: Scaling Up Soft Body Manipulation using Vision-Language Driven Differentiable Physics
Zhiao Huang, Feng Chen, Yewen Pu, Chunru Lin, Hao Su, Chuang Gan
NeurIPS 2023
DiffuseBot: Breeding Soft Robots With Physics-Augmented Generative Diffusion Models
Tsun-Hsuan Wang, Juntian Zheng, Pingchuan Ma, Yilun Du, Byungchul Kim, Andrew Spielberg, Joshua Tenenbaum, Chuang Gan, Daniela Rus
NeurIPS 2023 (Oral)
Physion++: Evaluating Physical Scene Understanding that Requires Online Inference of Different Physical Properties
Hsiao-Yu Tung, Mingyu Ding, Zhenfang Chen, Daniel Bear, Chuang Gan, Joshua B. Tenenbaum, Daniel LK Yamins, Judith E Fan, Kevin A. Smith
NeurIPS 2023 Dataset Track
TextPSG: Panoptic Scene Graph Generation from Textual Descriptions
Chengyang Zhao, Yikang Shen, Zhenfang Chen, Mingyu Ding, Chuang Gan
ICCV 2023
EfficientViT: Multi-Scale Linear Attention for High-Resolution Dense Prediction
Han Cai, Junyan Li, Muyan Hu, Chuang Gan, Song Han
ICCV 2023
Learning Vision-and-Language Navigation from YouTube Videos
Kunyang Lin, Peihao Chen, Diwei Huang, Thomas H. Li, Mingkui Tan, Chuang Gan
ICCV 2023
Sparse Universal Transformer
Shawn Tan, Yikang Shen, Zhenfang Chen, Aaron Courville, Chuang Gan
EMNLP 2023
Learning Neural Constitutive Laws from Motion Observations for Generalizable PDE Dynamics
Pingchuan Ma, Peter Yichen Chen, Bolei Deng, Joshua B. Tenenbaum, Tao Du, Chuang Gan, Wojciech Matusik
ICML 2023
Reparameterized Policy Learning for Multimodal Trajectory Optimization
Zhiao Huang, Litian Liang, Zhan Ling, Xuanlin Li, Chuang Gan, Hao Su
ICML 2023 (Oral)
On the Forward Invariance of Neural ODEs
Wei Xiao, Tsun-Hsuan Wang, Ramin Hasani, Mathias Lechner, Yutong Ban, Chuang Gan, Daniela Rus
ICML 2023
Roboninja: Learning an Adaptive Cutting Policy for Multi-material Objects
Zhenjia Xu, Zhou Xian, Xingyu Lin, Cheng Chi, Zhiao Huang, Chuang Gan, Shuran Song
RSS 2023
JECC: Commonsense Reasoning Tasks Derived from Interactive Fictions
Mo Yu*, Yi Gu*, Xiaoxiao Guo, Yufei Feng, Xiaodan Zhu, Michael Greenspan, Murray Campbell, Chuang Gan
ACL 2023 (Findings)
3D Concept Learning and Reasoning from Multi-View Images
Yining Hong, Chunru Lin, Yilun Du, Zhenfang Chen, Joshua B. Tenenbaum, Chuang Gan
CVPR 2023
Mod-Squad: Designing Mixture of Experts As Modular Multi-Task Learners
Zitian Chen, Yikang Shen, Mingyu Ding, Zhenfang Chen, Hengshuang Zhao, Erik Learned-Miller, Chuang Gan
CVPR 2023
EC^ 2: Emergent Communication for Embodied Control
Yao Mu, Shunyu Yao, Mingyu Ding, Ping Luo, Chuang Gan
CVPR 2023
Physics-Driven Diffusion Models for Impact Sound Synthesis from Videos
Kun Su, Kaizhi Qian, Eli Shlizerman, Antonio Torralba, Chuang Gan
CVPR 2023
Visual Dependency Transformers: Dependency Tree Emerges from Reversed Attention
Mingyu Ding, Yikang Shen, Lijie Fan, Zhenfang Chen, Zitian Chen, Ping Luo, Joshua B Tenenbaum, Chuang Gan
CVPR 2023
Masked Motion Encoding for Self-Supervised Video Representation Learning
Xinyu Sun, Peihao Chen, Liangwei Chen, Changhao Li, Thomas H. Li, Mingkui Tan, Chuang Gan
CVPR 2023
FluidLab: A Differentiable Environment for Benchmarking Complex Fluid Manipulation
Zhou Xian, Bo Zhu, Zhenjia Xu, Hsiao-Yu Tung, Antonio Torralba, Katerina Fragkiadaki, Chuang Gan
ICLR 2023 (Spotlight)
PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identification
Xuan Li, Yi-Ling Qiao, Peter Yichen Chen, Krishna Murthy Jatavallabhula, Ming Lin, Chenfanfu Jiang, Chuang Gan
ICLR 2023 (Spotlight)
SoftZoo: A Soft Robot Co-design Benchmark For Locomotion In Diverse Environments
Tsun-Hsuan Wang, Pingchuan Ma, Andrew Everett Spielberg, Zhou Xian, Hao Zhang, Joshua B Tenenbaum, Daniela Rus, Chuang Gan
ICLR 2023
Planning with Large Language Models for Code Generation
Shun Zhang, Zhenfang Chen, Yikang Shen, Mingyu Ding, Joshua B Tenenbaum, Chuang Gan
ICLR 2023
DexDeform: Dexterous Deformable Object Manipulation with Human Demonstrations and Differentiable Physics
Sizhe Li*, Zhiao Huang*, Tao Chen, Tao Du, Hao Su, Joshua B Tenenbaum, Chuang Gan
ICLR 2023
Hyper-Decision Transformer for Efficient Online Policy Adaptation
Mengdi Xu, Yuchen Lu, Yikang Shen, Shun Zhang, Ding Zhao, Chuang Gan
ICLR 2023
