Yansong Tang

I am a tenure-track Associate Professor of Shenzhen International Graduate School, Tsinghua University, where I direct the IVG@SZ (Intelligent Vision Group at Shenzhen, the sister group of the IVG at Beijing). Before that, I was a postdoctoral researcher at the Department of Engineering Science of the University of Oxford, working with Prof. Philip H. S. Torr. My current research interests lie in computer vision and pattern recognition.

I received my B.S. degree and Ph.D degree with honour from Tsinghua University, advised by Prof. Jie Zhou and Prof. Jiwen Lu. I have also spent time at Prof. Song-Chun Zhu's lab at University of California, Los Angeles (UCLA), and Microsoft Research Asia (MSRA), hosted by Dr. Han Hu and Dr. Xin Tong respectively.

I am looking for self-motivated Master/PhD/Postdoc. If you have top grades or coding skill, and are highly creative and interested in joining my group, please do not hesitate to send me your CV and transcripts of grades by Email after reading this file.

(*Equal Contribution, #Corresponding Author)

We propose QDMN, a memory-based dynamic framework for semi-supervised video object segmentation that performs adaptive quality awareness and dynamic update of reference information.

We propose LAVT, a Transformer-based universal referring image and video segmentation (RIS and RVOS) framework that performs language-aware visual encoding in place of cross-modal fusion post feature extraction.

COIN is currently the largest and most comprehensive instructional video analysis datasets with rich annotations.

OccNeRF is a LiDAR-free method for 3D occupancy prediction that leverages multi-camera images, temporal photometric consistency, and open-vocabulary segmentation for supervision.

We propose Momentum-GS, a self-distillation approach for more accurate and efficient 3D Gaussian Splatting.

We propose ScoreHOI, a framework for human-object interaction reconstruction via score-guided diffusion to enhance the physical plausibility.

We propose AnyBimanual, a plug-and-play method that transfers pretrained unimanual policy to general bimanual manipulation tasks with few demonstrations.

We propose Flash-VStream, an efficient VLM with a novel memory mechanism that enables real-time understanding and querying of extremely long video streams.

We introduce OpenBench, a new benchmark designed to more rigorously evaluate open-vocabulary segmentation models on diverse real-world concepts, and propose OVSNet for segmentation in diverse and open scenarios.

We propose KV-Edit to address the challenge of background preservation in image editing, thereby enhancing the practicality of AI editing.

We propose ManiGaussian++, a hierarchical Gaussian world model that captures multi-body dynamics for improved multi-task bimanual manipulation.

We achieve action transfer in heterogeneous scenarios with varying spatial structures or cross-domain subjects.

We propose VoCo-LLaMA, the first approach to compress vision information utilizing the LLMs' understanding paradigm, which can compress hundreds of vision tokens into a single VoCo token with minimal visual information loss.

We propose ATP-LLaVA, a framework that adaptively determines pruning ratios instance-wise and LLM layer-wise for effective vision token pruning on large vision language models.

We propose a novel framework called SAM2-LOVE to effectively segment the video objects referred by the audio and text and achieve significant improvement in Ref-AVS tasks.

We introduce FADE—a training-free yet highly effective video editing approach that fully leverages the inherent priors from pre-trained video diffusion models via frequency-aware factorization.

We propose a new fully AI-powered automatic reinforcement learning method, Reinforcement Learning from Evol-Instruct Feedback (RLEIF), alongside Math Evol-Instruct and Process Supervision, for improving reasoning performance.

We have presented a ThinkBot agent that reasons the thought chain for missing instruction recovery in embodied instruction following (EIF) tasks.

We propose InstaRevive, a straightforward yet powerful image enhancement framework that employs score-based diffusion distillation to harness potent generative capability and minimize the sampling steps.

This paper proposes a geometry-aware large reconstruction model for sparse-view reconstruction and 3D generation.

This paper aims to achieve efficient inference and memory saving for large vision-language models.

We propose a dynamic Gaussian Splatting method named ManiGaussian for multi-task robotic manipulation, which mines scene dynamics via future scene reconstruction.

We introduces MotionLCM, extending controllable motion generation to a real-time level.

We present a divide-and-conquer framework named PRO-Motion, which consists of three modules as motion planner, posture-diffuser and go-diffuser.

We propose a method to efficiently equip the Segment Anything Model (SAM) with the ability to generate regional captions by introducing a lightweight query-based feature mixer.

We propose a unified framework to achieve universal segmentation at a wide spectrum of granularities and levels.

We propose an open-vocabulary segmentation (OVS) method by calibrating in-vocabulary and domain-biased embedding space with generalized contextual prior of CLIP.

We propose an accurate data-free post-training quantization framework of diffusion models (ADP-DM) for efficient image generation.

We investigate a new problem called narrative action evaluation (NAE) and propose a prompt-guided multimodal interaction framework.

We propose a new method to exploit diffusion priors for human mesh recovery (HMR) in occlusion and crowded scenarios.

We proposed a unified framework for various efficient image enhancement tasks with generative diffusion priors.

We propose a hardware-algorithm co-optimizations method called MCUFormer to deploy vision transformers on microcontrollers with extremely limited memory, where we jointly design transformer architecture and construct the inference operator library to fit the memory resource constraint.

We present Tem-Adapter, a method that improves VQA by leveraging image-based knowledge and introducing temporal and semantic aligners.

We propose Skip-Plan, a condensed action space learning method for procedure planning in instructional videos.

We present FLAG3D, a large-scale 3D fitness activity dataset with language instruction.

LOGO is a new multi-person long-form video dataset for action quality assessment.

HorNet is a family of generic vision backbones that perform explicit high-order spatial interactions based on Recursive Gated Convolution.

We present a language-powered paradigm for ordinal regression.

We present an end-to-end hierarchical Transformer-based network for referring segmentation.

We present Bi-level Neural Volume Fusion, which leverages recent advances in neural implicit representations and neural rendering for dense 3D reconstruction. In order to incrementally integrate new depth maps into a global neural implicit representation, we propose a novel bi-level fusion strategy that considers both efficiency and reconstruction quality by design.

DenseCLIP is a new framework for dense prediction by implicitly and explicitly leveraging the pre-trained knowledge from CLIP.

We propose an uncertainty-aware score distribution learning method and extend it to a multi-path model for action quality assessment.

Website Template