Jingyang Lin

Bio

I am a second-year Ph.D. student at University of Rochester, advised by Prof. Jiebo Luo. Before that, I received the B.Eng. degree in software engineering and M.Sc. degree in computational mathematics all from the Sun Yat-sen University, supervised by Prof. Hongyang Chao. I also worked as a research intern at JD AI Research, co-advised by Dr. Yu Wang, Dr. Ting Yao, and Dr. Tao Mei.

My current research lies in Multimodal Learning with LLMs, AI for Health, and Self-Supervised Learning.

News

Jul. 2024	🔥 Released the 2nd version of the Video Understanding with LLMs survey .
May 2024	One paper is accepted by ACL 2024.
Jan. 2024	🔥 The 🤗VideoXum dataset and the 🤗VTSUM-BLIP model.
Dec. 2023	Released a survey of Video Understanding with LLMs.
Nov. 2023	One paper is accepted by IEEE TMM.
May 2023	One paper is accepted by ICDH 2023.
Sep. 2022	One paper is accepted by NeurIPS 2022.
Jun. 2022	One paper is accepted by IEEE TPAMI.
Sep. 2021	One paper is accepted by NeurIPS 2021.

Publications

* indicates equal contribution

CT-GLIP: 3D Grounded Language-Image Pretraining with CT Scans and Radiology Reports for Full-Body Scenarios Jingyang Lin, Yingda Xia, Jianpeng Zhang, Ke Yan, Le Lu, Jiebo Luo, Ling Zhang Preprint. [PDF]

Fine-Grained Image-Text Alignment in Medical Imaging Enables Explainable Cyclic Image-Report Generation Wenting Chen, Linlin Shen, Jingyang Lin, Jiebo Luo, Xiang Li, Yixuan Yuan. The 62nd Annual Meeting of the Association for Computational Linguistics. ACL 2024. [PDF]

Video Understanding with Large Language Models: A Survey University of Rochester, Southern University of Science and Technology, The University of Hong Kong Preprint. [PDF][Project Page]

VideoXum: Cross-modal Visual and Textural Summarization of Videos Jingyang Lin*, Hang Hua*, Ming Chen, Yikang Li, Jenhao Hsiao, Chiuman Ho, Jiebo Luo IEEE Transactions on Multimedia. IEEE TMM. [Project Page] [PDF] [Code] [Dataset] [Model]

Predicting Adverse Neonatal Outcomes for Preterm Neonates with Multi-Task Learning Jingyang Lin, Junyu Chen, Hanjia Lyu, Igor Khodak, Divya Chhabra, Colby L Day Richardson, Irina Prelipcean, Andrew M Dylag, Jiebo Luo IEEE International Conference on Digital Health. ICDH 2023. [PDF]

Out-of-Distribution Detection via Conditional Kernel Independence Model Yu Wang, Jingjing Zou, Jingyang Lin, Qing Ling, Yingwei Pan, Ting Yao, Tao Mei Neural Information Processing Systems. NeurIPS 2022. [PDF] [Code] Spotlight Presentation

Out-of-Distribution Detection with Hilbert-Schmidt Independence Optimization Jingyang Lin, Yu Wang, Qi Cai, Yingwei Pan, Ting Yao, Hongyang Chao, Tao Mei Preprint. [PDF] [Code]

A Low Rank Promoting Prior for Contrastive Learning Yu Wang, Jingyang Lin, Qi Cai, Yingwei Pan, Ting Yao, Hongyang Chao, Tao Mei IEEE Transactions on Pattern Analysis and Machine Intelligence. IEEE TPAMI. [PDF] [Code]

Improving Self-supervised Learning with Automated Unsupervised Outlier Arbitration Yu Wang, Jingyang Lin, Jingjing Zou, Yingwei Pan, Ting Yao, Tao Mei Neural Information Processing Systems. NeurIPS 2021. [PDF] [Code]

CORE-Text: Improving Scene Text Detection with Contrastive Relational Reasoning Jingyang Lin, Yingwei Pan, Rongfeng Lai, Xuehang Yang, Hongyang Chao, Ting Yao IEEE International Conference on Multimedia and Expo. ICME 2021. [PDF] [Code] Oral Presentation

Improving Fast Segmentation With Teacher-student Learning Jiafeng Xie, Bing Shuai, Jian-Fang Hu, Jingyang Lin, Wei-Shi Zheng British Machine Vision Conference. BMVC 2018. [PDF]

Internship

PAII, Palo Alto, CA Research Intern May 2024 - Aug. 2024 Advisors: Dr. Andy Wong, Dr. Tian Xia, and Dr. Mei Han Project: Long-Context LLMs

DAMO Academy, Alibaba Group, New York City, NY Research Intern Jun. 2023 - Sep. 2023 Advisors: Dr. Yingda Xia, Dr. Ling Zhang, and Dr. Le Lu Project: 3D Vision-Language Alignment for CT-Report Dataset

JD AI Research, Beijing, China Research Intern Jun. 2020 - Jul. 2022 Advisors: Dr. Yu Wang, Dr. Ting Yao, and Dr. Tao Mei Project: Self-Supervised Learning and Out-of-Distribution Detection

JD AI Research, Beijing, China Research Intern Jul. 2018 - Aug. 2019 Advisors: Rongfeng Lai, Dr. Ting Yao, and Dr. Tao Mei Project: Scene Text Detection

Service

• Conference: CVPR, NeurIPS, ICLR, ACM Multimedia, CICAI
• Journal: IEEE Transactions on Big Data, IEEE MultiMedia

Teaching

• University of Rochester
• CSC 240 Data Mining - Spring 2024 (Prof. Thaddeus E. Pawlicki)
• CSC 440 Data Mining - Fall 2023 (Prof. Jiebo Luo)
• CSC 249 Machine Vision - Spring 2023 (Prof. Jiebo Luo)

Last updated on Jul 2024.

Copyright © Jingyang Lin