I am a Ph.D. candidate at Human-Computer Communications Laboratory (HCCL), The Chinese University of Hong Kong (CUHK), supervised by Prof. Helen Meng. Previously, I received my M.Phil. degree from the Institute of Automation, Chinese Academy of Sciences (CASIA), where I was supervised by Prof. Jie Tian. And I received my Bachelor's degree from Harbin Institute of Technology (HIT).
I am currently a research intern at Microsoft Research Asia. My research interests focus on language modeling for speech synthesis and the integration of speech with large language models. I am also working on speech processing and recognition.
🟢 I am currently on the job market. Please drop me emails!
[Jul 2024]
We present [MELLE], a concise, continuous-value token-based language model for TTS, forgoing cumbersome NAR steps.
[Jun 2024]
We propose [WavLLM], a robust and adaptive speech LLM achieving SOTA performance on various speech-related tasks.
[Jun 2024]
Three papers has been accepted to ISCA INTERSPEECH 2024, including one first-authored [paper].
[Jan 2024]
Two papers has been accepted to IEEE ICASSP 2024.
[Nov 2023]
Started an internship at Microsoft Research Asia.
[May 2023]
One first-authored [paper] has been accepted to ISCA INTERSPEECH 2023.
[Feb 2023]
One first-authored [paper] has been accepted to IEEE ICASSP 2023 (Oral).
[Jun 2022]
Two papers have been accepted to ISCA INTERSPEECH 2022.
[Mar 2022]
Our [paper] was selected as an ⭐️ ESI Highly Cited Paper by Web of Science.
[Jan 2022]
We won the 2nd place on the diarization track of the ICASSP 2022 - M2Met Chanllenge. This is our [paper].
Since 2021, Ph.D. student, The Chinese University of Hong Kong (CUHK)
2018 - 2021, M.Phil., Pattern Recognition and Intelligent Systems, Institute of Automation, Chinese Academy of Sciences (CASIA)
2014 - 2018, B.Eng., Electrical Engineering, Harbin Institute of Technology (HIT)
MELLE: Autoregressive Speech Synthesis without Vector Quantization | [demo]
Lingwei Meng, Long Zhou, Shujie Liu, Sanyuan Chen, Bing Han, Shujie Hu, Yanqing Liu, Jinyu Li, Sheng Zhao, Xixin Wu,
Helen Meng, Furu Wei
arXiv, 2024
Empowering Whisper as a Joint Multi-Talker and Target-Talker Speech Recognition System | [code]
Lingwei Meng, Jiawen Kang, Yuejiao Wang, Zengrui Jin, Xixin Wu, Xunying Liu, Helen Meng
ISCA INTERSPEECH, Kos Island, Greece, 2024
A Sidecar Separator Can Convert a Single-Talker
Speech Recognition System to
a Multi-Talker One | [slides] [video]
Lingwei Meng, Jiawen Kang, Mingyu Cui, Yuejiao Wang, Xixin Wu, Helen Meng
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP),
Rhodes Island, Greece, 2023 (Oral)
Unified Modeling of Multi-Talker
Overlapped Speech Recognition and Diarization with a Sidecar Separator
Lingwei Meng, Jiawen Kang, Mingyu Cui, Haibin Wu, Xixin Wu, Helen Meng
ISCA INTERSPEECH, Dublin, Ireland, 2023
WavLLM: Towards Robust and Adaptive Speech Large Language Model | [code]
Shujie Hu, Long Zhou, Shujie Liu, Sanyuan Chen, Lingwei Meng, Hongkun Hao, Jing Pan, Xunying Liu, Jinyu Li, Sunit Sivasankaran, Linquan Liu, Furu Wei
arXiv, 2024
VALL-E R: Robust and Efficient Zero-Shot
Text-to-Speech Synthesis via Monotonic Alignment | [demo]
Bing Han, Long Zhou, Shujie Liu, Sanyuan Chen, Lingwei Meng, Yanming Qian, Yanqing Liu,
Sheng Zhao, Jinyu Li, Furu Wei
arXiv, 2024
Large Language Model-based FMRI Encoding of Language Functions for Subjects with Neurocognitive Disorder
Yuejiao Wang, Xianmin Gong, Lingwei Meng, Xixin Wu, Helen Meng
Sheng Zhao, Jinyu Li, Furu Wei
ISCA INTERSPEECH, Kos Island, Greece, 2024
LibriheavyMix: A 20,000-Hour Dataset for Single-Channel Reverberant Multi-Talker Speech Separation, ASR and Speaker
Diarization
Zengrui Jin, Yifan Yang, Mohan Shi, Wei Kang, Xiaoyu Yang, Zengwei Yao, Fangjun Kuang, Liyong Guo, Lingwei Meng, Long
Lin, Yong Xu, Shi-Xiong Zhang, Daniel Povey
ISCA INTERSPEECH, Kos Island, Greece, 2024
Cross-Speaker Encoding Network for Multi-Talker Speech Recognition | [code]
Jiawen Kang, Lingwei Meng, Mingyu Cui, Haohan Guo, Xixin Wu, Xunying Liu, Helen Meng
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP),
Seoul, Korea, 2024
UNIT-DSR: Dysarthric Speech Reconstruction System Using Speech Unit Normalization
Yuejiao Wang, Xixin Wu, Disong Wang, Lingwei Meng, Helen Meng
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP),
Seoul, Korea, 2024
The defender's perspective on automatic speaker verification: An overview
Haibin Wu, Jiawen Kang, Lingwei Meng, Helen Meng, Hung-yi Lee
International Joint Conferences on Artificial Intelligence (IJCAI) Workshop,
Macao SAR, China, 2023
Spoofing-Aware Speaker Verification
by Multi-Level Fusion
Haibin Wu, Lingwei Meng, Jiawen Kang,
Jinchao Li, Xu Li, Xixin Wu, Hung-yi Lee, Helen Meng
ISCA INTERSPEECH, Incheon, Korea, 2022
Exploring Linguistic Feature and Model Combination
for Speech Recognition Based Automatic AD Detection
Yi Wang, Tianzi Wang, Zi Ye, Lingwei Meng, Shoukang Hu, Xixin Wu, Xunying
Liu, Helen Meng
ISCA INTERSPEECH, Incheon, Korea, 2022
The CUHK-Tencent Speaker Diarization
System
for the ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Challenge
Naijun Zheng, Na Li, Xixin Wu, Lingwei Meng, Jiawen Kang, Haibin Wu, Chao
Weng, Dan Su, Helen Meng
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Singapore, 2022
Tackling Spoofing-Aware Speaker Verification with
Multi-Model Fusion
Haibin Wu, Jiawen Kang, Lingwei Meng, Yang Zhang, Xixin Wu, Zhiyong Wu,
Hung-yi Lee, Helen Meng
ISCA The Speaker and Language Recognition Workshop (Odyssey), Beijing, China, 2022
2D and
3D CT Radiomic
Features Performance Comparison in Characterization of Gastric Cancer: a Multi-Center
Study
Lingwei Meng, Di
Dong, Xin Chen, Mengjie Fang, Rongpin Wang, Jing Li, Zaiyi Liu, Jie Tian
IEEE Journal of Biomedical and Health Informatics (IEEE JBHI), 2021 (Impact Factor: 6.7)
(ESI Highly Cited Paper)
A Deep
Learning Prognosis
Model Help Alert for COVID-19 Patients at High-Risk of Death: a
Multi-Center Study
Lingwei Meng, Di
Dong, Liang Li, Meng Niu, Yan Bai, Meiyun Wang, Xiaoming Qiu, Yunfei Zha, Jie Tian
IEEE Journal of Biomedical and Health Informatics (IEEE JBHI), 2020 (Impact Factor: 6.7)
Noninvasive Model
for Predicting Future Ischemic Strokes in Patients with Silent Lacunar Infarction Using
Radiomics
Jiehua Su#, Lingwei
Meng#, Di Dong#, Wenyan Zhuo, Jianming Wang, Libin Liu, Yi
Qin, Ye Tian, Jie Tian, Zhaohui Li
BMC Medical Imaging, 2020 (# denotes the co-first authorship.) (Impact Factor: 2.9)
SEEM 3440, Operation Research II
AIST 3510 / SEEM 3510, Human-Computer Interaction
Some awards on control algorithm and circuit design competitions: