🤓 Hey folks! I am Hengyuan Zhang (张恒源 in Chinese). I am a master’s student at Tsinghua University from August 2022, under the supervision of Prof. Yong Jiang and Prof. Yujiu Yang.
My current research interests revolve around the application of Natural Language Processing (NLP) and Data Mining in specialized domains such as Multilingualism, Education, and Cognitive Science. I aim to approach these studies in an interpretable manner, seeking deeper insights into complex phenomena. I am particularly intrigued by the decision-making mechanisms integrated within models, eager to unravel their inner workings and enhance transparency.
Looking ahead, I am also exploring the possibility of enrolling in a Ph.D. program for the Fall of 2025. I am keen on exploring opportunities for collaboration in research or projects. Please do not hesitate to contact me at your convenience.
📝 Publications
ShifCon: Enhancing Non-Dominant Language Capabilities with a Shift-based Contrastive Framework
Hengyuan Zhang, Chenming Shang, Sizhe Wang, Dongdong Zhang, Feng Yao, Renliang Sun, Yiyao Yu, Yujiu Yang, Furu Wei
| [Paper] | [Code] | Natural Language Processing, Multilingual, Interpretability in Parameter | Conference Preprint |
- This paper aims to enhance the performance of non-dominant languages by projecting their representations into the dominant language space. We pinpoint the optimal layer area for shifting representations via a subspace distance metric.
Balancing Speciality and Versatility: a Coarse to Fine Framework for Supervised Fine-tuning Large Language Model
Hengyuan Zhang, Yanru Wu, Dawei Li, Zacc Yang, Rui Zhao, Yong Jiang, Fei Tan
| [Paper] | [Code] | Natural Language Processing, Fine-tuning Technique, Interpretability in Parameter | CCF-A Conference |
- This paper introduces a Coarse-to-Fine Fine-tuning framework (CoFiTune) that strikes a delicate balance between speciality and versatility. It pinpoints and updates specific modules that are crucial for speciality, while keeping other parameters frozen.
A Question-centric Multi-experts Contrastive Learning Framework for Improving the Accuracy and Interpretability of Deep Sequential Knowledge Tracing Models
Hengyuan Zhang, Zitao Liu, Chenming Shang, Dawei Li, Yong Jiang
| [Paper] | [Code] | Data Mining, Education Recommendation, Interpretability in Prediction | JCR Q1 Journal |
- This paper proposes Q-MCKT framework, which utilizes an item response theory-based prediction layer to generate interpretable prediction results by simultaneously modeling knowledge acquisition and question difficulty.
- Preprint 2024
BPO: Towards Balanced Preference Optimization between Knowledge Breadth and Depth in Alignment
Sizhe Wang, Yongqi Tong,Hengyuan Zhang, Dawei Li, Xin Zhang, Tianlong Chen
[Paper] | Natural Language Processing, Fine-tuning Technique | Conference Preprint
- Preprint 2024
Improving Low-Resource Knowledge Tracing Tasks by Supervised Pre-training and Importance Mechanism Fine-tuning
Hengyuan Zhang, Zitao Liu, Shuyan Huang, Chenming Shang, Bojun Zhan, Yong Jiang
[Paper] | [Code] | Data Mining, Education Recommendation | Journal Preprint
- Preprint 2024
Compositional Generalization Through Neuroscience-inspired Geometric Constraints on Representation Structure
Chenming Shang, Shiji Zhou,Hengyuan Zhang, Xinchen Zhang, Lei Ke, Yuwang Wang, Yujiu Yang
[Paper] | Computer Vision, Cognitive Science, Interpretability in Representation | Conference Preprint
- CogSci 2024
Understanding Multimodal Deep Neural Networks: A Concept Selection View
Chenming Shang,Hengyuan Zhang, Hao Wen, Yujiu Yang
[Paper] | Computer Vision, Cognitive Science, Interpretability in Prediction | CCF-B Conference
- CVPR 2024
Incremental Residual Concept Bottleneck Model
Chenming Shang, Shiji Zhou,Hengyuan Zhang, Yujiu Yang, Xinzhe Ni, Yuwang Wang
[Paper] | [Code] | Computer Vision, Cognitive Science, Interpretability in Prediction | CCF-A Conference
- EMNLP 2023
Multi-level Contrastive Learning for Script-based Character Understanding
Dawei Li,Hengyuan Zhang, Yanran Li, Shiping Yang
[Paper] | [Code] | Natural Language Processing, Cognitive Science | CCF-B Conference
- ACL 2023 BEA
Assisting Language Learners: Automated Trans-Lingual Definition Generation via Contrastive Prompt Learning
Hengyuan Zhang, Dawei Li, Yanran Li, Chenming Shang, Chufan Shi, Yong Jiang
[Paper] | [Code] | Natural Language Processing, Education, Interpretability in Representation | CCF-A Conference
- AACL 2022(Oral)
Fine-grained Contrastive Learning for Definition Generation
Hengyuan Zhang, Dawei Li, Shiping Yang, Yanran Li
[Paper] | [Code] | Natural Language Processing, Education, Interpretability in Representation | CCF-A Conference
- ACL 2022
Multitasking Framework for Unsupervised Simple Definition Generation
Cunliang Kong, Yun Chen,Hengyuan Zhang, Liner Yang, Erhong Yang
[Paper] | [Code] | Natural Language Processing, Education, Interpretability in Representation | CCF-A Conference
- NAACL 2022 SemEval
SemEval-2022 Task 1: Cross-Attention Multitasking Framework for Definition Modeling
Cunliang Kong, Yujie Wang, Ruining Chong, Liner Yang,Hengyuan Zhang, Erhong Yang
[Paper] | [Code] | Natural Language Processing, Education, Interpretability in Representation | CCF-B Conference
💻 Interships
Xiaomi, AI Lab, Beijing
- Mar. 2022 - Sept. 2022, Research Intern, working with Tong Chen and Yanran Li
Tencent, AI Lab, Shenzhen
- Mar. 2023 - Jul. 2023, Research Intern, working with Wei Bi
Sensetime, Research, Shenzhen
- Aug. 2023 - Mar. 2024, Research Intern, working with Fei Tan
Microsoft Research Asia, Natural Language Computing (NLC) Group, Beijing
- Mar. 2024 - Present, Research Intern, working with DongDong Zhang
🏅 Selected Honors and Awards
👉 Tsinghua University Comprehensive First-Class Scholarship (Top 3%, RMB ¥ 10,000) | 2024
👉 Tsinghua University General Excellence Scholarship (Top 10%, RMB ¥ 5,000) | 2023
👉 National Scholarship (Top 1%, RMB ¥ 8,000) | 2019, 2020, 2021
👉 Outstanding Graduate Student of Beijing (Top 3%) | 2021
👉 Excellent League Member of Beijing (Top 3%) | 2021
👉 Merit Student of Beijing (Top 3%) | 2021
👉 Meritorious Winner of Interdisciplinary Contest in Modeling (Top 7%) | 2021
👉 Computer Design Competition National Second Prize (Top 5%) | 2020
👉 CUMCM-Beijing Area First Prize (Top 5%) | 2020
👉 Xiaomi Third Hacker Marathon Excellence (Top 7%, RMB ¥ 3,000) | 2022
📌 Miscellaneous
- I am actually a person with a strong desire to share. In my spare time, I like writing blogs and sharing experiences on Redbook, Wechat Official Account and Bilibili (阿源的NLP碎碎念). The selected blogs are as follows:
- Linguistic Regions in the Large Language Model
- Prevent Catastrophic Forgetting via SoftMask Mechanism
- The Key Components in Transformer
- The Evaluation of Instruction Following
- Skill Localization of Large Language Model
- iMAge-guided Text GeneratIon with CLIP
- The Difference between GLM and UL2 Architectures
- I used to be a guitarist 🎸 in a band when I was in high school. Also, I love playing badminton 🏸, table tennis 🏓 and soccer ⚽️. During holidays, I will also seize any opportunity to travel around the world ⛳️.
