Ting-Han Fan
|
I am a Research Scientist at the LLM pre-train team of ByteDance. Previously, I was a machine learning engineer at TikTok E-commerce Recommendation, a summer associate at Goldman Sachs Asset Management, a research intern at Siemens and Microsoft. I completed my M.A. and Ph.D. degrees at Electrical and Computer Engineering, Princeton University, where I was very fortunate to be advised by Prof. Peter J. Ramadge. Before Princeton, I completed my B.S. degree at Electrical Engineering, National Taiwan University. email: tinghanfan at gmail dot com Find me at Google Scholar and LinkedIn. |
Publications:
Length Extrapolation of Transformers
- Attention Alignment and Flexible Positional Embeddings Improve Transformer Length Extrapolation, Ta-Chung Chi, Ting-Han Fan, Alexander I. Rudnicky, Findings of NAACL 2024.
- Dissecting Transformer Length Extrapolation via the Lens of Receptive Field Analysis, Ta-Chung Chi, Ting-Han Fan, Alexander I. Rudnicky, Peter J. Ramadge, ACL 2023. (Outstanding Paper Award) [link]
- Latent Positional Information is in the Self-Attention Variance of Transformer Language Models Without Positional Embeddings, Ta-Chung Chi, Ting-Han Fan, Li-Wei Chen, Alexander I. Rudnicky, Peter J. Ramadge, ACL 2023. (Honorable Mention) [link]
- KERPLE: Kernelized Relative Positional Embedding for Length Extrapolation, Ta-Chung Chi*, Ting-Han Fan*, Peter J. Ramadge, Alexander I. Rudnicky, NeurIPS 2022. [code]
Regular Language Reasoning
- Advancing Regular Language Reasoning in Linear Recurrent Neural Networks, Ting-Han Fan*, Ta-Chung Chi*, Alexander I. Rudnicky, NAACL 2024.
- Transformer Working Memory Enables Regular Language Reasoning and Natural Language Length Extrapolation, Ta-Chung Chi, Ting-Han Fan, Alexander I. Rudnicky, Peter J. Ramadge, Findings of EMNLP 2023.
Reparameterization for Discrete Deep Generative Models
- Training Discrete Deep Generative Models via Gapped Straight-Through Estimator, Ting-Han Fan*, Ta-Chung Chi*, Alexander I. Rudnicky, Peter J. Ramadge, ICML 2022. [code] [SlidesLive]
Reinforcement Learning for Power Distribution System Controls
- PowerGym: A Reinforcement Learning Environment for Volt-Var Control in Power Distribution Systems, Ting-Han Fan, Xian Yeow Lee, Yubo Wang, L4DC 2022. [code]
- Soft Actor-critic With Integer Actions, Ting-Han Fan, Yubo Wang, ACC 2022.
Model-based Reinforcement Learning
- A Contraction Approach to Model-based Reinforcement Learning, Ting-Han Fan, Peter J. Ramadge, AISTATS 2021. [SildesLive]
(* denotes equal contribution)
Patent Applications:
- System and method for controlling large scale power distribution systems using reinforcement learning, Ting-Han Fan, Yubo Wang, Ulrich Muenz, US Patent App. 17/814535.
- System and method for supporting execution of batch production using reinforcement learning, Ting-Han Fan, Yubo Wang, Ulrich Muenz, Mathias Hakenberg, WO Patent App. PCT/US2022/041711.
Academic Activities:
Reviewer
- NeurIPS 2022-2023, ICLR 2024, ICML 2022-2024, ISIT 2024, L4DC 2023, AISTATS 2021
- ACL Rolling Review: 2024 February (ACL 2024), 2023 December (NAACL 2024)
Teaching Assistant at Princeton University
- ECE 435/535: Machine Learning and Pattern Recognition, Fall 2019, Fall 2020, Fall 2022 (head TA)
- EGR 154: Foundations of Engineering: Linear Systems, Spring 2022
- COS 302: Mathematics for Numerical Computing and Machine Learning, Fall 2021 (head TA)
- SML 310: Research Projects in Data Science, Spring 2021 (head TA)
- ECE 201: Information Signals, Spring 2020, Spring 2023