Portrait of Chengyuan Yao

Chengyuan Yao

Ph.D. Candidate
Measurement, Evaluation, and Statistics
Columbia University

1230 Amsterdam Ave
New York, NY 10027

I am currently a Ph.D. candidate in Measurement, Evaluation, and Statistics at Columbia University, where I am fortunate to be advised by Dr. James E. Corter and Dr. Renzhe Yu. I am also a member of the AEQUITAS Lab. Previously, I earned a B.A. in Applied Mathematics with Minors in Data Science and Education from UC Berkeley.

Outside academia, I have also had lots of fun working at Microsoft AI and iFLYTEK.

I am looking for full-time Applied Scientist / Research Scientist roles; feel free to contact me regarding research collaborations and potential job opportunities.

Research

My research lies at the intersection of Responsible AI and Educational Data Science, examining how AI methods can be responsibly developed and applied in real-world contexts. I am currently exploring and interested in the following topics:

  • Validity-Centered Evaluation: Advancing more reliable and theoretically grounded evaluation methods for large language models (LLMs) and AI systems, with a focus on construct validity, consistency across tasks, and alignment with real-world goals.
  • Responsible Predictive Modeling: Examining challenges of algorithmic fairness and data privacy in transfer learning, and the responsible development and use of predictive models in educational data mining and learning analytics, aiming to democratize access to trustworthy educational models, especially in under-resourced contexts.
  • Impact of AI Technologies: Understanding how AI technologies reshape teaching and learning, and their broader impact on society.

Experience

Publications Full paper list

LAK 2026 workshop paper thumbnail From Leaderboards to Decisions: A Psychometric Perspective on Criterion-Referenced Evaluation of Large Language Models Chengyuan Yao, Zhen Xu, Jiayu Zheng, Renzhe Yu LAK '26 LLM Psychometrics Workshop
L@S 2026 paper thumbnail Who's More Biased? Calibrating LLM Generation Against Human Responses in Postsecondary Online Discussion Forums Daniel March, Zhen Xu, Chengyuan Yao, Renzhe Yu L@S 2026
L@S 2026 paper thumbnail Are Online Discussion Forums Falling Behind in the Age of AI? Preliminary Evidence from Students' Cognitive and Social Engagement Shifts in 370,000+ Posts Zhen Xu, Yijun Dai, Chenxi Shi, Siyan Li, Chengyuan Yao, Renzhe Yu L@S 2026
AIES 2025 paper thumbnail When the Past Misleads: Rethinking Training Data Expansion Under Temporal Distribution Shifts Chengyuan Yao, Yunxuan Tang, Christopher Brooks, Rene Kizilcec, Renzhe Yu AIES 2025
EDM 2025 paper thumbnail Understanding Predictive Models of Student Success with a Multiverse Analysis Yunxuan Tang, Emma Harvey, Chengyuan Yao, Renzhe Yu, Rene Kizilcec, Christopher Brooks EDM 2025
LAK 2025 paper thumbnail Towards Fair and Privacy-Aware Transfer Learning for Educational Predictive Modeling Chengyuan Yao, Carmen Cortez, Renzhe Yu LAK 2025 🏆 Best Full Research Paper Award
L@S 2024 paper thumbnail Technology-Based Instructional Strategies Show Promise in Improving Self-Regulated Learning Skills at Broad-Access Postsecondary Institutions Renzhe Yu, Hui Yang, Xiaoying Lin, Chengyuan Yao, Paul Burkander, Krystal Thomas, Jessica Mislevy L@S 2024

Non-Archival Presentations

  • Attending to Distribution Shifts: Decomposing Transfer Performance of Educational Predictive Models Chengyuan Yao, Rebecca Su, Christopher Brooks, René F. Kizilcec, Renzhe Yu Poster presentation, Education Data Science Conference (EDS), 2026
  • Why Educational Predictive Models Fail to Transfer: A Distribution Shift Decomposition Chengyuan Yao, Rebecca Su, Rene Kizilcec, Christopher Brooks, Renzhe Yu Oral presentation, 12th International Conference on Computational Social Science (IC2S2 '26)
  • Are Online Discussion Forums Falling Behind in the Age of AI? Preliminary Evidence from Students' Cognitive and Social Engagement Shifts in 370,000+ Posts Zhen Xu, Yijun Dai, Chenxi Shi, Siyan Li, Chengyuan Yao, Renzhe Yu Oral presentation, 12th International Conference on Computational Social Science (IC2S2 '26)
  • Responsible AI for Educational Equity Zhen Xu, Chengyuan Yao, Renzhe Yu Poster presentation, Columbia Data Science Day, 2025
  • Improving Student Success in Online Postsecondary STEM Courses through Technology-Based Interventions Paul Burkander, Renzhe Yu, Hui Yang, Qican Cao, Jessica Mislevy, Xiaoying Lin, Chengyuan Yao Paper presentation, Society for Research on Educational Effectiveness (SREE), 2024

Honors and Awards

Academic Services

Organizing & Program Committees

Reviewer

Community & Service

  • Student Volunteer, Festival of Learning 2026
  • Student Volunteer, alt-FAccT 2025
  • Co-Host, EAAMO NYC Meetup 2025
  • Student Representative, Student Conduct Committee, Teachers College, Columbia University. 2025–2027
  • E-board Member, Data Science and Education Association, Teachers College, Columbia University. 2023–2024
  • TEDx Organizer, TEDxYouth@YouyiWRoad. 2017

Teaching