Zachary Levonian

Machine Learning Engineer, Data Scientist, and Researcher
PhD in Computer Science
Envelope

zacharylevonian at gmail dot com

My research lies at the intersection of social computing and natural language processing. I am interested in the way humans communicate with and support one another online.

I completed my PhD with GroupLens at the University of Minnesota in October 2022. I was advised by Professors Lana Yarosh and Loren Terveen.

My current research is in the domains of education and human-AI interaction.

Research

Useful Machine Learning

A major theme of my research is creating machine learning models that are useful to humans. That starts with developing effective annotation and evaluation practices grounded in the detailed understanding of a problem. In my experience as a data scientist, I've written about how to incorporate expert knowledge during problem formulation and annotation [ICWSM 2020], how to evaluate models when very little data is available [IUI 2022], and how to create metrics appropriate for a specific user interface [CSCW 2025].

I am also a believer in open-sourcing research code: I've released Python packages for multi-modal deep learning recommender systems, for simulation experiments with active learning, and for retrieval-augmented generation with large language models.

Social Support in Online Health Communities

Online social support groups are important places for patients and caregivers to seek information, express themselves, and exchange support. I collaborated with CaringBridge, a prominent online community for writing about and sharing personal health journeys, to study these online health communities. My most recent work investigated how to foster supportive relationships between CaringBridge authors via peer-to-peer recommendation [CSCW 2025]. Our work has been cited in the Star Tribune [1, 2], with publications accepted to [ICWSM 2020], [CSCW 2020], and [TOCHI 2020].

Automatic Speech Recognition for Aviation

I worked with a research team at The MITRE Corporation to create real-time safety monitoring systems for air traffic control towers. These systems use automatic speech recognition to detect radio misunderstandings. We presented our work at [DASC 2015] and [ATM 2017]. I also received a fellowship through MITRE's Early Career Research Program to explore natural language understanding of air traffic controller and pilot radio transmissions.

Publications

Google Scholar profile

Star (*) indicates students I mentored.

Peer Recommendation Interventions for Health-related Social Support: a Feasibility Assessment [preprint] [code]
Zachary Levonian, Matthew Zent, Ngan Nguyen*, Matthew McNamara*, Loren Terveen, Svetlana Yarosh
28th ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW). 2025.
Designing Safe and Relevant Generative Chats for Math Learning in Intelligent Tutoring Systems [html]
Zachary Levonian, Owen Henkel, Chenglu Li, Millie-Ellen Postle
Journal of Educational Data Mining (JEDM). 2025.
This journal article combines the contents of Retrieval-augmented Generation to Improve Math Question-Answering: Trade-offs Between Groundedness and Human Preference (EDM 2024 Short Paper) and Safe Generative Chats in a WhatsApp Intelligent Tutoring System (EDM 2024 Workshop Paper).
Retrieval-augmented Generation to Improve Math Question-Answering: Trade-offs Between Groundedness and Human Preference [preprint] [code] [blog] [poster] [video] [slides] [demo]
Zachary Levonian, Chenglu Li, Wangda Zhu, Anoushka Gade, Owen Henkel, Millie-Ellen Postle, Wanli Xing
Educational Data Mining (EDM). 2024. Best Student Short Paper Award.
Previously presented in the NeurIPS'23 Workshop on Generative AI for Education (GAIED) (non-archival). 2023.
"Thoughts & Prayers" or "❤️ & 🙏": How the Release of New Reactions on CaringBridge Reshapes Supportive Communication During Health Crises [preprint] [blog]
C. Estelle Smith, Hannah Miller Hillberg, Zachary Levonian
26th ACM Conference On Computer-Supported Cooperative Work and Social Computing (CSCW). 2023.
Trade-offs in Sampling and Search for Early-stage Interactive Text Classification [pdf] [code] [video] [blog]
Zachary Levonian, Chia-Jung Lee, Vanessa Murdock, F. Maxwell Harper
27th Annual Conference on Intelligent User Interfaces (IUI). 2022.
Patterns of Patient and Caregiver Mutual Support Connections in an Online Health Community [preprint] [code] [video]
Zachary Levonian, Marco Dow*, Drew Erikson*, Sourojit Ghosh*, Hannah Miller Hillberg, Saumik Narayanan*, Loren Terveen, Svetlana Yarosh
23rd ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW). 2020.
Bridging Qualitative and Quantitative Methods for User Modeling: Tracing Cancer Patient Behavior in an Online Health Community [preprint] [code] [video] [slides] [summary]
Zachary Levonian, Drew Erikson*, Wenqi Luo*, Saumik Narayanan*, Sabirat Rubya, Prateek Vachher*, Loren Terveen, Svetlana Yarosh
Proceedings of the 14th International AAAI Conference on Web and Social Media (ICWSM). 2020.
"I Cannot Do All of This Alone": Exploring Instrumental and Prayer Support in Online Health Communities [preprint] [video]
C. Estelle Smith, Zachary Levonian, Haiwei Ma, Robert Giaquinto, Gemma Lein-Mcdonough, Zixuan Li, Susan O'Conner-Von, Svetlana Yarosh
ACM Transactions on Computer-Human Interaction (TOCHI). 2020.
What I See is What You Don't Get: The Effects of (Not) Seeing Emoji Rendering Differences across Platforms [preprint]
Hannah Miller Hillberg, Zachary Levonian, Daniel Kluver, Loren Terveen, Brent Hecht
Proceedings of the ACM on Human-Computer Interaction (CSCW). Volume 2 Issue CSCW, November 2018.
Readback Error Detection using Automatic Speech Recognition [slides]
Shuo Chen, Hunter Kopald, Ronald Chong, Yuan-Jun Wei, Zachary Levonian
Air Traffic Management Research & Development Seminar (ATM R&D). 2017.
Speech Inputs to Surface Safety Logic Systems
Shuo Chen, Hunter Kopald, Adel Elessawy, Zachary Levonian, Robert Tarakan
Digital Avionics Systems Conference (DASC). 2015

Posters & Non-archival Publications

Publications in this section received peer review for non-archival venues. See my blog for self-published writing.

Star (*) indicates students I mentored.

Safe Generative Chats in a WhatsApp Intelligent Tutoring System [preprint] [code]
Zachary Levonian, Owen Henkel
Educational Data Mining (EDM) Workshop: Leveraging Large Language Models for Next Generation Educational Technologies. 2024.
Can Large Language Models Make the Grade? An Empirical Study Evaluating LLMs Ability To Mark Short Answer Questions in K-12 Education
Owen Henkel, Libby Hills, Adam Boxer, Bill Roberts, Zachary Levonian
Learning @ Scale (L@S). 2024.
ORES-Inspect: A technology probe for machine learning audits on enwiki [preprint] [code] [video] [demo]
Zachary Levonian, Lauren Hagen*, Lu Li*, Jada Lilleboe*, Solvejg Wastvedt, Aaron Halfaker, Loren Terveen
Wiki Workshop. 2024.
Digital Trial Solutions eScreening: a software solution that ranks patients by their predicted clinical trial eligibility using Real-World Data and Machine Learning [pdf]
Sudip Bhandari, Zachary Levonian, Robert Annand, Jo-Zahn Baxter, Jericho Cain, Jen Flach, Jiby Joseph-Thomas, Judith Mueller
American Society of Clinical Oncology - ASCO (Abstract). 2023.
How much is a "like" worth? Engagement and retention in an online health community
Ruyuan Wan*, Zachary Levonian, Svetlana Yarosh
Companion of the 23rd ACM Conference on Computer Supported Cooperative Work and Social Computing, CSCW 2020.
Outstanding Poster Recognition.
Grandtotem: Supporting International and Intergenerational Relationships [preprint] [video]
Mark Butzer, Zachary Levonian, Yangyang Luo, Kathleen Watson, Ye Yuan, C. Estelle Smith, Svetlana Yarosh
Companion of the 23rd ACM Conference on Computer Supported Cooperative Work and Social Computing, CSCW 2020
Understanding Community-Level Conflicts Through Reddit r/place [code] [demo video]
Prateek Vachher*, Zachary Levonian, Hao-Fei Cheng, Svetlana Yarosh
Companion of the 23rd ACM Conference on Computer Supported Cooperative Work and Social Computing, CSCW 2020
Condition Unknown: Predicting Patients' Health Conditions in an Online Health Community
Changye Li*, Zachary Levonian, Haiwei Ma, Svetlana Yarosh
Companion of the 2018 ACM Conference on Computer Supported Cooperative Work and Social Computing, CSCW 2018

Blog

View all of my self-published writing here.

Most of my blog-style writings are on GitHub or Medium.

What's New

Recent research updates. I keep my professional updates on my LinkedIn.

May 2023, I joined the Digital Harbor Foundation as a Senior Machine Learning Engineer in the Learning Engineering Virtual Institute.
December 2022, my dissertation is now available on ProQuest. (available open-access via the UMN Conservancy and hosted here [pdf])
November 2022, I was officially awarded my degree and started an industry job. I am a Senior Machine Learning Engineer at ConcertAI.
July 2022, I defended my dissertation [pdf].
April 2021, I received the University of Minnesota's Doctoral Dissertation Fellowship for Fall 2021 and Spring 2022.
October 2020, my advisee Ruyuan Wan's CSCW2020 poster received an Outstanding Poster award. Check it out here.
August 2020, a Master's student I mentored, Marco Dow, successfully defended his thesis, which is now available online!
July 2020, I had 1 full paper (preprint) and 3 posters accepted to CSCW 2020! You can watch my pre-recorded presentation here.
July 2019, my first paper was accepted to ICWSM 2020. View here.
June 2019, I gave a guest lecture on "Why NLP matters to HCI researchers" in CS322: Natural Language Processing at Carleton College. Slides here.
March 2019, I presented a workshop on NLP for social computing researchers.
April 2018, Attended CHI'18, my first conference!
August 2017, I joined the GroupLens lab at the University of Minnesota.

Miscellaneous

In my spare time I visit breweries, see weird indie shows, and try to predict the outcome of The Bachelor. I have competed in exactly one curling tournament, where I was laughed at by a former Olympian. I am an occasional editor on English Wikipedia (2000 edits and counting).

I go by Zach or Zachary and my preferred pronouns are he/his/him.

I am on Mastodon, Twitter, Bluesky, LinkedIn, and GitHub.

This website is hosted on GitHub.

This page last edited on October 17, 2024.