I attended the second annual NCME Artificial Intelligence in Measurement and Education Conference (AIME-Con), which is an interesting space because it brings together more traditional education measurement/psychometrics researchers with applied machine learning researchers.

AIME-Con 2026 hype video, one-shotted by Claude Opus 5.5, with ElevenLabs for TTS and Python for layout and synthesized music.

I attended with the Renaissance Philanthropy Engineering Hub team. We had a few papers at the conference:

Overall, the conference was a good experience and I would recommend attending.

An assortment of thoughts:

  • For those used to computer-science-style conferences where every paper is fully baked, AIME-Con is more of an education conference and includes a lot of very preliminary and in-progress work. I find this pretty refreshing: a lot of interesting but flawed ideas, leading to good discussions!
  • The two best sessions I attended were panels featuring psychometricians and teachers. I get exposed to research enough in other places, but practitioner perspectives are irreplaceable.
  • A lot of LLM-generated language on slides! Notable because I noticed it in both good slide decks and bad slide decks. Even one of the keynotes had obviously AI-generated prose – and minimally edited at that! It definitely made me think less of the presenter.
  • Research timelines feel particularly brutal against the steady patter of new LLM releases. For studies based on older models, it can be hard to determine how the results do or do not generalize. In some cases, results based on working around the limitations of older models were pretty clearly obsoleted by newer releases.
  • A lot of discussion of automated assessment, which makes sense for a measurement conference. That led to a lot of discussions involving recently-released decision models like Jev and Clef. There’s excitement about what cheap, fast models can enable for in-the-moment formative assessments, and the calibrated probabilities can be used to direct teacher and tutor attention where it may be most helpful.
  • Interweaving Assessment is a great initiative.
  • I’m biased, but the two best talks I saw were “Machine-Verifiable Item Models for Geometry Diagram Generation” and “Fractional Information Gain: A Personalization Metric for Black-Box Predictive Models”. Hopefully the slides are online somewhere.
  • I live-posted a few of the talks and papers I enjoyed on Bluesky, which was fun. But, the days of academic Twitter are truly past; I only saw two other posts on Bluesky related to the conference (and none on X)! See all of them at the bottom of this post.
  • You can access the full paper proceedings here.

Pittsburgh was lovely. I enjoyed the Andy Warhol museum, the National Aviary, and the Cathedral of Learning. Apparently Warhol was quite the hoarder, and I enjoyed paging through materials in the time capsules he built during his life. Perhaps presciently, one of the articles Warhol saved was a March 1961 LIFE article called “The Machines Are Taking Over: Computers Outdo Man at His Work Now — and Soon May Outthink Him”.

LIFE magazine: The Machines Are Taking Over: Computers Outdo Man at his Work Now – and soon may outthink him LIFE: “The Machines Are Taking Over”; available on Google Books

And what an outro!

The Perceptron is so complex that even its inventor can no longer predict how it will react to a new problem. “If devices like the Perceptron,” says one expert, “can really learn effectively by themselves, we will be approaching the making of a true robot, fantastic as that sounds. But remember, all this was begun and devised by human brains, so humans—if they take care—will remain supreme.”

This is cheering news, no doubt. But there is another view of the future in a story that computer designers now tell only as a macabre joke: A weary programer who has spent his life tending a computer that always has the right answer for everything finally gets fed up. “All right,” he asks his machine, “if you’re so smart, tell me—is there a God?” The computer whirs gently, its lights flicker, its coils buzz and hum, and at last it clicks out its answer: THERE IS NOW.

Bluesky posts

I posted a few “in the moment” Bluesky posts during the conference, in part to try out Lea. I’m copying the posts here to put them all in one place.

Before the conference

I’ll be attending NCME’s “Artificial Intelligence in Measurement and Education Conference” next week for the first time! You can follow the conference in the Lea feed. #AIMECon2026

I’m not presenting anything, although I co-authored a paper exploring the use of Gemini-backed multi-modal transcripts to align events across learning systems. arxiv.org/abs/2609.36502

I’m expecting and hoping for deep, contentious conversations about the ethics of measurement in K-12 education! Say hi if you see me at the conference!

(thread, Oct 3)

Day 1: Hype video

AIME-Con 2026 starts today. This year’s theme is “Measurement Science in AI-Integrated Assessment and Pedagogy”. Here’s a one-shot Claude Opus 5.5 hype video to get in the mood (it is slop, but amusing).

Video thumbnail: AIME-Con 2026 hype video, one-shotted by Claude Opus 5.5, with ElevenLabs for TTS and Python for layout and synthesized music.

(post, with embedded video, Oct 5. This is the same video as at the top of this post.)

Day 1: keynote

Dragan Gašević keynote at AIME-Con 2026 osf.io/preprints/psyarxiv/s7dvk_v1

A 2×2 diagram of the SYNC model (Synergetic Notion of Human-AI Skills and Knowledge Construction), plotting learning gain against task performance with AI support, yielding four quadrants: Floundering, Effortful Learning, Autopilot Performance, and Synergetic Mastery.

(post, Oct 6)

Day 1: tutor-session transcription

Come see Dr. Danielle Thomas (CMU) talk about tutor-session transcription, today at 1:45pm #AIMECon2026

Video thumbnail: Claude Opus 5.5-generated hype video for research paper "Towards Breaking the Learning System Wall Using Multimodal Tutoring Transcriptions".

(post, with video, Oct 6)

Day 1: math metaphors

Using metaphors to elicit feelings about math. 1100 US middle-school students are asked “If math were a food, it would be ____ because ____.” and “If math were an animal, it would be ____ because ____.” Zhang et al. #AIMECon2026

(post, Oct 6)

Day 1: student use of chatbots

How do students use chatbots for online coursework? Qualitative work-in-progress at #AIMECon2026 Choi et al.

Summary card: How do students use chatbots for online coursework? Based on 1-on-1 naturalistic use sessions with thirteen students (grades 7–12). Students used the chatbot for practical, task-driven learning support. Chatbot responses were generally supportive, but boundary handling varied across interaction contexts. High-priority interactions revealed opportunities to strengthen pedagogical scaffolding. Students reported positive experiences, with important nuance around fact checking, effort, and future use.

(post, Oct 6)

Day 1: teacher praise during tutoring

In 157 teacher-facilitated tutoring sessions (grades 2 & 3), teachers include praise in 25% of their utterances. Work-in-progress at #AIMECon2026 Yoo et al.

(post, Oct 6)

Day 1: Students don’t read AI-generated writing feedback

Students don’t read writing feedback, and they don’t read AI writing feedback either! Study with GPT-5.1, undergrad psych class. ~10% revise in response to feedback. #AIMECon2026

Presentation slide from the University of Memphis Institute for Intelligent Systems, titled "…but most students don't spend time reading." A table shows how much of an AI reply students had time to read, the share of encounters in each group, and how often students then revised. Not even the verdict: 20% of encounters, 7% then revised. The verdict, but not the rest: 43% of encounters, 6% then revised. The whole reply: 37% of encounters, 19% then revised.

(post, Oct 6)

Day 2: taxonomy for AI-assisted human-to-human collaboration

Designing AI collaboration partners for classroom discussions. Work-in-progress from @pf-flyer.bsky.social at #AIMECon2026 aclanthology.org/2026.aimecon-wip.45

Photo of conference slide: Design taxonomy for collaboration partners. Taxonomy components: Context and setup, Modeling and evidence, Timing, Feedback and use, Scope and scale.

(post, Oct 7)

Day 2: AI literacy

Lot of interesting examples at #AIMECon2026 of how teachers are incorporating AI literacy into their lessons. Example from an ETS pilot study: students are encouraged to test the model’s boundaries and to compare their output to their neighbors.

Slide from an ETS presentation: "Making the system visible." The teacher guided every student to ask the AI the same task-clarification question, compare responses, and test its boundaries with questions of their choosing. Takeaway: transparency was created through guided experience, not merely by telling students how the AI worked.

(post, Oct 7)