Ryan Yen

Code Shaping: Iterative Code Editing with Free-form Sketching [pdf] [poster] [video] [bibtex]

Ryan Yen, Jian Zhao, Daniel Vogel

We present an initial step towards building a system for programmers to edit code using free-form sketch annotations drawn directly onto editor and output windows. Using a working prototype system as a technical probe, an exploratory study examines how programmers sketch to annotate Python code to communicate edits for an AI model to perform.

CoLadder: Manipulating Code Generation via Multi-Level Blocks [pdf] [doi] [video] [bibtex]

Ryan Yen, Jiawen Zhu, Sangho Suh, Haijun Xia, Jian Zhao

A system that assists programmers by enabling hierarchical task decomposition, incremental code generation, and verification of results during prompt authoring. Bridging the abstraction gap between programmers and LLMs.

Reifying the Reuse of User-AI Conversational Memories [pdf] [doi] [video] [bibtex]

Ryan Yen, Jian Zhao

To support users in recalling and reusing relevant user-AI conversational memories, we introduce Memolet, an interactive object that reifies memory reuse. Users can directly manipulate Memolet to specify which memories to reuse and how to use them. We developed a system demonstrating Memolet's interaction across various memory reuse stages.

Gait Gestures: Examining Stride and Foot Strike Variation as an Input Method While Walking [pdf] [doi] [video]

ChingYi Tsai, Ryan Yen, Day Kim, Daniel Vogel

We conduct a systematic examination of different kinds of intentional variations from a normal gait that could be used as input actions without interrupting overall walking progress. A design space of 22 candidate Gait Gestures is generated by adapting previous standing foot input actions and identifying new actions possible in a walking context.

Scholarly Exploration via Conversations with Scholars-Papers Embedding [pdf] [poster] [video]

Ryan Yen, Yelizaveta Brus, Leyi Yan, Jian Zhao, Jimmy Lin

We propose a system that supports contextually aware, controllable, and interactive exploration of academic publications and scholars, enabling bidirectional interaction between question-answering components and Scholets, the 2D projections of scholarly works' embeddings, demonstrated through an exploratory study with graduate researchers.

To Search or To Gen? Exploring the Synergy between Generative AI and Web Search in Programming [pdf] [poster] [doi] [video]

Ryan Yen, Nicole Sultanum, Jian Zhao

We have identified three major challenges and proposed three decision-making stages, each with its own relevant factors. Additionally, we present a thorough process model that captures programmers' interaction patterns.

CoPrompt: Supporting Prompt Sharing and Referring in Collaborative Natural Language Programming [pdf] [doi] [video]

Ryan Yen*, My girlfriend 😚Felicia Feng*, Yuzhe You, Mingming Fan, Jian Zhao, Zhicong Lu ^{*These authors contributed equally to this work.}

A system to support collaborative prompt engineering by providing referring, requesting, sharing, and linking mechanisms. It assists programmers in comprehending collaborators' prompts and building on their collaborators' work, reducing repetitive updates and communication costs.

StoryChat: Designing a Narrative-Based Viewer Participation Tool for Live Streaming Chatrooms [pdf] [doi] [video]

Ryan Yen, My girlfriend 😚Felicia Feng, Brinda Mehra, Ching Christie Pang, Siying Hu, Zhicong Lu

A narrative-based viewer participation tool that utilizes a dynamic graphical plot to reflect chatroom negativity. We discovered that StoryChat encouraged viewers to contribute prosocial comments, increased viewer engagement, and fostered viewers' sense of community.

Gist and Verbatim: Understanding Speech to Inform New Interfaces for Verbal Text Composition [pdf] [doi] [video]

Brinda Mehra, Kejia Shen, Ryan Yen, Can Liu

Exploring speech input in HCI, we address editing challenges. Our study combines Cognitive Science with HCI, revealing memory patterns and proposing new interaction concepts for efficient speech editing.

Wizundry: A Cooperative Wizard of Oz Platform for Simulating Future Speech-based Interfaces with Multiple Wizards [pdf] [doi] [video]

Siying Hu, Ryan Yen, Ziwei Yu, Mingjian Zhao, Katie Seaborn, Can Liu

A real-time, web-based WoZ platform that allows multiple Wizards to collaboratively operate a speech-to-text based system remotely. Our findings reveal the promises and challenges of the multi-Wizard approach and open up new research questions.

MIT

CSAIL

• MIT EECS Grad Fellowship
• Research Assistant in MIT VIS Lab

University of Waterloo

David R. Cheriton School of Computer Science

• UW Grad Scholarship
• International Masters Award Excellence Scholarship
• Graduate Research Studentship Scholarship
• TA Funding
• Research Assistant in WatVis Lab

City University of Hong Kong

Department of Computer Science

• Vice President of Computer Science Student Chapter
• 4 years Full CityU International Student Scholarship
• Awarded Dean's List
• HK Scholarship Fund
• HK STEM Internship Scholarship
• Department Overseas Learning Funding
• English Proficiency Scholarship
• Departmental Research Funding
• Research Assistant in DEER Lab
• Research Assistant in ERFI Lab