My research interest is systems for machine learning. I look for scalable and programmable abstractions that bind accelerators, models, and application logic together. I believe that a good system empowers its users, embraces heterogeneity, and adapts to change—all without fighting physics.

Recently, I have been applying these principles to PiePie, a programmable LLM serving system that enables dynamic application logic to run natively within the inference engine, unlocking new possibilities for AI agents.

My broader interests span programming languages, operating systems, neurosymbolic computing, and game engines. At Yale, I am fortunate to be advised by Prof. Lin Zhong.

News

Aug 2025
Cacheback accepted to EMNLP 2025 (Main).
Jul 2025
Pie accepted to SOSP 2025.
Apr 2025
Serve Programs, Not Prompts accepted to HotOS 2025.
Feb 2024
Prompt Cache accepted to MLSys 2024.

Publications

Cacheback: Speculative Decoding With Nothing But Cache
Zhiyao Ma*, In Gim*, and Lin Zhong
EMNLP 2025 PDF Code (*Equal contribution)
Pie: A Programmable Serving System for Emerging LLM Applications
In Gim, Zhiyao Ma, Seung-Seob Lee, and Lin Zhong
SOSP 2025 PDF Code
Serve Programs, Not Prompts
In Gim and Lin Zhong
HotOS 2025 PDF
Wiretapping LLMs: Network Side-Channel Attacks on Interactive LLM Services
Mahdi Soleimani, Grace Jia, In Gim, Seung-Seob Lee, Anurag Khandelwal
Preprint 2025 PDF
Asynchronous LLM Function Calling
In Gim, Seung-Seob Lee, and Lin Zhong
Preprint 2024 PDF
Confidential Prompting: Protecting User Prompts from Cloud LLM Providers
In Gim*, Caihua Li*, and Lin Zhong
Preprint 2024 PDF Code (*Equal contribution)
Prompt Cache: Modular Attention Reuse for Low-Latency Inference
In Gim, Guojun Chen, Seung-Seob Lee, Nikhil Sarda, Anurag Khandelwal, and Lin Zhong
MLSys 2024 PDF Code
Prior to Yale
Memory-Efficient DNN Training on Mobile Devices
In Gim and JeongGil Ko
MobiSys 2022 PDF Code
Fast Monte-Carlo Approximation of the Attention Mechanism
In Gim and JeongGil Ko
AAAI 2022 PDF Code

Education

Yale Yale University
2022 - Present
Ph.D. in Computer Science
M.S. in Computer Science (2024)
Yonsei Yonsei University
2021
B.S. in Integrated Technology

Work Experience

Apple
Summer 2025
AI/ML Research Intern

Teaching

Principles of Computer System Design
Fall 2024
Yale CPSC 429
Intro to Systems Programming
Spring 2024
Yale CPSC 323

Talks

Rethinking LLM Serving From the Application's Perspective
Oct 2025
Seoul National Univ, Yonsei Univ, NAVER, Furiosa AI
LLM Prompt Caching
Sep 2024
NVIDIA