Manish Shetty

Papers

Research publications and preprints.

ICLR 2026
Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
Mike A. Merrill, Alexander G. Shaw, ..., Manish Shetty, ..., Ludwig Schmidt
paper / website
NeurIPS 2025
GSO: Challenging Software Optimization Tasks for Evaluating SWE-Agents
Manish Shetty, Naman Jain, Jinjian Liu, Vijay Kethanaboyina, Koushik Sen, Ion Stoica
paper / website / code / blog / included in EpochAI's ECI
COLM 2025
R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents
Naman Jain*, Jaskirat Singh*, Manish Shetty, Liang Zheng, Koushik Sen, Ion Stoica
paper / website
ICML 2025
Challenges and Paths Towards AI for Software Engineering
Alex Gu, Naman Jain*, Wen-Ding Li*, Manish Shetty*, Yijia Shao, Ziyang Li, Diyi Yang, Kevin Ellis, Koushik Sen, Armando Solar-Lezama
paper
ICSE 2025
Syzygy: Dual Code-Test C to Rust Translation using LLMs and Dynamic Analysis
Manish Shetty*, Naman Jain*, Adwait Godbole*, Sanjit Seshia, Koushik Sen
paper / website
ICML 2024
R2E: Turning any GitHub Repository into a Programming Agent Environment
Manish Shetty*, Naman Jain*, Tianjun Zhang, King Han, Koushik Sen, Ion Stoica
paper / website / code
MLSys 2025
AIOpsLab: A Holistic Framework to Evaluate AI Agents for Enabling Autonomous Clouds
Yinfang Chen, Manish Shetty, Gagan Somashekar, Minghua Ma, Yogesh Simmhan, Jonathan Mace, Chetan Bansal, Rujia Wang, Saravan Rajmohan
paper / website / code
SoCC 2024
Building AI Agents for Autonomous Clouds: Challenges and Design Principles
Manish Shetty, Yinfang Chen, Gagan Somashekar, Minghua Ma, Yogesh Simmhan, Xuchao Zhang, Jonathan Mace, Dax Vandevoorde, Pedro Las-Casas, Shachee Mishra Gupta, Suman Nath, Chetan Bansal, Saravan Rajmohan
paper / featured on Microsoft Research Blog
ArXiv 2024
LM Assertions: Computational Constraints for Self-Refining LM Pipelines
Arnav Singhvi*, Manish Shetty*, Shangyin Tan*, Chris Potts, Koushik Sen, Matei Zaharia, Omar Khattab
paper / code
ArXiv 2023
CodeScholar: Growing Idiomatic Code Examples
Manish Shetty, Koushik Sen, Ion Stoica
paper / code
SoCC 2022
How to Fight Production Incidents? An Empirical Study on a Large-scale Cloud Service
Supriyo Ghosh, Manish Shetty, Chetan Bansal, Suman Nath
paper / Best Paper Award
FSE 2022
AutoTSG: Learning and Synthesis for Incident Troubleshooting
Manish Shetty, Chetan Bansal, Sai Upadhyayula, Arjun Radhakrishna, Anurag Gupta
paper
ICSE 2022
DeepAnalyze: Learning to Localize Crashes at Scale
Manish Shetty, Chetan Bansal, Suman Nath, Sean Bowles, Henry Wang, Ozgur Arman, Siamak Ahari
paper / slides
ICSE 2021
Neural Knowledge Extraction From Cloud Service Incidents
Manish Shetty, Chetan Bansal, Sumit Kumar, Nikitha Rao, Nachiappan Nagappan, Thomas Zimmermann
paper / slides / featured on VentureBeat