I am a PhD student at UC Berkeley in the Sky Lab studying AI4Code.

I work on turning language models into capable software agents.

My focus has been building benchmarks ("evals") and environments that either stage long-horizon and challenging coding tasks or elicit more from models by scaling compute. My work spans the software lifecycle: code completion, optimization, translation, and deployment.

From 2020 to 2022, I was a research fellow at Microsoft Research.

News

Papers

GSO: Challenging Software Optimization Tasks for Evaluating SWE-Agents
Manish Shetty, Naman Jain, Jinjian Liu, Vijay Kethanaboyina, Koushik Sen, Ion Stoica
Preprint, 2025
paper / website / dataset / code

R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents
Naman Jain^*, Jaskirat Singh^*, Manish Shetty, Liang Zheng, Koushik Sen, Ion Stoica
COLM 2025: To Appear at the 2nd Conference on Language Modeling
paper / website

Challenges and Paths Towards AI for Software Engineering
Alex Gu, Naman Jain^*, Wen-Ding Li^*, Manish Shetty^*, Yijia Shao, Ziyang Li, Diyi Yang,
Kevin Ellis, Koushik Sen, Armando Solar-Lezama
ICML 2025: Proceedings of the 42nd International Conference on Machine Learning (Position Paper)
paper

Syzygy: Dual Code-Test C to Rust Translation using LLMs and Dynamic Analysis
Manish Shetty^*, Naman Jain^*, Adwait Godbole^*, Sanjit Seshia, Koushik Sen
LLM4Code Workshop @ ICSE 2025
paper / website

R2E: Turning any GitHub Repository into a Programming Agent Environment
Manish Shetty^*, Naman Jain^*, Tianjun Zhang, King Han, Koushik Sen, Ion Stoica
ICML 2024: Proceedings of the 41st International Conference on Machine Learning
paper / code / website

AIOpsLab: A Holistic Framework to Evaluate AI Agents for Enabling Autonomous Clouds
Yinfang Chen, Manish Shetty, Gagan Somashekar, Minghua Ma, Yogesh Simmhan, Jonathan Mace, Chetan Bansal, Rujia Wang, and Saravan Rajmohan
MLSys 2025: Proceedings of the 9th Annual Conference on Machine Learning and Systems
paper / code / website

Building AI Agents for Autonomous Clouds: Challenges and Design Principles
Manish Shetty, Yinfang Chen, Gagan Somashekar, Minghua Ma, Yogesh Simmhan, Xuchao Zhang, Jonathan Mace, Dax Vandevoorde, Pedro Las-Casas, Shachee Mishra Gupta, Suman Nath, Chetan Bansal, Saravan Rajmohan
SoCC 2024: Proceedings of the 14th Symposium on Cloud Computing
paper / featured on Microsoft Research Blog

LM Assertions: Computational Constraints for Self-Refining LM Pipelines
Arnav Singhvi^*, Manish Shetty^*, Shangyin Tan^*, Chris Potts, Koushik Sen, Matei Zaharia, Omar Khattab
ArXiv, 2024
paper / code / tweet-1 / tweet-2

CodeScholar: Growing Idiomatic Code Examples
Manish Shetty, Koushik Sen, Ion Stoica
ArXiv, 2023
paper / code

How to Fight Production Incidents? An Empirical Study on a Large-scale Cloud Service
Supriyo Ghosh, Manish Shetty, Chetan Bansal, Suman Nath
SoCC 2022: Proceedings of the 13th Symposium on Cloud Computing
paper / Best Paper Award 🏆

AutoTSG: Learning and Synthesis for Incident Troubleshooting
Manish Shetty, Chetan Bansal, Sai Upadhyayula, Arjun Radhakrishna, Anurag Gupta
FSE 2022: Proceedings of the 30th ACM Symposium on the Foundations of Software Engineering
paper

DeepAnalyze: Learning to Localize Crashes at Scale
Manish Shetty, Chetan Bansal, Suman Nath, Sean Bowles, Henry Wang, Ozgur Arman, Siamak Ahari
ICSE 2022: Proceedings of the 44th International Conference on Software Engineering
paper / slides / talk-1 / talk-2 / talk-3

Neural Knowledge Extraction From Cloud Service Incidents
Manish Shetty, Chetan Bansal, Sumit Kumar, Nikitha Rao, Nachiappan Nagappan, Thomas Zimmermann
ICSE 2021: Proceedings of the 43rd International Conference on Software Engineering
paper / slides / talk / featured on VentureBeat

Awards

Teaching

• Spring 2023 CS164: Programming Languages and Compilers (Head TA with Koushik Sen)
  website / course evaluations

Service