Dhruv Kumar

Assistant Professor at BITS PilaniOn Leave From IIIT Delhi
Post-Doc from Microsoft ResearchPhD from UMN Twin Cities

prof_pic_cropped.jpeg

6120-N, NAB

BITS Pilani

Pilani, Rajasthan - 333031

I am currently an assistant professor at BITS Pilani. I am also an adjunct faculty at IIIT Delhi where I was previously a full-time faculty from 2022 to 2024. I continue to collaborate closely with students and faculty members at IIIT Delhi.

Before joining IIIT Delhi, I was a Post-Doc Researcher at Microsoft Research. Prior to Post-Doc, I completed my PhD under Abhishek Chandra in the Computer Science Department at the University of Minnesota, Twin Cities, USA. During my PhD, I also collaborated with Ramesh Sitaraman at University of Massachusetts, Amherst, USA. Long time back, I graduated in 2014 with a Bachelors in Computer Science from BITS, Pilani.

I consider myself a scientist before a computer scientist. Hence, I am interested in solving real life problems even if they are simple and/or not strictly related to computer science. I am particularly interested in solving problems which are specific to developing countries such as India.

As part of computer science research, I am primarily interested in Generative AI, Large Language Models, Agentic AI and Computing Education. Sometimes, I also work on topics related to Human-computer Interaction, ICT4D and Distributed Systems.

News

Aug 3, 2025 Our paper on “Improving LLM-Based Code Evaluation With Question-Specific Rubrics” accepted and published in ACM ICER 2025.
Aug 1, 2025 Preprint released for our research work on “A Hybrid Framework of Reasoning optimised LLMs and SMT Solvers for Loop Invariant Generation”
Aug 1, 2025 Preprint released for our research work on “Automated Type Annotation in Python Using Large Language Models”
May 25, 2025 Preprint released for our research work on “Investigating Pedagogical Teacher and Student LLM Agents”
May 20, 2025 Our paper on “Survey of Transformers Based Methods for Computer Vision” accepted and published in IEEE Access.
May 6, 2025 Preprint released for our research work on “The Impact of Large Language Models on K-12 Education in Rural India”
Mar 17, 2025 Our paper on “The role of generative AI tools in shaping mechanical engineering education” accepted and published in Nature Scientific Reports.
Mar 16, 2025 Preprint released for our research work on “AI-assisted Coding Tutors in collaboration with Disha AI”
Feb 18, 2025 Preprint released for our research work on “ReviewEval: An Evaluation Framework for AI-Generated Reviews”
Feb 13, 2025 Our paper titled “Analyzing LLM Usage in an Advanced Computing Class in India” accepted and presented at ACE 2025.
Feb 5, 2025 Preprint released for our research work on “DebateBench: A Challenging Long Context Reasoning Benchmark For Large Language Models”
Dec 4, 2024 Preprint released for our research work on “System Test Case Design from Requirements Specifications: Insights and Challenges of Using ChatGPT”
Sep 22, 2024 Preprint released for our research work on Prototyping Financially Responsible Interfaces for UPI Apps .
Aug 31, 2024 Received the Teaching Excellence Award for AY 2023-24 at IIIT Delhi.
Jul 25, 2024 Preprint released for our paper titled “TAMIGO: Empowering Teaching Assistants using LLM-assisted viva and code assessment in an Advanced Computing Class”
Jul 25, 2024 Preprint updated for our paper titled “Analyzing LLM Usage in an Advanced Computing Class in India”
May 5, 2024 Several newspapers and media outlets recently covered our research study on the impact of UPI on Spending Behavior among Indian Users. [Zee News] [Aaj Tak] [NDTV India] [Zee Business] [LallanTop] [India Today] [Aaj Tak] [NDTV Profit] [Full Details]
May 4, 2024 Our paper titled “A Comparative Analysis of Large Language Models for Code Documentation Generation” has been accepted to ACM AIware 2024
Apr 30, 2024 Our paper titled “Comuniqa: Exploring Large Language Models for improving English speaking skills” accepted in ACM COMPASS 2024.
Apr 19, 2024 Our paper titled “From Cash to Cashless: UPI’s Impact on Spending Behavior among Indian Users” accepted in ACM CHI 2024 - Late Breaking Work Track.

Selected Publications

  1. ICER 2025
    Rubric Is All You Need: Improving LLM-Based Code Evaluation With Question-Specific Rubrics
    Aditya Pathak, Rachit Gandhi, Vaibhav Uttam, Arnav Ramamoorthy, and 10 more authors
    In Proceedings of the 2025 ACM Conference on International Computing Education Research V.1, 2025
  2. arXiv 2025
    Automated Type Annotation in Python Using Large Language Models
    Varun Bharti, Shashwat Jha, Dhruv Kumar, and Pankaj Jalote
    2025
  3. arXiv 2025
    Investigating Pedagogical Teacher and Student LLM Agents: Genetic Adaptation Meets Retrieval Augmented Generation Across Learning Style
    Debdeep Sanyal, Agniva Maiti, Umakanta Maharana, Dhruv Kumar, and 3 more authors
    2025
  4. IEEE Access 2025
    Transformers for Vision: A Survey on Innovative Methods for Computer Vision
    Balamurugan Palanisamy, Vikas Hassija, Arpita Chatterjee, Arpita Mandal, and 4 more authors
    IEEE Access, 2025
  5. arXiv 2025
    The Impact of Large Language Models on K-12 Education in Rural India: A Thematic Analysis of Student Volunteer’s Perspectives
    Harshita Goyal, Garima Garg, Prisha Mordia, Veena Ramachandran, and 2 more authors
    2025
  6. Sci Rep 2025
    The role of generative AI tools in shaping mechanical engineering education from an undergraduate perspective
    Harshal Akolekar, Piyush Jhamnani, Vikash Kumar, Vinay Tailor, and 5 more authors
    Scientific Reports, 2025
  7. arXiv 2025
    Sakshm AI: Advancing AI-Assisted Coding Education for Engineering Students in India Through Socratic Tutoring and Comprehensive Feedback
    Raj Gupta, Harshita Goyal, Dhruv Kumar, Apurv Mehra, and 3 more authors
    2025
  8. arXiv 2025
    ReviewEval: An Evaluation Framework for AI-Generated Reviews
    Chavvi Kirtani, Madhav Krishan Garg, Tejash Prasad, Tanmay Singhal, and 2 more authors
    2025
  9. arXiv 2024
    TAMIGO: Empowering Teaching Assistants using LLM-assisted viva and code assessment in an Advanced Computing Class
    Anishka IIITD, Diksha Sethi, Nipun Gupta, Shikhar Sharma, and 3 more authors
    2024
  10. arXiv 2024
    Analyzing LLM Usage in an Advanced Computing Class in India
    Anupam Garg, Aryaman Raina, Aryan Gupta, Jaskaran Singh, and 9 more authors
    2024
  11. COMPASS 2024
    Comuniqa : Exploring Large Language Models for improving speaking skills
    Manas Mhasakar, Shikhar Sharma, Apurv Mehra, Utkarsh Venaik, and 3 more authors
    2024
  12. arXiv 2024
    AI as a Medical Ally: Evaluating ChatGPT’s Usage and Impact in Indian Healthcare
    Aryaman Raina, Prateek Mishra, Harshit goyal, and Dhruv Kumar
    2024
  13. CHI LBW 2024
    From Cash to Cashless: UPI’s Impact on Spending Behavior among Indian Users
    Harshal Dev, Raj Gupta, and Dhruv Kumar
    2024
  14. ACE 2024
    "It’s not like Jarvis, but it’s pretty close!" – Examining ChatGPT’s Usage among Undergraduate Students in Computer Science
    Ishika Joshi*, Ritvik Budhiraja*, Harshal D Akolekar, Jagat Sesh Challa, and 1 more author
    2024
  15. SIGCSE TS 2024
    ChatGPT in the Classroom: An Analysis of Its Strengths and Weaknesses for Solving Undergraduate Computer Science Questions
    Ishika Joshi*, Ritvik Budhiraja*, Harshal Dev, Jahnvi Kadia, and 4 more authors
    2024
  16. AIware 2024
    A Comparative Analysis of Large Language Models for Code Documentation Generation
    Shubhang Shekhar Dvivedi, Vyshnav Vijay, Sai Leela Rahul Pujari, Shoumik Lodh, and 1 more author
    2024
  17. arXiv 2023
    "With Great Power Comes Great Responsibility!": Student and Instructor Perspectives on the influence of LLMs on Undergraduate Engineering Education
    Ishika Joshi*, Ritvik Budhiraja*, Pranav Deepak Tanna, Lovenya Jain, and 6 more authors
    2023
  18. CCGrid 2023
    AggFirstJoin: Optimizing Geo-Distributed Joins using Aggregation-Based Transformations
    Dhruv Kumar, Sohaib Ahmad, Abhishek Chandra, and Ramesh K. Sitaraman
    In 23RD IEEE/ACM INTERNATIONAL SYMPOSIUM ON Cluster, Cloud and Internet Computing, Bengaluru, India, May 1-4, 2023, 2023
  19. IPDPS 2022
    HACCS: Heterogeneity-Aware Clustered Client Selection for Accelerated Federated Learning
    Joel Wolfrath, Nikhil Sreekumar, Dhruv Kumar, Yuanli Wang, and 1 more author
    In 2022 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2022, Lyon, France, May 30 - June 3, 2022, 2022
  20. SEC 2021
    AggNet: Cost-Aware Aggregation Networks for Geo-distributed Streaming Analytics
    Dhruv Kumar, Sohaib Ahmad, Abhishek Chandra, and Ramesh K. Sitaraman
    In 6th IEEE/ACM Symposium on Edge Computing, SEC 2021, San Jose, CA, USA, December 14-17, 2021, 2021
  21. SIGMETRICS 2019
    A TTL-based Approach for Data Aggregation in Geo-distributed Streaming Analytics
    Dhruv Kumar, Jian Li, Abhishek Chandra, and Ramesh K. Sitaraman
    SIGMETRICS, 2019