Saikat Dutta and Kevin Ellis, both assistant professors of computer science in the Cornell Ann S. Bowers College of Computing and Information Science, have received a grant from Meta to further develop the capabilities of artificial intelligence (AI) in software development.
Through this support, they will develop a next-generation benchmark that will enable researchers to measure their progress in using generative AI for software development and to expand the scope of the problems these models can address. The benchmark will cover areas such as fixing bugs, responding to human code reviews, generating software tests, and translating one programming language into another.
"The future impact of AI-automated software development will be far-ranging," said Dutta and Ellis in a statement. "Beyond building and improving apps, AI will help us write more comprehensive software tests, review code, and perform many other tasks that are part of the software development life cycle."
Dutta and Ellis received one of four grants awarded by Meta through their Large Language Model (LLM) Evaluation Research Grant program. The program was designed to create new benchmarks for the behavior of LLMs, which are AI-based models designed to process human language – both to ensure that the models can be trusted and to expand the scope of their performance.
"The teams at Meta are really excited to collaborate with Dr. Saikat Dutta and his team at Cornell on building new benchmarks for evaluations of large language models, and more broadly, to continue working with the broader academic community on pushing the boundaries of new research in the LLM space," said Nivi Obla, program manager, GenAI at Meta.
This work is in line with Dutta’s and Ellis' larger research programs. Dutta's team focuses on software engineering, specifically developing software testing and debugging techniques. He aims to build tools and techniques that improve the reliability of machine learning-based systems and is also exploring how to leverage the latest machine learning techniques to solve software engineering problems.
Ellis' work centers around neural program synthesis, and, more broadly, on ways of using machine learning to generate programs and other forms of human-understandable symbolic models.
Patricia Waldron is a writer for the Cornell Ann S. Bowers College of Computing and Information Science