Automated Clustering and Program Repair for Introductory Programming Assignments

Abstract

Providing feedback on programming assignments is a tedious task for the instructor, and even impossible in large MOOCs with thousands of students. In this paper, we present a novel technique for automatic feedback generation: (1) For a given programming assignment, we automatically cluster the correct student attempts using a dynamic program analysis. From each cluster we select one student attempt as a specification. (2) Given an incorrect student attempt we run a repair procedure against all specifications, and automatically generate the minimal repair based on one of them. We evaluated the approach on 56,807 student attempts over 18 problems from an undergraduate programming course and a MOOC course. In our evaluation we first automatically clustered the correct student attempts and generated specifications, and then used these specifications to repair the incorrect student attempts without any human intervention, finding repairs for 90% of the incorrect attempts.