CursorBench 3.1 introduces new tasks aimed at codebase understanding and bugfinding, enhancing the evaluation of coding skills. The update also includes improved grading criteria for specific editing tasks to better assess developer performance.
CursorBench 3.1 has added tasks that specifically target codebase understanding, bugfinding, planning, and code review. These tasks are designed to enhance evaluation metrics for developers and their coding capabilities.
Alongside the new tasks, the update incorporates improved grading criteria for certain edit tasks. This adjustment aims to refine how developers' performances are assessed during these tasks, potentially leading to more accurate evaluations.
The prior version, CursorBench 3.0, focused primarily on edit, refactor, and bugfix challenges. This groundwork has been expanded with the introduction of more diverse problem types in version 3.1.
CursorBench computes the average cost per task by applying published pricing for each model used in tasks. The calculation considers inputs, cache read/write, and outputs, providing a cost-benefit analysis of performance across tasks. Variance in results is acknowledged, indicating that small differences might not be statistically significant.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
CursorBench 3.1 introduces new tasks aimed at codebase understanding and bugfinding, enhancing the evaluation of coding skills. The update also includes improved grading criteria for specific editing tasks to better assess developer performance.