You will evaluate real JavaScript software projects to help measure how well AI systems handle software engineering and bug-fixing tasks. The work includes reviewing codebases, issues, tests, and AI-generated fixes.
- Analyze and triage issues in open-source JavaScript libraries.
- Set up and configure repositories, including Docker-based development environments.
- Review unit-test coverage and assess test quality.
- Modify and run code locally, then evaluate bug-fixing results.
- Assess how well large language models handle software engineering tasks.
- Work with researchers to find repositories and issues that challenge AI systems.