Google’s Angular team has created a new open-source tool called Web Codegen Scorer. This tool helps developers evaluate the quality of web code produced by large language models (LLMs). It works with any web library or framework, making it flexible for different projects.

The tool was introduced on September 16 and is designed specifically for web code generation. Simona Cotin, a senior engineering manager at Angular, explained that Web Codegen Scorer helps fine-tune prompts used with LLMs. These prompts, which can be found at angular.dev/ai, are optimized to get better results from AI models when generating code for Angular and other frameworks.

Using Web Codegen Scorer, developers can make smarter decisions about AI-created code. They can test different prompts to see which instructions produce the best results. It also allows comparing code quality from various models and tracking how generated code improves over time. This makes it easier to see what’s working and what needs fixing.

Unlike many other code benchmarks, Web Codegen Scorer focuses specifically on web code. It uses well-known measures of code quality to assess the output. The tool is versatile and compatible with any web framework or library, or even if no framework is used. It can also be used with any AI model.

Installing the tool is straightforward, with instructions available on GitHub. Once set up, developers can customize evaluations by choosing different models, frameworks, and tools. They can add system instructions and connect MCP (Model Context Protocol) servers for more advanced testing.

Web Codegen Scorer includes built-in checks for common issues like build failures, runtime errors, accessibility problems, security vulnerabilities, and adherence to coding best practices. If issues are detected, the tool can automatically try to fix them, saving time and effort.

The results of these evaluations can be viewed through a report viewer user interface, making it easy to compare different runs and see detailed insights. This helps teams improve their AI-generated code quality, ensuring better, more reliable web applications.

In summary, Web Codegen Scorer is a powerful addition for developers working with AI and web development. It provides a clear, evidence-based way to assess and improve the code generated by AI models, no matter what framework or tools they use.

Inspired by

• https://www.infoworld.com/article/4061080/web-codegen-scorer-evaluates-ai-generated-web-code.html

Sources

• gmpg.org
• angular.dev
• angular.dev
• podigee.com
• podigee.com