StarCoder 2: The New Era of Code Generation for Developers

In a world where developers are increasingly turning to AI-powered code generators to streamline their work, a new player has entered the scene. StarCoder 2, the latest iteration of an open source code generator created by AI startup Hugging Face and workflow automation platform ServiceNow, promises to make coding more efficient without sacrificing speed or quality.

StarCoder 2 isn’t a single model, but a family of three models with varying parameters, all capable of running on most modern consumer GPUs. The first model, a 3-billion-parameter (3B) one, was trained by ServiceNow. The second, a 7-billion-parameter (7B) model, was trained by Hugging Face. The third and largest model, a 15-billion-parameter (15B) one, was trained by Nvidia, the newest supporter of the StarCoder project.

Like most other code generators, StarCoder 2 can suggest ways to complete unfinished lines of code and summarize and retrieve snippets of code when asked in natural language. However, it was trained with 4 times more data than the original StarCoder, resulting in significantly improved performance at lower costs to operate.

The new code generator can be fine-tuned in a few hours using a GPU like the Nvidia A100 on first- or third-party data to create apps such as chatbots and personal coding assistants. It was trained on a larger and more diverse data set than the original StarCoder, allowing it to make more accurate, context-aware predictions.

However, not every developer may agree that StarCoder 2 can truly deliver on speed and quality. A recent Stanford study found that engineers who use code-generating systems are more likely to introduce security vulnerabilities in the apps they develop. Additionally, a poll from Sonatype shows that the majority of developers are concerned about the lack of insight into how code from code generators is produced and the potential for “code sprawl” from generators producing too much code to manage.

StarCoder 2’s license might also prove to be a roadblock for some. It is licensed under the BigCode Open RAIL-M 1.0, which aims to promote responsible use by imposing “light touch” restrictions on both model licensees and downstream users. While less constraining than many other licenses, RAIL-M isn’t truly “open” in the sense that it doesn’t permit developers to use StarCoder 2 for every conceivable application. Some commentators say RAIL-M’s requirements may be too vague to comply with in any case, and that RAIL-M could conflict with AI-related regulations like the EU AI Act.

Despite these potential drawbacks, StarCoder 2 appears to be more efficient than one of the versions of Code Llama, Code Llama 33B. Hugging Face says that StarCoder 2 15B matches Code Llama 33B on a subset of code completion tasks at twice the speed. Additionally, as an open source collection of models, StarCoder 2 has the advantage of being able to deploy locally and “learn” a developer’s source code or codebase, an attractive prospect to devs and companies wary of exposing code to a cloud-hosted AI.

Hugging Face, ServiceNow, and Nvidia also make the case that StarCoder 2 is more ethical and less legally fraught than its rivals. All GenAI models regurgitate data they were trained on, which can lead to developers unwittingly using copyrighted code. However, StarCoder 2 was trained only on data under license from the Software Heritage, a nonprofit organization providing archival services for code. Ahead of StarCoder 2’s training, code owners were given the chance to opt out of the training set if they wanted.

As with the original StarCoder, StarCoder 2’s training data is available for developers to fork, reproduce, or audit as they please. This transparency sets it apart from other code generators, which have been criticized for a lack of information about the data that went into training them and how they were trained.

StarCoder 2 isn’t perfect, of course. Like other code generators, it’s susceptible to bias and performs weaker on certain programming languages. However, it represents a step in the right direction towards building trust and accountability with AI models.

So why are Hugging Face, ServiceNow, and Nvidia investing in a project like StarCoder 2? It’s a tried-and-true strategy: foster goodwill and build paid services on top of the open source releases. ServiceNow has already used StarCoder to create Now LLM, a product for code generation fine-tuned for ServiceNow workflow patterns, use cases, and processes. Hugging Face, which offers model implementation consulting plans, is providing hosted versions of the StarCoder 2 models on its platform. So is Nvidia, which is making StarCoder 2 available through an API and web front-end.

For devs expressly interested in the no-cost offline experience, StarCoder 2 – the models, source code, and more – can be downloaded from the project’s GitHub page. Whether or not StarCoder 2 will become the go-to code generator for developers remains to be seen, but one thing is clear: the demand for efficient, ethical, and accessible AI-powered coding tools is only growing.

---

Grow your business with AI. Be an AI expert at your company in 5 mins per week with this Free AI Newsletter