There’s no two ways about it: (generative) AI is changing the way we work, and coding is no exception. On the contrary: besides general-purpose assistants like OpenAI’s ChatGPT and Google’s Gemini, we also have purpose-built tools for writing code and related documents. But what if you want to focus on not just development, but operations as well? 

Gilles Plaquet, one of our DevOps engineers, spent some time figuring out just that. He used Gemini Code Assist, GitHub Copilot, and Amazon CodeWhisperer for a few weeks to see how they perform in a wide range of DevOps-related tasks, ranging from infrastructure configuration to writing documentation. In this blog, he shares his findings.  

Summary: who’s good at what? 

GitHub Copilot: the programming prodigy 

If writing code is the name of your game, Copilot is probably your answer. That should hardly come as a surprise. Specifically developed by GitHub and trained on a boatload of code, Copilot excels at generating code snippets and understanding complex class structures.

However, it lacks deeper integration with specific environments and DevOps tools. So, while it’s fantastic for pure coding, don’t expect it to help much with environment-specific issues or detailed infrastructure tasks.

AWS CodeWhisperer: the in-house expert 

Amazon’s CodeWhisperer is an absolute beast within the AWS ecosystem. It’s the go-to for infrastructure tasks and seamlessly integrates with a bunch of AWS services. As soon as you leave the AWS bubble, however, things quickly take a turn for the worse.

If your work revolves around AWS, CodeWhisperer is a no-brainer. For multi-cloud or hybrid setups, though, you’re bound to hit some snags sooner rather than later, so we recommend looking for a less specialised alternative.

Google Gemini Code Assist: the jack of all trades 

Compared to our two other tools, Gemini Code Assist is pretty much the Swiss Army knife of AI assistants. Just like OpenAI’s ChatGPT or Anthropic’s Claude, it’s a general-purpose assistant that can also be used for coding but supports many other use cases.

It performs well in and out of the Google Cloud environment, and its ability to consider databases during log analysis is a game-changer for large enterprises. This makes it a robust option for organisations dealing with complex, multi-faceted infrastructures. As Gilles points out, Gemini Code Assist has come a long way and has some impressive prospects:

In the land of integration, Gemini is king 

When it comes to integrations, both CodeWhisperer and Copilot offer seamless integration with popular IDEs like Visual Studio Code, IntelliJ IDEA, and PyCharm through extensions – as you’d expect from developer-targeted tools.

Gemini Cloud Assist does things a bit different. It evaluates entire projects, providing prompts directly within the IDE, which can save a ton of time. However, this ability is a double-edged sword. It gives you a comprehensive view of your entire codebase, but it might point out problems in parts of the project you’re not ready to deal with, like a database you still have to set up. Not a deal breaker, but quite inconvenient.

However, licensing costs and access management for this feature remain a question mark. We expect to see more clarity on how organisation-level access and per-project enablement will work. This will be crucial for larger companies managing multiple teams and projects.

Are AI tools worth it for DevOps engineers and teams? 

The short answer? Definitely, if used responsibly. Copilot, CodeWhisperer, and Gemini Code Assist can all save significant time. Copilot potentially cuts coding time by as much as 25 to 30%, while Gemini Code Assist (and CodeWhisperer to a lesser degree) are great for infrastructure projects. No matter if you’re responsible for development, operations, or both: they will help you track down issues, letting you focus on more impactful work.

However, speed isn’t the only advantage. AI tools can also boost the quality of your code with (automated) testing and more consistency. They always operate at 100% focus, so they won’t make any syntax errors – although most IDEs nowadays also warn users of these simple mistakes. However, they are not infallible, so make sure that you understand their output. Copying and pasting code without understanding it is a recipe for disaster.

Of course, there’s more to being a DevOps engineer than writing code. Luckily, these tools can also help you report to stakeholders and write documentation. Instead of spending time writing those pesky stakeholder updates, commit messages, and other documentation, you’ll be able to focus on what’s important.

What about open-source models? 

Of course, you are not obligated to give your money and data to one of the large tech players. As always, the open-source community is hard at work to build AI coding assistants. There are plenty of alternatives available on aggregator sites like Hugging Face, but things can get overwhelming quickly. In particular, we recommend taking a look at WizardCoder and other LLM’s mentioned in this big code models leaderboard. 

However, the unfortunate conclusion is that the level of performance is currently not competitive with state-of-the-art models like Gemini Pro and Copilot. GPU computing capacity makes the AI world go round, and the differences make themselves felt.

Conclusion 

AI-driven tools like GitHub Copilot, AWS CodeWhisperer, and Google Gemini Code Assist are revolutionising DevOps and the IT industry as a whole by significantly reducing coding time and boosting efficiency. These tools not only accelerate development but also enhance code quality through consistent, automated testing and comprehensive debugging capabilities.

While each tool has its strengths—Copilot in pure coding, CodeWhisperer within the AWS ecosystem, and Gemini Code Assist in versatile integrations and log analysis—they all share the common benefit of freeing developers and operations from mundane tasks, allowing them to focus on more impactful work.

However, it’s key to use them responsibly. These tools are not infallible and still require an expert’s eye to review and understand the generated code. AI assistants can also help with non-coding tasks like documentation and stakeholder reporting, which frees up valuable time. With the right mindset, they can be a game-changer for any DevOps team.

Need advice from real DevOps experts for your environment? Drop us a line, and let’s see how we can help you out!