At Packmind, we’ve been working with AI coding assistants for over 18 months and, collaborating with clients across various stages of deployment—from initial POCs to large-scale rollouts involving over 5,000 users.

It’s clear to us that the rapid advancements in AI coding assistants, and now the emergence of autonomous agents, are transforming the role of developers. As these tools evolve, our approach to integrating them into our development teams must evolve as well.

While the landscape is constantly evolving, with over a year of research and REX, we’ve identified key trends and recurring feedback worth highlighting.

The Good

In the realm of AI coding assistants, GitHub Copilot remains the leader, while Jetbrains AI Assistant is quickly gaining traction for its exceptional developer experience within IDEs. Alternatives like Tabnine and Codeium stand out for their self-hosting capabilities and solutions to copyleft concerns. Meanwhile, AWS CodeWhisperer and Google’s Gemini Code Assist are optimized for their respective cloud ecosystems. The open-source tool Continue is also emerging as a popular IDE extension, offering flexibility by integrating with various models, including those running locally on a developer’s machine.

These AI coding assistants have already demonstrated value across several critical areas:

• Code Generation, Autocompletion, and Documentation: These tools efficiently generate code snippets, simple feature implementations, and corresponding documentation, streamlining the development process.
• Navigating a Codebase: AI excels at explaining complex code, helping developers quickly understand unfamiliar projects.
• Test Creation: AI can generate unit tests and improve test coverage. We recently hosted a session with Valentina Jemuović that explored how AI supports different stages of Test-Driven Development (TDD). You can watch the full AI in TDD webinar for more insights.
• Bug Detection: By analyzing logs, AI can identify patterns and potential causes of errors or crashes, providing actionable insights.
• Advanced Practices: With the right prompts, AI can even assist in more sophisticated practices like Domain-Driven Design. Check out our webinar (in French) with Dorra Bartaguiz for more.

While vendors and consulting firms often tout miraculous productivity gains from Gen AI, the reality is more nuanced. The Today was a Good Day: The Daily Life of Software Developers study reveals that coding accounts for, at most, 40% of a developer’s time. Therefore, the productivity gains from these tools are limited to that portion, with optimization typically ranging between 10% and 15%. This is without even considering the limitations that come with current AI coding assistants.

The Bad

Despite their advantages, AI coding assistants come with significant drawbacks:

• Lack of Context: AI often struggles to grasp the broader context of a project, making it difficult for it to generate code that aligns with overall architecture and design patterns.
• Subpar Code Quality: While AI-generated code may be functionally correct, it can still be buggy, insecure, or not adhere to best practices, leading to increased technical debt.
• Reinforcement of Bad Practices: AI tools can perpetuate poor coding practices if they’ve learned from suboptimal codebases. Indeed, they still mostly amplify what they see indiscriminately.
• Reduced Learning Opportunities: Over-reliance on AI can hinder the growth of junior developers who might miss out on critical learning experiences.

The tendency to accept AI-generated code without proper scrutiny leads to suboptimal codebases. While there is an initial velocity gain, the long-term cost of ownership for this code needs to be carefully considered.

The research paper “Coding on Copilot: 2024 Developer Research” by William Harding and Matthew Kloster provides insight into the impact of AI tools like GitHub Copilot on coding practices and code quality. Analyzing 153 million lines of code from 2020 to 2023, the study reveals that while GitHub Copilot boosts coding speed, it raises concerns about maintainability and quality. Specifically, it points to a rise in code churn, decreased refactoring, and an increase in copy-pasted code—trends that could have significant long-term repercussions.

Code churn—code pushed to a repository that is subsequently reverted, removed, or updated within two weeks—increased by 100% between 2021 and 2024. Considering that wasted hours on unnecessary code rework can cost medium-sized businesses upwards of $4.7 million annually—and $160 million for large companies—these impacts can’t be ignored.

The Ugly

Imagine today’s AI coding assistants as a car that a junior developer is driving for the first time. Now, think about what happens when you hand them a high-speed race car—they’re likely to lose control if they’re not prepared. This is the potential risk we face with the rise of autonomous agents. These agents will soon be capable of tackling more complex tasks, from handling GitHub or Jira tickets to managing entire workflows.

Initially, they might excel at simpler tasks, but as their capabilities grow, so too do the challenges in human-AI collaboration. With increased code churn and rapid changes across large portions of the codebase, what happens when a human developer needs to revisit and refactor that AI-generated code for a new feature?

Smooth collaboration between humans and AI is crucial. Success shouldn’t be measured by lines of code produced or tasks completed, but by the quality of the outcome and the total cost of code ownership—the team’s ability to maintain, improve, and ensure quality. That’s why it’s essential to add guardrails and ensure that the output from AI coding assistants and agents implements industry best practices and aligns with team guidelines.

You can learn more about this and see how Packmind can help here.

Shifting quality (really) left by upskilling developers

The takeaway from various studies and our own experiences is clear: While AI-assisted coding tools can enhance productivity, they also pose risks to code quality and maintainability. To balance these immediate benefits with the potential long-term costs, proactive strategies are essential.

It’s time to truly shift quality left by upskilling developers!

The human in the loop must be skilled and diligent to ensure quality and security. Prevention is better than remediation, and that means upskilling developers. Understanding the fundamentals, mastering design patterns, and adhering to good coding practices are essential. Octo Technology highlighted in their REX the importance of Craft practices, training, and using solutions like Packmind to upskill developers and get the best out of Gen AI in development.

Thoughtworks, in a recent article, also highlighted that “it’s important to note that coding assistants should be paired with effective engineering practices that act as guardrails”.

The good news is that, like every transformative technology, AI also presents an opportunity to elevate developers—not just generate code faster:

• The Internet: Made knowledge accessible to everyone. I remember the first time I used Netscape during my Phd; it felt like discovering fire!
• Social Networks: Platforms like YouTube have democratized learning. Did you know that Julius Yego, a javelin world champion, learned his discipline on YouTube?
• AI: The next wave, AI has made all knowledge accessible and queryable through natural conversations. It promises a future where every developer has a personalized coach at their fingertips.

By focusing on upskilling developers and truly shifting quality left, we unlock a powerful productivity lever for engineering organizations, maximizing the potential of GenAI.

At Packmind, we believe that the key to success lies in enhancing developer skills and improving collaboration between AI agents and developers. This is our mission. With Packmind AI Tech Coach, we elevate the craft of development teams to new heights by harnessing the power of GenAI and collective intelligence.