Hi Jan,

DX recently hosted a panel event in San Francisco with Developer Productivity leaders from Snowflake, Atlassian, Yelp, and Gusto. The discussion covered how they’re planning AI tooling budgets, handling tool sprawl, and what they’re seeing from using AI to assist with migrations, incidents, and customer support.

Sharing my notes below in case they’re helpful.

-Brook

****

Handling costs and contracts

• Atlassian can predict third-party AI tool spend, but costs for their own AI platforms and model tuning work are still unpredictable. They advise leaders to expect vendor pricing to stabilize early, while internal AI costs continue to fluctuate as usage grows.
• Snowflake’s leadership is willing to tolerate an increasing Cursor bill because they believe a potential 50% lift in developer productivity pays for itself and ties directly to business outcomes.
• Gusto is pushing vendors to move away from seat-based licensing to usage-based models, arguing that seats no longer match how AI is used and block experimentation.
• Yelp believes it’s too early to cleanly prove ROI at a tool level. As long as the overall AI budget is affordable, optimize for learning and capability instead of perfect cost attribution.
• An additional recommendation: Nearly half of all leaders are allocating between 1% and 3% of their engineering budget to AI tools, and 20% already spend more than $500 per developer per year. Plan for budgets to rise quarter over quarter as usage grows.

Modernization and migrations are now AI-assisted

• The pattern we saw across the panel: AI speeds the prep and planning, not the final production changes.
• Yelp tested fully automated end-to-end Java upgrades, and it failed. When engineers drove the flow, and the model had strong reference material to work with, AI sped up about 10% of migrations.
• Atlassian is also using AI to support major Java upgrades, and they are pushing toward platform-driven workflows so thousands of repos can be upgraded centrally instead of individually.
• Gusto uses AI to map boundaries and draft service templates for their monolith extraction, then relies on existing automation to run the actual migrations.

Tool sprawl and AI fatigue are creating friction

• Yelp keeps AI tools limited. They learned that adoption breaks when developers don’t know features exist. A single internal workshop took their usage from zero to thousands of connections in one day.
• Developers at Snowflake felt overwhelmed by the volume of AI review comments, even when the comments were technically right. The team is now focused on reducing volume and improving quality rather than adding more tools.
• Atlassian is seeing the same pattern at a different scale. Their 35GB front-end repo breaks most standard tools, so they’ve had to build custom search and coding assistants to work around it. This increases the AI surface area that their teams have to manage.

AI support bots depend on the quality of your knowledge base

• Yelp runs an internal Slack assistant across about 70 channels that answers 2,000 questions a month with a 50% success rate. They learned that the limiting factor wasn’t the model; it was whether internal docs and systems are accessible and structured enough for the bot to use.
• Gusto sees a similar pattern in their modernization work. AI can draft templates and map code boundaries, but the actual migrations still run through well-defined automation because their domain needs predictable, verifiable outcomes that the model can’t consistently produce.

AI is expanding who can code

• Gusto is seeing AI adoption spread beyond engineering. Customer care teams are now using tools like Cursor to build internal workflow tooling for regulated payroll and benefits work, which is something that never would have happened without AI lowering the barrier to entry.
• Snowflake reported roughly 80% of their solutions engineers now use Cursor to troubleshoot customer issues directly in the codebase, producing higher-quality reports before anything reaches core engineering.

AI in incidents, operations, and test health

• Snowflake is experimenting with AI to run the initial checks in their incident investigations, and they’re already seeing investigations drop from 20-30 minutes to just a few minutes when AI handles the first pass.
• Atlassian is using AI and mutation testing to improve test health. If a change gets past the tests, it becomes a Jira ticket, and an agent suggests how to fix it, which is already reducing misses.

unsubscribe