At Harness, our journey with AI coding tools followed a pretty predictable path.
First came the excitement. "Wow, this is going to change how software gets built."
Then came adoption. "Let's get these tools into every engineer's hands."
Then came the bill.
And eventually, Jyoti Bansal asked the question every CEO asks: "This is great, but show me the money."
For the last two years, the software industry has optimized for AI adoption. More seats. More prompts. More tokens. Measuring AI consumption is easy. Tying it to real outcomes is where most teams get stuck.
So we spent the last 12 months actually tracking AI's impact across our engineering organization and found something surprising: Some of our biggest productivity gains happened while AI spend was going down.
The highest-performing teams weren't the heaviest AI users. They were generating the highest return per token.
That's the shift happening right now, from AI adoption to AI ROI.
I wrote up what we learned: the metrics that mattered, and why engineering leaders need to start thinking beyond token counts and usage dashboards.
The companies that win the next phase of GenAI won't be the ones spending the most.
They'll be the ones that can prove the return.
#AI #EngineeringLeadership
#DeveloperProductivity
#GenAI
#SoftwareEngineering
#FinOps
#AIROI
Trevor Stuart Bill Koefoed Carlos Delatorre Pushkar Priyadarshi Prashant Verma Sachin Kanodia Kiran Gangadharappa Naresh Agarwal Kahnan Patel Akshay Phadke Indira T.
Every founder I talk to says the same thing , “AI is letting us build 10x faster” . True. AI is compressing the cost of writing code.
It is NOT compressing the cost of understanding systems.
That asymmetry is where most engineering teams are going to get hurt. There was this interesting blog I saw about laws of software engineering, some of those I have followed all my life. I went through all 56 Laws of Software Engineering and asked: Which ones does AI actually invalidate?
The answer: Basically none. If anything, many have become harder constraints.
Here is how IMO the landscape is shifting:
THE UNTOUCHABLES (AI doesn’t get a vote)
Conway’s Law: LLM-generated code still reflects your team’s communication boundaries. Bad org charts = bad AI architecture.
Brooks’s Law: More agents/bots ≠ less coordination cost. Adding "man-months" (even digital ones) to a late project still makes it later.
CAP Theorem & Amdahl’s Law: Physics doesn't care about your prompt. Throughput and consistency tradeoffs remain absolute.
THE SHIFTED (Same laws, worse failure modes)
Kernighan’s Law: We are now generating code faster than we can build mental models for it. If you aren't smart enough to debug it, you shouldn't have generated it.
The 90-90 Rule: AI collapses the time-to-first-version. It does NOT collapse the time-to-correctness.
Hyrum’s Law: Higher code volume creates more emergent, undocumented dependencies. The surface area for bugs is exploding.
THE AMPLIFIED (AI as a multiplier)
Goodhart’s Law: If you measure productivity by Lines of Code or PR Volume, your system will drift into chaos instantly.
Jevons Paradox: As code becomes cheaper, we won't write less of it. We will consume more of it, creating even more complex systems to manage.
The Bottom Line: Complexity never disappears. It just moves up the stack:
→ From Syntax to Architecture
→ From Implementation to Validation
→ From Writing to Reasoning
This is exactly what Tesler’s Law predicts, complexity is a constant. If you simplify the developer's interaction, the complexity just moves behind the curtain.
The new bottleneck isn't “Can we build this?” It is, “Do we actually understand what we just built?”
#SoftwareEngineering #AI #LLM #SystemDesign #Founders
Today, we shipped Harness Agent DLC and Harness AgentTrace, the framework we use internally to observe, evaluate, and govern AI agents across the Harness platform.
We should be questioning our agents the same way we question everything else in production: not just whether they succeeded, but why they got there and whether they’d get there the same way again.
Two agents can hit the same outcome and take completely different paths, different tools, different models, different sequences, with very different costs, latencies, and risks along the way. Traditional tracing tells you what code executed and which services were called. It doesn't tell you why an agent chose a particular tool, model, or sequence of actions on one run and not another.
That's why we built Harness AgentTrace: execution captured at the run level and across full sessions, so you can see both what happened in a single agent run and how behavior evolves across an entire interaction.
It's also the layer underneath everything else in Harness Agent DLC. Evals, testing, deployment gates, cost attribution, all of it consumes the telemetry AgentTrace produces. One audit trail instead of five tools, each keeping its own partial record.
We're open-sourcing the foundational pieces, harness-sdk and harness-evals, so teams don't have to wait on us to get this kind of visibility into their own agents.
𝗵𝗮𝗿𝗻𝗲𝘀𝘀-𝘀𝗱𝗸 provides OpenTelemetry-based instrumentation for AI agents with automatic instrumentation for OpenAI, Anthropic, and LiteLLM.
𝗵𝗮𝗿𝗻𝗲𝘀𝘀-𝗲𝘃𝗮𝗹𝘀 provides transparent evaluation metrics for correctness, groundedness, safety, trajectory, and performance, making it easier to bring evaluation into CI pipelines.
Both work with any OpenTelemetry-compatible backend and can be used independently of the Harness platform.
Proud of the team for building the tracing layer this entire platform runs on.
The full technical deep dive is available in our latest engineering blog. Link in the comments.
Jyoti Bansal Sunil Gattupalle Rohan Gupta Trevor Stuart Uri Scheiner Harish Doddala Rahul Sood Brad Rydzewski
#Harness #AgentTrace #AIAgents #LLMOps #OpenTelemetry #OpenSource #AgentEngineering #DeveloperTools #Observability #GenAI #devops
---
At Harness, we are our own first customer.
Every single capability we ship is battle-tested against the exact reality our engineering organization faces daily: exploding AI-generated code volume, hyper-accelerated dev cycles, and zero tolerance for broken delivery pipelines.
In June, our R&D team shipped 62 features in 30 days.
But the number isn’t the true headline. The pattern is.
AI coding tools have fundamentally broken the code-generation bottleneck. As a result, the bottleneck has shifted entirely downstream to everything required to safely ship that code: builds, testing, security scanning, governance, and cost management. Writing code takes seconds; managing its wake takes operational discipline.
A few June highlights that stand out to me personally:
𝗔𝘂𝘁𝗼𝗻𝗼𝗺𝗼𝘂𝘀 𝗪𝗼𝗿𝗸𝗲𝗿 𝗔𝗴𝗲𝗻𝘁𝘀- Every step in a Harness pipeline can now run as a reasoning agent. Crucially, they are governed by the exact enterprise-grade controls you already trust: scoped credentials, OPA policies, and audit trails. This moves AI from just assisting developers to safely executing complex work across the delivery system.
𝗣𝗮𝗿𝗮𝗹𝗹𝗲𝗹 𝗣𝗶𝗽𝗲𝗹𝗶𝗻𝗲 𝗘𝘅𝗲𝗰𝘂𝘁𝗶𝗼𝗻- With new DAG-based execution, teams can now run independent pipeline stages in parallel automatically. Complex fan-out, fan-in, and advanced delivery flows become incredibly easy to express, allowing you to optimize existing pipelines without a complete rebuild.
𝗔𝗜 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴 𝗜𝗻𝘀𝗶𝗴𝗵𝘁𝘀- Engineering leaders need to know if their AI investments are actually driving real business return. This feature directly tracks adoption, AI-committed code, spend per developer, and contrasts the PR cycle times of AI-assisted developers against everyone else.
𝗦𝗲𝗰𝘂𝗿𝗶𝘁𝘆 𝗳𝗼𝗿 𝗔𝗜- More code volume naturally means more security risk. We added AI-powered SAST, API Inspector, bulk Bitbucket onboarding for automated SBOM scanning, and AI-generated OPA policies. Security can no longer be an afterthought; it must be highly automated and contextual.
𝗧𝗲𝘀𝘁𝗶𝗻𝗴, 𝗙𝗶𝗻𝗢𝗽𝘀, 𝗮𝗻𝗱 𝗣𝗹𝗮𝘁𝗳𝗼𝗿𝗺 𝗕𝗿𝗲𝗮𝗱𝘁𝗵- From native Playwright execution and OpenAI/Anthropic cost visibility to auto-discovery in the internal developer portal (IDP), Module Registry 2.0, and scheduled IaC drift detection, it all landed in a single month.
The companies that win the next phase of GenAI won't just be the ones typing code faster. They’ll be the ones that build the end-to-end operating model to secure, govern, and optimize that code at the exact same pace.
Incredibly proud of the entire Harness R&D team for another relentless month of execution.
Read the full June product breakdown in the comments.
#GetShipDone #SoftwareDelivery #EngineeringLeadership #DevOps #GenerativeAI #FinOps
Jyoti Bansal Uri Scheiner Harish Doddala Trevor Stuart Brad Rydzewski Bala Venkatrao Rahul Sood
The pace of change right now means what you build is constantly evolving and we are all perpetual students.
We just published a look into our recent Harness MCP server redesign, including the mistakes we made and the lessons we learned while building for a world that changes every day.
Big takeaway for me:
AI agents aren’t useful without deep, structured access to real systems. That means rethinking how they connect to pipelines, logs, repos not just adding another layer on top.
This is one step in that direction. Not perfect, but progress.
If you’re building in this space, would love to hear what’s working (and what’s not)
Read the full breakdown here:
https://lnkd.in/eD3UtqdZ
Sunil Gattupalle Rohan Gupta Shubham Jindal
#Harness #DevOps #AI #Innovation