Sleuth wants to use AI to measure the performance of software developers

- Advertisement -

As knowledge workers, including software engineers, moved to work remotely during the pandemic, executives have expressed concern that productivity will suffer as a result. certificate is an mixed On this one, but especially in the software industry, remote work exacerbated many of the problems that employees were already facing. According to Sad 2021 poll, most developers find slow feedback loops in the software development process to be a source of frustration, second only to communication difficulties between teams and functional groups. Seventy-five percent said the time they spend on specific tasks is wasted, suggesting it could be used more strategically.

- Advertisement -

In search of a solution to improve developer productivity, three former Atlassian employees – Dylan Atkin, Michael Knighten and Don Brown – founded Detective, a tool that integrates with existing software development toolkits to provide information for performance measurement. Sleuth today announced it has raised $22 in Felicis-led Series A funding with Menlo Ventures and CRV, which CEO Atkin says will be used to develop the product and expand Sleuth’s workforce (in particular, engineers and product specialists). sales).

- Advertisement -

“Due to the pandemic-driven remote work avalanche, the need for developers, managers, and leaders to understand and communicate about engineering efficiency has skyrocketed,” Atkin told TechCrunch via email. “Developers who are no longer in the same room need a way to coordinate deployment and a quick way to detect when a deployment has gone wrong. Managers need a non-intrusive way to proactively learn about bottlenecks affecting their teams. Leaders need an unobtrusive way to understand the impact of their corporate initiatives and investments. Sleuth takes the burden of understanding and communicating engineering efficiency offline and making it understandable for everyone.”

Atkin, Knighten, and Brown were colleagues at Atlassian, where they claim to have helped the company’s engineering teams go from releasing software every nine months to releasing every day. Atkin was an architect on the Jira team before becoming Development Manager at Bitbucket and StatusPage, while Knighten and Brown were VP of Product and Architect/Team Leader, respectively.

- Advertisement -

While at Atlassian, which has grown from 50 to over 5,000 employees during the time Slate’s co-founders have been there, Atkin says it’s become “crystal clear” that many engineering teams lack a quantitative way to measure performance—and that this gap can keep them from growing and improving.

“Measuring engineering performance is a well-known, large and ever-growing problem that has now become solvable. As each company invests more in software development, the need for visibility into engineering efficiency has intensified,” Atkin said. “However, measuring performance has historically been a very difficult task for a variety of reasons, namely the complexity of the tools, the lack of access to data, and the use of dubious proxy measures that have bred micromanagement and mistrust.”

Sleuth’s solution is DevOps Research and Assessment (DORA) metrics, a new standard used by development teams to measure how long it takes to deploy code, the average service recovery time after failures, and how often a team’s fixes result in bugs. problems after deployment. DORA originated from a Google academic research group that surveyed more than 31,000 engineers about DevOps practices between 2013 and 2017 to identify key differences between “low performers” and “elite performers.”

Sleuth isn’t the only platform using DORA metrics to quantify performance. Linear B, Jellyfish and Athenian are among the competing solutions that have adopted the DORA standard. But Etkin claims that its competitors are “not fully and accurately” tracking these metrics.

“Sleuth is unique…because we use deployment tracking to model how engineers go from concept to launch,” he explained. “Accurately modeling how engineers work in their pre-production and production environments and how they interact with issue trackers, CI/CD, bug trackers, and metrics allows Sleuth to create a fully automated… view of team DORA metrics and their engineering performance. ”

Sleuth uses AI to try to determine a team’s base change failure rate (i.e., the percentage of changes that resulted in a degradation of service) and mean recovery time — two of four DORA metrics — from existing systems like Datadog and Sentry. The platform can automatically detect when a metric goes beyond this baseline, Atkin says, and even automate steps in the development process to potentially improve the metric.

On the Sleuth project dashboard, individual teams can track their DORA performance. Organization-wide dashboard showing trends across projects and teams.

“Customers just indicate detective at at … error data and detective lets engineers know when they’ve taken those numbers to failure levels. Using AI to determine these values ​​means that engineers can focus on their work without having to understand every metric in their system or what the “normal” for each of them looks like.”


Tracking DORA metrics with Sleuth.

DORA scores are of course not the final decision. They can become a nuisance when the organization’s focus on them becomes all-consuming. Sagar Bhujbal, VP of Technology at Macmillan Learning, told InfoWorld in a recent article: “Developer productivity should not be measured by bugs, delivery delays, or incidents. This causes unnecessary worry for development teams who are always forced to deliver more features faster and better.”

Atkin agrees, emphasizing that technical managers must avoid the temptation to micromanage.

“Engineering is a creative activity, and engineers are more like artists than assembly line workers,” Atkin said. Engineering Managers Need to… Keep Track of the Right Metrics [and] accurately track them [but also] give engineers the tools they need to improve performance.”

detective clients range from businesses like Atlassian to startups including Launchdarkly, Puma, Matillion and Monte Carlo. Atkin says the platform has tracked nearly a million deployments and taken over a million automated actions on behalf of developers. He declined to disclose earnings when asked, but said Sleuth, with 12 employees, grew 700% last year with “very healthy” margins and cash flow.

Credit: /

- Advertisement -

Stay on top - Get the daily news in your inbox

DMCA / Correction Notice

Recent Articles

Related Stories

Stay on top - Get the daily news in your inbox