Uncertainty code
Experienced open source developers expected AI to accelerate their work. However, the results of the latest study present a slightly different perspective — AI-based tools may actually slow things down.
On paper, everything looked promising. The latest AI tools — Cursor Pro, Claude 3.5 and Claude 3.7 Sonnet — were set to revolutionize the work of experienced programmers. The METR team (Model Evaluation & Threat Research) decided to check what this revolution looks like in practice.
Too beautiful to be fast
In July 2025, the results of a meticulously designed experiment were published: 16 senior developers with an average of 5 years of experience in specific repositories, 246 realistic tasks, randomized controlled trial. Some tasks were performed with the assistance of AI, some without. Before starting, participants predicted that AI would speed up their work by 24%. After completion, they evaluated the actual gain at 20%.
Share
You might be interested in
-
GPT-5 will make it necessary to rename it…
Because… ChatGPT will no longer just be a chat. It will become a full-fledged assistant that will operate without your involvement.
-
Comet has landed
A new-generation browser that integrates the classic search feature with artificial intelligence. You can download it on one condition…
-
OpenAI finalized the purchase of io. A mysterious AI device is coming
What will emerge from the combination of Ive’s craftmanship, Altman’s technological infrastructure and Cheyer’s unique conversational interface?
-
Grok is pushing the boundaries again
This time, it’s because of vulgar and politically incorrect comments. Is it a deliberate decision by the creators, or an uncontrolled escalation?
