Morning. A new benchmark just tested ChatGPT, Claude and Copilot on real construction drawings, and the best score was 34%. Today we look at how to tell when AI is the right tool, and when the old way still wins.
IN TODAY'S ISSUE
Can AI read your construction drawings? “In distribution” vs “out of distribution”, and why the best general models scored 2–34% on Construction’s Last Exam
Firms are putting $1M+ into AI and their teams are growing, not shrinking
Bedrock Robotics raises $600m to put autonomous excavators on site
Today's read: 5 minutes
FAVORITE FINDS
💰 37% of construction firms have invested $1M+ in AI, and it’s not cutting headcount
Why it matters: BRG surveyed 100 construction and development executives and AI adoption now spans the full project lifecycle. But it isn't replacing people with 18% stating certain teams actually grew and 38% are investing in reskilling the people they already have.
What’s most interesting is how work is being reshaped with 50% saying AI changed their reliance on outside vendors or consultants indicating more tasks are being brought in house.
🚜 Bedrock Robotics raises $600m in new funding to turn heavy construction equipment into autonomous machines
Why it matters: The skilled labor shortage isn’t getting better. Excavators often make up 25% of a contractor's fleet and it takes almost five years for an operator to master. Bedrock has already deployed their Operator System onto live infrastructure sites.
For those of us in the field, the job shifts from running the machine to managing the space around it, and that starts with exclusion zones. When a human operator is in the cab, there’s someone watching who steps inside the swing radius. When there isn’t, the exclusion line has to do that job. So if you end up on a site running autonomous plant, ask:
How is the exclusion zone set and marked? Is it a geofence in the machine’s software, or bunting and a spotter?
What happens when someone crosses the line? Does the machine stop on its own?
Who has the authority to pause it, and how does control hand back to a human operator?
🔍 Newforma acquires Lumi AI to provide explainable and verifiable AI responses
Why it matters: Newforma builds project information management software and one challenge they’ve seen with AI in AEC is that we need answers that can be verified. It’s because project decisions carry high contractual and liability risk, so we can’t take an AI chatbot’s answer at face value.
Lumi AI’s engine was built to fix this. It shows the underlying reasoning and provides verifiable answers. So before you stamp drawings for issue, you can interrogate the responses and ensure they can be defended if you’re ever questioned. As engineers, the risk is on us, not ChatGPT.
DEEP DIVE
CAN AI READ YOUR CONSTRUCTION DRAWINGS?
AI is great at writing emails for me or digesting a long report, but when it comes to drawings, it tends to fail.
A reason why is the idea of ‘In Distribution’ (ID) and ‘Out of Distribution’ (OOD).
An ID task matches the data and scenarios that AI is trained out while an OOD is unfamiliar or novel. As ChatGPT, Claude and Microsoft Copilot were trained on publicly available data on the internet, they’re great at creating written content or generating photos. But most construction content like drawings are hidden behind firewalls or private servers.
It’s an issue which came up on the new benchmarked report Construction's Last Exam which is designed to test if AI models can perform the kinds of drawing analysis tasks that contractors, estimators and quantity surveyors undertake every day.
According to Togal AI which ran the test, general AI models achieved accuracy scores ranging from 2% to 34% on tasks involving blueprint interpretation, quantity take-offs and measurement exercises.
What it shows is how challenging reading construction drawings is for general purpose models. Blueprint interpretation requires a combination of computer vision, spatial reasoning, engineering knowledge and mathematical precision.
That doesn’t mean that general purpose models won’t get there. In fact they are getting better and quite rapidly.
What’s important for us in the industry is understanding the strengths of each model. AI is a new tool in our toolbelt and we need to know when and where to use it. That means knowing when we can use general AI which is usually for tasks such as summarizing specification sections, listing submittal requirements or comparing subcontractor proposals.
But for specialized tasks, where there isn’t really any publicly available information, we’re going to have to do it the old way or we can find a specialized construction AI model trained in takeoffs, drawing reviews or whatever task it is we are looking for.
So the next time you use AI ask yourself, is this in or out of distribution? Have they been able to train their models on the same type of data and output you’re looking to generate? If the answer is yes, then AI might be the best option. But if not, the old way it is.
Note: Togal AI builds takeoff tools so their report does have an outside incentive but it does reflect what we’ve seen trying out general models on drawings.
Around the Watercooler
Sid just launched the Builders Brain on instagram and we’re up to 700 followers! In one his first videos discusses Solvera, a new startup that raised $6.3m to deliver MEP engineering end to end, with AI agents handling a growing share of the work and engineers making the judgment calls.
Check out the full video below and follow us here.
Well that was a lot. Our takeaway is that AI isn’t one tool. It’s a toolbelt, and the skill is knowing which one to grab. Before you use it on your next task, ask whether it’s in or out of distribution. Sometimes the answer is that the old way still wins.
If you found this useful, forward it to a mate who’s been burnt by ChatGPT trying to read a set of drawings. And if you haven’t yet, come say g’day on Instagram, where Sid is breaking down this stuff on video.
As always, hit reply if you have a question or want to chat. We reply to each one.
Till next week,
Sid & Bhragan