Benchmark programme
Independent benchmarks for construction estimating software
No one publishes controlled, repeatable measurements of how these tools perform on identical inputs. This is the protocol we intend to run, published before the results exist so the method cannot be adjusted to fit them.
Results
Every row in this table would carry the test it came from, the date it was run, and the drawing set used.
No benchmark results have been published
Methodology
The protocol
Each test states the question it answers, what is held constant, and what is recorded. Controls are fixed before any product is opened, and control measurements are never adjusted after seeing a product’s output.
- 01
Residential takeoff — time to complete
Not yet runHow long does it take a trained operator to produce a complete material takeoff from a standard residential drawing set?
Held constant
- One drawing set, used unmodified across every product tested
- The same operator, trained to the vendor's own documented workflow before timing begins
- Default cost data as shipped; no custom assembly libraries built in advance
- A fixed scope definition stating exactly which quantities must be produced
Recorded
- Wall-clock time from opening the drawing set to a complete quantity list
- Number of manual corrections required after automated detection
- Quantities that the tool could not produce at all
- 02
Quantity accuracy against a measured control
Not yet runHow closely do the quantities a tool produces match an independently measured control takeoff?
Held constant
- A control takeoff measured by hand and checked twice before any product is tested
- Identical scope boundaries for every product
- No adjustment of the control after seeing any product's output
Recorded
- Percentage deviation from control, per quantity category
- Direction of error — systematic over- or under-measurement
- Categories where the tool produced no quantity
- 03
Cost database currency
Not yet runHow current is the shipped cost data, and how far does it sit from regional supplier pricing on the test date?
Held constant
- One region and one test date for all products
- A fixed basket of common line items
- Supplier reference pricing collected before product data is examined
Recorded
- Deviation from reference pricing per line item
- Vendor-stated update frequency versus observed data age
- Line items absent from the shipped database
- 04
Estimate to client-ready proposal
Not yet runHow long does it take to turn a completed estimate into a document that could be sent to a client?
Held constant
- The same completed estimate as the starting point
- A fixed definition of client-ready, agreed before testing
- Default templates only; no custom template design time
Recorded
- Time from completed estimate to sendable document
- Steps requiring software outside the product under test
- Whether the output can be produced on a mobile device
What counts as a benchmark
The word is used loosely in this category. On this site it has one meaning, and the two weaker kinds of claim are labelled as what they are.
- Vendor-reported
- The vendor states a figure. We have not seen the method or the inputs. Never presented as a measurement.
- Verified
- We confirmed a fact against a primary source on a recorded date. This establishes that something exists, not how well it works.
- Independent benchmark
- We ran the published protocol ourselves on controlled inputs and published both the method and the result. Currently: none.