How We Test and Score
Our methodology, stated plainly — including the parts we do not do, because a review site that overstates its process is worse than one that has none.
Last updated September 2026
Please read
This page describes our editorial method, not laboratory testing. We do not operate a test facility and we do not claim to. If what follows sounds less impressive than other sites, that is intentional: it is accurate.
What we do not do
We do not run our own laboratory tests. We do not buy every product we write about, and we do not measure suction, runtime, CADR or debris capture ourselves. Several review sites claim to do all of that; some of them genuinely do, and many do not.
What that means for you is straightforward: our scores are an assessment of available evidence, not the output of our own bench. Where a number on this site comes from a third-party measurement rather than a manufacturer claim, the specification table says so. Where nobody has published a figure that we can verify, the table says “Not published” and we do not estimate it.
Our three evidence sources
1. Manufacturer-published specifications
This is the baseline for every claim we make. A specification either appears on the manufacturer’s own documentation or it does not go in the table. Publishing channels vary — support documentation, product pages, regional spec sheets — so we cross-check and note the month a figure was last verified.
2. Independently measured third-party results
For several of our categories, independent testers publish measured rather than claimed results: hair-tangle percentages for robot vacuums, airflow and runtime for handhelds, debris capture and battery runtime for pool robots, and AHAM-verified CADR for air purifiers. Where a credible measurement exists, it carries more weight in our scoring than a manufacturer figure.
3. Owner-reported reliability over time
Long-run reliability almost never appears on a specification sheet. We look at documented owner experience on the parts that actually fail: dock hardware, cables and swivels, battery packs, brush wear, and app support after several years. A model with excellent launch specifications and a poor reliability record does not keep a high score.
The five-factor rubric
Every product is scored out of 10 on the same five factors, in the same order, using the same evidence standard. The breakdown is published on each review so you can see the reasoning — and disagree with our weighting if your priorities are different.
Performance
Weighted highest in every categoryThe measured capability of the product on the job it is sold for — cleaning pickup and tangle behaviour for vacuums, debris capture and runtime for pool robots, airflow and filtration for purifiers.
Evidence used: Manufacturer-published figures first; independently measured third-party test results where they exist and are credible.
Ease & maintenance
Second-highest weightHow often you have to touch the product, and how unpleasant that is. Bins, filters, docks, cables, charge cycles, and whether the design removes a chore or moves it somewhere else.
Evidence used: Specification review (stated intervals, filter design, dock behaviour) plus documented owner experience over time.
Feature depth
Moderate weightThe capability set relevant to the category — navigation and obstacle handling, coverage modes, app usefulness — scored for usefulness rather than quantity.
Evidence used: Manufacturer documentation and independent descriptions of how the features behave in practice.
Value for the tier
Moderate weightSpecification and capability relative to comparable models in the same class. Not a price comparison — we do not publish prices — but a judgment about whether the engineering justifies its position in the range.
Evidence used: Cross-comparison of published specifications and independent test results within a class.
Support & reliability
Moderate weightWarranty length and terms, parts availability, consumable costs, and how the product tends to age. A two-year warranty on a machine with awkward spare-part supply scores below a shorter one with easy parts.
Evidence used: Manufacturer warranty terms, parts listings, and long-run owner-reported failure points.
The overall score is a weighted average of these factors, rounded to one decimal place. We do not use the overall score to compare across categories — a 9.0 air purifier and a 9.0 pool robot are both strong in their class, and nothing more.
Why stars appear on the page
Scores are displayed as stars because they are easier to scan than a table of numbers. They are our editorial score out of 10, converted to a five-star scale — not an average of customer ratings, and not a customer review score. We do not aggregate user ratings, because we cannot verify them. Star ratings on this site always sit next to the words “editorial score”.
What we leave out, and why
- Prices. They change by retailer, region and promotion, and a stale number misleads. We describe what a model is good at instead, and leave the commercial detail to the seller.
- Purchase links. This site has no affiliate links and no purchase buttons. Nobody pays us when you buy something, which is the entire point of the model.
- Absolute claims. No “best ever”, no “guaranteed”, no ranking that cannot be explained. Every recommendation states its reasoning and its limits, including who should buy something else.
- Estimates presented as facts. If a figure is unavailable, it stays unavailable.
Corrections
Specifications change, and manufacturers occasionally revise documentation without announcing it. If you spot a figure that no longer matches the manufacturer’s current published specification, tell us and we will check and correct it. Corrections are made to the page itself rather than in a separate notice, and the “last checked” date is updated.
Editorial independence
Our recommendations are based on independent research and are not sponsored. No brand has ever paid for coverage, ranking or placement on this site, and no manufacturer has seen a review before publication. This site carries no advertising today; display advertising is planned, and when it is introduced it will be sold and served separately from editorial. See our editorial policy for how that separation is enforced.