FAQ
How do I use this?
This tool is to calculate the reach of breakage issues, i.e. roughly the percentage of users affected on a particular platform. When combined with a severity level, you can look up a priority level for the issue in the Severity Assessment Matrix.
Where can I find API usage share?
Chrome Platform Status is useful for JavaScript APIs. Use this lookup table for mapping features to API names.
What is "reduction assumption"?
Should the reach of this breakage be reduced? This should be based on data if available, or your best estimate. 100% (or no input) = no reduction. Examples of when this could apply:
- A specific platform version, e.g. only Big Sur users, which might account for 5% of all macOS users.
- A specific user action, e.g. only when users pause a video, which might be 10% of all video plays.
What is "unaccounted-for breakage"?
How confident are you that this breakage is happening on other domains or affects other APIs? For API usage, unaccounted-for breakage is probably "low confidence" unless you're pretty sure there are knock-on effects elsewhere.
What if multiple domains are broken?
There are a couple of ways to handle this:
- Use the calculator for the highest ranked domain you're aware of, then set "unaccounted-for breakage" to medium or high confidence.
- Use the calculator for each broken domain, set the "unaccounted-for breakage" to low confidence, then sum the resulting percentages.
What are some example calculations?
Single domain example: example.com broken, only happens after 2+ clicks
- Domain rank:
- 171
- Initial % of all page loads:
- 0.0337%
- Further reduction assumption:
- 50% (because 2+ clicks are required.)
- Updated % of page loads:
- 0.0168%
- Additional unaccounted-for breakage:
- High confidence (because this kind of breakage is very likely to happen on other sites.)
- Final % of page loads:
- 1.68%
Single API example: WebShare API not supported
- API usage share:
- 0.07%
- Initial % of all page loads:
- 0.07%
- Further reduction assumption:
- [None]
- Updated % of page loads:
- 0.07%
- Additional unaccounted-for breakage:
- Low confidence
- Final % of page loads:
- 0.07%
Extreme example 1: Google is broken for everyone
- Domain rank:
- 1
- Initial % of all page loads:
- 21.352%
- Further reduction assumption:
- [None]
- Updated % of page loads:
- 21.352%
- Additional unaccounted-for breakage:
- Low confidence
- Final % of page loads:
- 21.352%
Extreme example 2: Videos not playing on my personal blog
- Domain rank:
- 1,000,000
- Initial % of all page loads:
- 0.00011%
- Further reduction assumption:
- 10%
- Updated % of page loads:
- 0.000011% (because this only happens on an estimated 10% of website page loads.)
- Additional unaccounted-for breakage:
- High confidence
- Final % of page loads:
- 0.0011% (because this is likely to be an issue on other websites too.)
How is the page load percentage calculated from domain rank?
The paper "A World Wide View of Browsing the World Wide Web", written by researchers at Google and Stanford University, presents data from "several hundred million global users" revealing the following traffic statistics:
- The top website has 17% of traffic
- The top 6 sites have 25% of traffic
- The top 100 sites have 40% of traffic
- The top 10K sites have 70% of traffic
- The top 1M sites have 95%+ of traffic
This is a logarithmic increase which, when plotted, gives us the following trend line formula:
Note that's a cumulative page load percentage — the running total for all domain ranks. To get individual domain ranks we can calculate the page load difference for each domain rank. I did this for the top 300 domains which can them be plotted to give another trend line formula:
These are the constants that this calculator uses.