May 17, 2021

How to Measure Anything

Techniques and ideas for measuring the things traditionally thought of as intangible.

Many of the things thought as intangible have simple and reliable methods of measurement.

Real life measurements often give fuzzy answers, but they help narrow down the choices to those most likely to succeed. By applying intuition to the smaller list, you get far superior results compared to intuition alone.

You can use simple measurements and some cleaver math to estimate values that would be impossible to measure accurately. Eratosthenes, an ancient Greek, measured the circumference of the earth using shadows and the distance between two cities.

You can combine already known quantities to gain new insights. By looking up the population of New York, estimating how many people own pianos, and figuring out how often an average piano needs tuning, you can get close to the number of piano tuners in New York.

You can prove or disprove premises by measuring a random sample. If the thing you're testing has any chance of success, you should see at least some effect on your sample.

No real world measurement is 100% accurate. There is always a further degree of accuracy you could achieve. Chasing more accuracy can quickly become a losing game. Therefore, you should only look for enough accuracy to reduce uncertainty to a point where you can make a decision.

Nominal measurements represent yes or no quantities. There is no scale between them.

Ordinal measurement provide a scale but the values on that scale do not represent exact quantities. For example, a 4 star movie isn't twice as good as a 2 star movie.

Homogeneous measurements represent concrete values that support a full range of computation.

Interval measurements have an arbitrary starting point. This allows the computation of differences, but does not support the full range of operations available with a Homogeneous measurement.

All measurements are inaccurate to some degree or require an educated guess. To get the full value of these measurements, you have to know the probability along with the measurement value.

A sample as small and five can give the range of median with a 93.75% accuracy.

Just a single sample has a 75% of being part of the majority of the population.

Knowing just the above methods greatly increases certainty when starting from knowing nothing.

People overestimate the chance of outliers.

As the number of participants increase, the chance of significant outlier drastically decreases.

Statistical models outperformed experts in numerous studies across diverse fields.

The uniqueness fallacy leads to believing that a situation is so unique that history does not apply to it. In reality, even unique situation share properties with historical data. Always look towards past work to help with the problem at hand.

You already have, or can easily find, useful data about the situation you want to analyze.

You need far less data than you expect. In fact, often time you have more data than you could possibly analyze. In either case sampling helps to get started with useful calculations.

A simple measurement can get you 80% of the information that you could get from a perfect measurement.

Always start by figuring the decision you need to make. This will show you what measurements you need, how you should measure it, and how it will impact the final decision.

A decision must:

- Have at least two alternatives
- Involve uncertainty
- Include at least one scenario with a risk attached
- Have a decision maker to make the final call

The measurement must have well defined consequences on the decision being made.

Decomposing a problem can yield superior results, even if the measurement is a guess.