Skip to content

Sampling bias; or, small n

Sampling bias is the error caused by assuming that a sample is representative of the whole population when it is not.

It is the error that says “Everything else like this must be similar to what I’m looking at” without a good foundation (empirical or logical) for doing so.

You might survey the Netherlands in intricate detail and rule out the possibility of the Himalayas.

You might catch and examine a thousand fish from shallow water and never imagine what deep-sea species are like.

You might make a million observations of water above 0°c and never guess the properties of ice.

You might think:

  • “The rest of this country must be like the <1% of it I’ve seen.”
  • “I caught lots of waves today – surfing is easy.”
  • “All people like this must behave in this (objectionable/endearing) way.”
  • “People are mostly smart, like my friends.”
  • “People are mostly dumb, like the idiots who around me.”
  • “I have never seen this problem solved, so it must be impossible.”
  • “We do this all the time, so it must be easy for everyone.”
  • “I’m rich / poor.”
  • “My 2 kids turned out okay / struggled in life, so I’m a good / bad parent.”
  • “I’m great at tennis – the best in my village.”
  • “That ended well / badly so it was a good / bad decision.”
  • “The world is a nice / horrible place.”
  • “This is how service is done everywhere.”
  • “I am a high / low achiever.”
  • “This small amount of pollution I’m making doesn’t matter.”
  • “I’m a good / generous / bad / selfish person.”
  • “Treating people like this is normal.”

See also:

Statistical Dojo

Galton Board

I'd love to hear your thoughts and recommended resources...