# Confidence in frequencies and sample sizes

LsDgh
Red Chipper Posts:

**21**✭✭
Do you guys have any rules of thumb when it comes to the level of trust you put in the information you have on opponents frequencies?

What sort of sample size do you look for before you first start to factor in things like vpip/pfr, 3-bet %, fold vs. cbet , check-raise % etc..

Obviously you're going to be able to glean more from stats like vpip where we get to see their decision every hand as opposed to say how often they've bluffed the river.

Got thinking about this because I read an example that if an opponent has played 121 hands and has 10% the 95% confidence interval for his range is around 4%-16%. A pretty wide gap which has me thinking I probably value hud stats too much with too small samples.

What sort of sample size do you look for before you first start to factor in things like vpip/pfr, 3-bet %, fold vs. cbet , check-raise % etc..

Obviously you're going to be able to glean more from stats like vpip where we get to see their decision every hand as opposed to say how often they've bluffed the river.

Got thinking about this because I read an example that if an opponent has played 121 hands and has 10% the 95% confidence interval for his range is around 4%-16%. A pretty wide gap which has me thinking I probably value hud stats too much with too small samples.

Tagged:

## Leave a Comment

#### Categories

- 11.9K All Categories
- 510 Come Say Hello
- 3.8K Live Poker Hands
- 507 Coaching & Commercial
- 2.5K Online Poker Hands
- 936 Tournament Poker Hands
- 49 Omaha Variants
- 141 Local Poker Groups
- 2.6K General Concepts
- 318 New To Poker Questions
- 182 Podcast, Articles, Freebies
- 100 CORE
- 336 PRO Products
- 789 Off Topic Chat
- 23 Nutrition & Fitness
- 107 Challenge Forum
- 17 Poker Workbooks
- 10 Preflop & Math Workbook
- 7 Postflop Workbook
- 281 Hand Reading
- 140 Live Workbook (Vol 1)
- 62 6max Workbook (Vol 1)
- 31 MTT Workbook (Vol 1)
- 99 Poker Software

## Comments

4,448-Maybe I should produce a table or something.

102✭✭To your question, larger n is always better, but you can probably make decent assumptions with 50 or more hands. Confidence intervals for VPIP and PFR tend to be accurate with this number of hands.

4,448-However, I'd still want to throw in some priors.

1,562✭✭✭✭You are right that there are sample size issues that I suspect many players ignore. This is even a bigger problem with post-flop stats.

Instead of trying to do a bunch of statistics, I try to look for other information to support or not the "HUD read"

-make sure to watch all showdowns as these can sometimes give you a very big clues

-consider what is typical in your player pool

-do they use bet sizes that make sense

-do other HUD stats support/confirm

1,876-Lets say you are figuring out VPIP at the table for your target.

You see her fold.

Her VPIP is 0%

Is she tight?

Is this significant?

A simple way to answer this significance question is:

If I changed one observation to the other direction, how much would the calculation change?

We change her last observation to a VPIP

Now she had a VPIP of 100%

The change of one observation slipped the result by 100%.

I do not trust this stat at all.

Next hand.

She folds:

0% VPIP

Flip the observation:

50% VPIP

the change is 50%

I do not trust this stat at all.

----

...

Hand 10

She folds:

20% VPIP

Flip the observation:

30% VPIP

the change is 10%

I am beginning to believe in this stat.

Next the question would be:

How sensitive is your decision to this stat you are estimating?

If you estimate a stat at 10% but you know you will fold if this stat is 50% and below. You guess that this stat would change by 5% if the last observation is flipped.

Under this condition, your estimate of this stat is very useful.

If you estimate you will fold if the stat is below 12%, you do not feel very confident in this.

This is all hand-wavy, but it gives you a decent gut check on stats you are estimating from small datasets.

Author Poker Plays You Can Use

Author Poker Workbook for Math Geeks

21✭✭Cheers Doug

4,448-4,448-The main point I'd take away from Doug's contribution is that it really matters how the stat in question impacts our decision. This is often overlooked, but is in fact critical.

As to the math point made in the OP and highlighted in the quote. As discussed above, there are tacit assumptions about distributions here, but I imagine the confidence limit was obtained as follows.

121 hands and a 10% VPIP means this player VPIPed 12 times. The square root of 12 is ~ 3.46. That's the standard error assuming counting stats. The 95% confidence limit is twice that, thus at that level, this player VPIPs 12 +/-7 or [5,19].

5/121 ~ 0.04.

19/121 ~ 0.16

Hence the claim the 95% CL is 4% to 16% VPIP.

Anyone who has taken a college stats class can find half a dozen things wrong with that estimate, but equally without employing Bayes' theorem it's not bad.

I'd also suggest that, when using HUD stats, focus on how many

opportunitiesa player had for a certain action. You can have 3,000 hands on someone, but if you try and dig out their 4-bet from HJ or something equally esoteric, whatever your HUD tells you is probably close to meaningless. Give or take.1,876-https://xkcd.com/435/

Engineers are somewhere around biologists, depending on how close they are to a milling machine or a screwdriver.

Author Poker Plays You Can Use

Author Poker Workbook for Math Geeks

1,562✭✭✭✭This is so true.

As an aside, I am a bit titled that the latest greatest HM3 doesn't allow you to filter this on HUD as HM2 does. They removed one of my favorite HUD features.