Confidence in frequencies and sample sizes

LsDghLsDgh Red Chipper Posts: 21 ✭✭
Do you guys have any rules of thumb when it comes to the level of trust you put in the information you have on opponents frequencies?
What sort of sample size do you look for before you first start to factor in things like vpip/pfr, 3-bet %, fold vs. cbet , check-raise % etc..
Obviously you're going to be able to glean more from stats like vpip where we get to see their decision every hand as opposed to say how often they've bluffed the river.

Got thinking about this because I read an example that if an opponent has played 121 hands and has 10% the 95% confidence interval for his range is around 4%-16%. A pretty wide gap which has me thinking I probably value hud stats too much with too small samples.

Comments

  • TheGameKatTheGameKat Posts: 4,448 -
    You can assume the parent distribution is Gaussian/Poissonian, in which case the number of opportunities gives you the standard deviation, then 2 s.d. gives you the 95% confidence limits. Seems like using a steam-hammer to crack a walnut though. Plus it ignores the priors we could use in a Bayesian approach.

    Maybe I should produce a table or something.
    Moderation In Moderation
  • PapaGiorgioPapaGiorgio Red Chipper Posts: 102 ✭✭
    The issue with percentages is bounding: they range from 0 to 1. You really can't assume gaussian distribution when the percent is outside of the 40-60 range. At 10%, the confidence interval is probably assymetric with the longer tail toward the larger values.

    To your question, larger n is always better, but you can probably make decent assumptions with 50 or more hands. Confidence intervals for VPIP and PFR tend to be accurate with this number of hands.
  • TheGameKatTheGameKat Posts: 4,448 -
    Agreed, we're almost certainly dealing with distributions plagued by skewness and kurtosis, so to do this properly I'd recommend something like the biweight.

    However, I'd still want to throw in some priors.
    Moderation In Moderation
  • kenaceskenaces Red Chipper Posts: 1,562 ✭✭✭✭
    LsDgh wrote: »
    I probably value hud stats too much with too small samples.

    You are right that there are sample size issues that I suspect many players ignore. This is even a bigger problem with post-flop stats.

    Instead of trying to do a bunch of statistics, I try to look for other information to support or not the "HUD read"

    -make sure to watch all showdowns as these can sometimes give you a very big clues
    -consider what is typical in your player pool
    -do they use bet sizes that make sense
    -do other HUD stats support/confirm

  • Doug HullDoug Hull RCP Coach Posts: 1,876 -
    Not at all rigorous but a decent first order approximation:

    Lets say you are figuring out VPIP at the table for your target.

    You see her fold.
    Her VPIP is 0%

    Is she tight?
    Is this significant?

    A simple way to answer this significance question is:
    If I changed one observation to the other direction, how much would the calculation change?

    We change her last observation to a VPIP
    Now she had a VPIP of 100%

    The change of one observation slipped the result by 100%.
    I do not trust this stat at all.

    Next hand.
    She folds:
    0% VPIP

    Flip the observation:
    50% VPIP

    the change is 50%

    I do not trust this stat at all.
    ----

    ...

    Hand 10
    She folds:
    20% VPIP

    Flip the observation:
    30% VPIP

    the change is 10%

    I am beginning to believe in this stat.

    Next the question would be:

    How sensitive is your decision to this stat you are estimating?

    If you estimate a stat at 10% but you know you will fold if this stat is 50% and below. You guess that this stat would change by 5% if the last observation is flipped.

    Under this condition, your estimate of this stat is very useful.

    If you estimate you will fold if the stat is below 12%, you do not feel very confident in this.

    This is all hand-wavy, but it gives you a decent gut check on stats you are estimating from small datasets.
    Co-founder Red Chip Poker,
    Author Poker Plays You Can Use
    Author Poker Workbook for Math Geeks
  • LsDghLsDgh Red Chipper Posts: 21 ✭✭
    That's a nice practical heuristic to use.
    Cheers Doug
  • TheGameKatTheGameKat Posts: 4,448 -
    Engineers, I swear.
    Moderation In Moderation
  • TheGameKatTheGameKat Posts: 4,448 -
    edited September 2020
    LsDgh wrote: »

    Got thinking about this because I read an example that if an opponent has played 121 hands and has 10% the 95% confidence interval for his range is around 4%-16%. A pretty wide gap which has me thinking I probably value hud stats too much with too small samples.

    The main point I'd take away from Doug's contribution is that it really matters how the stat in question impacts our decision. This is often overlooked, but is in fact critical.

    As to the math point made in the OP and highlighted in the quote. As discussed above, there are tacit assumptions about distributions here, but I imagine the confidence limit was obtained as follows.

    121 hands and a 10% VPIP means this player VPIPed 12 times. The square root of 12 is ~ 3.46. That's the standard error assuming counting stats. The 95% confidence limit is twice that, thus at that level, this player VPIPs 12 +/-7 or [5,19].
    5/121 ~ 0.04.
    19/121 ~ 0.16

    Hence the claim the 95% CL is 4% to 16% VPIP.

    Anyone who has taken a college stats class can find half a dozen things wrong with that estimate, but equally without employing Bayes' theorem it's not bad.

    I'd also suggest that, when using HUD stats, focus on how many opportunities a player had for a certain action. You can have 3,000 hands on someone, but if you try and dig out their 4-bet from HJ or something equally esoteric, whatever your HUD tells you is probably close to meaningless. Give or take.
    Moderation In Moderation
  • Doug HullDoug Hull RCP Coach Posts: 1,876 -
    TheGameKat wrote: »
    Engineers, I swear.

    https://xkcd.com/435/

    Engineers are somewhere around biologists, depending on how close they are to a milling machine or a screwdriver.
    Co-founder Red Chip Poker,
    Author Poker Plays You Can Use
    Author Poker Workbook for Math Geeks
  • kenaceskenaces Red Chipper Posts: 1,562 ✭✭✭✭
    TheGameKat wrote: »
    I'd also suggest that, when using HUD stats, focus on how many opportunities a player had for a certain action. .

    This is so true.

    As an aside, I am a bit titled that the latest greatest HM3 doesn't allow you to filter this on HUD as HM2 does. They removed one of my favorite HUD features.

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file