Homebrew Equity Calculator

AlpineCurtAlpineCurt Red Chipper Posts: 61 ✭✭
In addition to microstakes player, I am an amateur/hobbyist programmer. For my own amusement and practice, I’ve been working on an equity calculator.

So far, you can enter any number of players’ ranges, a board (a flop with or without a turn and/or river or no board at all), and it will run a specified number of games and calculate each player’s equity.

My results are slightly off (usually only 1-3 percent points) from what Equilab gives and I’m trying to figure out why.

Possibilities I can see:

1. My methodology is off. My program will create or complete a board, pull a random combo from each player’s range (check to make sure there are no card duplicates), determine the highest five card hand for each player, and determine the winner. Ties explained below.

2. Definition of Equity is incorrect. My program returns the percent of the pot each player is entitled to. Each game, the program finds the number of “claims to the pot”. Meaning, if one player has the highest hand, there is one claim. If two players chop, there are two pot claims. If a whole table is playing the board, there are 9 claims and 9 added to total pot claims. Each claim is considered a “win” for that player. After completing the specified number of games, equity for each player is determined by dividing their number of “wins” (“claims to the pot”) by the total number of claims to the pot made by all players. This percentage includes the times they win outright and tie. Equilab displays results as %win along with %tie and adds those together to get equity.

3. Small sample size. My program is much slower than Equilab, and I'm not able to run 100 million games. I’ll typically run 5,000-10,000 games to determine equities. While a bigger n is always better to have, I’ve noticed that after a few thousand games, many more games after that don’t adjust results more than about a 1/10th of a percent. Certainly not 3 percentage points. My results are consistently off by that 1-3 percentage points.

4. Bug in the program. Every 250th game run, I have the program spit out each player’s seven card hand (hole cards + completed board), what it calculated as the highest five card hand, and the winner among each player’s highest five card hand. I can then go through and check to make sure it’s determining highest hands and the winner(s) correctly, and that there are no duplicate cards anywhere. From my own testing, this is working correctly.

5. Equilab is wrong. This is highly unlikely, but however remote, still a possibility. I’ve tried finding out how equilab calculates equities, but could not find anything definite. I did find one post saying they pull from a huge pre-calculated array. I wonder if equilab is actually using very close estimates and using volume of games played to hone in on accuracy.

Example results comparison:

Player 1: 22+, ATs+, KTs+, QTs+, JTs, AQo+, KQo
Player 2: 22+, A2s+, K7s+, QTs+, JTs, T9s, 98s, 87s, 76s, A9o+, KQo
Player 3: 22+, A2s+, K8s+, Q8s+, J9s+, A8o+, KJo+
Player 4: 72o

Board: AhTs3c

Equilab results (~50 million games):
Player 1: 30.89%
Player 2: 32.96%
Player 3: 33.11%
Player 4: 3.05%

My Results (50,000 games):
Player 1: 31.36%
Player 2: 31.82%
Player 3: 30.76%
Player 4: 6.06%

Any CS majors or programmers out there who can see any glaring errors or need clarification on something? I’m certain there is a more efficient or elegant way to calculate equity, and I would love to hear it. Like I said, I’m relatively new to coding. I would love to owe someone a beer at this summer’s WSOP for some good feedback.

Comments

  • TheGameKatTheGameKat Posts: 2,840 -
    I think @Doug Hull came across a similar problem and we figured out what was going on by looking at very narrow ranges. Seem to recall it was something to do with card removal.
    Moderation In Moderation
  • TheGameKatTheGameKat Posts: 2,840 -
    AlpineCurt wrote: »
    1. My methodology is off. My program will create or complete a board, pull a random combo from each player’s range (check to make sure there are no card duplicates), determine the highest five card hand for each player, and determine the winner. Ties explained below.

    I'm guessing this might be the issue, but would need functional details to be sure. The "check to see there are no card duplicates" means it's possible that the same card may be drawn twice? In other words, you don't trim the remaining card pool when a card is in hand? If so, that would mess up the %s.
    Moderation In Moderation
  • AlpineCurtAlpineCurt Red Chipper Posts: 61 ✭✭
    TheGameKat wrote: »
    The "check to see there are no card duplicates" means it's possible that the same card may be drawn twice? In other words, you don't trim the remaining card pool when a card is in hand? If so, that would mess up the %s.

    After it pulls a random combo from a player's range, it checks each card in that combo against every other card currently in use (the board, and all hole cards already selected from other players) and if it detects a duplicate, it chooses another random combo. This repeats till a valid random combo is found. Also, looking at the random samples while it's running games, I can "manually" check to make sure no duplicates are being used (i.e. a player is holding As3s and the board also has the As). While it's possible there is an error here, it seems to be working correctly.

    Another example: AA vs KK all in preflop:

    Eqilab (~50 million games):
    AA: 81.93%
    KK: 18.07%

    My results (50,000 games):
    AA: 73.63%
    KK: 26.37%

    The problem is even greater. I recant my statement about consistently being off only 1-3%.
  • TheGameKatTheGameKat Posts: 2,840 -
    Yeah, I've seen this before. As noted above, for increasingly narrow ranges down to specific holdings the discrepancy gets worse.

    And I am 90% sure the issue is this "check for duplicates" thing. (In fact ubergeeks might like to check my hypothesis we're dealing with an amusing variant of the Monty Hall problem.)

    Give me one more bit of info and I may be able to crack this. How exactly does the check for duplicates and replace work? And at what stage?

    For example, in the AA vs KK example you've run, you'd only need to check for duplicates if the board has already run out, unless you're actually drawing 2 Ks from the 4 available and you don't remove the drawn one from the remaining pool. If you assign the hole cards first, it sounds like it's possible for the board to "draw" a card that's already in one of the hands?

    I'm pretty sure that's the heart of the problem. If you can pull duplicates and then throw one away and try again it's messing up the frequencies. Ideally what you'd prefer to do is have a simulation that deals out cards, but at the very least you need to remove cards from the remaining card pool once they've been allocated.

    Here's another test. What equity do you get if you run :KH::KS: against :AH::AS: ?
    Moderation In Moderation
  • AlpineCurtAlpineCurt Red Chipper Posts: 61 ✭✭
    General algorithm:
    • For each input range, parse through the input range text and create a list (array) of every combo. The length of this list equals the number of combos in that range.
    • Create a deck of cards, then remove any board cards from that deck.
    • Complete the board by pulling random cards from the deck till the board has five cards.
      - Board cards are added to a dead card list
    • For each player’s combo list do the following:
      - Remove blocked combos from their range using the dead card list.
      - Pull a random combo from this new list of unblocked combos.
      - Add those cards to the dead card list.
      - Recurse with updated dead card list till every player's combo list has had a random combo selected.
    • For each player’s randomly selected combo:
      -Make a seven card hand of the board + their random combo
      -Determine the highest five card hand of those seven cards.
    • Compare each player’s highest five card hand and determine the winner(s).
    • Add results to the running tally of each player’s claims to the pot and total pot claims.
    • Repeat from “Create a deck of cards...” till specified number of games has been completed.
    Any glaring errors in method or logic where removal is not correctly handled?

    AhAs vs KhKs all in preflop:

    Equilab results (~50 million games):
    AhAs: 82.65%
    KhKs: 17.35%

    My Results (50,000 games):
    AhAs: 82.23%
    KhKs: 17.77%

    Did you expect these results to be closer or further from Equliab?
  • TheGameKatTheGameKat Posts: 2,840 -
    edited January 11
    Yeah, thought so. When you give it exact hands it agrees more closely with Equilab. When you open up this "check for duplicates" thing it screws it up, with the offset being bigger the larger the number of potential duplicates. It's because it's basically creating an extra opportunity to draw an out.
    Moderation In Moderation
  • TheGameKatTheGameKat Posts: 2,840 -
    If this diagnosis is correct, what you should find is that your methodology always predicts higher equities than standard methods for the dominated hand.

    E.g. AQ vs AK is 26.4 vs 73.6 according to Equilab. I'd predict your method gives AQ higher equity.
    Moderation In Moderation
  • AlpineCurtAlpineCurt Red Chipper Posts: 61 ✭✭
    I genuinely appreciate the feedback.

    What would be a better way to handle card removal when selecting combos?

    I rearranged the order of operations to more closely resemble the way a poker hand is dealt and results seem much more accurate.

    General flow is now:
    • Create deck of cards
    • Remove board cards from deck
    • For each player's range:
      -Remove blocked combos
      -Select random combo
      -Remove randomly selected combo from deck
    • Complete the board using randomly selected cards from remaining deck
    • Find winner
    • Tally results, repeat till desired number of games played

    Now for AA vs KK AIPF, I'm getting 81.98% and 18.02%. (Equilab gives 81.95% and 18.05%)

    Another example:
    Player 1: 22+,A2s+,K5s+,Q8s+,J9s+,T8s+,97s+,86s+,76s,65s,54s,43s,A7o+,KTo+,QTo+,JTo
    Player 2: 22+,A2s+,A2o+
    Player 3: TT+,AJs+,KQs,AJo+,KQo
    Player 4: 7c8d
    Board: 6h 2s Td 3c

    My results (750,000 games):
    Player 1: 26.08%
    Player 2: 29.12%
    Player 3: 31.35%
    Player 4: 13.44%

    Equilab:
    Player 1: 27.44%
    Player 2: 29.14%
    Player 3: 30.41%
    Player 4: 13.01%

    This has been stumping me for a while, and the desire to move on to the next part is making me say "my results are close enough for now."
  • LeChiffreLeChiffre NetherlandsRed Chipper Posts: 581 ✭✭✭
    edited January 14
    AlpineCurt wrote: »
    General flow is now:
    • Create deck of cards
    • Remove board cards from deck
    • For each player's range:
      -Remove blocked combos
      -Select random combo
      -Remove randomly selected combo from deck
    • Complete the board using randomly selected cards from remaining deck
    • Find winner
    • Tally results, repeat till desired number of games played

    Hmm. This seems wrong.

    Suppose player A has :KS::KC: and :AS::4S:
    Player B has :KS::KC: and :AH::TH:.
    In Equilab we can see player A has 44.07% equity.

    If we always start by choosing a combo in player A's range and run out a board, we will run boards with the following probability distribution:
    - :KS::KC: vs :AH::TH: (50%, A has 67.16% equity)
    - :AS::4S: vs :KS::KC: (25%, A has 32.69% equity)
    - :AS::4S: vs :AH::TH: (25%, A has 32.35% equity)
    , whereas the probabilities should be evenly distributed as they are all equally likely.

    If we take the weighted average we get player A's equity = 0.5*0.6716+0.25*0.3269+0.25*0.3235 = 49.84%

    Note also if you start with always choosing a combo in player B's range it will go like this:
    - :KS::KC: vs :AH::TH: (25%)
    - :AS::4S: vs :KS::KC: (50%)
    - :AS::4S: vs :AH::TH: (25%)
    , which is a different distribution.

    If we take the weighted average we get player A's equity = 0.25*0.6716+0.5*0.3269+0.25*0.3235 = 41.22%

    Clearly the order of the players is significant.

    What if we randomize which player to choose from first?
    Then there's a 50% chance of the match-ups where A is first, and a 50% chance
    of the match-ups where B is first. This means that we get the following distribution:

    - :KS::KC: vs :AH::TH: (0.5*50% + 0.5*25% = 37.5%)
    - :AS::4S: vs :KS::KC: (0.5*25% + 0.5*50% = 37.5%)
    - :AS::4S: vs :AH::TH: (0.5*25% + 0.5*25% = 25%).

    As you see we still don't have a proper distribution.

    In fact, if we now look at player's A equity we get:
    0.375*0.6716+0.375*0.3269+0.25*0.3235 = 45.53%

    Note that we also get this by simply taking the average of 49.84% and 41.22% since they are equally likely to represent player A's equity, but I wanted to show the faulty distribution.

    So I think the approach of choosing combo after combo from each range is wrong and leads to these kind of errors. We are ignoring the crucial information which is the probability that a match-up should take place at. It hence seems we need to know the total number of possible match-ups.

    Might I suggest something different?

    First enumerate all possible match-ups.
    Until you've reached your maximum number of runs:
    - Pick a random match-up.
    - Run out the board from remaining deck, determine the winner, update tally

    This method unfortunately does require you to enumerate all possible match-ups, though it's probably possible to do it without doing so.

    Anyway, in my example this should converge to the required 44.07% since very match-up will happen 1/3 of the time.

    (1/3)*0.6716+(1/3)*0.3269+(1/3)*0.3235 = 44.07%
  • LeChiffreLeChiffre NetherlandsRed Chipper Posts: 581 ✭✭✭
    I guess this was the Monty Hall problem @TheGameKat refered to. By choosing a combo which blocks the other player's range, the probabilities shift from seemingly (1/3)-(1/3)-(1/3) to something else (wrong). Just like in the famous game though honestly this is much more fun.
  • AlpineCurtAlpineCurt Red Chipper Posts: 61 ✭✭
    edited January 15
    LeChiffre wrote: »
    Clearly the order of the players is significant.

    What if we randomize which player to choose from first?

    I'm glad you brought this up. I was wondering if randomizing the range order was necessary for reasons similar to what you stated. I just hadn't done the math or thought through it as thoroughly as you did.

    From a programming perspective, choosing a random order of the players, then a random combo from each shouldn't be too difficult.

    How would you suggest handling blocked combos? (I see why you suggest enumerating every possible match-up might be necessary). After choosing a random player order, then a random combo from each, check at that point for any "overlapping cards"? That would preserve the integrity of a proper distribution, but creates the possibility of, in theory, picking millions or an infinite number of impossible combos before landing on a usable selection from each range. Then that problem could repeat every game being run.

    @LeChiffre, I feel like I sound like a broken record, but thanks for that detailed breakdown. It's given me an interesting problem to think on.

    This makes me really wonder what shortcut Equilab is using.

    Also, every time The Monty Hall Problem is mentioned, I think of the movie "21". Check it out if you haven't seen it.
  • LeChiffreLeChiffre NetherlandsRed Chipper Posts: 581 ✭✭✭
    edited January 15
    AlpineCurt wrote: »

    I'm glad you brought this up. I was wondering if randomizing the range order was necessary for reasons similar to what you stated. I just hadn't done the math or thought through it as thoroughly as you did.

    From a programming perspective, choosing a random order of the players, then a random combo from each shouldn't be too difficult.

    Yes but as I showed this would also be a wrong approach. We simply can not start by choosing a combo from one player's range (regardless of whether that player was chosen randomly), as I have demonstrated in my example. We have to randomly select a complete match-up (combo vs combo vs combo vs combo etc.) to preserve an equal distribution of each match-up.

    By choosing a combo we alter the distribution of each match-up. Just like in the Monty Hall problem, the moment we choose a door and the game show host opens a door, the probabilities shift from equal to unequal (switching doors becomes a better option).
    AlpineCurt wrote: »
    This makes me really wonder what shortcut Equilab is using.

    My guess would be something close to this.
    LeChiffre wrote: »
    First enumerate all possible match-ups.
    Until you've reached your maximum number of runs:
    - Pick a random match-up.
    - Run out the board from remaining deck, determine the winner, update tally

  • AlpineCurtAlpineCurt Red Chipper Posts: 61 ✭✭
    LeChiffre wrote: »
    Yes but as I showed this would also be a wrong approach. We simply can not start by choosing a combo from one player's range (regardless of whether that player was chosen randomly), as I have demonstrated in my example. We have to randomly select a complete match-up (combo vs combo vs combo vs combo etc.) to preserve an equal distribution of each match-up.

    By choosing a combo we alter the distribution of each match-up. Just like in the Monty Hall problem, the moment we choose a door and the game show host opens a door, the probabilities shift from equal to unequal (switching doors becomes a better option).

    I'm pretty sure I now get what you're saying. Do you see any problems with doing this:
    • Enumerate every possible match-up
    • Remove impossible match-ups (i.e. check for card overlap)
    • Randomly select a match up
    • Run out the board
    • Find winner, tally, etc.
  • LeChiffreLeChiffre NetherlandsRed Chipper Posts: 581 ✭✭✭
    Sure! You could of course also check for overlap already when you are enumerating the match-ups. Should save some computational time.

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file