Can AI learn complex card games? One student researched it — and found surprising limitations

Jack Lamberg ’27 is a senior double-majoring in math and computer science at Binghamton University who studied how artificial intelligence can learn through complex card games. 

He worked on the 10-week project, "Cards on the Table: The Winning Move is Not in the Rules," as part of the University of Central Florida's Computer Vision Research Experience for Undergraduates (REU) program last summer.

AI has already surpassed humans in other games, most notably chess, where even the most talented grandmasters have virtually no chance against the best AI chess engines. One AI chess engine, Stockfish, has won every big competition since 2020. It works by simulating millions of branching outcomes of up to 50 future moves to determine the optimal decision, while the most gifted humans struggle to see into the double digits of future moves. 

When competing against humans in complex card games, however, they stand little to no chance of victory. There are fundamental differences between the games in which AI models dominate and the type of game Lamberg is attempting to teach AI, which involves too much randomness and unknown information. 

“In something like chess, of course, it's complicated,” he said. “But you know what moves your opponent could possibly make, and you know what moves you could possibly make. In chess, the bot always knows what to expect. It can calculate the best move options because it can also calculate what all of your move options are. However, with these multilayer card games, they don’t know the exact list of possible moves you can make. There's no knowledge, and you still have to make a good decision.” 

The types of card games that are a part of Lamberg’s research are not based on a standard 52-card playing deck like poker or blackjack, but instead are more complicated. Rather than a set deck for every game, in constructed card games like Magic: The Gathering and the Pokémon Trading Card Game, players build their decks before the game starts using a massive roster of possible cards. 

Although traditional 52-card games share similarities, there are key differences, Lamberg said: “A lot of 52-card games are ‘solved games,’ where there’s a step that you should take which is statistically the best move, and there are ‘brute-force’ AI engines that are built for it. There's no such thing for these complex card games because it's almost too complicated.” 

His goal is to teach the AI to understand the fundamentals of complex card games in a more human way, so it can comprehend the principles through both patterns and strategy. 

Aurora's Call

Although they aren’t skilled at them, AI engines already are aware of complicated card games, with some pseudo-understanding of their mechanics through catalogs of the cards found on the internet. This poses a problem for Lamberg’s research: If the AI program already has knowledge of the game, it becomes nearly impossible to identify what it learns from the teaching Lamberg provides.

The solution? Introduce the AI to an entirely new game. 

“We created our own complex card game," Lamberg said. "We called it Aurora's Call. There are elements of Pokémon, elements of Digimon, Magic: The Gathering, and more, all mixed together in new ways. This way, the AI would have no knowledge going in and no outside influence.” 

The next step was to give the AI a controlled database of information about this new game; to do so, deterministic bots needed to be created. While bots may be viewed as synonymous with AI, Lamberg explained the fundamental difference between the two. 

AI models, such as large language models, are too advanced for their responses to be directly coded by a human with a rigid list of rules. Instead, the base version of the model is introduced to a massive database that it analyzes to predict outcomes. Over time, the models improve through trial and error with the oversight of human trainers and supervisors. In contrast, bots operate according to a rigid set of protocols based on current conditions, most commonly along the lines of “if X then do Y.” Their decision-making, unlike AI, is strictly linear and without nuance; given a specific scenario, their response will always be the same. 

Lamberg designed different bots to follow different protocols, ranging from random moves to detailed instructions.

“By playing the bots against each other, we could simulate hundreds of thousands of games far faster than humans could," Lamberg said. "That creates a database of games to test how good the language models are.” 

It turns out that AI had difficulty learning the game. 

“With the giant database we made of their answers, we found that they were not very good,” Lamberg said. “They didn't understand it very well, although that was about what we had expected. Additionally, we found that the large language models can beat the random bot, but that’s because that bot doesn't really make any smart moves. Against any other even slightly sophisticated bot, the models struggled a lot.” 

With the baseline established, Lamberg sought to improve the AI model's performance by adding expert-level games that a group member played against the bots. This enabled the AI to learn strategies for winning the game. 

While Lamberg has already made progress in the AI’s ability to understand and learn these complex card games, he's looking to push these engines even further and ultimately publish his research. 

“The goal would be to create an AI that could be as good as, and even surpass, pro players of these complex card games," he said. "However, I’m not sure that’s attainable, as I think that there will always be some sort of vulnerability that can be exploited by a human. It’s true that AI models can change their answers and are much more fluid than bots, but their algorithm is still a prediction matrix, and it can become fairly predictable in ways that a human could exploit. AI still has its limits.”