Artificial intelligence is being used to tackle some of the world's biggest challenges, ranging from controlling environmental damages to enhancing the study of computer science, medicine and human resources. It's also used to spur on the development of finance, education, music, customer service, transportation, telecommunications, aviation and health care administration.
In a recent whimsical, yet fascinating application of Artificial Intelligence, Grande Vegas casino players and other poker enthusiasts had an opportunity to observe AI compete in a poker battle with some of the world's best poker champions to test the limits of this mix of technologies.
AI and Poker
Most people have heard of AI and are aware that it has the potential to help mankind in many different ways but they aren't sure exactly what it is or how it works. In essence, AI is a constellation of technologies—from natural language processing to machine learning. These technologies are geared to allow machines to sense, learn, comprehend and act.
AI has already become part of our every days lives. Some of the AI-assisted services are recognizable to everyone including the virtual assistant Siri, MOOCs (massive, open, online courses), market analysis and data mining, analysis of user-created content and devices that assist people with their personal finances.
Recently, a group of scientists at Carnegie Mellon University developed an AI system that competed in a poker battle against four of the world's top human pros. The tournament, called “Brains Vs. Artificial Intelligence: Upping the Ante” pitted the Libratus AI computer against the humans for a $200,000 pot in 20 days of matches of Heads-Up, No-Limit Texas Hold’em.
The competition was held at the Rivers Casino in Pittsburgh, Pennsylvania. It involved a total of 120,000 hands played and the implications of the win (spoiler – AI won $1,766,250 in chips) go beyond who built the best hand.
Imperfect Information
The poker pros who joined the games included Jason Les, Dong Kim, Jimmy Chou and Daniel McAulay. They vied against Libratus, the AI machine that was built by Carnegie Mellon, in an effort to win – but also as part of an experiment that aimed to test the limits of Artificial Intelligence.
The MIT Technology Review explained the rationale of the experiment. The game, the Review wrote, has unique rules and thus it may not be as easily dominated by computerized players. "Unlike board games such as Go or chess, poker is a game of 'imperfect information,' and for this reason it has proved even more resistant to computerization than Go."
Carnegie Mellon computer science researcher Tuomas Sandholm expanded by explaining that in games like chess, "you know exactly what has happened in the game so far." Sandholm developed Libratus along with Ph.D. student Noam Brown. But, he said, this type of advantage doesn't exist in poker because in poker there's crucial unknown information -- your opponents' hands.
The decision to arrange the competition in Heads-Up, No-Limit Texas Hold’em was no accident – the scientists wanted to test AI in a game that deals with a huge scale of probabilities. For that reason, they eschewed Limit Texas Hold'em and decided on the No Limit game.
Game Time
The game of 120,000 hands was played in a marathon 20-day competition and the results – artificial intelligence beat the pros – has deep significance. The findings showed that artificial intelligence can spread into other realms of information gathering including fields such as military strategy, cybersecurity, medical treatment planning and business negotiation.
Sandholm summed it up by saying “The best AI's ability to do strategic reasoning with imperfect information has now surpassed that of the best humans." Frank Pfenning, head of the computer science department at Carnegie Mellon, elaborated.“The computer can't win at poker if it can't bluff. Developing an AI that can do that successfully is a tremendous step forward scientifically and has numerous applications. Imagine that your smartphone will someday be able to negotiate the best price on a new car for you. That's just the beginning.”
For their part the poker pros – who will split a $200,000 purse based on their performance in the competition, agreed that Libratus was tougher than they expected. McAulay said “Whenever you play a top player, you learn from it.”
Strategy
Libratus was able to play 4 players simultaneously by computing its strategy on the Pittsburgh Supercomputing Center's Bridges computer. It used an algorithm to determine its own weaknesses from the other pros rather than computing its opponent's weaknesses. The end-game strategy also had a perfect analysis of the cards which allowed the AI to solve an imperfect information game.
Nick Nystrom, Pittsburgh Supercomputing Center’s senior director of research and principal investigator for the National Science Foundation-funded Bridges system said “We designed Bridges to converge high-performance computing and artificial intelligence. Libratus' win is an important milestone toward developing AIs to address complex, real-world problems.”
AI and Gaming
In the past, game-playing AI has machine learning or programmed human knowledge strategies. Today however AI programmers can approximate equilibrium strategies and rational play to compete with and sometimes beat human players.
Libratus uses the Nash Equilibrium. Nash is named for the famed mathematics scholar John Nash (subject of the movie A Beautiful Mind) which involves a pair of strategies (one per player) where "neither player can benefit from changing strategy as long as the other player’s strategy remains the same…..Libratus employs new technologies that facilitate a faster equilibrium-finding method through identification of different paths for playing different types of hands. After coming across many bad paths it learns to ignore those bad paths."
Libratus has been improved over Claudico, last year's AI system from Carnegie Mellon. More core hours were put into programming the system and two new algorithms that allowed for better play and more computing hardware and resources were employed.
Libratus was programmed to know the rules of poker but doesn't rely on any other information such as advice from poker experts or referenced materials. This means that the AI might play the game in a totally new way and could make moves that may seem unfavorable to the best of traditional human players. In commenting on the Libratus approach Sandholm said "It's very different from how humans play this game. It plays like a martian. It’s deriving its own strategy from just the rules of the game."
