Predicting opponent moves for improving hearthstone AI

Publikation: Beitrag in Buch/Bericht/Sammelwerk/KonferenzbandAufsatz in KonferenzbandForschungPeer-Review

Autoren

Externe Organisationen

  • Otto-von-Guericke-Universität Magdeburg
Forschungs-netzwerk anzeigen

Details

OriginalspracheEnglisch
Titel des SammelwerksInformation Processing and Management of Uncertainty in Knowledge-Based Systems. Theory and Foundations - 17th International Conference, IPMU 2018, Proceedings
Herausgeber/-innenJesus Medina, Bernadette Bouchon-Meunier, Ronald R. Yager, Jose Luis Verdegay, David A. Pelta, Manuel Ojeda-Aciego, Inma P. Cabrera
Herausgeber (Verlag)Springer Verlag
Seiten621-632
Seitenumfang12
ISBN (Print)9783319914756
PublikationsstatusVeröffentlicht - 18 Mai 2018
Extern publiziertJa
Veranstaltung17th International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems, IPMU 2018 - Cadiz, Spanien
Dauer: 11 Juni 201815 Juni 2018

Publikationsreihe

NameCommunications in Computer and Information Science
Band854
ISSN (Print)1865-0929

Abstract

Games pose many interesting questions for the development of artificial intelligence agents. Especially popular are methods that guide the decision-making process of an autonomous agent, which is tasked to play a certain game. In previous studies, the heuristic search method Monte Carlo Tree Search (MCTS) was successfully applied to a wide range of games. Results showed that this method can often reach playing capabilities on par with humans or even better. However, the characteristics of collectible card games such as the online game Hearthstone make it infeasible to apply MCTS directly. Uncertainty in the opponent’s hand cards, the card draw, and random card effects considerably restrict the simulation depth of MCTS. We show that knowledge gathered from a database of human replays help to overcome this problem by predicting multiple card distributions. Those predictions can be used to increase the simulation depth of MCTS. For this purpose, we calculate bigram-rates of frequently co-occurring cards to predict multiple sets of hand cards for our opponent. Those predictions can be used to create an ensemble of MCTS agents, which work under the assumption of differing card distributions and perform simulations according to their assigned distribution. The proposed ensemble approach outperforms other agents on the game Hearthstone, including various types of MCTS. Our case study shows that uncertainty can be handled effectively using predictions of sufficient accuracy, ultimately, improving the MCTS guided decision-making process. The resulting decision-making based on such an MCTS ensemble proved to be less prone to errors by uncertainty and opens up a new class of MCTS algorithms.

ASJC Scopus Sachgebiete

Zitieren

Predicting opponent moves for improving hearthstone AI. / Dockhorn, Alexander; Frick, Max; Akkaya, Ünal et al.
Information Processing and Management of Uncertainty in Knowledge-Based Systems. Theory and Foundations - 17th International Conference, IPMU 2018, Proceedings. Hrsg. / Jesus Medina; Bernadette Bouchon-Meunier; Ronald R. Yager; Jose Luis Verdegay; David A. Pelta; Manuel Ojeda-Aciego; Inma P. Cabrera. Springer Verlag, 2018. S. 621-632 (Communications in Computer and Information Science; Band 854).

Publikation: Beitrag in Buch/Bericht/Sammelwerk/KonferenzbandAufsatz in KonferenzbandForschungPeer-Review

Dockhorn, A, Frick, M, Akkaya, Ü & Kruse, R 2018, Predicting opponent moves for improving hearthstone AI. in J Medina, B Bouchon-Meunier, RR Yager, JL Verdegay, DA Pelta, M Ojeda-Aciego & IP Cabrera (Hrsg.), Information Processing and Management of Uncertainty in Knowledge-Based Systems. Theory and Foundations - 17th International Conference, IPMU 2018, Proceedings. Communications in Computer and Information Science, Bd. 854, Springer Verlag, S. 621-632, 17th International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems, IPMU 2018, Cadiz, Spanien, 11 Juni 2018. https://doi.org/10.1007/978-3-319-91476-3_51
Dockhorn, A., Frick, M., Akkaya, Ü., & Kruse, R. (2018). Predicting opponent moves for improving hearthstone AI. In J. Medina, B. Bouchon-Meunier, R. R. Yager, J. L. Verdegay, D. A. Pelta, M. Ojeda-Aciego, & I. P. Cabrera (Hrsg.), Information Processing and Management of Uncertainty in Knowledge-Based Systems. Theory and Foundations - 17th International Conference, IPMU 2018, Proceedings (S. 621-632). (Communications in Computer and Information Science; Band 854). Springer Verlag. https://doi.org/10.1007/978-3-319-91476-3_51
Dockhorn A, Frick M, Akkaya Ü, Kruse R. Predicting opponent moves for improving hearthstone AI. in Medina J, Bouchon-Meunier B, Yager RR, Verdegay JL, Pelta DA, Ojeda-Aciego M, Cabrera IP, Hrsg., Information Processing and Management of Uncertainty in Knowledge-Based Systems. Theory and Foundations - 17th International Conference, IPMU 2018, Proceedings. Springer Verlag. 2018. S. 621-632. (Communications in Computer and Information Science). doi: 10.1007/978-3-319-91476-3_51
Dockhorn, Alexander ; Frick, Max ; Akkaya, Ünal et al. / Predicting opponent moves for improving hearthstone AI. Information Processing and Management of Uncertainty in Knowledge-Based Systems. Theory and Foundations - 17th International Conference, IPMU 2018, Proceedings. Hrsg. / Jesus Medina ; Bernadette Bouchon-Meunier ; Ronald R. Yager ; Jose Luis Verdegay ; David A. Pelta ; Manuel Ojeda-Aciego ; Inma P. Cabrera. Springer Verlag, 2018. S. 621-632 (Communications in Computer and Information Science).
Download
@inproceedings{8a94fedc65e84e95b9e6848b09958358,
title = "Predicting opponent moves for improving hearthstone AI",
abstract = "Games pose many interesting questions for the development of artificial intelligence agents. Especially popular are methods that guide the decision-making process of an autonomous agent, which is tasked to play a certain game. In previous studies, the heuristic search method Monte Carlo Tree Search (MCTS) was successfully applied to a wide range of games. Results showed that this method can often reach playing capabilities on par with humans or even better. However, the characteristics of collectible card games such as the online game Hearthstone make it infeasible to apply MCTS directly. Uncertainty in the opponent{\textquoteright}s hand cards, the card draw, and random card effects considerably restrict the simulation depth of MCTS. We show that knowledge gathered from a database of human replays help to overcome this problem by predicting multiple card distributions. Those predictions can be used to increase the simulation depth of MCTS. For this purpose, we calculate bigram-rates of frequently co-occurring cards to predict multiple sets of hand cards for our opponent. Those predictions can be used to create an ensemble of MCTS agents, which work under the assumption of differing card distributions and perform simulations according to their assigned distribution. The proposed ensemble approach outperforms other agents on the game Hearthstone, including various types of MCTS. Our case study shows that uncertainty can be handled effectively using predictions of sufficient accuracy, ultimately, improving the MCTS guided decision-making process. The resulting decision-making based on such an MCTS ensemble proved to be less prone to errors by uncertainty and opens up a new class of MCTS algorithms.",
keywords = "Bigrams, Ensemble, Hearthstone, Monte Carlo Tree Search Knowledge-base, Uncertainty",
author = "Alexander Dockhorn and Max Frick and {\"U}nal Akkaya and Rudolf Kruse",
year = "2018",
month = may,
day = "18",
doi = "10.1007/978-3-319-91476-3_51",
language = "English",
isbn = "9783319914756",
series = "Communications in Computer and Information Science",
publisher = "Springer Verlag",
pages = "621--632",
editor = "Jesus Medina and Bernadette Bouchon-Meunier and Yager, {Ronald R.} and Verdegay, {Jose Luis} and Pelta, {David A.} and Manuel Ojeda-Aciego and Cabrera, {Inma P.}",
booktitle = "Information Processing and Management of Uncertainty in Knowledge-Based Systems. Theory and Foundations - 17th International Conference, IPMU 2018, Proceedings",
address = "Germany",
note = "17th International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems, IPMU 2018 ; Conference date: 11-06-2018 Through 15-06-2018",

}

Download

TY - GEN

T1 - Predicting opponent moves for improving hearthstone AI

AU - Dockhorn, Alexander

AU - Frick, Max

AU - Akkaya, Ünal

AU - Kruse, Rudolf

PY - 2018/5/18

Y1 - 2018/5/18

N2 - Games pose many interesting questions for the development of artificial intelligence agents. Especially popular are methods that guide the decision-making process of an autonomous agent, which is tasked to play a certain game. In previous studies, the heuristic search method Monte Carlo Tree Search (MCTS) was successfully applied to a wide range of games. Results showed that this method can often reach playing capabilities on par with humans or even better. However, the characteristics of collectible card games such as the online game Hearthstone make it infeasible to apply MCTS directly. Uncertainty in the opponent’s hand cards, the card draw, and random card effects considerably restrict the simulation depth of MCTS. We show that knowledge gathered from a database of human replays help to overcome this problem by predicting multiple card distributions. Those predictions can be used to increase the simulation depth of MCTS. For this purpose, we calculate bigram-rates of frequently co-occurring cards to predict multiple sets of hand cards for our opponent. Those predictions can be used to create an ensemble of MCTS agents, which work under the assumption of differing card distributions and perform simulations according to their assigned distribution. The proposed ensemble approach outperforms other agents on the game Hearthstone, including various types of MCTS. Our case study shows that uncertainty can be handled effectively using predictions of sufficient accuracy, ultimately, improving the MCTS guided decision-making process. The resulting decision-making based on such an MCTS ensemble proved to be less prone to errors by uncertainty and opens up a new class of MCTS algorithms.

AB - Games pose many interesting questions for the development of artificial intelligence agents. Especially popular are methods that guide the decision-making process of an autonomous agent, which is tasked to play a certain game. In previous studies, the heuristic search method Monte Carlo Tree Search (MCTS) was successfully applied to a wide range of games. Results showed that this method can often reach playing capabilities on par with humans or even better. However, the characteristics of collectible card games such as the online game Hearthstone make it infeasible to apply MCTS directly. Uncertainty in the opponent’s hand cards, the card draw, and random card effects considerably restrict the simulation depth of MCTS. We show that knowledge gathered from a database of human replays help to overcome this problem by predicting multiple card distributions. Those predictions can be used to increase the simulation depth of MCTS. For this purpose, we calculate bigram-rates of frequently co-occurring cards to predict multiple sets of hand cards for our opponent. Those predictions can be used to create an ensemble of MCTS agents, which work under the assumption of differing card distributions and perform simulations according to their assigned distribution. The proposed ensemble approach outperforms other agents on the game Hearthstone, including various types of MCTS. Our case study shows that uncertainty can be handled effectively using predictions of sufficient accuracy, ultimately, improving the MCTS guided decision-making process. The resulting decision-making based on such an MCTS ensemble proved to be less prone to errors by uncertainty and opens up a new class of MCTS algorithms.

KW - Bigrams

KW - Ensemble

KW - Hearthstone

KW - Monte Carlo Tree Search Knowledge-base

KW - Uncertainty

UR - http://www.scopus.com/inward/record.url?scp=85056829819&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-91476-3_51

DO - 10.1007/978-3-319-91476-3_51

M3 - Conference contribution

AN - SCOPUS:85056829819

SN - 9783319914756

T3 - Communications in Computer and Information Science

SP - 621

EP - 632

BT - Information Processing and Management of Uncertainty in Knowledge-Based Systems. Theory and Foundations - 17th International Conference, IPMU 2018, Proceedings

A2 - Medina, Jesus

A2 - Bouchon-Meunier, Bernadette

A2 - Yager, Ronald R.

A2 - Verdegay, Jose Luis

A2 - Pelta, David A.

A2 - Ojeda-Aciego, Manuel

A2 - Cabrera, Inma P.

PB - Springer Verlag

T2 - 17th International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems, IPMU 2018

Y2 - 11 June 2018 through 15 June 2018

ER -

Von denselben Autoren