The 2K BotPrize : 2008

2008 BotPrize contest

The first BotPrize contest was held in Perth, on 17 December 2008, as part of the 2008 IEEE Symposium on Computational Intelligence and Games. The aim of the contest was to see if a computer game playing bot could convince a panel of expert judges that it was actually a human player.

The Judging Panel

• David Fogel - President of IEEE Computational Intelligence Society
• Robert (RJ) Spencer - COO of Interzone Entertainment
• Penny Sweetser - senior game designer at 2K Australia and an AI expert
• Cam Atkinson - a l33t game player
• John Wiese - senior project manager in Special Operations Command Support Systems at Thales Australia

The Humans

• Andrew Smith
• Roderick Baker
• Byron Pogson
• Keith Johnson
• Seb Davidson

The Teams

Team	Affiliation	Members
Amis	Charles University in Prague	Michal Stolba Jakub Gemrot Juraj Simlovic
ICE-UT@RITS	Ritsumeikan University, Japan	Daichi Hirono Yuna Dei Ruck Thawonmas
Intelligent Systems Centre	Nanyang Technological University	Budhitama Subagdja Ah Hwee Tan Di Wang
Underdog	University of Western Australia	Oren Nachman
University of Texas at Austin	University of Texas at Austin	Jacob Schrum Igor Karpov Risto Miikulainen

* The team of Jon Wright, David Nichol and Russel Thom from Glasgow Caledonian University made the finals but was unable to attend.

Results

The prize winners were:

• Best bot: AMIS (Michal Stolba, Jakub Gemrot, Juraj Simlovic)
• Best human: Roderick Baker
• Best Judge: John Wiese

Here is a summary of the results in the form of a table of bots and human confederates, sorted by human-ness, as rated by the judges.

Identity	Mean rating	#Judges convinced
Byron (human)	4	5
Andrew (human)	3.8	4
Roderick (human)	3.8	4
Keith (human)	3	2
Seb (human)	2.6	2
AMIS (bot)	2.4	2
ICE (bot)	2.2	1
ISC (bot)	2	2
UTexas (bot)	0.8	0
Underdog (bot)	0.4	0

All the humans were rated as more human than all the bots.

Complete round by round results can be found here.

The table below contains videos made from the games in the final, from the judge's viewpoint. Within each video, all players appear identical, but can be distinguished by name (e.g. Player123), which appears when the player is targeted by the judge. The table entries give the player names used in each game. You might like to try your hand at picking the bots! Can you do better than the judges?

Each video is between 30MB and 50MB. Warning: these videos depict simulated violence.

Judge 1	Judge 2	Judge 3	Judge 4	Judge 5
Round 1	video Player144=Judge Player443=Human Player932=Bot	video Player502=Judge Player212=Human Player773=Bot	video Player830=Judge Player136=Human Player415=Bot	video Player875=Judge Player433=Human Player441=Bot	video Player388=Judge Player471=Human Player639=Bot
Round 2	video Player787=Judge Player859=Human Player470=Bot	video Player752=Judge Player739=Human Player148=Bot	video Player952=Judge Player760=Human Player884=Bot	video Player173=Judge Player408=Human Player389=Bot	video Player640=Judge Player961=Human Player178=Bot
Round 3	video Player673=Judge Player367=Human Player833=Bot	video Player711=Judge Player431=Human Player189=Bot	video Player231=Judge Player247=Human Player703=Bot	video Player845=Judge Player525=Human Player352=Bot	video Player568=Judge Player275=Human Player794=Bot
Round 4	video Player325=Judge Player959=Human Player228=Bot	video Player165=Judge Player707=Human Player818=Bot	video Player720=Judge Player160=Human Player822=Bot	video Player475=Judge Player859=Human Player739=Bot	video Player467=Judge Player755=Human Player886=Bot
Round 5	video Player261=Judge Player357=Human Player217=Bot	video Player237=Judge Player980=Human Player157=Bot	video Player635=Judge Player955=Human Player494=Bot	video Player373=Judge Player642=Human Player189=Bot	video Player683=Judge Player178=Human Player578=Bot