Project Plan for project ‘DeepPoker: Training a Neural Network for Strategic Decision Making in No-Limit Texas Hold'em Poker’
Declaration of academic integrity
By submitting this work, I confirm that I have read and understood the University’s Academic Integrity Policy.
By submitting this work, I confirm that I have acted honestly, ethically and professionally in conduct leading to assessment for the programme of study.
By submitting this work, I confirm that the work I am submitting is my own. I have not commissioned production of the work from a third party or used artificial intelligence software in an unacceptable manner to generate the work*. I have not copied material from another person or source, nor committed plagiarism, nor fabricated, falsified or embellished data when completing the attached piece of work. I have not colluded with any other student in the preparation and/ or production of this work.
*Software applications include but are not limited to, ChatGPT, Bing Chat, DALL.E, Bard
Tables of contents, figures, etc.
|
Declaration of academic integrity
|
1
|
|
Tables of Contents, Figures, etc
|
2
|
|
Project Scope
|
3
|
|
Project Objectives
|
4
|
|
Project Requirement
|
5
|
|
Work Packages
|
6
|
|
Skills and Resource Audit
|
8
|
|
Appendix 1: Gantt Chart
|
9
|
|
Appendix 2: Risk Assessment Form
|
10
|
|
Appendix 3: FYP Ethics Self-Assessment
|
10
|
|
Appendix 4: References
|
11
|
Project scope
Overall aim
To design, train, and evaluate a deep neural-network system capable of making optimal strategic decisions—including betting, raising, folding, and bluffing—in No-Limit Texas Hold’em Poker. The model will be developed using deep reinforcement-learning techniques and assessed through 10,000 simulated games, with the goal of achieving at least a 60% win rate against existing baseline poker AI agents by April 2026, thereby demonstrating the effectiveness of deep learning for complex, real-time strategic decision-making tasks.
Current State of the Art
In the field of No-Limit Texas Hold’em (NLTH) artificial intelligence, two milestone systems dominate the literature: DeepStack and Libratus.
DeepStack introduced a deep reinforcement learning approach that uses neural networks to estimate the expected value of decision states, allowing the agent to approximate equilibrium strategies in imperfect-information games [1]. It significantly reduced the need for full game-tree traversal by learning value functions, but still required large computational resources and expert-designed state abstractions.
Libratus, on the other hand, achieved superhuman performance against top professional players in heads-up NLTH Poker through a game-theoretic search architecture rather than supervised learning [2]. Its strategy was built using recursive counterfactual regret minimization (CFR) and enhanced by online subgame solving and self-improvement modules that dynamically refined strategies during play. The system was trained on a supercomputer for over 15 million core-hours and won decisively against four professional players in a 120,000-hand match.
Most open-source frameworks today (e.g., PokerRL) reuse simplified versions of these systems, focusing on experimentation and simulation rather than full-scale equilibrium solving. However, these implementations are either computationally demanding or not fully accessible.
In contrast, DeepPoker aims to combine DeepStack’s neural-based valuation with a lightweight, directly trainable policy model. By using a hybrid reinforcement learning approach—integrating policy gradients with value estimation—the system can achieve competitive performance on standard university hardware while avoiding the heavy resource costs associated with DeepStack and Libratus.
Benefits
Gaming Industry: Offers a low-resource AI framework for poker platforms (e.g., amateur player training tools, dynamic in-game opponents).
AI Practitioners: Serves as a template for training decision systems in dynamic, opponent-dependent environments (e.g., finance, robotics).
Project objectives
|
Specific
|
Develop a fully functional, lightweight neural network (NN) system for No-Limit Texas Hold'em (NLTH) Poker that can make strategic decisions (fold, bet, raise) based on game state (player cards, community cards, pot size, opponent betting history). The system will include three key components: (1) an NLTH-tailored NN architecture, (2) model training on a labeled dataset, and (3) performance validation via simulations—plus a final technical report documenting the entire process.
|
|
Measurable
|
Success is verified by 3 quantifiable criteria:1. NN inference speed ≥100 game state samples/second (on Intel i7 CPU + 16GB RAM); 2. Training accuracy ≥85% (alignment with optimal actions from CFR benchmarks);3. Win rate ≥60% in 100-round PokerRL simulations against 2 baselines (rule-based agent, open-source NLTH AI);
|
|
Achievable
|
Feasible with available resources:1. Software: Open-source tools (PyTorch for NN, Pandas for data prep, PokerRL for simulations) – no cost, compatible with personal/university devices; 2. Data: Public PokerRL dataset (30k initial samples, expandable to 50k+ via augmentation) – free access via GitHub;4. Skills: Learning needs (PyTorch basics, PokerRL setup) can be covered in 2–3 weeks via free online tutorials (Udacity, GitHub docs).
|
|
Relevant
|
This objective directly ties to the DeepPoker project’s core purpose—building a competitive NLTH AI—and focuses only on critical, non-negotiable components (architecture, training, simulation, report). It excludes "nice-to-have" features (e.g., multi-player support, real-world poker platform. integration) because those do not impact the project’s main goal.
|
|
Time- bound
|
NN architecture design: Weeks 1–2;
2. Dataset preprocessing & model training: Weeks 2–4;
3. Simulation & performance testing: Weeks 4–8;
4. Final report writing & revision: Weeks 8–10
Buffer time (2 weeks) is included to address unexpected issues (e.g., GPU booking delays, simulation bugs)..
|
Project Requirements
|
Category
|
|
Identifier
|
Description
|
|
DAT -01
|
NLTH dataset must include 5 features: player cards (2/52), community cards (0-5/52), pot size ($0-$10,000), opponent betting history (last 3 actions), optimal action label (fold/bet/raise).
|
|
DAT -02
|
Dataset split: 80% training, 20% validation (stratified to preserve action label distribution).
|
|
MOD -01
|
Architecture: Input layer (size=TBD), 2 hidden layers (64-128 neurons each, ReLU activation), output layer (3 neurons, softmax activation for action probabilities).
|
|
SIM -01
|
Environment: PokerRL (heads-up NLTH; blinds=$5/$10; max bet=pot size).
|
|
SIM-02
|
Win rate ≥60% over 100 rounds vs two baselines (rule-based and PokerRL agent).
|
|
ETH-01
|
Only open-source datasets are used. No human subjects or real-money poker data involved.
|
Work packages
|
WORK PACKAGE DESCRIPTION
|
|
WP Title:
|
Literature Review & NN Architecture Design
|
WP number
|
WP-01
|
|
Start Event:
|
Project plan approval
|
|
Duration:
|
4 weeks
|
|
Objectives:
1. Review 15+ papers on poker AI/Deep Learning.
2. 2. Design NLTH-tailored NN architecture.
3. 3. Draft architecture diagram (Figure 1).
|
|
Task Description:
Task 1.1: Search IEEE Xplore/Google Scholar for papers.
Task 1.2: Define input features (game state) and output (actions).
Task 1.3: Code architecture draft in PyTorch.
|
|
Inputs:
University library access (papers).
PyTorch documentation.
|
|
Interfaces/links with other tasks or WPs:
None
|
|
Deliverables:
NN architecture code (PyTorch).
|
|
WORK PACKAGE DESCRIPTION
|
|
WP Title:
|
Model Training & Optimization
|
WP number
|
WP-02
|
|
Start Event:
|
Completion of WP-01
|
|
Duration:
|
4 weeks
|
|
Objectives:
1. Train the NN using a hybrid loss function (policy gradient + value estimation) on the university GPU.
2. 2. Validate training accuracy (≥85% vs. optimal actions).
|
|
Task Description:
Task 2.1: Split dataset (80% train, 20% validation)
Task 2.2: Prune redundant parameters to reduce model size.
Task 2.3: Test inference speed and validate training accuracy.
|
|
Inputs:
Cleaned/augmented dataset (WP-01).
|
|
Interfaces/links with other tasks or WPs:
Depends on: WP-01
|
|
Deliverables:
Trained, optimized model (PyTorch checkpoint).
|
|
WORK PACKAGE DESCRIPTION
|
|
WP Title:
|
Simulation, Reporting & Finalization
|
WP number
|
WP-03
|
|
Start Event:
|
Completion of WP-02
|
|
Duration:
|
4 weeks
|
|
Objectives:
1. Evaluate the model via 100-round PokerRL simulations against 2 baselines.
2. Finalize and submit all deliverables.
|
|
Task Description:
Task 3.1: Set up PokerRL and run simulations against 2 baselines.
Task 3.2: Calculate win rate (≥60% target) and document results.
Task 3.3: Write the final report (include architecture diagram, results tables).•
Task 3.4: Review and submit all deliverables.
|
|
Inputs:
Optimized model (WP-02).
PokerRL simulation environment.
Report template (Canvas).
|
|
Interfaces/links with other tasks or WPs:
Depends on: WP-02
|
|
Deliverables:
Simulation results.
Final project report.
|
Skills and Resource Audit
Skills Audit:
|
Skill
|
Level
|
Training needs
|
|
Deep Learning with PyTorch
|
Beginner
|
Complete “PyTorch for Deep Learning” online course
|
|
Data Preprocessing (Pandas)
|
Moderate
|
Review Pandas documentation for missing value imputation/augmentation
|
|
PokerRL Simulation Environment
|
Beginner
|
Follow PokerRL GitHub tutorial + test sample simulations
|
|
Model Evaluation and Debugging
|
Beginner
|
Learn to use tensorboard and sklearn metrics for tracking accuracy/loss.
|
|
Research Report Writing (Academic)
|
Intermediate
|
Attend Academic Writing workshop (English Language Centre / Library Hub).
|
Resource Audit:
|
Resource
|
Availability
|
Alternatives
|
|
NLTH Dataset (PokerRL)
|
Publicly available on GitHub (free access).
|
Combine with smaller datasets
|
|
PyTorch/PokerRL Software
|
Free open-source; compatible with Windows/macOS
|
|
|
IEEE Reference Guide
|
University library provides free access to IEEE Xplore documentation.
|
Use “IEEE Citation Reference” PDF
|