Leela Chess Zero
Leela Chess Zero [1,2] is a project that learns to play chess from scratch using a combination of neural networks and reinforcement learning. Except for the fundamental rules of chess, no human knowledge or data is used. Everything "she" learns is learned through self play. Leela Chess Zero is a forked , open source implementation of the AlphaZero  project by Google's DeepMind  for chess and designed to run on consumer grade hardware.
This is a distributed effort, meaning many users around the world donate their CPU/GPU computations in the form of self-play games. These games are then collected to train the next generation neural network, iteratively improving the chess engine. The goal is to become the most powerful chess engine in the world with a radical new interesting play style.
Training these neural networks requires powerful, dedicated hardware to run 24/7. Therefore we initiated this gofundme. The money collected here will go straight into vital hardware upgrades like the graphical processing unit, memory and fast random access harddrives.
P.S. You can already play her here: http://play.lczero.org/ She will automatically upgrade her "brain" as new networks are trained.
Gofundme kindly requests a withdrawal plan from us to access the funds. I personally live in the Netherlands, the servers and server room are hosted by a volunteer in Finland. This is how we plan on doing it:
1. We're constructing a list of hardware to buy (in progress)
2. The server admin from Finland will order the hardware from the list, send a copy of the receipts to me and pictures for us to post in updates here.
3. I will extract the required money consistent with the receipts and transfer the amount to his bank account.
Hereby another update, sorry to keep you waiting. We had a string of disasters in our pipeline (see github and discord for details). Also I fell quite ill which disabled me working on stuff for about a week in total (now back at full speed).
The hardware is almost complete. We are currently in possession of:
- AMD Ryzen Threadripper 1950X
- Cooling for the Threadripper
- 128G RAM
- 2x 400G SSD for data (donated)
- 250G M.2 for os
- Thermaltake Core P3 casing
Still underway is the new 1080Ti. The one we have already is paid through this funding for 50%. Once finalized we'll have 2x 1080Ti for training and 2x 1060 for play.lczero.org (or for generating games).
We're now in the process of setting up the system and the new training pipeline.
Hereby another update on where we stand with the training hardware. The hardware donation has been shipped and is now at customs in Finland (see attached photo). It requires a 24% VAT which we'll be paying using the gofundme, approximately 400 USD.
AWS and NVIDIA have both been so kind to let us try a V100 and Titan V respectively for a few weeks. I'm currently preparing the training pipeline to support 16 bit float as this will give huge performance improvements with these cards. After testing we will order the remaining hardware and I will update with a few pictures of the installation.
We're also doing a lot of performance improvements on the client side which will ensure a huge boost in games per day. This will nicely coincide with the trainingpipeline upgrades and will let Lc0 progress faster and faster! Awesome!
Thanks everyone for their support!
P.S. Unfortunately gofundme requires me to have a facebook account in order to reply to the comments below. I refuse to do so, but you can email me or find me on our discord server at https://discord.gg/pKujYxD.
The speed at which this goal was reached is amazing! Thank you all! We are currently heavily debating the exact hardware in question and I'd like to update you on the current situation. From here this will become a very technical update. You have been warned :-)
For our training purposes there are a few important factors to consider:
- We'll create a ramdisk for the active training window, this gives by far the most efficient throughput. We shuffle chunks many many times during a training run.
- We need 2x SSD in RAID-1 configuration for redundancy, nothing more annoying than losing your OS install when a disk fails.
- We decided on consumer grade hardware for superior price to performance.
- This will run 24/7, heating issues should be dealt with using consumer grade hardware.
Interestingly, as this campaign took off, one person physically donated 128G RAM and 2x 400G SSD (hardware is underway). This shifted the budget and now more money can go into the GPU(s). This does mean we are back to the drawing board somewhat. One of the options is buying a Titan V (designed for these kind of training purposes). But before we do, we want to test its performance first. Let me know if you have access to one and are willing to help out testing our training pipeline.
As we come to concrete decisions I will update here and post pictures of the actual hardware installed in the serverroom somewhere in Finland (host likes to remain anonymous), where our current training hardware resides.
Many thanks again and kind regards,
Folkert Huizinga (Error323)