自己対局と訓練の処理が実装できたので、初期値のモデルから自己対局と学習を繰り返して、モデル精度が上がるか検証する。
学習方法
- 初期値のモデルから自己対局を開始する
- 1回の自己対局で、100万局面生成する
- 自己対局で生成した訓練データを1エポックだけ学習する
- 学習したモデルで再び自己対局→学習を繰り返す
- 学習には、過去分は使用しないで直近の自己対局の訓練データのみを使用する
- シミュレーション数は16とする
実験結果
自己対局→学習を7サイクル繰り返した結果は以下の通り。
(base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '53731' '--' '-m' 'gumbel_dlshogi.selfplay' 'initial_model.pt' '--batch_size' '64' '--num_simulations' '16' '--num_positions' '1000000' '--amp' '--num_processes' '16' 1000164pos [28:43, 580.26pos/s, Games=2779, AverageMoves=360, Nyugyoku=15, Draw=116, MaxMoves=898] All processes terminated. (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '58655' '--' '-m' 'gumbel_dlshogi.train' 'training_data' '--test_file' 'F:\hcpe3\floodgate.hcpe' '--initial_model' 'initial_model.pth' '--num_files' '1' '-e' '1' '--amp' '-b' '256' '--num_workers' '4' '--save_torchscript' 'model.pt' Using device: cuda Initial model loaded from initial_model.pth Epoch 1/1 Training: 100%|████████████████████████████████████████████████████| 3906/3906 [01:40<00:00, 38.93batch/s, Loss=4.4705, Policy=3.8479, Value=0.6226] Train - Loss: 4.6249, Policy Loss: 3.9572, Value Loss: 0.6677, Time: 100.34s Evaluating: 100%|████████████████████████████████████████████| 837/837 [00:28<00:00, 29.81batch/s, Loss=5.3158, Policy Acc=0.0395, Value Acc=0.5013] Eval - Loss: 5.1230, Policy Loss: 4.3936, Value Loss: 0.7295, Policy Acc: 0.0395, Value Acc: 0.5013, Time: 28.08s Checkpoint saved to checkpoints\checkpoint_epoch_1.pth TorchScript model saved to model.pt (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '59213' '--' '-m' 'gumbel_dlshogi.selfplay' 'model.pt' '--batch_size' '64' '--num_simulations' '16' '--num_positions' '1000000' '--amp' '--num_processes' '16' 1000063pos [29:43, 560.64pos/s, Games=2599, AverageMoves=385, Nyugyoku=274, Draw=144, MaxMoves=1172] All processes terminated. (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '64525' '--' '-m' 'gumbel_dlshogi.train' 'training_data' '--test_file' 'F:\hcpe3\floodgate.hcpe' '--num_files' '1' '-e' '1' '--amp' '-b' '256' '--num_workers' '4' '--resume' 'checkpoints\checkpoint_epoch_001.pth' '--save_torchscript' 'model.pt' Using device: cuda Checkpoint loaded from checkpoints\checkpoint_epoch_001.pth, epoch 0, loss 4.6249 Epoch 2/2 Training: 100%|████████████████████████████████████████████████████| 3906/3906 [01:53<00:00, 34.54batch/s, Loss=4.5151, Policy=3.8803, Value=0.6348] Train - Loss: 4.5546, Policy Loss: 3.8911, Value Loss: 0.6635, Time: 113.08s Evaluating: 100%|████████████████████████████████████████████| 837/837 [00:28<00:00, 29.36batch/s, Loss=5.2803, Policy Acc=0.0682, Value Acc=0.5022] Eval - Loss: 5.0259, Policy Loss: 4.2929, Value Loss: 0.7330, Policy Acc: 0.0682, Value Acc: 0.5022, Time: 28.51s Checkpoint saved to checkpoints\checkpoint_epoch_002.pth TorchScript model saved to model.pt (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '64944' '--' '-m' 'gumbel_dlshogi.selfplay' 'model.pt' '--batch_size' '64' '--num_simulations' '16' '--num_positions' '1000000' '--amp' '--num_processes' '16' 1000268pos [28:15, 590.00pos/s, Games=2715, AverageMoves=368, Nyugyoku=476, Draw=122, MaxMoves=888] All processes terminated. (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '54038' '--' '-m' 'gumbel_dlshogi.train' 'training_data' '--test_file' 'F:\hcpe3\floodgate.hcpe' '--num_files' '1' '-e' '1' '--amp' '-b' '256' '--num_workers' '4' '--resume' 'checkpoints\checkpoint_epoch_002.pth' '--save_torchscript' 'model.pt' Using device: cuda Checkpoint loaded from checkpoints\checkpoint_epoch_002.pth, epoch 1, loss 4.5546 Epoch 3/3 Training: 100%|████████████████████████████████████████████████████| 3907/3907 [01:52<00:00, 34.64batch/s, Loss=4.2463, Policy=3.6966, Value=0.5497] Train - Loss: 4.3279, Policy Loss: 3.7261, Value Loss: 0.6018, Time: 112.80s Evaluating: 100%|████████████████████████████████████████████| 837/837 [00:28<00:00, 29.43batch/s, Loss=5.2504, Policy Acc=0.0718, Value Acc=0.5177] Eval - Loss: 5.0242, Policy Loss: 4.2581, Value Loss: 0.7661, Policy Acc: 0.0718, Value Acc: 0.5177, Time: 28.44s Checkpoint saved to checkpoints\checkpoint_epoch_003.pth TorchScript model saved to model.pt (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '54255' '--' '-m' 'gumbel_dlshogi.selfplay' 'model.pt' '--batch_size' '64' '--num_simulations' '16' '--num_positions' '1000000' '--amp' '--num_processes' '16' 1000259pos [28:42, 580.61pos/s, Games=3031, AverageMoves=330, Nyugyoku=1343, Draw=84, MaxMoves=447] All processes terminated. (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '64605' '--' '-m' 'gumbel_dlshogi.train' 'training_data' '--test_file' 'F:\hcpe3\floodgate.hcpe' '--num_files' '1' '-e' '1' '--amp' '-b' '256' '--num_workers' '4' '--resume' 'checkpoints\checkpoint_epoch_003.pth' '--save_torchscript' 'model.pt' Using device: cuda Checkpoint loaded from checkpoints\checkpoint_epoch_003.pth, epoch 2, loss 4.3279 Epoch 4/4 Training: 100%|████████████████████████████████████████████████████| 3907/3907 [01:52<00:00, 34.69batch/s, Loss=3.9954, Policy=3.5646, Value=0.4308] Train - Loss: 4.1161, Policy Loss: 3.5936, Value Loss: 0.5224, Time: 112.62s Evaluating: 100%|████████████████████████████████████████████| 837/837 [00:28<00:00, 29.56batch/s, Loss=5.2106, Policy Acc=0.0778, Value Acc=0.5001] Eval - Loss: 5.0862, Policy Loss: 4.2288, Value Loss: 0.8573, Policy Acc: 0.0778, Value Acc: 0.5001, Time: 28.32s Checkpoint saved to checkpoints\checkpoint_epoch_004.pth TorchScript model saved to model.pt (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '65377' '--' '-m' 'gumbel_dlshogi.selfplay' 'model.pt' '--batch_size' '64' '--num_simulations' '16' '--num_positions' '1000000' '--amp' '--num_processes' '16' 1000419pos [28:14, 590.47pos/s, Games=3149, AverageMoves=318, Nyugyoku=1659, Draw=109, MaxMoves=349] All processes terminated. (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '50956' '--' '-m' 'gumbel_dlshogi.train' 'training_data' '--test_file' 'F:\hcpe3\floodgate.hcpe' '--num_files' '1' '-e' '1' '--amp' '-b' '256' '--num_workers' '4' '--resume' 'checkpoints\checkpoint_epoch_004.pth' '--save_torchscript' 'model.pt' Using device: cuda Checkpoint loaded from checkpoints\checkpoint_epoch_004.pth, epoch 3, loss 4.1161 Epoch 5/5 Training: 100%|████████████████████████████████████████████████████| 3907/3907 [01:53<00:00, 34.57batch/s, Loss=3.7367, Policy=3.3257, Value=0.4110] Train - Loss: 3.8112, Policy Loss: 3.3286, Value Loss: 0.4827, Time: 113.03s Evaluating: 100%|████████████████████████████████████████████| 837/837 [00:28<00:00, 29.19batch/s, Loss=5.4564, Policy Acc=0.0806, Value Acc=0.4971] Eval - Loss: 5.2447, Policy Loss: 4.3031, Value Loss: 0.9415, Policy Acc: 0.0806, Value Acc: 0.4971, Time: 28.68s Checkpoint saved to checkpoints\checkpoint_epoch_005.pth TorchScript model saved to model.pt (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '51741' '--' '-m' 'gumbel_dlshogi.selfplay' 'model.pt' '--batch_size' '64' '--num_simulations' '16' '--num_positions' '1000000' '--amp' '--num_processes' '16' 1000301pos [28:15, 589.94pos/s, Games=3276, AverageMoves=305, Nyugyoku=1996, Draw=137, MaxMoves=225] All processes terminated. (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '53867' '--' '-m' 'gumbel_dlshogi.train' 'training_data' '--test_file' 'F:\hcpe3\floodgate.hcpe' '--num_files' '1' '-e' '1' '--amp' '-b' '256' '--num_workers' '4' '--resume' 'checkpoints\checkpoint_epoch_005.pth' '--save_torchscript' 'model.pt' Using device: cuda Using 1 files for training: - training_data\20250718_203648_224.data Checkpoint loaded from checkpoints\checkpoint_epoch_005.pth, epoch 4, loss 3.8112 Epoch 6/6 Training: 100%|████████████████████████████████████████████████████| 3907/3907 [01:52<00:00, 34.69batch/s, Loss=3.4616, Policy=3.0972, Value=0.3644] Train - Loss: 3.6076, Policy Loss: 3.1595, Value Loss: 0.4481, Time: 112.65s Evaluating: 100%|████████████████████████████████████████████| 837/837 [00:28<00:00, 29.28batch/s, Loss=5.4391, Policy Acc=0.0973, Value Acc=0.4966] Eval - Loss: 5.2198, Policy Loss: 4.3597, Value Loss: 0.8601, Policy Acc: 0.0973, Value Acc: 0.4966, Time: 28.59s Checkpoint saved to checkpoints\checkpoint_epoch_006.pth TorchScript model saved to model.pt (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '54088' '--' '-m' 'gumbel_dlshogi.selfplay' 'model.pt' '--batch_size' '64' '--num_simulations' '16' '--num_positions' '1000000' '--amp' '--num_processes' '16' 1000087pos [27:36, 603.80pos/s, Games=3729, AverageMoves=268, Nyugyoku=2466, Draw=179, MaxMoves=62] All processes terminated. (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '56630' '--' '-m' 'gumbel_dlshogi.train' 'training_data' '--test_file' 'F:\hcpe3\floodgate.hcpe' '--num_files' '1' '-e' '1' '--amp' '-b' '256' '--num_workers' '4' '--resume' 'checkpoints\checkpoint_epoch_006.pth' '--save_torchscript' 'model.pt' Using device: cuda Using 1 files for training: - training_data\20250718_211455_471.data Checkpoint loaded from checkpoints\checkpoint_epoch_006.pth, epoch 5, loss 3.6076 Epoch 7/7 Training: 100%|████████████████████████████████████████████████████| 3906/3906 [01:52<00:00, 34.57batch/s, Loss=3.2260, Policy=2.9259, Value=0.3001] Train - Loss: 3.2836, Policy Loss: 2.8842, Value Loss: 0.3994, Time: 112.99s Evaluating: 100%|████████████████████████████████████████████| 837/837 [00:28<00:00, 29.25batch/s, Loss=5.5593, Policy Acc=0.1144, Value Acc=0.5011] Eval - Loss: 5.3612, Policy Loss: 4.4565, Value Loss: 0.9047, Policy Acc: 0.1144, Value Acc: 0.5011, Time: 28.62s Checkpoint saved to checkpoints\checkpoint_epoch_007.pth TorchScript model saved to model.pt
精度
評価データにfloodgateの棋譜を使用した際の、方策と価値の正解率は以下の通り。
方策は、順調に精度が上がり、7エポックで11.4%になっている。
価値は、3エポック目がピークでそこから下がっている。
