前回、初期値のモデルから自己対局と学習を繰り返すと、入玉宣言の割合が増え過ぎる問題の対策として、最大手数で終局した対局を除くことを検証した。
詰みで終局する割合が増えることを期待したが、効果は薄かった。
今回は、3手詰めを導入し、3手以内の詰みが見つかった場合、そこで終局とすることを検証する。
学習方法
- 基本的には前回と同様
- 3手詰めで詰みが見つかった場合、勝ちとする
結果
10サイクル学習した結果は以下の通り。
(base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '63219' '--' '-m' 'gumbel_dlshogi.selfplay' 'initial_model.pt' '--batch_size' '64' '--num_simulations' '16' '--num_positions' '1000000' '--amp' '--num_processes' '16' '--skip_max_moves' 1000128pos [26:24, 631.22pos/s, Games=6764, AverageMoves=148, Nyugyoku=0, Draw=145, MaxMoves=5] All processes terminated. (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '50454' '--' '-m' 'gumbel_dlshogi.train' 'training_data' '--test_file' 'F:\hcpe3\floodgate.hcpe' '--initial_model' 'initial_model.pth' '--num_files' '1' '-e' '1' '--amp' '-b' '256' '--num_workers' '4' '--save_torchscript' 'model.pt' Using device: cuda Initial model loaded from initial_model.pth Using 1 files for training: - training_data\20250719_082526_977.data Epoch 1/1 Training: 100%|████████████████████████████████████████████████████| 3906/3906 [01:45<00:00, 37.04batch/s, Loss=4.2917, Policy=3.6865, Value=0.6053] Train - Loss: 4.4466, Policy Loss: 3.7838, Value Loss: 0.6627, Time: 105.46s Evaluating: 100%|████████████████████████████████████████████| 837/837 [00:28<00:00, 29.48batch/s, Loss=5.5105, Policy Acc=0.0394, Value Acc=0.4991] Eval - Loss: 5.2581, Policy Loss: 4.5015, Value Loss: 0.7565, Policy Acc: 0.0394, Value Acc: 0.4991, Time: 28.39s Checkpoint saved to checkpoints\checkpoint_epoch_001.pth TorchScript model saved to model.pt (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '51572' '--' '-m' 'gumbel_dlshogi.selfplay' 'model.pt' '--batch_size' '64' '--num_simulations' '16' '--num_positions' '1000000' '--amp' '--num_processes' '16' '--skip_max_moves' 1000071pos [25:58, 641.86pos/s, Games=1e+4, AverageMoves=99.5, Nyugyoku=4, Draw=86, MaxMoves=3] All processes terminated. (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '53153' '--' '-m' 'gumbel_dlshogi.train' 'training_data' '--test_file' 'F:\hcpe3\floodgate.hcpe' '--num_files' '1' '-e' '1' '--amp' '-b' '256' '--num_workers' '4' '--resume' 'checkpoints\checkpoint_epoch_001.pth' '--save_torchscript' 'model.pt' Using device: cuda Using 1 files for training: - training_data\20250719_085629_249.data Checkpoint loaded from checkpoints\checkpoint_epoch_001.pth, epoch 0, loss 4.4466 Epoch 2/2 Training: 100%|████████████████████████████████████████████████████| 3906/3906 [01:54<00:00, 34.14batch/s, Loss=4.0406, Policy=3.4559, Value=0.5847] Train - Loss: 4.1482, Policy Loss: 3.5182, Value Loss: 0.6300, Time: 114.43s Evaluating: 100%|████████████████████████████████████████████| 837/837 [00:28<00:00, 29.18batch/s, Loss=5.4996, Policy Acc=0.0629, Value Acc=0.5175] Eval - Loss: 5.1861, Policy Loss: 4.4088, Value Loss: 0.7773, Policy Acc: 0.0629, Value Acc: 0.5175, Time: 28.68s Checkpoint saved to checkpoints\checkpoint_epoch_002.pth TorchScript model saved to model.pt (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '53502' '--' '-m' 'gumbel_dlshogi.selfplay' 'model.pt' '--batch_size' '64' '--num_simulations' '16' '--num_positions' '1000000' '--amp' '--num_processes' '16' '--skip_max_moves' 1000003pos [25:45, 647.08pos/s, Games=10390, AverageMoves=96.2, Nyugyoku=2, Draw=53, MaxMoves=1] All processes terminated. (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '55084' '--' '-m' 'gumbel_dlshogi.train' 'training_data' '--test_file' 'F:\hcpe3\floodgate.hcpe' '--num_files' '1' '-e' '1' '--amp' '-b' '256' '--num_workers' '4' '--resume' 'checkpoints\checkpoint_epoch_002.pth' '--save_torchscript' 'model.pt' Using device: cuda Using 1 files for training: - training_data\20250719_092637_952.data Checkpoint loaded from checkpoints\checkpoint_epoch_002.pth, epoch 1, loss 4.1482 Epoch 3/3 Training: 100%|████████████████████████████████████████████████████| 3906/3906 [01:55<00:00, 33.90batch/s, Loss=3.9647, Policy=3.4036, Value=0.5611] Train - Loss: 3.9242, Policy Loss: 3.3056, Value Loss: 0.6186, Time: 115.24s Evaluating: 100%|████████████████████████████████████████████| 837/837 [00:28<00:00, 29.24batch/s, Loss=5.5584, Policy Acc=0.0830, Value Acc=0.5207] Eval - Loss: 5.1890, Policy Loss: 4.3779, Value Loss: 0.8110, Policy Acc: 0.0830, Value Acc: 0.5207, Time: 28.63s Checkpoint saved to checkpoints\checkpoint_epoch_003.pth TorchScript model saved to model.pt (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '55430' '--' '-m' 'gumbel_dlshogi.selfplay' 'model.pt' '--batch_size' '64' '--num_simulations' '16' '--num_positions' '1000000' '--amp' '--num_processes' '16' '--skip_max_moves' 1000047pos [25:18, 658.65pos/s, Games=13044, AverageMoves=76.7, Nyugyoku=1, Draw=36, MaxMoves=0] All processes terminated. (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '57744' '--' '-m' 'gumbel_dlshogi.train' 'training_data' '--test_file' 'F:\hcpe3\floodgate.hcpe' '--num_files' '1' '-e' '1' '--amp' '-b' '256' '--num_workers' '4' '--resume' 'checkpoints\checkpoint_epoch_003.pth' '--save_torchscript' 'model.pt' Using device: cuda Using 1 files for training: - training_data\20250719_095644_087.data Checkpoint loaded from checkpoints\checkpoint_epoch_003.pth, epoch 2, loss 3.9242 Epoch 4/4 Training: 100%|████████████████████████████████████████████████████| 3906/3906 [01:56<00:00, 33.64batch/s, Loss=3.4878, Policy=2.9025, Value=0.5852] Train - Loss: 3.5742, Policy Loss: 2.9616, Value Loss: 0.6126, Time: 116.11s Evaluating: 100%|████████████████████████████████████████████| 837/837 [00:28<00:00, 29.09batch/s, Loss=5.7290, Policy Acc=0.0937, Value Acc=0.4934] Eval - Loss: 5.3691, Policy Loss: 4.4690, Value Loss: 0.9001, Policy Acc: 0.0937, Value Acc: 0.4934, Time: 28.78s Checkpoint saved to checkpoints\checkpoint_epoch_004.pth TorchScript model saved to model.pt (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '58115' '--' '-m' 'gumbel_dlshogi.selfplay' 'model.pt' '--batch_size' '64' '--num_simulations' '16' '--num_positions' '1000000' '--amp' '--num_processes' '16' '--skip_max_moves' 1000001pos [25:12, 661.03pos/s, Games=13381, AverageMoves=74.7, Nyugyoku=3, Draw=6, MaxMoves=0] All processes terminated. (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '60224' '--' '-m' 'gumbel_dlshogi.train' 'training_data' '--test_file' 'F:\hcpe3\floodgate.hcpe' '--num_files' '1' '-e' '1' '--amp' '-b' '256' '--num_workers' '4' '--resume' 'checkpoints\checkpoint_epoch_004.pth' '--save_torchscript' 'model.pt' Using device: cuda Using 1 files for training: - training_data\20250719_102709_893.data Checkpoint loaded from checkpoints\checkpoint_epoch_004.pth, epoch 3, loss 3.5742 Epoch 5/5 Training: 100%|████████████████████████████████████████████████████| 3906/3906 [01:54<00:00, 34.09batch/s, Loss=3.2059, Policy=2.5935, Value=0.6124] Train - Loss: 3.1950, Policy Loss: 2.5833, Value Loss: 0.6118, Time: 114.57s Evaluating: 100%|████████████████████████████████████████████| 837/837 [00:28<00:00, 29.34batch/s, Loss=5.9040, Policy Acc=0.0947, Value Acc=0.5266] Eval - Loss: 5.4579, Policy Loss: 4.5783, Value Loss: 0.8796, Policy Acc: 0.0947, Value Acc: 0.5266, Time: 28.53s Checkpoint saved to checkpoints\checkpoint_epoch_005.pth TorchScript model saved to model.pt (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '60459' '--' '-m' 'gumbel_dlshogi.selfplay' 'model.pt' '--batch_size' '64' '--num_simulations' '16' '--num_positions' '1000000' '--amp' '--num_processes' '16' '--skip_max_moves' 1000046pos [25:19, 657.99pos/s, Games=14693, AverageMoves=68.1, Nyugyoku=1, Draw=3, MaxMoves=0] All processes terminated. (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '62558' '--' '-m' 'gumbel_dlshogi.train' 'training_data' '--test_file' 'F:\hcpe3\floodgate.hcpe' '--num_files' '1' '-e' '1' '--amp' '-b' '256' '--num_workers' '4' '--resume' 'checkpoints\checkpoint_epoch_005.pth' '--save_torchscript' 'model.pt' Using device: cuda Using 1 files for training: - training_data\20250719_105930_992.data Checkpoint loaded from checkpoints\checkpoint_epoch_005.pth, epoch 4, loss 3.1950 Epoch 6/6 Training: 100%|████████████████████████████████████████████████████| 3906/3906 [01:55<00:00, 33.90batch/s, Loss=2.9196, Policy=2.3399, Value=0.5797] Train - Loss: 2.8815, Policy Loss: 2.2728, Value Loss: 0.6086, Time: 115.21s Evaluating: 100%|████████████████████████████████████████████| 837/837 [00:28<00:00, 28.97batch/s, Loss=6.1045, Policy Acc=0.0991, Value Acc=0.5134] Eval - Loss: 5.7797, Policy Loss: 4.8579, Value Loss: 0.9218, Policy Acc: 0.0991, Value Acc: 0.5134, Time: 28.90s Checkpoint saved to checkpoints\checkpoint_epoch_006.pth TorchScript model saved to model.pt (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '62777' '--' '-m' 'gumbel_dlshogi.selfplay' 'model.pt' '--batch_size' '64' '--num_simulations' '16' '--num_positions' '1000000' '--amp' '--num_processes' '16' '--skip_max_moves' 1000051pos [24:52, 670.04pos/s, Games=15892, AverageMoves=62.9, Nyugyoku=0, Draw=3, MaxMoves=0] All processes terminated. (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '51958' '--' '-m' 'gumbel_dlshogi.train' 'training_data' '--test_file' 'F:\hcpe3\floodgate.hcpe' '--num_files' '1' '-e' '1' '--amp' '-b' '256' '--num_workers' '4' '--resume' 'checkpoints\checkpoint_epoch_006.pth' '--save_torchscript' 'model.pt' Using device: cuda Using 1 files for training: - training_data\20250719_112909_400.data Checkpoint loaded from checkpoints\checkpoint_epoch_006.pth, epoch 5, loss 2.8815 Epoch 7/7 Training: 100%|████████████████████████████████████████████████████| 3906/3906 [01:53<00:00, 34.54batch/s, Loss=2.3766, Policy=1.8493, Value=0.5273] Train - Loss: 2.5317, Policy Loss: 1.9367, Value Loss: 0.5951, Time: 113.09s Evaluating: 100%|████████████████████████████████████████████| 837/837 [00:28<00:00, 29.17batch/s, Loss=6.3695, Policy Acc=0.1045, Value Acc=0.5140] Eval - Loss: 6.0530, Policy Loss: 5.0455, Value Loss: 1.0075, Policy Acc: 0.1045, Value Acc: 0.5140, Time: 28.70s Checkpoint saved to checkpoints\checkpoint_epoch_007.pth TorchScript model saved to model.pt (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '52470' '--' '-m' 'gumbel_dlshogi.selfplay' 'model.pt' '--batch_size' '64' '--num_simulations' '16' '--num_positions' '1000000' '--amp' '--num_processes' '16' '--skip_max_moves' 1000051pos [24:23, 683.30pos/s, Games=15560, AverageMoves=64.3, Nyugyoku=3, Draw=10, MaxMoves=0] All processes terminated. (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '54062' '--' '-m' 'gumbel_dlshogi.train' 'training_data' '--test_file' 'F:\hcpe3\floodgate.hcpe' '--num_files' '1' '-e' '1' '--amp' '-b' '256' '--num_workers' '4' '--resume' 'checkpoints\checkpoint_epoch_007.pth' '--save_torchscript' 'model.pt' Using device: cuda Using 1 files for training: - training_data\20250719_115924_530.data Checkpoint loaded from checkpoints\checkpoint_epoch_007.pth, epoch 6, loss 2.5317 Epoch 8/8 Training: 100%|████████████████████████████████████████████████████| 3906/3906 [01:52<00:00, 34.84batch/s, Loss=2.2126, Policy=1.6910, Value=0.5216] Train - Loss: 2.3227, Policy Loss: 1.7335, Value Loss: 0.5892, Time: 112.11s Evaluating: 100%|████████████████████████████████████████████| 837/837 [00:28<00:00, 29.33batch/s, Loss=6.3465, Policy Acc=0.1228, Value Acc=0.5297] Eval - Loss: 6.0754, Policy Loss: 5.0802, Value Loss: 0.9952, Policy Acc: 0.1228, Value Acc: 0.5297, Time: 28.54s Checkpoint saved to checkpoints\checkpoint_epoch_008.pth TorchScript model saved to model.pt (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '54280' '--' '-m' 'gumbel_dlshogi.selfplay' 'model.pt' '--batch_size' '64' '--num_simulations' '16' '--num_positions' '1000000' '--amp' '--num_processes' '16' '--skip_max_moves' 1000054pos [24:24, 682.99pos/s, Games=15941, AverageMoves=62.7, Nyugyoku=2, Draw=21, MaxMoves=0] All processes terminated. (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '56894' '--' '-m' 'gumbel_dlshogi.train' 'training_data' '--test_file' 'F:\hcpe3\floodgate.hcpe' '--num_files' '1' '-e' '1' '--amp' '-b' '256' '--num_workers' '4' '--resume' 'checkpoints\checkpoint_epoch_008.pth' '--save_torchscript' 'model.pt' Using device: cuda Using 1 files for training: - training_data\20250719_122903_720.data Checkpoint loaded from checkpoints\checkpoint_epoch_008.pth, epoch 7, loss 2.3227 Epoch 9/9 Training: 100%|████████████████████████████████████████████████████| 3906/3906 [01:52<00:00, 34.70batch/s, Loss=2.1163, Policy=1.5655, Value=0.5508] Train - Loss: 2.1283, Policy Loss: 1.5334, Value Loss: 0.5948, Time: 112.58s Evaluating: 100%|████████████████████████████████████████████| 837/837 [00:28<00:00, 29.29batch/s, Loss=6.5337, Policy Acc=0.1429, Value Acc=0.5312] Eval - Loss: 6.0839, Policy Loss: 5.1235, Value Loss: 0.9604, Policy Acc: 0.1429, Value Acc: 0.5312, Time: 28.58s Checkpoint saved to checkpoints\checkpoint_epoch_009.pth TorchScript model saved to model.pt (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '57102' '--' '-m' 'gumbel_dlshogi.selfplay' 'model.pt' '--batch_size' '64' '--num_simulations' '16' '--num_positions' '1000000' '--amp' '--num_processes' '16' '--skip_max_moves' 1000005pos [24:33, 678.77pos/s, Games=14293, AverageMoves=70, Nyugyoku=2, Draw=28, MaxMoves=0] All processes terminated. (base) PS D:\src\gumbel-dlshogi> d:; cd 'd:\src\gumbel-dlshogi'; & 'anaconda3\python.exe' '.vscode\extensions\ms-python.debugpy-2025.10.0-win32-x64\bundled\libs\debugpy\launcher' '58730' '--' '-m' 'gumbel_dlshogi.train' 'training_data' '--test_file' 'F:\hcpe3\floodgate.hcpe' '--num_files' '1' '-e' '1' '--amp' '-b' '256' '--num_workers' '4' '--resume' 'checkpoints\checkpoint_epoch_009.pth' '--save_torchscript' 'model.pt' Using device: cuda Using 1 files for training: - training_data\20250719_130459_316.data Checkpoint loaded from checkpoints\checkpoint_epoch_009.pth, epoch 8, loss 2.1283 Epoch 10/10 Training: 100%|████████████████████████████████████████████████████| 3906/3906 [01:52<00:00, 34.61batch/s, Loss=2.0998, Policy=1.4934, Value=0.6064] Train - Loss: 2.0163, Policy Loss: 1.4069, Value Loss: 0.6095, Time: 112.86s Evaluating: 100%|████████████████████████████████████████████| 837/837 [00:28<00:00, 29.44batch/s, Loss=6.4689, Policy Acc=0.1526, Value Acc=0.5309] Eval - Loss: 6.0699, Policy Loss: 5.1528, Value Loss: 0.9171, Policy Acc: 0.1526, Value Acc: 0.5309, Time: 28.43s Checkpoint saved to checkpoints\checkpoint_epoch_010.pth TorchScript model saved to model.pt
棋譜
10サイクル学習時点の棋譜は以下の通り。
シミュレーション回数16で探索深さが1とは言え、初手から角頭の歩を突くなどまだまだ弱そうである。
startpos moves 8g8f 2c2d 3g3f 8b6b 8f8e 6c6d 2h4h 3c3d 5i5h 2a3c 7g7f 5a5b 8h6f 6d6e 8i7g 6e6f 6g6f 6b6f 6i6h B*4d P*6g 6f7f 8e8d 8c8d 7i8h 2d2e 9g9f 7a6b 4i5i 6b6c 5g5f 7f5f 5h6i 3c4e 4g4f 5f5i+ 6i5i G*5g 6h5g 4e5g+ 4f4e P*6h 4e4d 2b4d B*3c 4d3c 7g6e 3c4d 4h4d
