[ArchawinL]

In v2.0.2 and on master, pytorch.dqn.DQN.load and pytorch.nfsp.NFSP.save/restore fail on every call.

Reproduction (v2.0.2):

import pathlib, tempfile
from open_spiel.python.pytorch import dqn, nfsp

tmp = pathlib.Path(tempfile.mkdtemp())

agent = dqn.DQN(0, state_representation_size=11, num_actions=2)
agent.save(tmp / "dqn.pt")
agent.load(tmp / "dqn.pt")
# KeyError: 'model_state_dict'

agent = nfsp.NFSP(0, state_representation_size=11, num_actions=2,
                  hidden_layers_sizes=[16], reservoir_buffer_capacity=10,
                  anticipatory_param=0.1)
agent.save(tmp / "nfsp")
# RuntimeError: Parent directory .../nfsp does not exist.
# If the directory exists: RuntimeError: File .../nfsp cannot be opened.
# With a str path: TypeError: unsupported operand type(s) for /: 'str' and 'str'

Cause (introduced in #1419):

1. save writes the keys model/optimiser, but load/restore read model_state_dict/optimizer_state_dict.
2. NFSP.save uses checkpoint_dir both as a directory (for the inner DQN) and as a file (for torch.save).

Fix:

• DQN.load reads the keys save writes. save now also stores the target network, and load restores it, so resuming between target updates no longer resets it. Files written by v2.0.2 still load; they never stored the target, so it starts from the online network.
• NFSP treats checkpoint_dir as a directory: it creates the directory, writes q_network.pt (inner DQN) and avg_network.pt (average network), and accepts str paths as the JAX version does.
• nfsp_leduc_pytorch.py now checkpoints each player to its own subdirectory. Before, both players shared one path.

I chose this NFSP layout; happy to change it.

Tests: new save→load/restore round-trip tests in dqn_pytorch_test.py and nfsp_pytorch_test.py, including the target networks. Failed before this change and now passing.

Not included: saving the replay and reservoir buffers.

[sylvesterkaczmarek]

The DQN checkpoint still does not preserve the target network. save() writes only _q_network, and load() copies that state into both networks, so resuming between target updates silently resets the lagged target to the online network and changes subsequent learning. Could the target state be saved/restored separately, with the regression comparing the restored target to the original _target_q_network?

[ArchawinL]

Fair point. Tests are updated, and save now also stores the target network.

One related gap I've left out of this PR: pytorch.boltzmann_dqn.BoltzmannDQN inherits DQN.save/load, which don't include its _prev_q_network. After a load, it keeps the fresh agent's initial weights, so a restored agent doesn't evaluate like the saved one.

It's not visible right now because _boltzmann_action_probs passes the legal action indices to torch.BoolTensor as if they were a mask (on Kuhn, [0, 1] becomes [False, True]), so the previous network never affects the evaluation policy. Fixing the checkpoint only makes sense together with that, and it needs a way for the subclass to add to the checkpoint. Happy to open a PR after this one if that sounds right.

[sylvesterkaczmarek]

Rechecked current b8aa626. DQN checkpoints now store the target network separately, load restores it when present, and older checkpoints fall back to the online model for compatibility. The regression directly compares the restored target state to the original lagged target network. This resolves the learning-state reset I raised.