关于DGM网络中Steps per sample双层训练循环的技术问询

Great question—this inner loop over steps_per_sample is a common, practical trick in physics-informed neural networks (PINNs) like the Deep Galerkin Method (DGM), and it serves three key purposes tailored to how these models are trained:

• Efficiently amortizes sampling costs
  Generating collocation points (the t1, x1, t2, x2, t3, x3 sets here) for PINNs isn't free. These points need to follow specific distributions (e.g., covering PDE domains, boundary conditions, and initial conditions for the Merton model). By reusing the same batch of sampled points for multiple gradient updates, you spread out the computational cost of sampling across several training steps. Instead of paying the sampling overhead for every single optimizer step, you pay it once per sampling_stage and use those points steps_per_sample times—this adds up to significant savings over long training runs.

• Stabilizes training against sampling randomness
  PINN loss functions combine multiple terms (like PDE residual loss L1 and boundary/initial condition loss L3 here). Sampling new points every optimizer step introduces randomness into the loss signal, which can cause noisy, unstable training dynamics. Reusing the same batch of points for several updates lets the model converge more smoothly within the constraints of that specific point set before moving to a new batch, reducing unnecessary loss fluctuations.

• Mimics fixed mini-batch training for PINNs
  In standard deep learning, we often use fixed mini-batches to let the model learn consistent patterns before switching data. For PINNs, since we don't have a static dataset (we generate points on the fly), this inner loop acts like using a fixed mini-batch. It gives the model time to adjust its weights to satisfy the PDE and boundary conditions for the current set of collocation points, rather than constantly shifting to entirely new points with every weight update.

To put it in context with your code: each sampling_stage generates a fresh set of points that cover the problem's domain and constraints. The inner loop then runs steps_per_sample rounds of gradient descent using those points, making the most of the sampled data and stabilizing the training process before moving to the next set of points.

内容的提问来源于stack exchange，提问作者Christopher J McCoy