DAE-PINN: a physics-informed neural network model for simulating differential algebraic equations with application to power networks

Abstract

Deep learning-based surrogate modeling is becoming a promising approach for learning and simulating dynamical systems. However, deep-learning methods find it very challenging to learn stiff dynamics. In this paper, we develop DAE-PINN, the first effective physics-informed deep-learning framework for learning and simulating the solution trajectories of nonlinear differential-algebraic equations (DAE). DAEs are used to model complex engineering systems, e.g., power networks, and present a “form” of infinite stiffness, which makes learning their solution trajectories challenging. Our DAE-PINN bases its effectiveness on the synergy between implicit Runge–Kutta time-stepping schemes (designed specifically for solving DAEs) and physics-informed neural networks (PINN) (deep neural networks that we train to satisfy the dynamics of the underlying problem). Furthermore, our framework (i) enforces the neural network to satisfy the DAEs as (approximate) hard constraints using a penalty-based method and (ii) enables simulating DAEs for long-time horizons. We showcase the effectiveness and accuracy of DAE-PINN by learning the solution trajectories of a three-bus power network.

Appendix A: The Van der Pol system

In this appendix, we consider the classical Van der Pol (VDP) system [16, 49]:

$$\begin{aligned} \ddot{x} + \mu (x^2 - 1) + x = 0, \end{aligned}$$

where $(x, \dot{x})$ is the state vector and $\mu$ is the system’s parameter. To facilitate our analysis, let us transform the VDP system as follows. Consider the following identity:

$$\begin{aligned} \ddot{x} + \mu \dot{x}(x^2 - 1) = \frac{d}{dt}\left( \dot{x} + \mu \left[ \frac{1}{3}x^3 - x \right] \right) . \end{aligned}$$

If we let $\varphi (x) := \frac{1}{3}x^3 - x$ and $z = \dot{x} + \mu \varphi (x)$, the above identity and the VDP system imply

$$\begin{aligned} \dot{z} = \ddot{x} + \mu \dot{x}(x^2 - 1) = -x. \end{aligned}$$

If we also let $y:= \frac{z}{\mu }$, then we can write the VDP system as follows:

$$\begin{aligned} \dot{x}&= \mu [y - \varphi (x)] \end{aligned}$$

(A1)

$$\begin{aligned} \dot{y}&= - \frac{x}{\mu }. \end{aligned}$$

(A2)

Now suppose the initial condition $(x_0,y_0)$ is not too close to the line described by the function $y = \varphi (x)$, i.e., assume $y - \varphi (x) \sim O(1)$. Then Eq. (A1) implies that $|\dot{x} |\sim O(\mu )$ and Eq. (A2) implies that $|\dot{y} |\sim O(\mu ^{-1})$.

1.1 A.1 Non-stiff Van Der Pol system

It is well known [49] that whenever $\mu = 1$, the Van Der Pol system is non-stiff, i.e., $\vert \dot{x} |, \vert \dot{y} |\sim O(1)$, the time-scales are similar. Figure 8 presents the non-stiff solution trajectories for the VDP system with initial condition $(x_0, y_0) = (2,0)$. Figure 8 also presents the predicted trajectory using a modified version of DAE-PINN. Note that this version of DAE-PINN is trained to satisfy Eqs. (5a) and (5c) for the VDP system. The results illustrate that the proposed DAE-PINN can easily simulate the dynamic response of the non-stiff VDP system.

Fig. 8

Comparing the long-time simulation accuracy of DAE-PINN for the non-stiff Van der Pol system (parameter $\mu = 1$ and time-step $h = 0.2$). a x(t) trajectory. b y(t) trajectory

1.2 A.2 Increasing stiffness

To increase the stiffness of the VDP system, one must increase the value of the parameter $\mu$. Clearly, $|\dot{x} |\sim O(\mu ) \gg 1$ and $|\dot{y} |\sim O(\mu ^{-1}) \ll 1$ when $\mu$ increases and is much bigger than one. In particular, the velocity in the horizontal direction is considerable, while the velocity in the vertical direction is minimal [49]. These two widely separated time scales are a distinctive property of stiff systems. Figure 9 illustrates the solution trajectories for the VDP system with initial condition $(x_0, y_0) = (2.0, 0.0)$ and parameter $\mu \in \{10, 100, 1000\}$. Note that we can observe both time scales in the graph of the x(t) trajectory. Figure 9 also illustrates the prediction of a modified DAE-PINN. This DAE-PINN can effectively predict/simulate the trajectory even for the very stiff VDP system, i.e., when $\mu = 1000$.

Fig. 9

Comparing the long-time simulation accuracy of DAE-PINN for the stiff Van der Pol system ($\mu \in \{10,100,1000\}$ and time-step $h = 0.2$). a x(t) trajectory for $\mu =10$. b y(t) trajectory for $\mu = 10$. c x(t) trajectory for $\mu =100$. d y(t) trajectory for $\mu = 100$. e x(t) trajectory for $\mu =1000$. e y(t) trajectory for $\mu = 1000$

1.3 A.3 Differential algebraic equations and infinite stiffness

If we let the parameter increase to infinity, i.e., if we let $\mu \rightarrow \infty$, then the stiffness increases to infinity. Then, the differential equations of the VDP system become the following differential-algebraic equations:

$$\begin{aligned} 0&= y - \varphi (x) \\ \dot{y}&= - \frac{1}{\mu } x \end{aligned}$$

Note that the claim that DAEs present a “form” of infinite stiffness [23] can be easily inferred from the previous analysis.