Dynamical Methods in Data-based Exploration of Complex Systems

The human organism is an integrated network where complex physiological systems continuously interact to optimize and coordinate their function. Organ-to-organ interactions occur at multiple levels and spatiotemporal scales to produce distinct physiologic states. Disrupting organ communications can lead to dysfunction of individual systems or to collapse of the entire organism. Yet, we do not know the nature of interactions among systems and sub-systems, and their collective role as a network in maintaining health. The new field of Network Physiology aims to address these fundamental questions. Through the prism of concepts and approaches from statistical and computational physics and nonlinear dynamics, we will present a new framework to identify and quantify dynamic networks of organ interactions. We will demonstrate how physiologic network topology and systems connectivity lead to integrated global behaviors representative of distinct states and functions. The presented investigations are initial steps in building a first Atlas of dynamic interactions among organ systems and the Human Physiolome, a new kind of BigData of blue-print reference maps that uniquely represent physiologic states and functions under health and disease. *We acknowledge support from W M Keck Foundation.

Epilepsy is a complex malfunction of the brain that affects 50 million people worldwide. Epileptic seizures are the cardinal symptom of this multi-facetted disease and are usually characterized by an overly synchronized firing of neurons. Seizures cannot be controlled by any available therapy in about 25% of individuals, and knowledge about mechanisms underlying generation, spread, and termination of the extreme event seizure in humans is still fragmentary. Over the last decades, an improved characterization of the spatial-temporal dynamics of the epileptic process could be achieved with time series analysis tools from nonlinear dynamics, statistical physics, synchronization and network theory. I will summarize these research findings that already have opened promising directions for the development of new therapeutic possibilities.

Until recently, the resolution of ice-core records was inadequate for nonlinear time-series analysis. Advances in laboratory techniques have greatly improved this situation. The isotopic content of the WAIS Divide core, for instance-the longest continuous and highest-resolution such record yet recovered from Antarctica-was measured at 0.5 cm intervals. This is an order of magnitude better than older cores from both poles, which lump years or even decades worth of climate information into each data point. These particular isotopic measurements-ratios of the heavy to light isotopes of hydrogen and oxygen-are considered to be proxies for Earth's temperature. Taking a nonlinear dynamics view of this, we consider these two traces as the outputs of two different measurement functions sampling the dynamics of the paleoclimate and use the method of delays to reconstruct those dynamics over the past 31,000 years. There are a number of unique challenges involved in this analysis. The measurements are spaced unevenly in time because of the progressive downcore thinning of the ice. The relationship between depth and age, and hence the timeline of the data, is uncertain. And the ice itself has undergone tens of thousands of years of unknown natural processes, which can affect both the data values and the timeline. We discuss all of these effects from the standpoint of nonlinear time-series analysis, including change-point detection, fractal dimension, and Lyapunov exponents.

Reconstruction of coupling from time series of oscillatory systems is a problem, having applications in different fields of science, including neurophysiology (and biology in general), climatology, econometrics and radio-engineering. There are two main classes of approaches to this problem. The first one declares that a sufficiently good model for each node of the network is hardly to be constructed; therefore, the general (usually, autoregressive) models are used. Different realizations of Granger causality and partial directed coherence follow this idea. The other class declares that a physically/chemically/biologically driven equations can be written for each node, and these equations are simple enough to be fitted to experimental data. The different techniques for reconstruction of ensembles of phase oscillators, stochastic oscillators, time-delayed systems, first order neurooscillators, van der Pol, Rössler and Lorenz systems and some other models were proposed, with these techniques using specifics of the considered type of nodes to increase their efficiency. Here, we propose some special modifications of previously developed techniques for reconstruction of ensembles of nodes described by ODEs of 1st and 2nd order, targeting problems of: • Partial effective synchronization of some elements of ensembles (not strict synchronization, but close dynamics during the measurement time interval). We use Fisher criterion to determine the element, driving from which cannot be separated, and then the whole cluster is considered as a source of signal for the network. • Hidden series of some nodes. We propose to try using the signal of other nodes in case of partial synchronization, and to separate some part of network if the hidden node affects the dynamics of some subnetwork only. • Transients. We show that using long transients is sometimes as much efficient as using stationary chaotic series. Also we study coupling changes due to short transients between stationary regimes and compare the results with results previously reported for this task using other approaches such as Granger causality. This study was funded by Russian Science Federation, grant number 19-12-00201.

We investigate the chaotic diffusion of dissipative solitons in the one-dimensional Cubic-Quintic Complex Ginzburg-Landau Equation, a prototypical system from nonlinear optics. It turns out that the incremental process of the soliton motion is governed for a wide range of parameters by Hidden Markov Processes with continuous output. We show that a 4-state Hidden Markov Modell with Gaussian output density functions approximates very well the observed soliton dynamics with respect to correlation functions, certain waiting time distributions and the distribution of jump widths. In limiting cases of the parameters the dynamics reduces to an Anti-Persistent Random Walk with fluctuating jump widths.

We propose a method for the reconstruction of dynamical systems with time-delayed feedback, having several hidden variables, including a hidden variable with a time delay. The method is based on the original approach, which allows one to significantly reduce the number of starting guesses for a hidden variable with a delay. The main idea is to assign only a small number of starting guesses on the delay time interval and find the remaining initial conditions for the hidden variable by interpolating the trajectory with a cubic spline. As the objective function of the method, we use the sum of squares of the distances between the points of the observed variable and its reconstruction. By minimizing the objective function, we obtain an estimation of the unknown parameters and recover the time series of hidden variables. The method is applied to the reconstruction of the model system of Lang-Kobayashi equations, which describes the dynamics of a single-mode semiconductor laser with time-delayed feedback, from periodic and chaotic time series. The dependence of the quality of the system reconstruction on the accuracy of the assignment of starting guesses for unknown parameters and hidden variables is investigated. It is shown that for periodic regimes, the region of starting guesses, which provides high quality of reconstruction, is greater than for chaotic regimes. This work was supported by the Russian Foundation for Basic Research, Grant No. 19-02-00071.

In the fields of complex dynamics and complex networks, the reverse engineering, systems identification, or inverse problem is generally regarded as hard and extremely challenging mathematically as complex dynamical systems and networks consists of a large number of interacting units. However, our ideas based on compressive sensing, in combination with innovative approaches, generates a new paradigm that offers the possibility to address the fundamental inverse problem in complex dynamics and networks. In particular, in this talk, I will argue that evolutionary games model a common type of interactions in a variety of complex, networked, natural systems and social systems. Given such a system, uncovering the interacting structure of the underlying network is key to understanding its collective dynamics. Based on compressive sensing, we develop an efficient approach to reconstructing complex networks under game-based interactions from small amounts of data. The method is validated by using a variety of model networks and by conducting an actual experiment to reconstruct a social network. While most existing methods in this area assume oscillator networks that generate continuous-time data, our work successfully demonstrates that the extremely challenging problem of reverse engineering of complex networks can also be addressed even when the underlying dynamical processes are governed by realistic, evolutionary-game type of interactions in discrete time. I will also touch on the issue of detecting hidden nodes, on how to ascertain its existence and its location in the network, this being highly relevant to metabolic networks. ----------- Data based identification and prediction of nonlinear and complex dynamical systems, W.-X. Wang, Y.-C. Lai, and C. Grebogi, Phys. Reports 644, 1-76 (2016) Predicting catastrophe in nonlinear dynamical systems by compressive sensing, W.-X. Wang, R. Yang, Y.-C. Lai, V. Kovanis, and C. Grebogi, Phys. Rev. Lett. 106, 154101 (2011) Network reconstruction based on evolutionary-game data via compressive sensing, W.-X. Wang, Y.-C. Lai, C. Grebogi, and J. Ye, Phys. Rev. X 1, 021021 (2011) Forecasting the future: Is it possible for adiabatically time-varying nonlinear dynamical systems? R. Yang, Y.-C. Lai, and C. Grebogi, Chaos 22, 033119 (2012) Optimizing controllability of complex networks by minimum structural perturbations, W.-X. Wang, X. Ni, Y.-C. Lai, and C. Grebogi, Phys. Rev. E 85, 026115 (2012)

Synchronization of neural activity in the brain is involved in a variety of brain functions including perception, cognition, memory, and motor behavior. Excessively strong, weak, or otherwise improperly organized patterns of synchronous oscillatory activity may contribute to the generation of symptoms of different neurological and psychiatric diseases. However, synchrony in cortical and subcortical circuits is frequently variable in time and not perfect. Few long intervals of desynchronized dynamics may be functionally different from many short desynchronized intervals although the average synchrony may be the same. Analysis of experimental recordings reveals a rich temporal structure of intermittent neural synchronization. Some of its properties appear to be potentially universal, while some a specific to particular brain states and other conditions. We will discuss the techniques to analyze the temporal patterning of synchronized activity in the experimental data and its implications for neural model building.

Spatiotemporal systems are ubiquitous in nature, ranging from biological systems, active matter, turbulence, electromagnetism and wave propagation, to name a few. It is advantageous to be able to temporally predict the evolution of such systems, in an iterative fashion. Also, it is often necessary to be able to cross estimate one field of a pair of coupled fields, given the complementary one. What makes this task difficult is that the complex dynamics of spatiotemporal systems typically lead to high-dimensional chaos and are thus hard to predict. An additional difficulty comes from the nature of spatiotemporal data, which when sampled discretely lead to high dimensional data points. This can make some numeric prediction techniques unfeasible, from a computational perspective. Here we will present a method for both cross estimation and iterated timeseries prediction of spatiotemporal systems based on estimating the local dynamics. Our method works using reconstructed local states, PCA dimension reduction, and nearest neighbour prediction methods. The effectiveness of this approach will be shown for (noisy) data from a cubic Barkley model, the Bueno-Orovio-Cherry-Fenton model, and the Kuramoto-Sivashinsky model.

We first review the basic idea of using machine learning in conjunction with a limited duration of time series data to construct a closed-loop, autonomous, dynamical system that can predict the future evolution of the state of the unknown system that generated the data [1]. Using the reservoir computing type of machine learning, we then present examples of extensions and applications of this idea. These will include a parallel implementation enabling forecasting of the states of very large spatiotemporally chaotic systems with local interactions [2], a hybrid scheme where an imperfect knowledge-based model component is combined with a limited-size machine learning component to achieve prediction performance much better than that of either of the components acting alone [3], an architecture combining the parallel and hybrid schemes, and generalization of ensemble Kalman filtering to cyclic prediction using the parallel/hybrid machine learning schemes. As an example of the potential utility of these elements for large complex spatiotemporally chaotic systems, their use in our ongoing project on improving weather forecasting [4] will be outlined. [1] Jaeger, Haas, Science (2004). [2] Pathak, Hunt, Girvan, Lu, Ott, Phys Rev Lett (2018). [3] Pathak, Wikner, Fussell, Chandra, Hunt, Girvan. Ott, CHAOS (2018). [4] Collaborators on our current weather forecasting project: T. Acomano, M. Girvan, B. Hunt, G. Katz, J. Reggia, I. Szunyogh, C.-Y. Wang, A. Wikner.

When a complex network of $N_n$ node with a $d$-dimensional dynamics is investigated, the goal is to retrieve the dynamics of the whole system by only measuring a specific signal at a particular reduced set of nodes (sensors). Determining the sensors to measure is indeed one of the main problems for investigating complex networks, since too often, its dimensionality is too large to allow measurements of the whole set of variables required for determining each of its states. As a prior knowledge, we limit ourselves to know the adjacency matrix. The equations governing the node dynamics are not known and are retrieved from data by using a global modelling technique. The observability of an isolated node and of a dyad of nodes are then determined from such global model. The nature of the coupling between two nodes of the studied network is also determined using our global modelling technique. Once the observability of the dyad is assessed, the observability of the whole network can be estimated. All of these results are used to design observers for the node, the dyad and the network, respectively. The accuracy of the observers will be quantified. The example of a complex network made of $N_n = 28$ Rössler nodes will be explicitly treated as well as an electronic realization of it. This work is mainly performed with Irène Sendina-Nadal (RJC University, Madrid) and Sylvain Mangiarotti (CESBIO, Toulouse).

We present a use of modern data-based machine learning approaches to suppress self-sustained collective oscillations typically signaled by the ensembles of degenerative neurons in the brain. The proposed hybrid model relies on two components: the medium of oscillators block and the reward-based reinforcement learning block. We consider four types of reward functions to provide a quick and efficient suppression of the self-oscillating activity. The synchrony suppression is demonstrated on two models: the simple globally coupled neuronal oscillators model and the bursting Hindmarsh–Rose neurons model.

We investigate the predictive power of recurrent neural networks for oscillatory systems not only on the attractor but in its vicinity as well. For this, we consider systems perturbed by an external force. This allows us to not merely predict the time evolution of the system but also study its dynamical properties, such as bifurcations, dynamical response curves, characteristic exponents, etc. It is shown that they can be effectively estimated even in some regions of the state space where no input data were given. We consider several different oscillatory examples, including self-sustained, excitatory, time-delay, and chaotic systems. Furthermore, with a statistical analysis, we assess the amount of training data required for effective inference for two common recurrent neural network cells, the long short-term memory and the gated recurrent unit.

Interactions among discrete oscillatory units (e.g., cells) can result in partially synchronized states when some of the units exhibit phase locking and others phase slipping. Such states are typically characterized by a global order parameter that expresses the extent of synchrony in the system. In this talk we show that such states carry data-rich information of the system behavior, and a local order parameter analysis reveals universal relations through a semicircle representation. The universal relations are derived from thermodynamic limit analysis of a globally coupled Kuramoto-type phase oscillator model. The relations are confirmed with the partially synchronized states in numerical simulations with a model of circadian cells, and in laboratory experiments with chemical oscillators. The application of the theory allows direct approximation of coupling strength, the natural frequency of oscillations, and the phase lag parameter without extensive nonlinear fits as well as a self-consistency check for presence of network interactions and higher harmonic components in the phase model.

Extreme events as floods or draughts do often occur in temporal clusters. In this talk I will present the analysis and modelling of the timings of observed series of extreme events. The data series are presented as binary symbol sequences where the symbol 1 presents time intervals with an extreme event and the symbol 0 time intervals without. The symbol sequences are characterised by the following measures from statistical physics and time series analysis: the Shannon entropy which is estimated in terms of the Lempel-Ziv complexity, the shape parameter of the Weibull distribution that best fits the event return times, and the strength of long-range correlations quantified by detrended fluctuation analysis (DFA). The event series will be modelled by peaks over threshold models, where the background signal will be chosen appropriately. These methodical concepts will be applied to two sets of observational data: (i) to heart beat annotations obtained from 24-h electrocardiogram recordings of post-infarction patients where the symbol sequences represent arrhythmic and normal beats, and (ii) to a record of palaeofloods which occurred over a period of 9.3 thousand years in the southern Alps.

The speaker will discuss two topics in the interdisciplinary field of machine learning and nonlinear dynamics. Firstly, a general principle will be introduced for choosing the complex neural network underlying reservoir computing machines, a class of recurrent neural networks, for predicting nonlinear dynamical systems. A strategy for significantly expanding the prediction horizon will be described. Secondly, predicting rare intense or extreme events in spatiotemporal dynamical systems using deep convolutional neural networks will be discussed.

Nowadays, unprecedented advances in machine learning for biomedicine have led to a variety of unsupervised methods for remote image analysis, allowing for cost-effective early detection of diseases. Here I will present several methods for the analysis of retina images, which exploit nonlinear data analysis tools and network theory. First, I will discuss an unsupervised algorithm for the analysis of optical coherence tomography (OCT) anterior chamber images, which extracts features that discriminate between healthy subjects and patients with angle-closure [1]. Then, I will discuss how to extract relevant features from the structure of the tree-like vessel network in the retina [2]. I will show that these features allow to discriminate between healthy subjects and those with glaucoma or diabetic retinopathy. Finally, I will discuss how the percolation transition in networks can be exploited for outlier mining in different types of datasets including anterior chamber OCT images [3]. [1] P. Amil, L. Gonzalez, E. Arrondo, C. Salinas, J.L. Guell, C. Masoller, and U. Parlitz, “Unsupervised feature extraction of anterior chamber OCT images for ordering and classification”, Sci. Rep. 9, 1157 (2019). [2] P. Amil, F. Reyes-Manzano, L. Guzmán-Vargas, I. Sendiña-Nadal and C. Masoller, “Novel network-based methods for retinal fundus image analysis and classification”, submitted (2019). [3] P. Amil, N. Almeira and C. Masoller, “Outlier mining methods based on network structure analysis”, submitted (2019).

Sparse regression has recently emerged as an attractive approach to discover models of spatiotemporally complex dynamics directly from data. In many instances, such models are in the form of nonlinear partial differential equations, hence sparse regression requires evaluation of various partial derivatives. Accurate evaluation of derivatives, especially of high order, is however infeasible when the data is noisy, which has a dramatic negative effect on the result of regression. We present an approach that solves this difficulty by using a weak formulation of the problem. For instance, it allows accurate reconstruction of the Kuramoto-Sivashinsky equation, which involves fourth-order derivatives, from data with a considerable amount of noise. Our approach also allows reconstruction of models which involve latent variables that cannot be measured directly with acceptable accuracy. This is illustrated by reconstructing a model for a forced turbulent flow in a thin fluid layer, where neither the forcing nor the pressure field is known.

The Hilbert transform is a building block of nonlinear time series analysis for oscillatory processes. As long as the deviation from the corresponding limit cycle remain small, one is able to extract the phase of the dynamic system from the data. The Hilbert transform is applied once to obtain the analytic signal and the corresponding amplitude and phase. The properties of the later are analyzed with a vast number of techniques. However, the phase of the signal and the phase of the dynamical system do not coincide. The reason for this is that the Hilbert transform imposes artificial modulations on the obtained phase. The following problem is solved by iterative application of the Hilbert transform: Given a scalar time series of an arbitrarily complex wave form with constant amplitude and arbitrarily complex and monotonous phase, construct this phase from the data. The following problem is partly solved: Given a time series from a (perturbed) dynamic system, construct the underlying phase of this dynamic system. The final phase result will be obtained after iterative reuse of the previous phase from the Hilbert embedding. The method allows for major improvements in case of signals for information transmission. For dynamic systems it allows for major improvements if the phase dynamics dominates. Dynamic processes with high phase modulation frequency and stronger deviation from the limit cycle have to be treated with more caution.

The equivalence of data assimilation and supervised machine learning will be explained and used to address problems in both fields. Methods including precision annealing in variational and Monte Carlo estimations of important conditional expected values of model state variables will be discussed.

This contribution will consider data assimilation in infinite dimensional models given by dissipative PDE's (e.g. reaction diffusion, 2-dimensional Navier Stokes). The observations, however, are assumed to be finite dimensional only as is the case in practice. Relatively simple schemes employing linear error feedback will be investigated, and the long time behaviour of the error dynamics will be analysed. In the case without observational noise, conditions are identified which guarantee zero asymptotic error (i.e. complete synchronisation). In the case of observational error, bounds on the long-term synchronisation error are derived, both for individual realisations of the noise as well as on average. In addition to being of practical interest, the results confirm the view of these equations having ``only finitely many asymptotic degrees of freedom''.

Estimating the network structure underlying complex dynamical systems just based on observation of the data they exhibit is a key challenge in many fields of research. This contribution provides a start of the art overview with particular emphasis on challenges faced with this complex task. Real-world applications will be used as examples to demonstrate the problems arising when using the various approaches.

We are interested in Reinforcement Learning for the control of turbulent flows. A key ingredient of this application consists in learning a dynamical model of the system under consideration directly from limited measurements. To this end, we are exploring different approaches for the prediction of nonlinear chaotic systems. Specifically, we consider a convolutional-in-time GAN relying on a regularized optimal transport metric. Alternatively, we learn a model via an online discounted boosting approach with function trains from the time-streaming observations. These methods are assessed in terms of various metrics, such as the accuracy of the Lyapunov exponents for a given amount of training data. The results are illustrated with the reconstruction and long-term prediction of the solution of the Kuramoto-Sivashinsky equation in the hyper-chaotic regime observed through a few local sensors.

We introduce the bivariate jump-diffusion process, comprising two-dimensional diffusion and two-dimensional jumps, that can be coupled to one another. We present a data-driven, non-parametric estimation procedure of higher-order Kramers–Moyal coefficients that allows one to reconstruct relevant aspects of the underlying jump-diffusion processes and to recover the underlying parameters. The procedure is validated with numerically integrated data using synthetic bivariate continuous and discontinuous time series. We further evaluate the possibility of estimating the parameters of the jump-diffusion model via data-driven analyses of the higher-order Kramers–Moyal coefficients, and the limitations arising from the scarcity of points in the data or disproportionate parameters in the system.

There is an abundant physics literature on methods for reconstructing the dynamical networks from node trajectories. Those methods often rely on strong assumptions about the properties of the network and/or its collective dynamics. While this makes them elegant and efficient, it also limits their practical usage. Relying on insights from machine learning, we introduce a couple of concepts that enable design of new reconstruction methods that work with no such assumptions, while still being fairly robust to noise, trajectory length and dynamical regime.

Chimera states are an intriguing interplay of synchronous and asynchronous motion in networks of coupled oscillators. While chimera states were traditionally studied in one-layer networks, recent work studies interactions of chimera states across coupled layers in multi- layer networks. We here review our recent work in which we applied different types of couplings between pairs of networks that individually show chimera states when there is no coupling between them. We show that these couplings across network layers can lead to generalized syn- chronization [1] and phase synchronization [2] between networks, while both layers continue to exhibit distinct chimera states. We show that these synchronization phenomena are in close analogy to those found for low-dimensional chaotic dynamics. References: [1] Andrzejak, R. G., Ruzzene, G., & Malvestio, I. (2017). Generalized synchronization between chimera states. Chaos, 27(5), 053114. [2] Andrzejak, R. G., Ruzzene, G., Malvestio, I., Schindler, K., Schöll, E., & Zakharova, A. (2018). Mean field phase synchronization between chimera states. Chaos, 28(9), 091101.

In physics when we deal with dynamical systems we often assume that they result from closed systems and apply mathematical concepts within the framework of autonomous dynamics. In reality, however, many systems are open and so cannot usefully be considered in that way. Traditionally, their openness is modelled purely statistically as openness to external random noise. A new approach, which we discuss here, is to consider them as a response to deterministic perturbations that are external in origin, and then to identify their properties within the framework of nonautonomous dynamical systems. In this talk we will take the view that for many open systems in biology and physics, the external influences may be regarded as consisting of deterministic components that interact in a tractable manner with each other and with the system under investigation. Contrary to the closed systems assumptions of conservation of mass and energy, our study of open systems focuses on the question of conservation of phase differences among interacting oscillatory components. We will take nonautonomicity due to the openness explicitly into account. We will show that when this nonautonomicity takes the form of frequency modulation of oscillators within a phase-oscillator network used to model the system, this can stabilise the system’s dynamics. This principle can occur independently of the number of oscillatory components involved, making non-stochastic cause-consequence relations obtainable even when the system is too complex for individual contributions to be tractable. Hence, important information that is invisible to the statistical noise approach to dealing with openness can now be uncovered. Data from real biological examples, at the cellular and systems levels, such as cancer and dementia, will be used to illustrate both the necessity and the potential for success of our dynamical methods for exploring complex systems taking explicit account of their openness.

The beat of heart cells is preceded by a fast change of their transmembrane voltage. The distribution of cell-transmembrane voltages over the heart tissue is made visible by electrocardiogram (ECG) measurements at the body surface. The calculation of ECG signals based on the voltage distribution on the heart, also known as the forward problem, can be solved straight-forwardly. However the inverse problem turns out to be ill-posed. I approach the inverse problem by applying machine learning techniques to multi-ECG data obtained from Langendorff perfused hearts in order to reconstruct the underlying electrical state. This is further supported by applying these techniques to numerical simulations of the experimental setup.

System identification (SID) is central in science and engineering applications whereby a general model form is assumed, but active terms and parameters must be inferred from observations. Most methods for SID rely on optimizing some metric-based cost function that describes how a model fits observational data. A commonly used cost function employs a Euclidean metric and leads to a least squares estimate, while recently it becomes popular to also account for model sparsity such as in compressed sensing and Lasso. While the effectiveness of these methods has been demonstrated in previous studies including in cases where outliers exist, it remains unclear whether SID can be accomplished under more realistic scenarios where each observation is subject to non-negligible noise and sometimes contaminated by large noise and outliers. We show that existing sparsity-focused methods such as compressive sensing, when used applied in such scenarios, can result in “over sparse” solutions that are brittle to outliers. In fact, metric-based methods are prone to outliers because outliers by nature have an unproportionally large influence. To mitigate such issues of large noise and outliers, we develop an entropic regression approach for nonlinear SID, whereby true model structures are identified based on relevance in reducing information flow uncertainty, not necessarily (just) sparsity. The use of information-theoretic measures as opposed to a metric-based cost function has a unique advantage, due to the asymptotic equipartition property of probability distributions, that outliers and other low-occurrence events are conveniently and intrinsically de-emphasized.

We present two approaches for data driven modelling. The first one [1] combines deep convolutional neural networks (CNNs) for feature extraction and dimension reduction with an adapted conditional random field (CRF) in order to model the properties of temporal sequences. The second one, which is not published yet, extends our work by Neural Ordinary Differential Equations [2]. To validate the proposed method we used the BOCF model describing electrical excitation waves in cardiac tissue where chaotic dynamics is associated with cardiac arrhythmias and the Lorenz 96. Running the trained network in a closed loop(feedback) configuration iterated prediction provided forecasts of the complex dynamics that turned out to follow the true evolution. [1] S. Herzog, F. Wörgötter, U. Parlitz, Data-driven modelling and prediction of complex spatio-temporal dynamics in excitable media, in: Front. Appl. Math. Stat. - Dynamical Systems (2018), doi: 10.3389/fams.2018.00060 [2] Chen, Tian Qi, et al. "Neural ordinary differential equations." Advances in Neural Information Processing Systems. 2018.

We explore the relationship between the topological relevance of a node in a complex network and the individual dynamics it exhibits. When the system is weakly coupled and therefore not synchronised, the effect of the coupling strength upon the dynamical complexity of the nodes is found to be a function of their topological roles, with nodes of higher degree displaying lower levels of complexity. We provide several examples of theoretical models of chaotic oscillators, pulse-coupled neurons, and experimental networks of nonlinear electronic circuits and cultured neuronal networks evidencing such a hierarchical behavior. Importantly, our results imply that it is possible to infer the degree distribution of a network only from individual signal recordings. Finally, we revisit the definition of complexity and provide a marker which is able to distinguish between organized from disorganized complexity.

This paper presents a “structured” learning approach for the identification of continuous partial differential equation (PDE) models with both constant and spatial-varying coefficients. The identification problem of parametric PDEs can be formulated as a mixed optimization problem by explicitly using block structures. Block-sparsity is used to ensure parsimonious representations of parametric spatiotemporal dynamics. In particular, the estimated values of varying coefficients are further used as data to identify functional forms of the coefficients. By exploring the restricted isometry properties of the structured random dictionary matrices, we derive a recovery condition that relates the number of samples to the sparsity and the probability of failure in the Lasso scheme. Various examples, such as the Schrödinger equation, the Fisher-Kolmogorov-Petrovsky-Piskunov equation, the Burger equation, and the Fisher equation, suggest that the proposed algorithm is fairly effective, especially when using a limited amount of measurements.

During life threatening cardiac arrhythmias like ventricular fibrillation the electrical excitation dynamics inside the heart is governed by chaotic spiral/scroll wave propagation. Due to the significant side-effects of the current medical treatment of such arrhythmias (the application of a high energy defibrillation shock) improved control concepts and strategies are required. In numerical simulations and experimental studies it was found, that the chaotic spiral/scroll wave dynamics is often transient, rather than persistent. We show how the state space structure (determined by a chaotic saddle) which leads to the self-termination of chaotic episodes can also have a significant impact on how efficient control strategies may be developed. Furthermore, we demonstrate how such efficient algorithms could benefit from novel machine learning concepts.

Understanding and modelling of real world complex dynamic systems is often made difficult by incomplete knowledge about the interactions between systems states and by unkown disturbances to the system. We address both problems by considering dynamic networks as open systems receiving inputs from their environment. To understand a system and to estimate the state dynamics, these inputs need to be reconstructed from output measurements, which is only possible for invertible sytems. In our work, we exploit the fact that invertibility of a dynamic system is a structural property, which depends only on the network topology. We analyse synthetic and real networks to find structural features influencing invertibility and show that a careful experimental design is required to select outputs nodes in the network which render the system invertible. As an important result we present a new sensor node placement algorithm to achieve invertibility with a minimum set of measured states. We discuss why our results have important conceptual as well as practical implications for the analysis and modelling of incompletey specified dynamic networks and for the design of synthetic systems. Authors: Dominik Kahl^(1), Philipp Wendland^(1), Matthias Neidhardt^(2) Andreas Weber^(2) and Maik Kschischo^(1∗) 1: Department of Mathematics and Technology, University of Applied Sciences Koblenz, Joseph-Rovan-Allee 2, 53424 Remagen, Germany 2: Institute for Computer Science, University of Bonn, Endenicher Allee 19a, 53115 Bonn, Germany * Corresponding author Preprint: Kahl, D., Wendland, P., Neidhardt, M., Weber, A. and Kschischo, M. Structural Invertibility and Optimal Sensor Node Placement for Error and Input Recon- struction in Dynamic Systems Under review in Phys. Rev. X preprint: arXiv:1904.11828 (https://arxiv.org/abs/1904.11828)

Cell surface receptors convert extracellular cues into receptor activation, hereby triggering intracellular signaling networks and controlling cellular decisions. A major unresolved issue is the identification of receptor properties that critically determine processing of ligand-encoded information. We show by mathematical modeling of time-resolved data and experimental validation that rapid ligand depletion and replenishment of the cell surface receptor are characteristic features of the erythropoietin (Epo) receptor (EpoR). Epo is a hormone that triggers progenitor cells to become red blood cells. As this system is under fast turnover, it is often damaged in cancer patients treated with chemotherapy leading to anemia which is treated by applying Epo or some manufactured equivalent. However, it has been observed that the EpoR is also expressed on tumor cells leading to resistance of the cancer cell against chemotherapy. Furthermore, over-dosing Epo treatment leads to cardiovascular events. To pave the way for medical applications of our basic insight, we apply two extensions of the model. First, we couple the receptor model to a downstream signalling model and show that there might be a therapeutic window for which Epo-treatment stimulates the blood-building system but does not protect cancer cells. Second, we couple the cell-scale model to a whole-body pharmacokinetic/pharmacodynamic model and derive patient-individual optimal strategies for Epo treatment of chemotherapy-induced anemia.

Data assimilation in highly nonlinear high-dimensional systems is an unsolved challenge, and it is easy to show that we will always have to rely on approximations of the full Bayesian Inference problem. The question then becomes what the best approximation is given the specific problem at hand. In this talk recent developments involving fully nonlinear particle filters will be diuscussed. Specifically, proposal density methods exploring typical sets and so-called particle flows will be explored in high-dimensional settings. The latter involved methods from machine learning to speed up parts of the calculations. If time permits a discussion of the merging of data-assimilation and machine learnbing will be priovided.

The number of units of a network dynamical system, its size, arguably constitutes its most fundamental property. Many units of a network, however, are typically hidden, i.e. experimentally inaccessible such that the network size is often unknown. Here we introduce a detection matrix that suitably arranges multiple transient time series from the subset of perceptible units to detect network size via matching rank constraints. The proposed method is model-free, applicable across system types and interaction topologies and applies to non-stationary dynamics near fixed points, as well as periodic and chaotic collective motion. Even if only a small minority of units is perceptible and for systems simultaneously exhibiting nonlinearities, heterogeneities and noise, exact size detection is feasible. We illustrate applicability for a paradigmatic class of biochemical reaction networks. This is work with Hauke Haehne, Jose Casadiego and Joachim Peinke. in press at Physical Review Letters (2019).

Hate speech and more generally the spread of hate online (cyberhate) is a growing concern in many countries around the world with very real psychological consequences for victims as well as bystanders, and may even give rise to real-world violence. It is believed that ``counterspeech" or user-generated direct responses to cyberhate, is emerging as a promising intervention strategy. Unfortunately, little is known about the actual effectiveness of various counterspeech strategies. This, in part, stems from a lack of knowledge about how cyberhate and counterspeech interact. Here we leverage a unique situation in Germany, where there currently exist (at least) two competing groups with opposing political narratives which frequently engage in polarizing conversations on Twitter: a hate group ``Reconquista Germanica" (RG) and a counter group ``Reconquista Internet" (RI). Both of these groups maintain a plethora of constantly evolving strategies for disseminating their messages as well as strategies to block and discredit the opposing groups ideology from being broadcast to the general public. The aim of this study is to understand how these strategies evolve and to quantify their effectiveness as they change over time. To this end, we examine the nonstationary coupled dynamics of cyberhate and counterspeech through their direct interactions in Twitter based conversations or ``reply trees." We have collected over 230,000 reply trees involving these two groups which took place from 2013 to present. Using tens of millions of German tweets combined with expert domain knowledge we trained neural networks in a semi-supervised framework to construct a representation of ``typical language" for both of these groups. We then use a set of classifiers on this representation space to label each tweet as cyberhate or counterspeech and to discern which strategies are present. As this is a fully temporally resolved dataset, it is useful to think about the data as a nested time series, where we have a time series of reply trees and where each tree is itself a time series of tweets. In this view, we can use the macro scale time series to understand how these strategies evolve over time and quantify whether the adaptations are effective. On the micro level we can study the local interactions and strategies between these two groups. Utilizing this framework we are able to leverage a variety of tools from time series analysis, machine learning and network theory to dissect and investigate the strategies being used on the micro- and macro-scales by both groups. For example, on a micro-level we can study network attachment strategies within a reply tree, i.e., does a participant preferentially reply to the root tweet (a root-biased strategy) or to the tweet with the most replies (a degree-biased strategy). We find that cyberhate being spread by RG members is more root- and degree-biased compared to counterspeech of RI members. Even this simple result has implications for successful design of interventions for cyberhate. For example, the latest versions of Twitter's display algorithms are root-, degree- and media-biased. This, coupled with the behavior of RG members which is root- and degree-biased, implies that hate speech is more likely to be displayed to a generic user than another random reply. Ultimately, understanding the strategies of cyberhate groups could suggest new algorithmic design principles and inform successful counterspeech strategies.

We will report on a novel framework for network identification (and more generally for nonlinear system identification), which is based on the so-called Koopman operator. This framework relies on the key idea that identifying a nonlinear dynamical system in the state space is equivalent to identifying the linear Koopman operator in the space of observables. Two dual identification methods will be presented in this context and complemented with theoretical convergence results. They will be shown to be efficient with a general class of systems, including chaotic systems, well-suited to low sampling rate data sets, and capable of identifying the nature of the coupling between nodes. The performance of the methods will be illustrated with several examples.