Disciplina Discipline PCS5042

Inteligência Artificial Informada por Leis Físicas e Modelos de Ordem Reduzida para Engenharia
Physics-informed Artificial Intelligence and Reduced-order Models for Engineering

Apresentar os conceitos de redes neurais informadas por leis físicas e modelos de ordem reduzida como ferramentas modernas de solução de problemas governados por equações diferenciais parciais (EDPs) lineares e não-lineares. Comparar esse tipo de abordagem com métodos clássicos de discretização de EDPs e levantar suas principais qualidades e limitações. Explorar as abordagens propostas em sistemas dinâmicos de aplicação em engenharia com foco particular em sistemas caóticos usando python notebooks como bancadas virtuais de experimentação teórico-numérica.

Introduce physics-informed machine learning (ML) and reduced- order models as modern tools for solving problems governed by linear and nonlinear partial differential equations (PDEs). Compare both approaches with classical discretization methods for solving PDEs and stress their strengths and weaknesses. Explore both ML-based methods in engineering applications with focus on chaotic systems using python notebooks as virtual environments for theoretical and numerical experimentation.

Os algoritmos de aprendizado de máquina e inteligência artificial transformaram amplamente a ciência e a engenharia, especialmente em aplicações de visão computacional e processamento de linguagem natural. No entanto, a extensão desses algoritmos para resolver problemas baseados em física só recentemente ganhou maturidade com a introdução de redes neurais informadas por física (PINNs) e modelos de ordem reduzida (ROMs) baseados em aprendizado de máquina. Tais esquemas podem emular sistemas multifísicos complexos governados por equações diferenciais parciais (PDEs). Os PINNs incorporam perfeitamente dados ruidosos em algoritmos existentes, contornam a tirania da geração de malha e lidam com problemas de alta dimensão associados a sistemas governados por PDEs parametrizados. Da mesma forma, modelos de ordem reduzida baseados em projeção têm desempenhado um papel essencial na compactação de conjuntos de dados de alta dimensão que fornecem aproximações computacionalmente eficientes de sistemas dinâmicos complexos. PINNs e ROMs tornaram-se ferramentas de engenharia indispensáveis que combinam o melhor dos princípios físicos e paradigmas de IA.

Machine learning and artificial intelligence algorithms have broadly transformed science and engineering, especially in computer vision and natural language processing applications. However, the extension of these algorithms to solve physics-based problems has only recently gained maturity with the introduction of physics- informed neural networks (PINNs) and machine learning-based reduced-order models (ROMs). Such schemes can emulate complex multi-physics systems governed by partial differential equations (PDEs). PINNs seamlessly incorporate noisy data into existing algorithms, bypass the tyranny of mesh generation, and tackle high-dimensional issues associated with systems governed by parametrized PDEs. Likewise, projection-based reduced-order models have played an essential role in compressing high-dimensional datasets that provide computationally efficient approximations of complex dynamical systems. PINNs and ROMs have become indispensable engineering tools that combine the best in physical principles and AI paradigms.

1. Equações Diferenciais Parciais (PDEs) e Métodos de Discretização 1.1 - Métodos de Diferença Finita, Elemento Finito e Volume Finito para PDEs 1.2 - Convecção linear e não linear 1D 1.3 - Condição CFL 1.4 - Equação de difusão 1D 1.5 - Equação de Burger 1D 1.6 - Equação 1D Kuramoto-Sivashinsky (KS) 1.7 - Equação de Buckley-Leverett 1D (transporte de duas fases fluidas em meios porosos) 2. Redes Neurais Artificiais (RNA): Uma visão geral 2.1 - O conceito inicial de neurônios artificiais: perceptrons 2.2 - Neurônios sigmóides: suavizando a saída binária dos perceptrons 2.3 - A arquitetura das redes neurais 2.4 - Aprendizagem com gradiente descendente 2.5 - Abordagem baseada em matriz 2.6 - Função custo 2.7 - O algoritmo de retropropagação 3. Redes Neurais Informadas por Física 3.1 - Diferenciação automática como ingrediente chave para resolver PDEs usando RNAs 3.2 - O PINN básico 3.3 - O PINN baseado em quadratura 3.4 - O PINN Variacional (VPINN) e o hp-VPINN 3.5 - O PINN Conservador (CPINN) e o PINN Estendido (XPINN) 3.6 - Identificação e problemas inversos usando PINNs 3.7 - Redes de Operador Profundo (DeepONets) 4. Modelos de Ordem Reduzida (ROM) 4.1 - Redução de dimensionalidade usando decomposição ortogonal adequada (POD) 4.2 - Redução de dimensionalidade usando redes neurais de autoencoder 4.3 - Modelos substitutos baseados em ANN: arquiteturas DNN, LSTM e ESN-RC 4.4 - Decomposição em Modo Dinâmico 4.5 - Redução do modelo não intrusivo via inferência do operador (OpInf) 4.6 - Previsão de dinâmicas espaço-temporais complexas 4.7 - Modelos substitutos para problemas diretos e inversos

1. Partial Differential Equations (PDEs) and Discretization Methods 1.1 - Finite Difference, Finite Element, and Finite Volume Methods for PDEs 1.2 - 1D linear and nonlinear convection 1.3 - CFL condition 1.4 - 1D diffusion equation 1.5 - 1D Burgers equation 1.6 - 1D Kuramoto–Sivashinsky (KS) equation 1.7 - 1D Buckley-Leverett equation (transport of two fluid phases in porous media) 2. Artificial Neural Networks (ANN): An overview 2.1 - The early concept of artificial neurons: perceptrons 2.2 - Sigmoid neurons: smoothing the binary output of perceptrons 2.3 - The architecture of neural networks 2.4 - Learning with gradient descent 2.5 - Matrix based approach of neural networks 2.6 - Cost function 2.7 - The backpropagation algorithm 3. Physics Informed Neural Networks 3.1 - Automatic differentiation as the key ingredient to solve PDEs using ANNs 3.2 - The Vanilla PINN 3.3 - The Quadrature-based PINN 3.4 - The Variational PINN (VPINN) and the hp-VPINN 3.5 - The Conservative PINN (CPINN) and the Extended PINN (XPINN) 3.6 - Identification and Inverse problems using PINNs 3.7 - Deep Operator Networks (DeepONets) 4. Reduced Order Models (ROM) 4.1 - Dimensionality reduction using Proper orthogonal decomposition (POD) 4.2 - Dimensionality reduction using autoencoder neural networks 4.3 - ANN-based surrogate models: DNN, LSTM and ESN-RC architectures 4.4 - Dynamic Mode Decomposition 4.5 - Non-intrusive Model Reduction via Operator Inference (OpInf) 4.6 - Forecasting of complex spatiotemporal dynamics 4.7 - Surrogate models for direct and inverse problems

1. Entrega de 3 projetos no formato pyhton notebook associados aos tópicos 1, 3 e 4 da ementa cobrindo os seguintes temas: a. Discretização e solução das PDEs de Buckley-Leverett (BL), para escoamento bifásico, e Kuramoto-Sivashinsky (KS), que modela a propagação de chama. b. Aplicação de PINNs para a obtenção de soluções aproximadas das PDEs do item a. c. Aplicação de ROMs a partir dos outputs gerados no item a para a obtenção de soluções aproximadas das PDEs do item a. 2. Apresentação de um dos artigos que faz parte da bibliografia da disciplina.

1. 3 projects in pyhton notebook format associated with topics 1, 3 and 4 of the syllabus covering the following topics: a) Discretization and solution of Buckley-Leverett (BL) PDEs, for two-phase flow, and Kuramoto-Sivashinsky (KS), which models flame propagation. b) Application of PINNs to obtain approximate solutions for the PDEs in item a. c) Application of ROMs from the outputs generated in item a to obtain approximate solutions for the PDEs in item a. 2. Presentation of one of the articles that is part of the bibliography of the discipline.

1. Michael A. Nielsen, Neural Networks and Deep Learning, Determination Press, 2015. (available online: http://neuralnetworksanddeeplearning.com/index.html) 2. M. Raissi, P. Perdikaris, and G. E. Karniadakis, Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations, arXiv: 1711.10561v1, 28-Nov-2017. 3. M. Raissi, P. Perdikaris, and G. E. Karniadakis, Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations, arXiv: 1711.10556v1, 28-Nov-2017. 4. V. Buljak, Inverse Analysis with Model Reduction: POD in Structural Mechanics, Springer, 2012. 5. S. Brunton and J. Kutz, Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control. Cambridge: Cambridge University Press, 2019. Textos e artigos diversos: 6. O. Fuks and H. A. Tchelepi, Limitations of Physics Informed Machine Learning for Nonlinear Two-Phase Transport in Porous Media, ResearchGate preprint publication https://www.researchgate.net/publication/343111185, 2020. 7. S. Mishra and R. Molinaro, Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEs, arXiv:2006.16144v1, 29 Jun 2020. 8. E. Kharazmi, Z. Zhang, and G. E. Karniadakis, VPINNs: Variational Physics-Informed Neural Networks for Solving Partial Differential Equations, arXiv:1912.00873v1, 27 Nov 2019. 9. E. Kharazmi, Z. Zhang, and G. E. Karniadakis, hp-VPINNs: Variational Physics- Informed Neural Networks with Domain Decomposition, arXiv:2003.05385v1, 11 Mar 2020. 10.D. Jagtap, E. Kharazmi, and G. E. Karniadakis, Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems, Comput. Methods Appl. Mech. Engrg. 365 (2020) 113028. 11.D. Jagtap, and G. E. Karniadakis, Extended Physics-Informed Neural Networks (XPINNs): A Generalized Space-Time Domain Decomposition Based Deep Learning Framework for Nonlinear Partial Differential Equations, Commun. Comput. Phys., 2020. 12.Hugo F. S. Lui, Construction of Reduced Order Models for Fluid Flows using Deep Neural Networks, MSc dissertation, 2019. 13.S. H. Rudy, S. L. Brunton, J. L. Proctor, and J. N. Kutz, Data-driven discovery of partial differential equations, arXiv: 1609.06401v1, 21-Sept-2016. 14.S. L. Brunton, J. L. Proctor, and J. N. Kutz, Discovering governing equations from data: Sparse identfication of nonlinear dynamical systems, PNAS, 2016. 15.K. Champion, B. Lusch, J. N. Kutz, and S. L. Brunton Data-driven discovery of coordinates and governing equations, PNAS, 2019. 16.J. S. Hesthaven, S. Ubbiali, Non-intrusive reduced order modeling of nonlinear problems using neural networks, Journal of Computational Physics (2018) 363, 55-78. 17.Z. Gao, Q. Liu, J. S. Hesthaven, B. S. Wang, W. S. Don, X. Wen, Non-intrusive reduced order modeling of convection dominated flows using artificial neural networks with application to Rayleigh-Taylor instability, ResearchGate publication, June-2019. 18.P.R. Vlachas, J. Pathak, B.R. Hunt, T.P. Sapsis, M. Girvan, E. Ott, and P. Koumoutsakos, Backpropagation algorithms and Reservoir Computing in Recurrent Neural Networks for the forecasting of complex spatiotemporal dynamics, Neural Networks (2020) 126, 191-217. 19.L. Lu, P. Jin, Pengzhan, P. Guofei, Z. Zhang, and G. E. Karniadakis, Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators, Nature Machine Intelligence, (2021) 3, 218–229. 20.C. Lin, Z. Li, L. Lu, S. Cai, M. Maxey, and G. E. Karniadakis, Operator learning for predicting multiscale bubble growth dynamics, The Journal of Chemical Physics, (2021) 154, 104118. 21.O. Ghattas and K. Willcox, Learning physics-based models from data: Perspectives from inverse problems and model reduction. Acta Numerica, (2021) 30, 445-554. 22.E. Qian, B. Kramer, B. Peherstorfer and Karen Willcox, Lift & Learn: Physics-informed machine learning for large-scale nonlinear dynamical systems, Physica D, (2021) 406, 132401. 23.S. A. McQuarrie, C. Huang and Karen E. Willcox, Data-driven reduced-order models via regularized operator inference for a single-injector combustion process, Journal of the Royal Society of New Zealand, (2021) 51, 194–211. 24.E. Qian, I-G. Farcas and Karen E. Willcox, Reduced operator inference for nonlinear partial differential equations, arXiv 2102.00083, 2021.