Numerical Methods using Python

This set of tutorials are written at an introductory level for an engineering or physical sciences major. It is ideal for someone who has completed college level courses in linear algebra, calculus and differential equations. While prior experience with programming is certainly an advantage, it is not expected. At UC Davis, this is aimed at sophomore level Chemical and Biochemical Engineers and Materials Scientists: examples and the language used here might reflect this. At the same time, this is not meant to be an exhaustive course in Python or in numerical methods. The focus is on introducing the mathematical techniques and developing an insight for scientific computation, independent of programming language.

Interactive tutorials using the Jupyter framework are an engaging alternative to learning numerical methods from a static textbook. Yet I was unable to find a set of pedagogic and interactive code notebooks that covered the range of topics suitable for this level of instruction. I have hoped to fill this gap. These notebooks were developed and tested using the Anaconda distribution.

Introduction

• 0.1 Context and Scope
• 0.2 Getting Used to Python
  • 0.2.1 Installing and Using Python
  • 0.2.2 Useful Tips

1. Basic Operations and Control

• 1.1 Data Types and Basic Operations
  • 1.1.1 Integers
  • 1.1.2 Floats
  • 1.1.3 String
  • 1.1.4 Tuples
  • 1.1.5 Lists
  • 1.1.6 Numbers and Math: Numpy
• 1.2 User-defined Functions
  • 1.2.1 Defining Functions
  • 1.2.2 Single-line lambda function
• 1.3 Logic and Repetition
  • 1.3.1 The IF condition
  • 1.3.2 The FOR and WHILE loops
• 1.4 Vectorization: Manipulating Scientific Data
  • 1.4.1 Introduction to Arrays
  • 1.4.2 Sequences
  • 1.4.3 Slicing
  • 1.4.4 Vectorizing
• 1.5 Formatting and Storing Data
  • 1.5.1 Formatting Output
  • 1.5.2 Storing Arrays
• Practice Problems

2. Plotting and Visualization

• 2.1 Plotting 1D Data Sets
  • 2.1.1 Basic Plotting: The pyplot Interface
  • 2.1.2 Multiple graphics: Subplots
  • 2.1.3 Object-oriented plotting
  • 2.1.4 Scatter plots
• 2.2 Multi-dimensional Data
  • 2.2.1 2D data: Pseudocolors and contours
  • 2.2.2 3D visualization: Surface plotting
• Practice Problems

3. Statistics and Data Analysis

• 3.1 Elementary Statistics
  • 3.1.1 Measures of Spread
  • 3.1.2 Distributions and histograms
• 3.2 Pandas: a powerful data platform
  • 3.2.1 Introduction to data frames
  • 3.2.2 A real example: the data science of lead in lipsticks
• Practice Problems

4. Systems of Linear Equations

• 4.1 Elimination Methods
  • 4.1.1 Linear Systems and Computational Cost
  • 4.1.2 Gauss Elimination
  • 4.1.3 Pivoting: Choosing the Right Row
  • 4.1.4 Gauss-Jordan: Gauss on steroids!
  • 4.1.5 LU Decomposition
  • 4.1.6 Computational Cost Revisited
• 4.2 Inbuilt Python Routines
  • 4.2.1 Solving Linear Systems: linalg.solve
  • 4.2.2 Fast and Easy Linear Algebran
• Practice Problems

5. Fitting and Interpolation

• 5.1 Line and Curve Fitting
  • 5.1.1 Least-squares linear fit
  • 5.1.2 Goodness of fit
  • 5.1.3 Nonlinear fits using linear regression
  • 5.1.4 Polynomial Fits
• 5.2 Interpolation
  • 5.2.1 Lagrange Interpolation
  • 5.2.2 Splines
• Practice Problems

6. Solutions of Nonlinear Equations

• 6.1 Numerical Root Finding
  • 6.1.1 Exact and Numerical Roots
  • 6.1.2 Errors and Tolerance
• 6.2 Nonlinear equations in one variable
  • 6.2.1 Bisection Method
  • 6.2.2 Regula Falsi Method
  • 6.2.3 Newton’s Method
  • 6.2.4 Halley’s and higher-order methods
• 6.3 More general root-finding
  • 6.3.1 Equations with multiple solutions
  • 6.3.2 Systems of nonlinear equations
  • 6.3.3 Inbuilt Python routines
• Practice Problems

7. Filtering Data and Images

• 7.1 Convolutions
  • 7.1.1 Moving averages as low-pass filters
  • 7.1.2 Matrix convolutions
• 7.2 Image Processing
  • 7.2.1 Image matrices and color schemes
  • 7.2.2 Image filters: box blur and Gaussian blur
  • 7.2.3 Edge detection
• Practice Problems

8. Numerical Differentiation

• 8.1 Finite Differences
  • 8.1.1 Derivatives from discrete points
  • 8.1.2 Finite differences from Taylor series expansions
  • 8.1.3 End corrections
  • 8.1.4 Higher derivatives and higher-order methods
• 8.2 Numerical Partial Differentiation
  • 8.2.1 Partial finite differences
  • 8.2.2 An application: heat conduction
  • 8.2.3 Inbuilt Python routines
• Practice Problems

9. Numerical Integration

• 9.1 Integrals as Finite Sums
  • 9.1.1 Rectangle methods or Riemann sums
  • 9.1.1 Taylor series expansions and truncation errors
  • 9.1.1 Trapezoidal rule
  • 9.1.1 Simpson’s rule
• 9.2 More Integration Strategiess
  • 9.2.1 Multidimensional integrals
  • 9.2.1 Inbuilt Python routines
• Practice Problems

10. Ordinary Differential Equations: Initial Value Problems

• 10.1 Ordinary Differential Equations
  • 10.1.1 First Order Initial Value Problems
  • 10.1.2 Euler’s method: explicit versus implicit
  • 10.1.3 Multi-stage solutions: the Runge-Kutta methods
• 10.2 Coupled and Higher-order ODEs
  • 10.2.1 Systems of first-order ODEs: predators and prey
  • 10.2.2 Higher-order initial value problems
  • 10.2.3 Inbuilt python routines
• Practice Problems

11. Ordinary Differential Equations: Boundary Value Problems

• 11.1 Linear Boundary Value Problems
  • 11.1.1 Shooting method
  • 11.1.2 Finite difference method

• 11.2 Inbuilt Python Function

• 11.1 Extending to Partial Differential Equations

• Practice Problems

License Requirements. This repository is maintained on GitHub at hmanikantan/ECH60 and published under an MIT license. This means you are free to use, copy, modify and adapt any part of this module for non-commercial purposes. The license terms require you to give attribution and share your work under the same terms. I welcome feedback of any kind, including pull requests for corrections and additions.

Acknowledgements. My ECH 60 students beta tested these tutorials. Their learning styles, feedback and comments crafted the structure of this series. The world of Python is a fantastic testament to the power of open-source science and learning. I thank the countless selfless nameless strangers whose stackoverflow comments have informed me, and whose coding styles have inadvertently crept into these modules. And I thank the generous online notes of John Kitchin, Patrick Walls, Charles Jekel, Jeffrey Kantor, and Lorena Barba whose works directly or indirectly inspired and influenced this project. I am happy to contribute to this collective knowledge base, free for anyone to adapt, build on, and make their own.