Foundations of computational materials science on Sibasankar Panigrahy

A history of interatomic potentials

Mon, 01 Jan 0001 00:00:00 +0000

The problem every simulation must solve

Every atomistic simulation rests on one function: the potential energy $E(\mathbf r_1, \dots, \mathbf r_N)$ of a set of atoms as a function of their positions. Its negative gradient gives the forces, the forces give the motion, and the motion, sampled over time, gives every property a simulation can predict, from the melting point of a metal to the binding free energy of a drug. The chapter on potential energy surfaces explains where this function comes from: in the Born–Oppenheimer approximation, it is the ground-state energy of the electrons for fixed nuclei, a solution of the Schrödinger equation.

Analysing trajectories

Mon, 01 Jan 0001 00:00:00 +0000

Averages, and why the error bar is the hard part

Statistical mechanics says an equilibrium property is an ensemble average, $\langle A\rangle = \int A(\mathbf x)\,p(\mathbf x)\,\mathrm d\mathbf x$ , over all microstates $\mathbf x$ weighted by their Boltzmann probability. A simulation replaces this with a time average over $N$ saved frames,

Classical mechanics and phase space

Mon, 01 Jan 0001 00:00:00 +0000

Why start with a ball on a hill

A molecular dynamics simulation of a protein or a crystal follows millions of atoms, but each atom obeys the same laws as a single ball rolling in a valley. If you understand one particle moving in one dimension, its position, velocity, momentum, force and energy, and the picture called phase space, you understand what every MD code computes, and why its results can be trusted. This chapter builds that picture.

Energy landscapes and transition states

Mon, 01 Jan 0001 00:00:00 +0000

The timescale gap

A molecular dynamics simulation advances in steps of about one femtosecond, $10^{-15}$ s, because it must resolve the fastest motions of atoms, bond vibrations with periods of 10 to 100 fs. With a good GPU code and a few hundred thousand atoms, a day of computing yields perhaps a few hundred nanoseconds. Many of the processes that matter in chemistry and materials science are vastly slower. A vacancy in a metal at room temperature hops once every microsecond to millisecond; a small protein folds in microseconds to seconds; a crystal nucleates from a supercooled liquid after seconds to years; a catalytic turnover takes milliseconds; ions in a battery cathode move between sites on timescales from nanoseconds to hours. Between the femtosecond time step and these events lie nine to fifteen orders of magnitude.

Entropy and temperature

Mon, 01 Jan 0001 00:00:00 +0000

The question statistical mechanics answers

A glass of water contains about $10^{25}$ molecules. Following every one is hopeless, and also unnecessary: we want the temperature, the pressure, the heat capacity, whether ice melts. Statistical mechanics connects the microscopic world of quantum levels and phase-space trajectories to those few macroscopic numbers. The whole subject rests on a single idea, which this chapter makes concrete by counting.

Free-energy perturbation and thermodynamic integration

Mon, 01 Jan 0001 00:00:00 +0000

The problem: a ratio of partition functions

Free energies are not averages of anything a simulation records. The free energy of a state is $-k_BT\ln Z$ , and the partition function $Z$ is an integral over all of configuration space, which a simulation never samples completely. What can be computed are free-energy differences between two states, A and B, defined by two potential energy functions $U_A$ and $U_B$ on the same configuration space, because their ratio of partition functions can be written as an average:

Functions and their graphs

Mon, 01 Jan 0001 00:00:00 +0000

Functions

Science is largely the study of how one quantity depends on another. The pressure of a gas depends on its temperature, the energy of two atoms depends on the distance between them, the number of undecayed nuclei in a sample depends on how long you have waited. The mathematical object that captures “depends on” is the function.

Generative models for crystals and molecules

Mon, 01 Jan 0001 00:00:00 +0000

From prediction to generation

Everything so far has been discriminative: given a structure, predict a property. Discovery asks the inverse question: which structures have the property I want? Screening a database answers it only among materials already listed. A generative model learns the probability distribution $p(\mathbf x)$ of known structures well enough to draw new samples from it, ideally conditioned on a target, $p(\mathbf x\mid y)$ .

How Python runs

Mon, 01 Jan 0001 00:00:00 +0000

The processor only understands machine code

A processor executes machine code: sequences of very simple binary instructions such as “load the 8 bytes at this memory address into register 3”, “add register 3 to register 5” or “if the last result was zero, continue at that address”. The chapter on computer architecture shows a toy processor doing exactly this. Every program that runs on a computer, whether it was written in Python, Fortran or JavaScript, must in the end be turned into such instructions, because nothing else can drive the hardware.

Inside a computer

Mon, 01 Jan 0001 00:00:00 +0000

A machine that follows instructions

Every simulation, every neural network and every plot you will make runs on the same kind of machine, and it is simpler than it looks. A computer does exactly one thing, billions of times a second: it reads an instruction from memory, carries it out, and moves on to the next one. Everything else (operating systems, Python, GROMACS, PyTorch) is built from that.

Kernel and linear potentials

Mon, 01 Jan 0001 00:00:00 +0000

Regression on fixed descriptors

The models in this chapter share a structure. Each atom’s environment is first turned into a fixed vector of numbers, a descriptor $\mathbf q_i$ , that respects the symmetries described in the chapter on what an MLIP must learn. The atomic energy is then a function of that descriptor, chosen from a family whose parameters can be fitted reliably: either a linear combination of basis functions, or a kernel expansion that compares the environment with stored reference environments. Both families have a decisive practical property: the prediction is linear in the parameters being fitted, so training is a linear least-squares problem with a unique solution, no random initialisation and no learning-rate schedule. Both also come with principled uncertainty estimates. Their limitation is that the descriptor is fixed in advance, so the model can be no better than the description of the environment it is given.

Learning from data

Mon, 01 Jan 0001 00:00:00 +0000

The question

You measured a rate constant at six temperatures and want the activation energy. You computed the adsorption energy of a molecule on twenty surfaces and want to predict it for a twenty-first. In both cases you have pairs $(x_i, y_i)$ and want a rule $f$ that turns a new $x$ into a good guess of $y$ . That is supervised learning, and almost everything in this course is a version of it: the rule gets more flexible (a line, a neural network, an interatomic potential), but the recipe stays the same.

Machine-learned collective variables

Mon, 01 Jan 0001 00:00:00 +0000

Learning the coordinates

Every collective-variable method depends on choosing the right variables, and the chapter on collective variables showed how a poor choice hides barriers. For simple processes, physical intuition suffices: a distance, a dihedral, a coordination number. For complex ones, such as the folding of a protein, the nucleation of a crystal or a reaction in a disordered solvent, the relevant combination of hundreds of possible coordinates is not obvious. Since about 2010, a family of methods has learned collective variables from simulation data instead. They differ in what they ask the variable to capture: the largest fluctuations, the difference between known states, the slowest dynamics, or the ability to predict the future.

Metadynamics

Mon, 01 Jan 0001 00:00:00 +0000

Filling the wells

A bias built from history

Umbrella sampling needs the bias to be designed in advance. A more adaptive idea is to let the simulation build its own bias from its history: wherever the system has already been, add a little repulsive potential, so that it is gradually pushed to explore somewhere new. Once the bias has filled the well the system started in, it spills over the barrier into the next well, fills that too, and eventually the whole free-energy landscape is levelled. The accumulated bias is then a mirror image of the free energy.

Monte Carlo simulation

Mon, 01 Jan 0001 00:00:00 +0000

Origins

The averages of statistical mechanics are integrals over all configurations, weighted by the Boltzmann factor, in a space of thousands of dimensions. The chapter on integrals showed why such integrals cannot be done on a grid and why random sampling, with an error falling as $1/\sqrt N$ whatever the dimension, is the only practical approach. The question is how to sample. Drawing configurations uniformly at random is useless: almost all of them have overlapping atoms and a Boltzmann weight of essentially zero. The samples must be drawn from the Boltzmann distribution itself.

NumPy arrays

Mon, 01 Jan 0001 00:00:00 +0000

Why NumPy exists

The Python data model is flexible, and the chapters on the language showed what that flexibility costs. A list of a million numbers is a million separate float objects scattered across memory, each 24 bytes with its own type pointer and reference count, plus a million 8-byte pointers in the list. Adding two such lists element by element runs a million iterations of the interpreter loop, each checking types, dispatching an addition and allocating a new object. For numerical work, that is hopelessly slow and wasteful.

Path sampling

Mon, 01 Jan 0001 00:00:00 +0000

Sampling paths instead of states

The methods of the previous part bias the dynamics along a chosen collective variable and then correct for the bias. They work well when a good collective variable is known. When it is not, when the mechanism itself is the question, a different philosophy is attractive: leave the dynamics unbiased and instead sample trajectories, concentrating effort on the rare trajectories that actually make a transition. Path-sampling methods do this. Because the trajectories they produce are true dynamical trajectories, they give rates and mechanisms directly, without assuming a reaction coordinate.

Potential energy surfaces and force fields

Mon, 01 Jan 0001 00:00:00 +0000

Atoms move on a landscape

Almost every atomistic simulation, classical or machine-learned, rests on one approximation. Nuclei are thousands of times heavier than electrons, so on the timescale of nuclear motion the electrons relax instantly to their ground state. This Born–Oppenheimer approximation separates the problem: for each fixed arrangement of nuclei $\mathbf R = (\mathbf r_1, \dots, \mathbf r_N)$ , solving the electronic Schrödinger equation gives one number, the energy $E(\mathbf R)$ . As the nuclei move, that number traces out a $3N$ -dimensional landscape, the potential energy surface (PES).

Probability and statistics

Mon, 01 Jan 0001 00:00:00 +0000

Chance and frequency

Toss a coin and you cannot say whether it will land heads. Toss it ten thousand times and you can say with great confidence that close to half the tosses will be heads. This combination, unpredictability in each case and regularity in the aggregate, is what probability theory describes, and it is everywhere in science. A single molecule in a gas moves erratically, but the pressure of $10^{23}$ molecules is steady. A single radioactive nucleus decays at an unpredictable moment, but a sample decays with a precise half-life. A single run of a stochastic training algorithm gives a slightly different network each time, but the distribution of results is reproducible.

The artificial neuron

Mon, 01 Jan 0001 00:00:00 +0000

Where the idea came from

In 1943 the neurophysiologist Warren McCulloch and the logician Walter Pitts asked a bold question: could a brain cell be described as a tiny logical device? A biological neuron receives signals from thousands of others through synapses, some exciting it and some inhibiting it; if the combined stimulation crosses a threshold, it fires a pulse down its axon. McCulloch and Pitts wrote this as arithmetic: add up the inputs, each multiplied by a strength, and output 1 if the total exceeds a threshold, 0 otherwise. In 1958 Frank Rosenblatt built a machine, the perceptron, that could learn those strengths from examples.

The Linux system

Mon, 01 Jan 0001 00:00:00 +0000

Why there is an operating system at all

The previous chapters showed a processor executing one program. A real computer runs hundreds at once: your shell, an editor, a simulation on 64 cores, a network daemon, someone else’s job. Each believes it has the machine to itself. Something must share out the processor time, keep the programs from overwriting each other’s memory, and give them a simple way to use disks, networks and GPUs without knowing how each device works. That something is the operating system, and its core, the part that is always running with full control of the hardware, is the kernel.

Training data and validation

Mon, 01 Jan 0001 00:00:00 +0000

The training set is the model

An MLIP knows only what its training data show it. Within the region of configuration space covered by the data it interpolates, and its errors are typically a few meV per atom; outside that region it extrapolates, and its predictions can be wrong by electronvolts. The architecture decides how efficiently the model learns from data, but the data decide what it can describe. Most of the effort in building a reliable potential, and most of the expertise, goes into the training set.

Vectors and matrices

Mon, 01 Jan 0001 00:00:00 +0000

Vectors

Some quantities are fully described by a single number: a temperature, a mass, an energy. These are scalars. Others need a direction as well as a size. A displacement of 3 m means nothing until you say which way; a force of 10 N pushes in some particular direction; the velocity of an atom has a speed and a heading. Such quantities are vectors.

Atomic neural-network potentials

Mon, 01 Jan 0001 00:00:00 +0000

The high-dimensional neural network potential

The neural-network potential of Jörg Behler and Michele Parrinello (2007) is the template from which most of the field grew, and it is worth understanding completely. Its three ingredients, introduced in the history and justified in the chapter on what an MLIP must learn, are the atomic decomposition of the energy, a symmetric description of each atom’s neighbourhood, and one feed-forward neural network per chemical element that maps that description to an atomic energy. Behler called it a high-dimensional neural network potential (HDNNP) because, unlike the networks of the 1990s, it describes systems with any number of atoms and therefore any number of dimensions.

Building a potential from scratch

Mon, 01 Jan 0001 00:00:00 +0000

What we will build

This chapter builds a complete machine-learned interatomic potential for copper, from raw data to molecular dynamics, using the ideas of the preceding chapters: the atomic decomposition and symmetry functions of neural-network potentials, the energy-and-force loss of what an MLIP must learn, and the validation strategy of training and validation. Every line of code below was run while writing this chapter, on a laptop CPU, and every number quoted is the actual output. The whole exercise takes about an hour of computer time.

Derivatives and Taylor series

Mon, 01 Jan 0001 00:00:00 +0000

Average rates of change

A car leaves at noon and has travelled 120 km by 2 pm. Its average speed was 60 km/h. That number is a ratio of two changes, the change in position divided by the change in time, and it says nothing about what happened in between: the car may have waited at traffic lights and then driven at 90 km/h on the motorway. The speedometer, by contrast, shows the speed at this instant. Making the idea of an instantaneous rate precise is the purpose of the derivative, and it is one of the most useful ideas in all of mathematics.

Differentiable simulation

Mon, 01 Jan 0001 00:00:00 +0000

A simulation is a function

A molecular dynamics run maps parameters to observables. Give it the Lennard-Jones $\sigma$ and $\varepsilon$ , partial charges, or the weights of a neural-network potential, and after many integration steps it returns a density, a radial distribution function, a diffusion coefficient. Usually we evaluate that function and compare it with experiment by hand. Differentiable simulation also computes its derivatives, $\partial\langle O\rangle/\partial\boldsymbol\theta$ , exactly and cheaply, so that parameters can be fitted to observables by gradient descent, just as network weights are fitted to labels (the gradient-descent lesson).

Generative models for equilibrium sampling

Mon, 01 Jan 0001 00:00:00 +0000

Sampling without dynamics

Why trajectories are slow

Every method in the previous chapters generates samples of the Boltzmann distribution $p(\mathbf x) \propto e^{-U(\mathbf x)/k_BT}$ by moving through configuration space one small step at a time. A molecular dynamics step changes positions by a fraction of an ångström; a Monte Carlo move is accepted only if it does not raise the energy too much. Consecutive samples are therefore strongly correlated, and the time needed to pass from one metastable state to another grows exponentially with the barrier between them, as the chapter on rates showed. Enhanced sampling methods attack this by biasing the dynamics, heating the system or pulling it along a collective variable, but they all keep the basic picture of a walker moving through the landscape.

Gradient descent

Mon, 01 Jan 0001 00:00:00 +0000

Why we need an iterative method

The straight line in the previous lesson had a closed-form answer because its loss is quadratic in the parameters. Most models do not: a nonlinear curve fit, a neural network or an interatomic potential has a loss $L(\boldsymbol\theta)$ with no formula for its minimum. What we can compute cheaply is the gradient $\nabla L(\boldsymbol\theta)$ , the vector of partial derivatives that points in the direction of steepest increase. Gradient descent takes repeated small steps the other way:

Gradients and the Hessian

Mon, 01 Jan 0001 00:00:00 +0000

Functions of several variables

The energy of a diatomic molecule depends on one number, the bond length. The energy of a water molecule depends on two bond lengths and an angle, and the energy of a protein on the positions of thousands of atoms. The loss of a neural network depends on every one of its weights, often millions of them. Almost every function that matters in simulation and machine learning has many inputs, and this chapter extends the calculus of derivatives to them.

Graph neural networks

Mon, 01 Jan 0001 00:00:00 +0000

From hand-made descriptors to learned ones

The descriptors of the previous lesson are designed by hand and then fixed. Graph neural networks (GNNs) keep the same symmetries but learn the description of each atom’s environment, end to end, from the training data. Since about 2017 they have become the dominant architecture for molecular and materials property prediction and for machine-learned interatomic potentials.

How a neuron learns

Mon, 01 Jan 0001 00:00:00 +0000

From choosing weights to learning them

In the previous chapter you set the weights by hand. With two inputs that is a game; with two hundred it is impossible. Learning means letting the data choose the weights. The recipe has three parts, and every network in this volume uses the same three:

Learning a potential energy surface

Mon, 01 Jan 0001 00:00:00 +0000

The quantity being learned

A machine-learned interatomic potential is a regression model. Its input is a set of atoms, each with a chemical element $Z_i$ and a position $\mathbf r_i$ , plus the periodic cell for a crystal. Its output is the potential energy $E$ , and, by differentiation, the forces on every atom and the stress on the cell. It is trained on examples whose energies and forces were computed by an electronic-structure method, and its purpose is to reproduce that method at a cost lower by a factor of a thousand to a million.

Markov state models

Mon, 01 Jan 0001 00:00:00 +0000

Long kinetics from short trajectories

Enhanced-sampling methods change the dynamics to obtain thermodynamics; path sampling targets particular transitions. Markov state models (MSMs) take a third approach that has become central in biomolecular simulation: run many ordinary, unbiased simulations, each much shorter than the processes of interest, and stitch their statistics together into a model of the long-time dynamics. Because the simulations can be run independently and in parallel, on many GPUs or on distributed computing projects such as Folding@home, MSMs reach timescales of milliseconds and beyond from simulations of microseconds.

Molecular dynamics

Mon, 01 Jan 0001 00:00:00 +0000

The idea

Molecular dynamics (MD) solves Newton’s equations of motion for every atom,

m_i\,\ddot{\mathbf r}_i = \mathbf F_i = -\nabla_{\mathbf r_i} E(\mathbf R), \qquad i = 1,\dots,N,

one small time step at a time, starting from some positions and velocities. The result is a trajectory, a movie of the atoms. Averages over the trajectory give thermodynamic and structural properties; correlations in time give transport properties. The method dates from Alder and Wainwright’s hard-sphere simulations (1957) and Rahman’s simulation of liquid argon (1964); today it runs routinely on millions of atoms.

Numbers in a computer

Mon, 01 Jan 0001 00:00:00 +0000

Everything is bits

Memory stores only two states per cell, written 0 and 1: a bit. Eight bits make a byte. Every number, letter, image and instruction is a pattern of bits, and the meaning of a pattern depends entirely on how a program agrees to read it. The same 32 bits can be an integer, a floating-point number, four letters of text or an instruction. Most of the numerical surprises in scientific computing come from forgetting how these patterns work.

Objects, types and names

Mon, 01 Jan 0001 00:00:00 +0000

Every value is an object

The previous chapter ended with a fact about CPython’s machinery: every value lives in memory as an object. This chapter builds the whole of Python’s data model on that fact, because nearly everything that confuses newcomers about the language, and a good deal that confuses experienced programmers, follows from it.

OPES and adaptive biasing force

Mon, 01 Jan 0001 00:00:00 +0000

Target distributions instead of hills

Well-tempered metadynamics, viewed from the end, drives the collective variable towards a definite target distribution, $P(s)^{1/\gamma}$ , by depositing hills until it gets there. Two later methods start from that target and ask directly what bias produces it. If the unbiased distribution of the collective variable is $P(s)$ and we want the system to sample a target $p_{\text{tg}}(s)$ , the required bias is

Quantum energy levels

Mon, 01 Jan 0001 00:00:00 +0000

Why a simulation chapter needs quantum mechanics

Molecular dynamics moves nuclei with Newton’s laws, as in the previous chapter. Yet the forces between atoms come from electrons, which obey quantum mechanics, and even the nuclei are quantum particles. This chapter answers two questions every simulator should be able to answer. Where does quantisation come from? And when is it safe to forget about it?

Reaction rates

Mon, 01 Jan 0001 00:00:00 +0000

Rates and waiting times

When a system sits in a basin A and occasionally escapes to basin B, and the escapes are rare compared with the time it takes to forget how it got into A, every moment spent in A carries the same small chance of escape. The process is then memoryless: the probability of still being in A after time $t$ decays exponentially, and the waiting times are exponentially distributed.

Scientific data

Mon, 01 Jan 0001 00:00:00 +0000

From raw output to a result

A typical computational study produces files: trajectories, energy logs, structure files, tables of computed properties. Turning those files into a number with an error bar, or a figure in a paper, follows the same sequence almost every time. The files are read and parsed into arrays; the arrays are cleaned and checked; a quantity is computed or fitted; its uncertainty is estimated; and the result is plotted and written out. Each step is easy on its own. Doing all of them correctly, in a way that can be rerun next month on new data and give the same answer for the same input, is the real skill, and Python’s scientific libraries make it straightforward. This chapter goes through the steps in order.

The Boltzmann distribution

Mon, 01 Jan 0001 00:00:00 +0000

From an isolated system to one at a temperature

The previous chapter treated an isolated system: fixed energy, every microstate equally likely. Real samples, and most simulations, are instead in contact with surroundings at a fixed temperature, which can lend and borrow energy. What is the probability of finding a small system in a particular microstate then? The answer, the Boltzmann distribution, is used in every corner of chemistry and materials science.

The Linux shell

Mon, 01 Jan 0001 00:00:00 +0000

Why the shell, in 2026

Nearly every scientific computer you will work on, from a cluster with ten thousand cores to a GPU workstation or a cloud machine, runs Linux and is reached through a text shell. Simulation codes are launched from it, job schedulers are driven from it, and the files they produce, often thousands of them, are far quicker to inspect, filter and move with a few commands than with any graphical tool. The shell also makes work reproducible: a command you typed can be saved in a script and run again next year, on another machine, with identical results.

Transport coefficients

Mon, 01 Jan 0001 00:00:00 +0000

From fluctuations to transport

Transport coefficients describe how a system responds to gradients: the diffusion coefficient relates a flux of particles to a concentration gradient (Fick’s law), the shear viscosity relates a momentum flux to a velocity gradient, the thermal conductivity relates a heat flux to a temperature gradient, and the electrical conductivity relates a current to an electric field. They can be measured in simulations by imposing a gradient and measuring the flux, but the fluctuation–dissipation theorem offers a more elegant route: in equilibrium, the spontaneous fluctuations of the fluxes already contain the transport coefficients. This chapter shows how, starting with the oldest example, diffusion.

Umbrella sampling

Mon, 01 Jan 0001 00:00:00 +0000

The idea: make the improbable probable

A free-energy profile $F(s) = -k_BT\ln P(s)$ along a collective variable can be computed from a histogram of $s$ , but only where the simulation goes. Across a barrier of 10 or 20 $k_BT$ , the probability density at the top is $e^{-10}$ to $e^{-20}$ times that in the well, so an unbiased simulation never samples it. In 1977 Glenn Torrie and John Valleau proposed adding a bias potential that makes the improbable regions probable, and then removing the effect of the bias analytically. They called the bias an umbrella because it spreads sampling over a region that the natural distribution would leave uncovered.

Attention and transformers

Mon, 01 Jan 0001 00:00:00 +0000

Attention as a soft lookup

In a sequence, a protein, a SMILES string, a sentence, the meaning of one element depends on others, sometimes far away. Recurrent networks pass information along the sequence one step at a time and struggle with long distances. Attention lets every element look directly at every other element and decide how much each matters.

Collective variables

Mon, 01 Jan 0001 00:00:00 +0000

Describing a transition with a few numbers

A transition in a system of thousands of atoms involves, in principle, all $3N$ coordinates. Yet chemists describe a reaction by a bond that forms and another that breaks, a biophysicist describes folding by how many native contacts have formed, and a materials scientist describes crystallisation by the size of the largest crystalline cluster. Each of these is a collective variable (CV): a function $\xi(\mathbf x)$ of the atomic coordinates, chosen to capture the progress of the process while ignoring the irrelevant details.

Control flow

Mon, 01 Jan 0001 00:00:00 +0000

Statements, blocks and indentation

A Python program is a sequence of statements executed one after another, top to bottom. Assignments, function calls and import are simple statements that occupy a line. Compound statements, such as if, while, for, def and try, control the execution of other statements: they consist of a header line ending in a colon, followed by an indented block of statements that belong to it.

Ensembles and free energy

Mon, 01 Jan 0001 00:00:00 +0000

Ensembles

An experiment never controls the microscopic state of a sample. It controls a few macroscopic quantities, such as the temperature, the pressure or the number of molecules, and the sample explores all the microstates compatible with those constraints. Statistical mechanics describes this exploration with an ensemble: an imagined collection of infinitely many copies of the system, each in a different microstate, with the copies distributed according to the probability of each microstate under the given conditions. The average of any quantity over the ensemble is what an experiment measures.

Generalisation and regularisation

Mon, 01 Jan 0001 00:00:00 +0000

Fitting is not the goal

A model is useful only if it predicts data it has not seen. With enough parameters any data set can be fitted exactly: a polynomial of degree $N-1$ passes through $N$ points, noise included. Such a model has memorised the training set rather than learned the relationship behind it. The difference between error on the training data and error on new data is the generalisation gap, and controlling it is the central practical problem of machine learning.

How the shell reads a command

Mon, 01 Jan 0001 00:00:00 +0000

The shell is a program like any other

bash is an ordinary user program (it lives at /bin/bash) whose whole job is a loop: print a prompt, read a line, work out what it means, start the programs it names, wait for them, and repeat. Understanding the middle step, working out what a line means, removes almost all of the mystery from the command line: why spaces in file names break scripts, why * sometimes stays a star, why cd cannot be a separate program, and why 2>&1 must come after >.

Integrals

Mon, 01 Jan 0001 00:00:00 +0000

Accumulation

The derivative answers the question “how fast is this changing?”. The integral answers the reverse question: “if I know how fast something changes at every moment, how much has accumulated in total?”. If a car travels at a constant 60 km/h for 2 hours, it covers $60 \times 2 = 120$ km. On a graph of speed against time, that product is the area of a rectangle 2 hours wide and 60 km/h tall. The units confirm it: (km/h) × h = km. Area under a rate curve is accumulated quantity.

Layers and representations

Mon, 01 Jan 0001 00:00:00 +0000

Many neurons side by side

A single neuron draws one straight cut (chapter on the neuron). Put several neurons side by side, all reading the same inputs, and you have a layer: each neuron computes its own weighted sum and activation, so a layer of $m$ neurons turns an input vector into an output vector of $m$ numbers. In matrix form, a whole layer is one line:

Message-passing and equivariant potentials

Mon, 01 Jan 0001 00:00:00 +0000

From fixed descriptors to learned representations

The potentials of the previous two chapters describe each atom with a descriptor chosen before training. Choosing it well requires expertise, its resolution is fixed, and for many elements it becomes very long. Deep learning’s central lesson, learned on images and text, is that representations learned from data for the task at hand usually beat hand-designed ones. Message-passing neural networks bring this lesson to atoms. A structure is treated as a graph whose nodes are atoms and whose edges connect atoms closer than a cutoff; every atom carries a feature vector that starts as a learned embedding of its element and is refined, layer by layer, by exchanging messages with its neighbours. The final features are mapped to atomic energies and summed, so the atomic decomposition, locality within a finite range, and forces by differentiation all survive.

Performance

Mon, 01 Jan 0001 00:00:00 +0000

Measure before optimising

Most code is fast enough, and most of the time spent in slow code is spent in a small part of it. Donald Knuth’s warning that premature optimisation is the root of much evil is about exactly this: effort spent speeding up code that does not dominate the run time is wasted, and usually makes the code harder to read. The first step is therefore always to measure, both to find where the time goes and to know whether a change helped.

Replica exchange and tempering

Mon, 01 Jan 0001 00:00:00 +0000

Tempering

Heating without losing the low-temperature ensemble

The collective-variable methods of the previous chapters accelerate sampling along chosen coordinates. A blunter but more general way to cross barriers is to raise the temperature: at high $T$ , barriers that are many $k_BT$ high at room temperature become small, and the system explores freely. The difficulty is that the high-temperature ensemble is not the one we want. The idea of tempering is to run at high temperature to explore, and at low temperature to sample correctly, letting configurations move between the two in a way that preserves the correct distribution at every temperature.

Symmetry and atomic descriptors

Mon, 01 Jan 0001 00:00:00 +0000

The input problem

A neural network takes a vector of numbers. A configuration of atoms is a list of positions $\mathbf r_i$ and element types $Z_i$ . The obvious encoding, concatenating the Cartesian coordinates $(x_1, y_1, z_1, x_2, \dots)$ , is a bad one, for a reason that has nothing to do with machine learning: the energy of a molecule does not change when you move it, turn it round or relabel its atoms, but the coordinate vector changes completely. A model fed raw coordinates would have to learn, from data, that millions of different input vectors describe the same physical object. With finite data it never quite does.

Thermostats and barostats

Mon, 01 Jan 0001 00:00:00 +0000

Why control temperature

Newton’s equations conserve energy, so plain molecular dynamics samples the microcanonical (NVE) ensemble: constant number of particles, volume and energy. Experiments are almost always done at constant temperature, and usually at constant pressure. Simulations also need temperature control for practical reasons: to equilibrate a system prepared far from equilibrium, to remove the heat produced by a driving force (as in shear or current flow), and to compensate for the slow energy drift caused by numerical integration errors over long runs.

Universal potentials

Mon, 01 Jan 0001 00:00:00 +0000

From one material to the periodic table

Until about 2021, every machine-learned potential was a specialist. A potential for silicon knew nothing about oxygen; a potential for water could not describe a salt solution unless salt had been in its training set. Building one took months of DFT calculations, training and validation, and the result applied to one chemical system. This made MLIPs powerful tools for groups studying a particular material in depth, but useless for the broad questions of materials discovery: which of ten thousand candidate compounds are stable, which conduct lithium ions quickly, which might catalyse a reaction.

Active learning

Mon, 01 Jan 0001 00:00:00 +0000

The question every lab faces

You can afford twenty more syntheses, or two hundred more DFT calculations. Which ones? Choosing by intuition, on a grid or at random wastes most of the budget. Sequential experimental design uses everything measured so far to choose the next experiment, so that each one is as informative or as promising as possible. Two closely related goals use the same machinery.

Backpropagation

Mon, 01 Jan 0001 00:00:00 +0000

The credit-assignment problem

For one neuron, the gradient was a single neat formula. A network has layers of neurons, and a weight deep inside it affects the loss only through everything downstream of it. If the prediction is wrong, how much is each of a million weights to blame, and in which direction should each move? This is the credit-assignment problem, and its solution, backpropagation, is the algorithm that made deep learning possible.

Fluctuations and response

Mon, 01 Jan 0001 00:00:00 +0000

Averages are not the whole story

Thermodynamics treats the energy of a beaker of water in a thermostat, or the number of molecules in a cubic millimetre of air, as definite numbers. Statistical mechanics says they are not. A system in contact with a heat bath constantly exchanges energy with it, so its energy fluctuates about its average; a region of a gas constantly exchanges molecules with its surroundings, so the number inside it fluctuates too. For a macroscopic sample these fluctuations are far too small to notice, which is why thermodynamics works. For the systems studied in molecular simulation, a few thousand atoms, and for the nanoparticles, single molecules and small droplets of modern experiments, they are large enough to see and to matter.

Fortran

Mon, 01 Jan 0001 00:00:00 +0000

Why learn Fortran now?

Fortran (FORmula TRANslation, 1957) is the oldest language still in heavy scientific use, and for a good reason: it was designed for one job, fast numerical work on arrays, and compilers have had seventy years to get good at it. Much of the code a computational materials scientist runs every day is written in it: VASP, Quantum ESPRESSO, CP2K, CASTEP, SIESTA and ABINIT for electronic structure, and many weather, climate and engineering codes. Reading, modifying and compiling Fortran is part of the job, even if your own analysis is in Python.

Functions and modules

Mon, 01 Jan 0001 00:00:00 +0000

Why functions

A function packages a computation under a name so that it can be used without thinking about how it works. A function that returns the Lennard-Jones energy of two atoms can be called a million times inside a simulation loop, tested once on its own, and replaced by a faster or more accurate version without touching the code that calls it. Functions are the main tool for controlling complexity in a program, in the same way that lemmas control complexity in a proof: each one does one clearly stated job, and larger results are built from them.

Kernels and Gaussian processes

Mon, 01 Jan 0001 00:00:00 +0000

Why uncertainty is not optional

A model predicts that a new alloy has a formation energy of −0.21 eV/atom. Should you synthesise it? That depends entirely on whether the prediction is good to ±0.01 or ±0.3 eV/atom, and on whether the alloy resembles anything in the training data. Scientific uses of machine learning, such as choosing the next experiment, flagging an MD configuration the potential has never seen, or deciding when to stop a screening campaign, all need the model to say how sure it is.

Linux for simulation work

Mon, 01 Jan 0001 00:00:00 +0000

How a cluster is organised

A high-performance computing (HPC) cluster is many computers (nodes) connected by a fast network and sharing file systems. You log in to a login node, which is shared by everyone and meant for editing, compiling and submitting work. The calculations run on compute nodes, which you reach only through the scheduler (usually Slurm, sometimes PBS or LSF): you describe the resources you need in a job script, submit it, and the scheduler starts it when the resources are free.

Representing materials

Mon, 01 Jan 0001 00:00:00 +0000

What is special about crystals

A molecule is a finite set of atoms. A crystal is infinite: a lattice of translations $\mathbf T = n_1\mathbf a_1 + n_2\mathbf a_2 + n_3\mathbf a_3$ (integers $n_k$ , lattice vectors $\mathbf a_k$ ) repeats a basis of atoms at fractional coordinates $\mathbf f_j$ , so that every atom sits at $\mathbf r = \sum_k (f_{jk} + n_k)\mathbf a_k$ . A crystal structure file (CIF, POSCAR) stores just one unit cell: the three lattice vectors, the species and fractional coordinates of the basis, and, implicitly, the rule that it repeats forever.

Collections

Mon, 01 Jan 0001 00:00:00 +0000

Containers and the cost of operations

Most programs spend their time manipulating collections of things: the atoms of a structure, the frames of a trajectory, the lines of a file, the energies of a set of candidate materials. Python has four general-purpose built-in containers: the list, the tuple, the dictionary and the set. Each is good at some operations and poor at others, and choosing well can make the difference between a script that runs in a second and one that runs for an hour.

Phase transitions and the Ising model

Mon, 01 Jan 0001 00:00:00 +0000

Phases and order parameters

Water freezes at 0 °C, iron loses its magnetism above 770 °C, and the copper and zinc atoms of β-brass, arranged in an ordered alternating pattern at room temperature, mix randomly on the same lattice above about 460 °C. These are phase transitions: changes in the collective organisation of matter that happen abruptly as a control parameter, usually temperature or pressure, passes a definite value. They are the most dramatic consequence of the competition between energy and entropy described in the chapter on free energy. Below the transition, the ordered arrangement has the lower free energy because its energy is lower; above it, the disordered arrangement wins because its entropy, multiplied by the temperature, outweighs the energy cost.

Training neural networks

Mon, 01 Jan 0001 00:00:00 +0000

From fixed features to learned features

A linear model predicts with fixed features: $\hat y = \sum_k w_k\,\phi_k(x)$ . Someone has to choose the $\phi_k$ , and for a complex relationship (the energy of a configuration of 200 atoms, say) nobody knows the right ones. A neural network keeps the same form but makes the features themselves adjustable:

Classes

Mon, 01 Jan 0001 00:00:00 +0000

Types of your own

Every object belongs to a type, and the built-in types, int, str, list, dict and the rest, cover a great deal. But the things a scientific program reasons about are atoms, molecules, crystal structures, potentials, trajectories and simulations. A class lets you define a new type whose objects bundle together the data describing such a thing (its attributes) and the operations that make sense on it (its methods). An Atom object might carry an element symbol, a mass and a position; a Potential object might know how to compute an energy and a force. Code that uses them can then speak the language of the problem, atom.mass and potential.force(r), rather than juggling parallel lists and remembering that index 3 means the mass.

The structure of liquids

Mon, 01 Jan 0001 00:00:00 +0000

Order without a lattice

A crystal is easy to describe: give the lattice and the contents of one unit cell, and every atomic position follows. A gas is easy too: the atoms are almost independent. Liquids, glasses, amorphous solids and dense fluids lie in between. They have no long-range order, yet they are far from random: each atom is surrounded by a well-defined first shell of neighbours at nearly fixed distance, a less sharply defined second shell, and so on, with order fading over a few molecular diameters. The theory of liquids, developed from the 1930s by John Kirkwood, Joseph Mayer, Jean-Pierre Hansen, John Barker, Douglas Henderson and many others, and tested at every stage by computer simulation, describes this statistical order with a few correlation functions. The most important is the radial distribution function.