Seismology is a science that aims at providing tomographic images of the Earth’s interior, similar to X-ray images of the human body. These images can be used as snapshots of the current state of flow patterns inside the mantle. The main way we communicate, from tomographer to geodynamicist, is through publication of some tomographic image. We seismologists, however, make countless choices, approximations and assumptions, which are limited by poor data coverage, and ultimately never fit our data perfectly. These things are often overlooked, or taken for granted and poorly communicated. Inevitably, this undermines the rigour and usefulness of subsequent interpretations in terms of heat or material properties. This post will give an overview of what can worry a seismologist/tomographer. Our goal is not to teach seismic tomography, but to plant a seed that will make geodynamicists push seismologists for better accuracy, robustness, and communicated uncertainty!

A typical day in a seismologist’s life starts with downloading some data for a specific application. Then we cry while looking at waveforms that make no sense (compared to the clean and physically meaningful synthetics calculated the day before). After a sip, or two, or two thousand sips of freshly brewed coffee, and some pre-processing steps to clean up the mess that is real data, the seismologist sets up a measurement of the misfit between synthetics and observed waveforms. Do we try to simulate the entire seismogram, just its travel time, its amplitude? The choice we make in defining this misfit can non-linearly affect our outcome, and there’s no clear way to quantify that uncertainty.

After obtaining the misfit measurements, the seismologist starts thinking about best inversion practices in order to derive some model parameters. There are two more factors to consider now: how to mathematically find a solution that fits our data, and the choice of how to choose a subjectively unique solution from the many solutions of the problem… The number of (quasi-)arbitrary choices can increase dramatically in the course of the poor seismologist’s day!

The goal is to image seismic anomalies; to present a velocity model that is somehow different from the assumed background. After that, the seismologist can go home, relax and write a paper about what the model shows in geological terms. Or… More questions arise and doubts come flooding in. *Are the choices I made sensible? Should I make a calculation of the errors associated with my model?* Thermodynamics gives us the basic equations to translate seismic to thermal anomalies in the Earth but how can we improve the estimated velocity model for a more realistic interpretation?

Figure 1 is one such example of a velocity model, constructed through seismic tomography (specifically from ambient-noise surface waves). The paper reviews the tectonic history of the crust and upper mantle in this offshore region. We are proud of this model, and sincerely hope it can be of use to those studying tectonics or dynamics. We are also painfully aware of the assumptions that we had to make, however. This picture could look drastically different if we had used a different amount of regularization (smoothing), had made different prior assumptions about where layers may be, had been more or less restrictive in cleaning our raw data observations, or made any number of other changes. We were careful in all these regards, and ran test after test over the course of several months to ensure the process was up to high standards, but for the most part… you just have to take our word for it.

There’s a number of features we interpret here: thinning of the crust, upwelling asthenosphere, the formation of volcanic seamounts, etc. But it wouldn’t shock me if some other study came out in the coming years that told an entirely different story; indeed that’s part of our process as scientists to continue to challenge and test hypotheses. But what if this model is used as an input to something else as-of-yet unconstrained? In this model, could the Lithosphere-Asthenosphere Boundary (LAB) shown here be 10 km higher or deeper, and why does it disappear at 200km along the profile? Couldn’t that impact geodynamicists’ work dramatically? Our field is a collaborative effort, but if we as seismologists can’t properly quantify the uncertainties in our pretty, colourful models, what kind of effect might we be having on the field of geodynamics?

Another example comes from global scale models. Taking a look at figures 6 and 7 in Meier et al. 2009, ”Global variations of temperature and water content in the mantle transition zone from higher mode surface waves” (DOI:10.1016/j.epsl.2009.03.004), you can observe global discontinuity models and you are invited to notice their differences. Some major features keep appearing in all of them, which is encouraging since it shows that we may be indeed looking at some real properties of the mantle. However, even similar methodologies have not often converged to same tomographic images. The sources of discrepancies are the usual plagues in seismic tomography, some of them mentioned on top.

In an effort to improve imaging of mantle discontinuities, especially those at 410 and 660 km depths which are highly relevant to geodynamics (I’ve been told…), we have put some effort into building up a different approach. Usually, traveltime tomography and one-step interpretation of body wave traveltimes have been the default for producing images of mantle transition zone. We proposed an iterative optimisation of a pre-existing model, that includes flat discontinuities, using traveltimes in a full-waveform inversion scheme (see figure 2). The goal was to see whether we can get the topography of the discontinuities out using the new approach. This method seems to perform very well and it gives the potential for higher resolution imaging. Are my models capable of resolving mineralogical transitions and thermal variations along the depths of 410 and 660 km?

The most desired outcome would be not only a model that represents Earth parameters realistically but also one that provides error bars, which essentially quantify uncertainties. Providing error bars, however, requires extra computational work, and as every pixel-obsessed seismologist, we would be curious to know the extent to which uncertainties are useful to a numerical modeller! Our main question, then, remains: how can we build an interdisciplinary approach that can justify large amounts of burnt computational power?

As (computational) seismologists we pose questions for our regional or global models: Are velocity anomalies good enough, intuitively coloured as blue and red blobs and representative of heat and mass transfer in the Earth, or is it essential that we determine their shapes and sizes with greater detail? Determining a range of values for the derived seismic parameters (instead of a single estimation) could allow geodynamicists to take into account different scenarios of complex thermal and compositional patterns. We hope that this short article gave some insight into the questions a seismologist faces each time they derive a tomographic model. The resolution of seismic models is always a point of vigorous discussions but it could also be a great platform for interaction between seismologists and geodynamicists, so let’s do it!

*For an overview of tomographic methodologies the reader is referred to Q. Liu & Y. J. Gu, Seismic imaging: From classical to adjoint tomography, 2012, Tectonophysics. https://doi.org/10.1016/j.tecto.2012.07.006*

Tomography… wait, isn’t that what happens in your CAT scan? Although the general public might associate tomography with medical imaging, Earth scientists are well aware that ‘seismic tomography’ has enabled us to peer deeper, and with more clarity, into the Earth’s interior (Fig. 1). What are some of the ways we can download and display tomography to inform our scientific discoveries? Why has seismic tomography been a valuable tool for plate reconstructions? And what are some new approaches for incorporating seismic tomography within plate tectonic models?

**Downloading and displaying seismic tomography**

Seismic tomography is a technique for imaging the Earth’s interior in 3-D using seismic waves. For complete beginners, IRIS (Incorporated Research Institutions for Seismology) has an excellent introduction that compares seismic tomography to medical CT scans.

A dizzying number of new, high quality seismic tomographic models are being published every year. For example, the IRIS EMC-EarthModels catalogue currently contains 64 diverse tomographic models that cover most of the Earth, from global to regional scales. From my personal count, at least seven of these models have been added in the past half year – about one new model a month. Aside from the IRIS catalog, a plethora of other tomographic models are also publicly-available from journal data suppositories, personal webpages, or by an e-mail request to the author.

Downloading a tomographic model is just the first step. If one does not have access to custom workflows and scripts to display tomography, consider visiting an online tomography viewer. I have listed a few of these websites at the end of this blog post. Of these websites, a personal favourite of mine is the Hades Underworld Explorer built by Douwe van Hinsbergen and colleagues at Utrecht University, which uses a familiar Google Maps user interface. By simply dragging a left and right pin on the map, a user can display a global tomographic section in real time. The displayed tomographic section can be displayed in either a polar or Cartesian view and exported to a .svg file. Another tool I have found useful are tomographic ‘vote maps’, which provide indications of lower mantle slab imaging robustness by comparison of multiple tomographic models (Shephard et al., 2017). Vote maps can be downloaded from the original paper above or from the SubMachine website (Hosseini et al. (2018); see more in the website list below).

**Using tomography for plate tectonic reconstructions**

Tomography has played an increasing role in plate tectonic studies over the past decades. A major reason is because classical plate tectonic inputs (e.g. seafloor magnetic anomalies, palaeomagnetism, magmatism, geology) are independent from the seismological inputs for tomographic images. This means that tomography can be used to augment or test classic plate reconstructions in a relatively independent fashion. For example, classical plate tectonic models can be tested by searching tomography for slab-like anomalies below or near predicted subduction zone locations. These ‘conventional’ plate modelling workflows have challenges at convergent margins, however, when the geological record has been significantly destroyed from subduction. In these cases, the plate modeller is forced to describe details of past plate kinematics using an overly sparse geological record.

**A ‘tomographic plate modelling’ workflow** (Fig. 2) was proposed by Wu et al. (2016) that essentially reversed the conventional plate modelling workflow. In this method, slabs are mapped from tomography and unfolded (i.e. retro-deformed) (Fig. 2a). The unfolded slabs are then populated into a seafloor spreading-based global plate model. Plate motions are assigned in a hierarchical fashion depending on available kinematic constraints (Fig. 2b). The plate modelling will result in either a single unique plate reconstruction, or several families of possible plate models (Fig. 2c). The final plate models (Fig. 2c) are fully-kinematic and make testable geological predictions for magmatic histories, palaeolatitudes and other geological events (e.g. collisions). These predictions can then be systematically compared against remnant geology (Fig. 2d), which are independent from the tomographic inputs (Fig. 2a).

The proposed** 3D slab mapping workflow **of Wu et al. (2016) assumed that the most robust feature of tomographic slabs is likely the slab center. The slab mapping workflow involved manual picking of a mid-slab ‘curve’ along hundreds (and sometimes thousands!) of variably oriented 2D cross-sections using software GOCAD (Figs. 3a, b). A 3-D triangulated mid-slab surface is then constructed from the mid-slab curves (Fig. 3c). Inspired by 3D seismic interpretation techniques from petroleum geoscience, the tomographic velocities can be extracted along the mid-slab surface for further tectonic analysis (Fig. 3d).

For relatively undeformed upper mantle slabs, a pre-subduction slab size and shape can be estimated by unfolding the mid-slab surface to a spherical Earth model, minimizing distortions and changes to surface area (Fig. 3e). Interestingly, the slab unfolding algorithm can also be applied to shoe design, where there is a need to flatten shoe materials to build cut patterns (Bennis et al., 1991). The three-dimensional slab mapping within GOCAD allows a self-consistent 3-D Earth model of the mapped slabs to be developed and maintained. This had advantages for East Asia (Wu et al., 2016), where many slabs have apparently subducted in close proximity to each other (Fig. 1).

**Web resources for displaying tomography**

Hades Underworld Explorer : http://www.atlas-of-the-underworld.org/hades-underworld-explorer/

Seismic Tomography Globe : http://dagik.org/misc/gst/user-guide/index.html

SubMachine : https://www.earth.ox.ac.uk/~smachine/cgi/index.php

]]>ReferencesBennis, C., Vezien, J.-M., Iglesias, G., 1991. Piecewise surface flattening for non-distorted texture mapping. Proceedings of the 18th annual conference on Computer graphics and interactive techniques 25, 237-246. Hosseini, K. , Matthews, K. J., Sigloch, K. , Shephard, G. E., Domeier, M. and Tsekhmistrenko, M., 2018. SubMachine: Web-Based tools for exploring seismic tomography and other models of Earth's deep interior. Geochemistry, Geophysics, Geosystems, 19. Li, C., van der Hilst, R.D., Engdahl, E.R., Burdick, S., 2008. A new global model for P wave speed variations in Earth's mantle. Geochemistry, Geophysics, Geosystems 9, Q05018. Shephard, G.E., Matthews, K.J., Hosseini, K., Domeier, M., 2017. On the consistency of seismically imaged lower mantle slabs. Scientific Reports 7, 10976. Wu, J., Suppe, J., 2018. Proto-South China Sea Plate Tectonics Using Subducted Slab Constraints from Tomography. Journal of Earth Science 29, 1304-1318. Wu, J., Suppe, J., Lu, R., Kanda, R., 2016. Philippine Sea and East Asian plate tectonics since 52 Ma constrained by new subducted slab reconstruction methods. Journal of Geophysical Research: Solid Earth 121, 4670-4741

One integral part of doing estimations on parameters is an uncertainty analysis. The aim of a general inverse problem is to find the value of a parameter, but it is often very helpful to indicate the measure of certainty. For example in the last figure of my previous post, the measurement values at the surface are more strongly correlated to the upper most blocks. Therefore, the result of an inversion in this set up will most likely be more accurate for these parameters, compared to the bottom blocks.

In linear deterministic inversion, the eigenvalues of the matrix system provide an indication of the resolvability of parameters (as discussed in the aforementioned work by Andrew Curtis). There are classes of methods to compute exact parameter uncertainty in the solution.

From what I know, for non-linear models, uncertainty analysis is limited to the computation of second derivatives of the misfit functional in parameter space. The second derivatives of X (the misfit function) are directly related to the standard deviations of the parameters. Thus, by computing all the second derivatives of X, a non-linear inverse problem can still be interrogated for its uncertainty. However, the problem with this is its linearisation; linearising the model and computing derivatives may not be truly how the model reacts in model space. Also, for strongly non-linear models many trade-offs (correlations) exist which influence the final solution, and these correlations may very strongly depending on the model to be inverted.

Enter reverend Thomas Bayes

This part-time mathematician (he only ever published one mathematical work) from the 18th century formulated the Bayes’ Theorem for the first time, which combines knowledge on parameters. The mathematics behind it can be easily retrieved from our most beloved/hated Wikipedia, so I can avoid getting to caught up in it. What is important is that it allows us to combine two misfit functions or probabilities. Misfits and probabilities are directly interchangeable; a high probability of a model fitting our observations corresponds to a low misfit (and there are actual formulas linking the two). Combining two misfits allows us to accurately combine our pre-existing (or commonly: prior) knowledge on the Earth with the results of an experiment. The benefits of this are two-fold: we can use arbitrarily complex prior knowledge and by using prior knowledge that is *bounded* (in parameter space) we can still invert underdetermined problems without extra regularisation. In fact, the prior knowledge *acts* as regularisation.

Let’s give our friend Bayes a shot at our non-linear 1D bread. We have to come up with our prior knowledge of the bread, and because we did not need that before I’m just going to conjure something up! We suddenly find the remains of a packaging of 500 grams of flour

This is turning in quite the detective story!

However, the kitchen counter that has been worked on is also royally covered in flour. Therefore, we estimate that probably this pack was used; about 400 grams of it, with an uncertainty (standard deviation) of 25 grams. Mathematically we can formulate our prior knowledge as a Gaussian distribution with the aforementioned standard deviation and combine this with our misfit of the inverse problem (often called the likelihood). The result is given here:

One success and one failure!

First, we successfully combined the two pieces of information to make an inverse problem that is no longer non-unique (which was a happy coincidence of the prior: it is not guaranteed). However, we failed to make the problem more tractable in terms of computational requirements. To get the result of our combined misfit, we still have to do a systematic grid search, or at least arrive at a (local) minimum using gradient descent methods.

We can do the same in 2D. We combine our likelihood (original inverse problem) with rather exotic prior information, an annulus in model space, to illustrate the power of Bayes’ theorem. The used misfit functions and results are shown here:

This might also illustrate the need for non-linear uncertainty analysis. Trade-offs at the maxima in model space (last figure, at the intersection of the circle and lines) distinctly show two correlation directions, which might not be fully accounted for by using only second derivative approximations.

Despite this ‘non-progress’ of still requiring a grid search even after applying probability theory, we can go one step further by combining the application of Bayesian inference with the expertise of other fields in *appraising* inference problems…

Up to now, using a probabilistic (Bayesian) approach has only (apparently) made our life more difficult! Instead of one function, we now have to perform a grid search over the prior *and* the original problem. That doesn’t seem like a good deal. However, a much used technique in statistics deals with exactly the kind of problems we are facing here: given a very irregular and high dimensional function

How do we extract interesting information (preferably without completely blowing our budget on supercomputers)?

Let’s first say that with interesting information I mean minima (not necessarily restricted to global minima), correlations, and possibly other statistical properties (for our uncertainty analysis). One answer to this question was first applied in Los Alamos around 1950. The researches at the famous institute developed a method to simulate equations of state, which has become known as the Metropolis-Hastings algorithm. The algorithm is able to draw samples from a complicated probability distribution. It became part of a class of methods called Markov Chain Monte Carlo (MCMC) methods, which are often referred to as samplers (technically they would be a subset of all available samplers).

The reason that the Metropolis-Hastings algorithm (and MCMC algorithms in general) is useful, is that a complicated distribution (e.g. the annulus such as in our last figure) does not easily allow us to generate points proportional to its misfit. These methods overcome this difficulty by starting at a certain point in model space and traversing a random path through it – *jumping around* – but visiting regions only proportional to the misfit. So far, we have only considered directly finding optima to misfit functions, but by generating samples from a probability distribution proportional to the misfit functions, we can readily compute these minima by calculating statistical modes. Uncertainty analysis subsequently comes virtually for free, as we can calculate any statistical property from the sample set.

I won’t try to illustrate any particular MCMC sampler in detail. Nowadays many great tools for visualising MCMC samplers exist. This blog by Alex Rogozhnikov does a beautiful job of both introducing MCMC methods (in general, not just for inversion) and illustrating the Metropolis Hastings Random Walk algorithm as well as the Hamiltonian Monte Carlo algorithm. Hamiltonian Monte Carlo also incorporates gradients of the misfit function, thereby even accelerating the MCMC sampling. Another great tool is this applet by Chi Feng. Different target distributions (misfit functions) can be sampled here by different algorithms.

The field of geophysics has been using these methods for quite some time (Malcom Sambridge writes in 2002 in a very interesting read that the first uses were 30 years ago), but they are becoming increasingly popular. However, strongly non-linear inversions and big numerical simulations are still very expensive to treat probabilistically, and success in inverting such a problem is strongly dependent on the appropriate choice of MCMC sampler.

In the third part of this blog we saw how to combine any non-linear statistical model, and how to sample these complex functions using MCMC samplers. The resulting sample sets can be used to do an inversion and compute statistical moments of the inverse problem.

If you reached this point while reading most of the text, you have very impressively worked your way through a huge amount of inverse theory! Inverse theory is a very diverse and large field, with many ways of approaching a problem. What’s discussed here is, to my knowledge, only a subset of what’s being used ‘in the wild’. These ramblings of a aspiring seismologist might sound uninteresting to the geodynamiscists at the other side of the geophysics field. Inverse methods seem to be not nearly discussed as much in geodynamics as they are in seismology. Maybe it’s the terminology that differs, and that all these concepts are well known and studied under different names and you recognise some of the methods. Otherwise, I hope I have given an insight in the wonderful and sometimes ludicrous mathematical world of (some) seismologists.

Interested in playing around with inversion yourself? You can find a toy code about baking bread here.

]]>

The idea of inversion is to literally invert the forward problem. For the aforementioned problems our knowns become unknowns, and vice versa. We wish to infer physical system parameters from measurements. Note that in our forward problem there are multiple classes of knowns; we have the forcing parameters, the material properties and the boundary conditions. All of these parameters could be inverted for in an inversion, but it is typically one only inverts for one class. Most of the time, medium parameters are the target of inversions. As we will go through the examples, I will gradually introduce some inversion methods. Every new method is marked in **blue**.

Now, let’s consider our first example: the recipe for a bread. Let’s say we have a **0.5 kilogram bread**. For our first case (case **1a**), we will assume that the amount of water is ideal. In this case, we have *one* free variable to estimate (the used amount of flour), from *one* observable (the resulting amount of bread). We have an analytical relationship that is both **invertible** and **one-to-one**. Therefore, we can use the** direct inversion ** of the relationship to compute the amount of flour. The process would go like this:

1. Obtain the forward expression G(m) = d;

2. Solve this formula for G^-1 (d) = m;

3. Find m by plugging d into the inverse function.

Applying this direct inversion shows us that **312.5 grams of flour** must have been used.

The two properties of the analytical relationship (**invertible** and **one-to one**) are very important for our choice of this inversion method. If our relationship was sufficiently complex, we couldn’t solve the formula analytically (though the limitation might then lie by the implementer 😉 ). If a function is not one-to-one, then two different inputs could have the same output, so we cannot analytically invert for a **unique** solution.

In large linear forward models (as often obtained in finite difference and finite element implementations), we have a matrix formula similar to

A **m** = **d**.

In these systems, **m** is a vector of model parameters, and **d** a vector of observables. If the *matrix is invertible*, we can directly obtain the model parameters from our observables. The invertibility of the matrix is a very important concept, which I will come back to in a bit.

Let’s see what happens when the first condition is not satisfied. I tried creating a non-linear function that at least *I* could not invert. Someone might be able to do it, but that is not the aim of this exercise. This new relationship is given here:

The actual forward model is included in the script at the bottom of the post. When we cannot analytically solve the forward model, one approach would be to iteratively optimise the input parameters such that the end result becomes as close as possible to the observed real world. But, before we talk about how to propose inputs, we need a way to compare the outputs of our model to the observables!

When we talk about our bread, we see one end-product. It is very easy to compare the result of our simulation to the real world. But what happens when we make a few breads, or, similarly, when we have a lot of temperature observations in our volcanic model? One (common) way to combine observables and predictions into a single measure of discrepancy is by defining a misfit formula which takes into account all observations and predictions, and returns a single – you guessed it – **misfit**.

The choice of misfit directly influences which data is deemed more important in an inversion scheme, and many choices exist. One of the most intuitive is the L2-norm (or L2 misfit), which calculates the vector distance between the predicted data (from our forward model) with the observed data as if in Euclidean space. For our bread it would simply be

X = | predicted bread – observed bread |

Let’s try to create a misfit function for our 1D bread model. I use the relationship in the included scripts. Again, we have 500 grams of bread. By calculating the quantity | G(m) – 500 |, we can now make a graph of how the misfit varies as we change the amount of flour. I created a figure which shows the observed and predicted value:

and a figure which shows the actual misfit at each value of m

Looking at the two previous figures may result in some questions and insights to the reader. First and foremost, it should be obvious that this problem does **not** have a unique solution. Different amounts of flour give exactly the same end result! It is thus impossible to say with certainty how much flour was used.

We have also recast our inversion problem as an optimisation function. Instead of thinking about fitting observations we can now think of our problem as a minimization of some function X (the misfit). This is very important in inversion.

Iterative optimizations schemes such as **gradient descent methods (and the plethora of methods derived from it) work in the following fashion:**

1. Pick a starting point based on the best prior knowledge (e.g., a 500 gram bread could have been logically baked using 500 grams of flour);

2. Calculate the misfit (X) at our initial guess;

3. If X is not low enough (compared to some arbitrary criterion):

• Compute the gradient of X;

• Do a step in the direction of the steepest gradient;

• Recompute X;

• Repeat from 2.

4. If X is low enough:

• Finalise optimisation, with the last point as solution.

These methods are **heavily influenced by the starting point** of the inversion scheme. If I would have started on the left side of the recipe-domain (e.g. 100 grams of flour), I might well have ended up in a different solution. Gradient descents often get ‘stuck’ in local solutions, which might not even be the optimal one! We will revisit this non-uniqueness in the 2D problem, and give some strategies to mitigate creating more than one solution. Extra material can be found here.

One thing that often bugged me about the aforementioned gradient descent methods is the seemingly complicated approach for such simple problems. Anyone looking at the figures could have said

Well, duh, there’s 3 solutions, here, here and here!

Why care about such an complicated way to only get one of them?

The important realisation to make here is that I have precomputed all possible solution for this forward model in the 0 – 700 grams range. This precomputation on a 1D domain was very simple; at a regular interval, compute the predicted value of baked bread. Following this, I could have also programmed my Python routine to extract all the values with a sufficiently low misfit as solutions. This is the basis of a ** grid search**.

Let’s perform a grid search on our second model (**1b**). Let’s find all predictions with 500 grams of bread as the end result, plus-minus 50 grams. This is the result:

The original 312.5 grams of flour as input is part of the solution set. However, the model actually has infinitely many solutions (extending beyond the range of the current grid search)! The reason that a grid search might not be effective is the inherent computational burden. When the forward model is sufficiently expensive in numerical simulation, exploring a model space completely with adequate resolution might take very long. This burden increases with model dimension; if more model parameters are present, the relevant model space to irrelevant model space becomes very small. This is known as the **curse of dimensionality** (very well explained in Tarantola’s textbook).

Another reason one might want to avoid grid searches is our inability to appropriately process the results. Performing a 5 dimensional or higher grid searches is sometimes possible on computational clusters, but visualizing and interpreting the resulting data is very hard for humans. This is partly why many supercomputing centers have in-house departments for data visualisation, as it is a very involved task to visualise complex data well.

Now: towards solving our physical inversions!

One big problem in inversion is non-uniqueness: the same result can be obtained from different inputs. The go-to way to combat this is to add extra information of any form to the forward problem. In our bread recipe we could think of adding extra observables to our problem, such as the consistency of the bread, its taste, color, etc. Another option could be to add constraints on the parameters, such as using the minimum amount of ingredients. This is akin to asking the question: given this amount of bread, how much flour and water was *minimally* used to make it?

Diffusion type problems are notorious for their non-uniqueness. Many different subsurface heat conduction distributions might result in the observations (imagine differently inclined volcanic conduits). An often used method of regularisation (not limited to diffusion type studies!) is spatial smoothing. This method requires that among equally likely solutions, the smoothest solutions are favoured, for it is more ‘natural’ to have smoothly varying parameters. Of course, in many geoscientific settings one would definitely expect sharp contrasts. However, in ‘underdetermined’ problems (i.e., you do not have enough observations to constrain a unique solution), we favour Occam’s Razor and say

The simplest solution must be assumed

When dealing with more parameters than observables (non-uniqueness) in linear models it is interesting to regard the forward problem again. If one would parameterize our volcanic model using 9 parameters for the subsurface and combine that with the 3 measurements at the surface, the result would be an underdetermined inverse problem.

This forward model (the Laplace equation) can be discretised by using, for example, finite differences. The resulting matrix equation would be A**m** = **d**, with A a 3 x 9 matrix, **m** a 9 dimensional vector and **d** a 3 dimensional vector. As one might recall from linear algebra classes, for a matrix to have an inverse, it has to be square. This matrix system is not square, and therefore not invertible!

Aaaaahhh! But don’t panic: there is a solution

By adding either prior information on the parameters, smoothing, or extra datapoints (e.g., taking extra measurements in wells) we can make the 3 x 9 system a perfect 9 x 9 system. By doing this, we condition our system such that it is invertible. However, many times we end up overdetermining our system which could result in a 20 x 9 system, for example. Note that although neither the underdetermined nor the overdetermined systems have an exact matrix inverse, both *do* have pseudo-inverses. For underdetermined systems, I have not found these to be particularly helpful (but some geophysicists do consider them). Overdetermined matrix systems on the other hand have a very interesting pseudo-inverse: the **least squares solution**. Finding the least squares solution in linear problems is the same as minimising the L2 norm! **Here, two views on inversion come together: solving a specific matrix equation is the same as minimising some objective functional** (at least in the linear case). Other concepts from linear algebra play important roles in linear and weakly non-linear inversions. For example, matrix decompositions offer information on how a system is probed with available data, and may provide insights on experimental geophysical survey design to optimise coverage *(see “Theory of Model-Based Geophysical Survey and Experimental Design Part A – Linear Problems” by Andrew Curtis)*.

I would say it is common practice for many geophysicists to pose an inverse problem that is typically underdetermined, and keep adding regularization until the problem is solvable in terms of matrix inversions. I do not necessarily advocate such an approach, but it has its advantages towards more agnostic approaches, as we will see in the post on probabilistic inversion next week!

We’ve seen how the forward model determines our inversion problem, and how many measurements can be combined into a single measure of fit (the misfit). Up to now, three inversion strategies have been introduced:

• **Direct inversion**: analytically find a solution to the forward problem. This method is limited to very specific simple cases, but of course yields near perfect results.

• **Gradient descent methods**: a very widely used class of algorithms that iteratively update solutions based on derivatives of the misfit function. Their drawbacks are mostly getting stuck in local minima, and medium computational cost.

• **Grid searches**: a method that searches the entire parameter space systematically. Although they can map all the features of the inverse problem (by design), they are often much too computationally expensive.

What might be even more important, is that we have seen how to reduce the amount of possible solutions from infinitely many to at least a tractable amount using regularisation. There is only one fundamental piece still missing… Stay tuned for the last blog post in this series for the reveal of this mysterious missing ingredient!

Interested in playing around with inversion yourself? You can find a toy code about baking bread here.

]]>

Inversion methods are at the core of many physical sciences, especially geosciences. The term itself is used mostly by geophysicists, but the techniques employed can be found throughout investigative sciences. Inversion allows us to infer our surroundings, the physical world, from observations. This is especially helpful in geosciences, where one often relies on observations of physical phenomena to infer properties of the (deep) otherwise inaccessible subsurface. Luckily, we have many, many algorithms at our disposal! (…cue computer scientists rejoicing in the distance.)

The process of moving between data and physical parameters goes by different names in different fields. In geophysics we mostly use **inversion** or **inference**. Inversion is intricately related to optimization in control theory and some branches of machine learning. In this small series of blog posts, I will try to shed a light on how these methods work, and how they can be used in geophysics and in particular geodynamics! In the end, I will focus a little on Bayesian inference, as it is an increasingly popular method in geophysics with growing potential. Be warned though:

Math ahead!

(although as little as possible)

First, for people who are serious about learning more about inverse theory, I strongly suggest Albert Tarantola’s “Inverse Problem Theory and Model Parameter Estimation”. You can buy it or download it (for free!) somewhere around here: http://www.ipgp.fr/~tarantola/.

I will discuss the following topics in this blog series (roughly in order of appearance):

1. Direct inversion

2. Gradient descent methods

3. Grid searches

4. Regularization

5. Bayesian inference

6. Markov chain Monte Carlo methods

Rather backwards, I will start the first blog post with a component of inversion which is needed about halfway through an inversion algorithm. However, this piece determines the physics we are trying to investigate. Choosing the forward ‘problem’ or model is posing the question of how a physical system will evolve, typically in a deterministic setting. This can be any relationship. The forward problem is roughly the same as asking:

I start with these materials and those initial conditions, what will happen in this system?

In this post I will consider two examples of forward problems (one of which is chosen because I have applied it everyday recently. Hint: it is not the second example…)

1. **The baking of a bread** with **known** quantities of water and flour, but the mass of the end product is **unknown**. A *recipe is available* which predicts how much bread will be produced (let’s say under all ratios of flour to water).

2. **Heat conduction** through a volcanic zone where geological and thermal properties and base heat influx are **known**, but surface temperatures are **unknown**. A *partial differential equation is available* which predicts the surface temperature (heat diffusion, Laplace’s equation).

Both these systems are forward problems, as we have all the necessary ingredients to simulate the real world. Making a prediction from these ‘input’ variables is a forward model run, a forward simulation or simply a forward solution. These simulations are almost exclusively deterministic in nature, meaning that no randomness is present. Simulations that start exactly the same, will always yield the same, *predictable* result.

For the first system, the recipe might indicate that 500 grams of flour produce 800 grams of bread (leaving aside the water for a moment, making case **1a**), and that the relationship is linear in flour. The forward model becomes:

flour * 800 / 500 = bread

This is a plot of that relationship:

Of course, if we add way too much water to our nice bread recipe, we mess it up! So, the correct amount of water is also important for making our bread. Hence, our bread is influenced by both quantities, and the forward model is a function of 2 variables:

bread = g(flour, water)

Let’s call this case **1b**. This relationship is depicted here:

It is important to note that the distinction between these two cases creates two different physical systems, in terms of invertibility.

Modelling a specific physical system is a more complicated task. Typically, physical phenomena can be described by set of (partial) differential equations. Other components that are required to predict the end result of a physical deterministic system are

1. The domain extent, suitable discretisation and parameter distribution in the area of investigation, and the relevant **material properties (sometimes parameters)** at every location in the medium

2. The boundary conditions, and when the system is time varying (transient), additional initial conditions

3. A simulation method

As an example, the relevant property that determines heat conduction is the heat conductivity at every location in the medium. The parameters control much of how the physical system behaves. The boundary and initial conditions, together with the simulation methods are very important parts of our forward model, but are not the focus of this blog post. Typically, simulation methods are **numerical methods** (e.g., finite differences, finite elements, etc.). The performance and accuracy of the numerical approximations are very important to inversion, as we’ll see later.

The result of the simulation of the volcanic heat diffusion is the temperature field throughout the medium. We might however be only interested in the temperature at real-world measurement points. We consider the following conceptual model:

The resulting temperatures at the measurement locations will be different for each point, because they are influenced by the heterogeneous subsurface. Real-world observations would for example look something like this

Now that you have a good grasp of the essence of the forward problem, I will discuss the inverse problem in the next blog post. Of course, the idea of inversion is to literally invert the forward problem, but if it were as easy as that, I wouldn’t be spending 3 blog posts on it, now would I?

]]>First of all, by looking at the number of mentions in books, you can quickly determine when a certain research field was first established as illustrated in the graph below. (Structural) Geology has always been an older branch of Earth sciences, as it originated in field observations rather than instruments or computers. James Hutton (1726-1797) was the Father of Modern Geology, as he developed the theory of uniformitarianism (= the processes on Earth that we see today also occurred in the past). His work was popularised in the 1830s by Charles Lyell, who also coined the phrase ‘The present is the key to the past’. Indeed, mentions of structural geology start to occur somewhere in the 1850s when Hutton’s and Lyell’s ideas have been firmly established in the scientific community.

According to our graph seismology originated in the 1850s. Indeed, a quick google search will tell you that the word ‘seismology’ was coined in 1857 by Robert Mallet, who also laid the foundation of instrumental seismology.

Both geodynamics and tectonophysics (fields that only really originated after the general acceptance of plate tectonics in the 1960s and benefited greatly from advances in computer science) are only starting to flourish in the late 1960s. Note that all graphs show a decline around the 1990s…

Apart from discovering when a certain field was established, it is also possible to see the direct effect of certain global events on the publishing history of a particular field. One of the most convincing cases stems from the tsunami research area. After the devastating 2004 Sumatra Boxing day earthquake and tsunami, interest in tsunamis surged in 2005 and has since remained much more popular than before (at least up until 2008).

Now on to the really fun part: karaoke. A phenomemon in the geodynamics community that is hard to get around. We have all been working and networking very seriously at conferences before we suddenly got swept away to the nearest karaoke bar. If you look at the graph comparing the amount of mentions of seismology, geodynamics etc., you will notice that there is a decline in mentions starting roughly in the 1990s. There could of course be many reasons for this: maybe the Google database is, as of yet, still incomplete, so the ongoing upward trend cannot be seen; maybe too much has been published, so that the relative percentages of words mentioned has declined, even though the absolute amount of published works containing these works has increased; but maybe.. just *maybe*, the reason is karaoke, as it started to spread around the world in the 1990s.

Now, I am not saying I did any fancy statistics on these results. I am also not saying that there is any *causality* involved (because we all know how hard it is to determine causality). I am just saying: **look at the graph** and draw your own conclusions…