I started developing the Fatiando a Terra Python library in 2010. Since then, many other open-source Python libraries for geophysics have appeared, each with unique capabilities. In this post, I'll explore where I think Fatiando fits in this larger ecosystem and how we can better fill our niche.
Fatiando is a Python library for modeling and inversion in geophysics. It's composed of different subpackages:
fatiando.gridder: functions for dealing with spatial data. It's mostly used to generate point scatters or coordinate arrays for regular grids. Both are required as inputs for modeling or creating synthetic datasets.
fatiando.mesher: classes that represent geometric objects (polygons, prisms, spheres, etc) and regular meshes. These classes are used to define the geometry and physical properties of our models. They are often the inputs for gravity and magnetic modeling functions.
fatiando.vis: utilities for plotting data using matplotlib and 3D models using Mayavi. Mostly deprecated but there is a lot of useful code for displaying
fatiando.mesherelements in Mayavi.
fatiando.inversion: classes for solving inverse problems. The idea is that the user needs only to implement the forward problem (the forward function and the Jacobian matrix) and the classes take care of the rest. Ideally, this would form the basis for all inversions in Fatiando.
fatiando.datasets: functions for loading data from common file formats and loading sample datasets packaged with Fatiando.
fatiando.seismic: functions and classes for modeling seismic data and some basic inversions. Mostly toy problems.
fatiando.geothermal: geothermal modeling functions. Has a single module for modeling how temperature perturbations at the surface propagate down into the Earth.
fatiando.gravmag: functions for gravity and magnetic processing, modeling, and inversion. By far the most developed package, though some components have lagged behind.
We set out with the goal of modeling the whole Earth using all geophysical methods. Humble, right? Turns out this is extremely hard and way beyond what a couple of grad students can do in a couple of years. Back then, there were very few Python geophysical modeling libraries. A decade later, the ecosystem has expanded. The five currently on going projects of which I'm aware are (let me know in the comments if I missed any):
The two projects that are most similar to us (SimPEG and pyGIMLi) implement flexible partial differential equation solvers that they use to run all forward modeling calculations. This makes a lot of sense because it gives them a unified framework to model most geophysical methods. It is the most sensible approach to build joint inversions of multiple geophysical datasets. However, there are some inverse problems that don't fit this paradigm, like inverting Moho relief from gravity data and some non-conventional inversion algorithms (see the animation below).
It's no coincidence that Fatiando mostly contains the tools needed to implement this type of inverse problem (i.e., analytical solutions for the gravity and magnetic fields of geometric objects). This is precisely the type of research that we do at the PINGA lab. We also develop processing methods for gravity and magnetics.
The niche I see for Fatiando is in gravity and magnetic methods, particularly using these analytical solutions. The processing functions are an important feature because there are hardly any open-source alternatives out there to commercial software like Oasis Montaj and Intrepid.
Fatiando has grown over the years as I slowly learned how to develop and maintain an open-source Python project. As a result, the codebase is littered with the bad choices that I made along the way. The most urgent problems that need to be fixed are:
fatiandoas a metapackage (like Jupyter).
The best way forward for Fatiando that I can see, is to become an ecosystem of
specialized tools and libraries, rather than a single Python package.
Having things in separate libraries allows us to better indicate what is robust
and professional and what is experimental or meant as a teaching tool.
In particular, the meshing library has some overlap with
discretize and we should be considering
a merger of our projects.
Separating what we have in a library will help us articulate the
requirements of Fatiando so that we can see if a merger is beneficial.
We can also include experimental libraries (like
CLI or GUI programs as independent projects.
This is how I envision the Fatiando ecosystem in the future (I have already started working on some of these projects):
fatiando: A metapackage that can be used to install all the whole stack (like the
deeplook: the inversion package. Should define a scikit-learn like interface and provide all of the standard tools (regularization, optimization, etc).
geometric: the geometric objects and meshes. Includes an optional way of plotting them on Mayavi and matplotlib. The way physical properties are handled needs to be redesigned and meshes need to support slicing and fancy indexing.
verde: the gridding package. It will include some new Green's functions based interpolation on which I've been working. Should also include the functions for calculating derivatives that are currently in
harmonica: the gravity and magnetic methods package. Will port over most of the code from
sismica: a package for seismics and seismology. For now, will include some of the toy examples from the
wavefd: the experimental 2D FD wave propagation code (useful for teaching but I don't trust it enough for research).
moulder: GUI for 2D gravity and magnetic modeling.
All of these packages will be tied together in the
fatiando Github organization
and the fatiando.org website, which will include
instructions for installing the entire stack.
The website will also link to individual packages (as is done right now for the
subpackages) and any other project in the
Members of the organization will be free to create new repositories and we'll
provide a template for doing so.
The requirements and goals for these new packages are:
This is how I think we could implement this:
setup.py, docs, continuous integration configuration,
Makefile, testing, etc).
fatiando/fatiandowhile ensuring that everything is tested and documented.
The goal of all these changes is to make Fatiando better for users and developers by making the code more robust and well documented. I'm curious to know what the Python geophysics community thinks about all of this. Do I have it all wrong? What should be done differently? And most importantly, would you like to help?