Software

FALL3D is an open source community code for atmospheric transport of particles, aerosols and radionuclides particularly targeting volcanological applications. During the first phase of the ChEESE Center of Excellence (ChEEE-1P, 2018-2022), the pure MPI version of the code was heavily refactored and accelerated using OpenACC directives, and a dwarf version (mini-app) implemented to facilitate co-design and performance improvement activities.

 

FALL3D solves the so-called Advection-Diffusion-Sedimentation (ADS) equation on a generalised orthogonal system of coordinates. Coordinate mappings exist to map physical domains (e.g. accounting for Earth’s curvature and topography) to a «brick-like” computational domain by using coordinate-dependent horizontal and vertical mappings. The model can be solved on various horizontal (cartesian, spherical, Mercator, polar stereographic, etc.) and vertical (terrain following, etc.) coordinate systems. The finite volume based solver uses a high-resolution Kurganov-Tadmor (KT) scheme combined with an explicit solver time marching (4th order Runge-Kutta or first order Euler) and considering a simple structured mesh with 3D brick-like domain decomposition. FALL3D uses MPI for 3D domain decomposition with freedom for users to choose the number of processors along each spatial direction. The «brick-like» computational domain is discretised using a variation of the staggered Arakawa-D grid, in which the wind velocity components are evaluated at the respective cell faces and the rest of scalar quantities at the cell centres. This configuration is very convenient for solving the 3D equation in a fractional manner because, when solving each one-dimensional case, wind velocities are already aligned with the boundaries of the corresponding one dimensional cells. In the accelerated version (currently relying on OpenACC + CUDA-aware MPI) each subdomain is assigned to a single GPU.