In spaceborne gravimetry we usually have a very high number of both observations and the unknowns we want to derive from these observations.
The GRACE satellites have now been in orbit for over 13 years. This means that we have about 87 million intersatellite range observations, 43 million GPS positions for each satellite, as well as accelerometer and attitude measurements. Considering global daily gravity field solutions we estimate about 2600 parameters per day, resulting in 13 million parameters for the whole time series. Processing such a huge amount of data in a sensible time frame (read: days) is quite challenging from two perspectives:
The first obstacle is the sheer volume of working memory required to determine the gravity field parameters. When computing daily gravity field solutions for the complete GRACE time series, about 510 gigabytes of memory are required for the equation system we need to solve. This is obviously quite a lot more than an average desktop computer can provide, so we have to use multiple, more powerful machines to form a so called distributed system. At the Institute of Geodesy at TU Graz we have four units with a total of 208 cores and about 1150 gigabytes of working memory for this purpose (for comparison: the fastest super computer in the world has over 3 million cores).
Now that parts of our problem are stored on different computers, solving it requires communication between each machine involved. Herein lies the second challenge, which is to develop an algorithm that solves the equation system as fast as possible in a distributed environment. Matrix computations used to solve such equation systems are very well understood and are able to operate at near peak processor performance. The tricky part is to parallelize these algorithms so that all machines can work in a concurrent fashion. In the case at hand this leads to a parallel Cholesky decomposition followed by forward and backward substitution, each tailored to the structure of the normal equation system.
With all these efforts, we are able to compute daily gravity field solutions from preprocessed daily normal equations for the complete GRACE time series within a few hours. This in turn means that new ideas can be tested quite rapidly and then discarded or pursued depending on their impact.
