This is a very specific post, most people will want to skip over it entirely. It is about how to install the ionosphere/thermosphere model TIE-GCM on Ubuntu computers using gfortran.
I get a couple of emails a week asking me for help with installing NCARs TIE-GCM v2.0 with GFortran (rather than the standard IFORT). There are a couple of steps that are needed, but the method below is tested with versions of Ubuntu from 14.04-18.04. You can install the model on Linux systems other than Ubuntu but you will need to download and build some packages which are included in the Ubuntu repository.
First we make sure we have the necessary packages installed:
sudo apt install csh libnetcdf* libopenmpi* curl libcurl3-dev gfortran
And that we can use gmake
sudo ln -s /usr/bin/make /usr/bin/gmake
Before we can build the model we need the Earth System Modeling Framework (ESMF) library. This step can cause a lot of trouble because using the wrong version of ESMF seems to cause lots of problems. I have found that TIE-GCM on Ubuntu will only work with ESMF version 7.0.2 (this is not the latest release). It can be downloaded from the ESMF website (this link should be to the right version).
Once it is downloaded extract it to a location of choice, in this example I’ll just drop it in my home directory:
mv ~/Downloads/esmf* ~/ tar -xf ~/esmf_*.tar
This will create a folder called ‘esmf’ containing everything we need (so there is a folder in the location ‘/home/sean/esmf’ in my case). We then need to update some environment variables.
You need to add to the bottom of your bashrc file the ESMF directory (ESMF_DIR) and the type of mpi being used (ESMF_COMM). You can do this with the following commands:
echo 'export ESMF_DIR=~/esmf' >> ~/.bashrc echo 'export ESMF_COMM=openmpi' >> ~/.bashrc source ~/.bashrc
The final line will make sure the bashrc file is reloaded. You can test this has worked by typing into a terminal:
echo $ESMF_COMM
and ‘openmpi’ should be returned. We’re now read to build ESMF. First navigate to the ESMF directory and you can check what settings are going to be used:
cd ~/esmf gmake info
You should see at the top our non-default options for ESMF_DIR and ESMF_COMM. You can then build the library, and check that it built correctly:
gmake gmake check
Every test should pass. If not something has gone wrong and do not proceed until you have worked out why it failed (for me this has only happened when using the ‘wrong’ version of ESMF – remember we want 7.0.2).
Now, if you haven’t already, download the TIE-GCM model and data from the NCAR website. You’ll want TIE-GCM version 2.0 (30 MB) and the 5.0 degree (1.8 GB) and 2.5 degree (3.8 GB) resolution data files. Untar them and place them in any directory structure you want (but remember where you keep put them). I usually make a folder called ~/data and combine both 5.0 and 2.5 degree files in there and another folder simply called ~/tiegcm. To create the same folder structure as I do (for the TIE-GCM model) simply:
mkdir ~/tiegcm mv ~/Downloads/tiegcm2.0.tar ~/tiegcm/ cd ~/tiegcm tar -xf tiegcm2.0.tar
Inside ~/tiegcm you should then have a folder called tiegcm2.0 (which will contain the source code and scripts) as well as a series of ‘job’ and ‘inp’ files and a ‘readme’. Next we need another environment variable this time with the details of where the TIE-GCM data directory is, assuming it is ~/data then:
echo 'export TGCMDATA=~/data' >> ~/.bashrc source ~/.bashrc
Again we can check this worked by from a terminal running ‘echo $TGCMDATA’. Next we need to ensure our makefiles are correct. First make a copy of one of the default Make info files for us to edit
cp ~/tiegcm/tiegcm2.0/scripts/Make.gfort_hao64 ~/tiegcm/tiegcm2.0/scripts/Make.gfort_ubuntu
Now we need to edit the file to make sure the paths are correct. Specifically you need to make sure you’ve updated the paths for F90, MPIF90, MPIRUN, LIB_NETCDF, INC_NETCDF and LIB_ESMF. If you’re on Ubuntu, and you installed the packages from the repos then these will simply be:
F90 = gfortran MPIF90 = mpif90 MPIRUN = mpirun LIB_NETCDF = /usr/lib -lnetcdff -lnetcdf INC_NETCDF = /usr/include LIB_ESMF = ~/esmf/lib/libO/Linux.gfortran.64.openmpi.default
Make sure you keep all of the other parts of the file, and only edit the above. You can download my Make info file here, which should work for you if you installed ESMF in ~/esmf. If not, you’ll need to update the paths.
Finally we need need to update the Makefile for compiling TIE-GCM. I really struggle understanding Makefiles but the one that is included fails because an if clause is outside of any rule scope. So I’ve found that 3 lines have to be commented out for the model to build and work. The file, ‘~/tiegcm/tiegcm2.0/scripts/Makefile’ as downloaded from NCAR looks like this (lines 51-54):
ifeq ($(F90),/usr/bin/gfortran) ESMF_F90COMPILER=$(MPIF90) @echo "set ESMF_F90COMPILER=$(ESMF_F90COMPILER)" endif
We need to comment out all of but the ESMF_F90COMPILER lines. Specifically making the files look like this:
#ifeq ($(F90),/usr/bin/gfortran) ESMF_F90COMPILER=$(MPIF90) # @echo "set ESMF_F90COMPILER=$(ESMF_F90COMPILER)" #endif
Alternatively you can download the updated file here, and replace the ‘Makefile’ in your ‘~/tiegcm/tiegcm2.0/scripts’ directory.
Now everything is set up and good to go. We just need to make sure the paths in our ‘job’ and ‘inp’ file are correct and the model can be run. In ~/tiegcm/tiegcm-linux.job update the path locations. If you’ve installed everything where I have then the following should work (obviously change things like the number of processors (nproc) if you want to):
set modeldir = ~/tiegcm/tiegcm2.0 set execdir = ~/tiegcm/tiegcm.exec #set tgcmdata = tiegcm_res5.0_data set input = ~/tiegcm/tiegcm_res5.0.inp set output = ~/tiegcm/tiegcm_res5.0.out set modelres = 5.0 set make = Make.gfort_ubuntu set mpi = TRUE set nproc = 4 set debug = FALSE set exec = TRUE set utildir = $modeldir/scripts
That should be everything and now you can run TIE-GCM by:
cd ~/tiegcm ./tiegcm-linux.job
As I said, these instructions are designed for Ubuntu (where everything you need, like NETCDF) are provided in the repos. If you want to install this on another Linux distribution you may have to compile these yourself. If anyone needs help with this, drop me a message in the comments below and I’ll make a guide for that as well.