I am preparing new input data for a fortran-based DGVM, but I am having trouble adjusting the data according to the DGVM format that will be used. I'm doing this task on a linux server.
Here is the climate data that I will use: download climate data MIROC6 historical monthly air temperature data from 1851 to 1860 (0.5 x 0.5).
I need to these steps in order
I tried converting the netcdf data by several method: CDO, NCO and ncdump comamnd and I'm able to get in one .txt format for whole grid value. But it will take huge time to extract/ classify the individual grid value according to the needed format manually. I hope there is other efficient way to convert those data, and I hope there is someone who willing to help me with this matter.
Thank you.
I found a solution by using the CDO outputtab
operator in a loop, written in a simple Linux bash script. The conversion process is done every 20 points to avoid putting a heavy load on the server's CPU (this can be adjusted).
The code I created is as follows:
#!/bin/bash
# Define input file and log file
input_file="/path/canesm5_r1i1p1f1_w5e5_ssp126_tas_global_daily_2015_2100_05gc.nc"
log_file="log.txt"
# Remove existing log file if it exists
rm -f $log_file
# Function to extract data for a specific latitude
extract_latitude() {
lat=$1
lon_index=1
max_jobs=20
job_count=0
# Loop through longitudes from -180 to 180 in 0.5 degree increments
for lon in $(seq -180 0.5 179.5); do
# Define the output file name based on the current latitude and longitude index
output_file="${lat}_${lon_index}.txt"
# Extract data for the grid point at the current latitude and longitude, save only values, and remove the header
(cdo -outputtab,value -remapnn,lon=${lon}_lat=${lat} $input_file | sed '1d' > $output_file) &>> $log_file &
# Increment the longitude index and job count
lon_index=$((lon_index + 1))
job_count=$((job_count + 1))
# Check if the max number of jobs has been reached
if [ $job_count -ge $max_jobs ]; then
wait # Wait for all background jobs to complete
job_count=0 # Reset job count
fi
done
# Wait for any remaining background processes to finish
wait
}
# Loop through latitudes from 90N to -90S in 0.5 degree increments
for lat in $(seq 90 -0.5 -90); do
echo "Extracting data for latitude $lat"
extract_latitude $lat
done
echo "Data extraction completed. Check $log_file for details."
Thank you