I am trying to convert a .nc
file to a .csv
file for further analysis in R as I am used to working with .csv
.
Basically I think to solve my problem (more detail below) I need to add a _FillValue
into the .nc
file but everything I have tried doesn't work.
I have successfully managed to do this for many .nc
files following the steps taken in http://geog.uoregon.edu/bartlein/courses/geog490/week04-netCDF.html#replace-netcdf-fillvalues-with-r-nas up until section 3.4.3.
However, I recently gained access to another .nc
file and the same process does not work correctly. I think I have narrowed it down to the fact that there is no _FillValue
in the new .nc
file.
From the looks of it the _FillValue
should be "9.97e+36". I have tried adding this number as a missing value using
ncin <- nc_open(ncfname, write=T)
dname <- "tas"
Mvalue <- 9.97e+36
ncvar_change_missval(ncin, dname, Mvalue)
This seems to add missing_value:9.97e+36
into the .nc
file. However, when I run: tmp_array <- ncvar_get(ncin,dname)
the tmp_array still has 9.97e+36.
I expect tmp_array to have replaced the 9.97e+36 to NA
as it does for the files where it works.
Is there a way I can add a _FillValue to my file so it replaces these values with NA
?
This is the info of the file that isn't working if needed:
> print(ncin)
File ./data/UKCP18/Mean_air_temperature_(tas)/.nc_files/tas_hadukgrid_uk_1km_mon_201801-201812.nc (NC_FORMAT_NETCDF4):
9 variables (excluding dimension variables):
double tas[projection_x_coordinate,projection_y_coordinate,time] (Contiguous storage)
standard_name: air_temperature
long_name: Mean air temperature
units: degC
description: Mean air temperature
label_units: C
level: 1.5m
plot_label: Mean air temperature at 1.5m (C)
cell_methods: time: mid_range within days time: mean over days
grid_mapping: transverse_mercator
coordinates: latitude longitude month_number season_year
missing_value: 9.97e+36
int transverse_mercator[] (Contiguous storage)
grid_mapping_name: transverse_mercator
longitude_of_prime_meridian: 0
semi_major_axis: 6377563.396
semi_minor_axis: 6356256.909
longitude_of_central_meridian: -2
latitude_of_projection_origin: 49
false_easting: 4e+05
false_northing: -1e+05
scale_factor_at_central_meridian: 0.9996012717
double time_bnds[bnds,time] (Contiguous storage)
double projection_y_coordinate_bnds[bnds,projection_y_coordinate] (Contiguous storage)
double projection_x_coordinate_bnds[bnds,projection_x_coordinate] (Contiguous storage)
8 byte int month_number[time] (Contiguous storage)
units: 1
long_name: month_number
8 byte int season_year[time] (Contiguous storage)
units: 1
long_name: season_year
double latitude[projection_x_coordinate,projection_y_coordinate] (Contiguous storage)
units: degrees_north
standard_name: latitude
double longitude[projection_x_coordinate,projection_y_coordinate] (Contiguous storage)
units: degrees_east
standard_name: longitude
4 dimensions:
time Size:12
axis: T
bounds: time_bnds
units: hours since 1800-01-01 00:00:00
standard_name: time
calendar: gregorian
projection_y_coordinate Size:1450
axis: Y
bounds: projection_y_coordinate_bnds
units: m
standard_name: projection_y_coordinate
projection_x_coordinate Size:900
axis: X
bounds: projection_x_coordinate_bnds
units: m
standard_name: projection_x_coordinate
bnds Size:2
11 global attributes:
_NCProperties: version=1|netcdflibversion=4.6.1|hdf5libversion=1.10.2
comment: Monthly resolution gridded climate observations
creation_date: 2019-08-09T20:34:33
frequency: mon
institution: Met Office
references: doi: 10.1002/joc.1161
short_name: monthly_meantemp
source: HadUK-Grid_v1.0.1.0
title: Gridded surface climate observations data for the UK
version: v20190808
Conventions: CF-1.5
I found the soultion. I thought I would post up here incase anyone finds themselves stuck too!
I realised that perhaps the missing_value
was not simply 9.97e+36
but had more decimal points. I ran this to find out the full missing_value
which I then set as the missing_value
so then ncvar_get()
worked correclty.
ncin <- nc_open(ncfname, write=T)
print(ncin)
tmp_array <- ncvar_get(ncin,dname) # This produced an array with the missing value inserted - should be replaced with NAs
# What is the missing value up to 100 decimal points?!
sprintf("%.100f", tmp_array[1,1,1])
# Set missing value
Mvalue <- 9.969209968386869047442886268468442020e+36
# insert missing_value to .nc file
ncvar_change_missval(ncin, dname, Mvalue)
print(ncin)
# make new array with values replaced with NAs
tmp_array <- ncvar_get(ncin,dname)
I then continue to follow the process outlined in http://geog.uoregon.edu/bartlein/courses/geog490/week04-netCDF.html#replace-netcdf-fillvalues-with-r-nas up until 3.4.3 to produce my .csv
Phew! Thanks all :)