I tried to search nearest values for 34 locations with given lat-lon as reference. Set of data files for 30 days (consists of thousands data) arranged in array forms with nearest lat-lon and desired data. How to collocate the lat-lon to find respective desirable data by looping method and saved to text files like below:
Expected output text files:
Lat Lon Date1 Date2 Date3 …………. Date30
1 6.483 100.267 0 ….. ….. ….. …..
2 6.333 99.733 0 ….. ….. ….. …..
3 6.2 100.4 1.237 ….. ….. ….. …..
4 5.457 100.388 0 ….. ….. ….. …..
5 5.35 100.4 0 ….. ….. ….. …..
6 5.297 100.272 0 ….. ….. ….. …..
7 4.221 100.701 0 ….. ….. ….. …..
8 4.567 101.1 2.003 ….. ….. ... …..
9 4.467 101.367 9.161 ….. ….. ….. …..
10 ….. ….. …..
11 ….. ….. …..
. ….. ….. …..
. ….. ….. …..
34 ….. ….. …..
Index on the left is not necessary in output
The desired data comes from this files under list files "trmm-0.25"
1. Combine.3B42.20161101.Daily.0.25.dat
2. Combine.3B42.20161102.Daily.0.25.dat
3. Combine.3B42.20161103.Daily.0.25.dat
.
.
30. Combine.3B42.20161130.Daily.0.25.dat
Thousands of desired data for each files are arranged like this :
Lat Lon Desired
-5.000 95.000 9.420
-5.000 95.250 13.470
-5.000 95.500 14.790
-5.000 95.750 12.840
................
9.750 119.750 17.310
9.750 120.000 19.650
10.000 95.000 15.480
10.000 95.250 15.690
The script I used is as follows:
import numpy as np
#MAIN REFERENCE LAT AND LON
data1=np.loadtxt('location-new.dat')
in_lats1=data1[:,0]
in_lons1=data1[:,1]
#SEARCH CLOSE LOCATION AND FIND RESPECTIVE VALUES IN SEVERAL FILES
files3=np.loadtxt('trmm-0.25',dtype=str)
for x in range(len(files3)):
file = files3[x]
data = np.loadtxt(file)
lats=data[:,0]
lons=data[:,1]
trmm=data[:,2]
ind=[]
for i in range(len(data1)):
dist=(lats-in_lats1[i])**2+(lons-in_lons1[i])**2
ind.append(np.where(dist==np.min(dist))[0][0])
lat2=lats[ind]
lon2=lons[ind]
trmm2=trmm[ind]
data3=np.array([in_lats1,in_lons1,trmm2])
data3=np.transpose(data3)
np.savetxt('Ground-match-trmm-0.25-loop2.dat',data3,fmt='%9.3f')
main reference of 34 locations are given under location-new.dat as follows:
Lat Lon
6.483 100.267
6.333 99.733
6.2 100.4
5.457 100.388
5.35 100.4
5.297 100.272
.
.
.
Problem: The output saved text appear only for Date1 not at other dates..
You really have 3-dimensions of data here (lat-lon, date, trmm) so as a first pass work with three nested for loops instead of two. You can decide which will be the outer loop based on your memory/speed requirements, but I'll give an example with lat-lon being the outer loop so you don't have to keep the big date files in memory while you're writing your output file.
In this example I've excluded a lot of the heavy lifting to concentrate on the three nested loop iteration you need to get the data from all the dates.
import numpy as np
result = []
latlons=np.loadtxt('location-new.dat')
for latlon in latlons:
daily_closest_trmms = []
daily_trmm_files =np.loadtxt('trmm-0.25',dtype=str)
for daily_trmm_file in daily_trmm_files:
trmms = np.loadtxt(daily_trmm_file)
closest_trmm = 0
for trmm in trmms:
pass # calculate closest trmm
daily_closest_trmms.append(closest_trmm)
result.append(np.concatenate([latlon, daily_closest_trmms]))
np.savetxt('Ground-match-trmm-0.25-loop2.dat',result,fmt='%9.3f')