Suppose I have two datasets. The first one is:
t1<-sample(1:10,10,replace = T)
t2<-sample(1:10,10,replace = T)
t3<-sample(1:10,10,replace = T)
t4<-sample(11:20,10,replace = T)
t5<-sample(11:20,10,replace = T)
xtrain<-rbind(t1,t2,t3,t4,t5)
xtrain
[,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
t1 7 3 9 10 4 9 2 1 6 9
t2 5 1 1 6 5 3 10 2 6 3
t3 8 6 9 7 9 2 3 5 1 8
t4 16 18 14 17 19 20 15 15 20 19
t5 13 14 18 13 11 19 13 17 16 14
The second one is:
t6<-sample(1:10,10,replace = T)
t7<-sample(11:20,10,replace = T)
xtest<-rbind(t6,t7)
xtest
[,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
t6 1 5 8 2 10 2 3 4 8 5
t7 14 18 15 12 17 20 17 13 16 17
What I did like to do is to calculate the sum of the distance between each row of xtest and each row of xtrain. For example:
sum((7-1)^2+(3-5)^2+(9-8)^2+.....(9-5)^2)
sum((5-1)^2+(1-5)^2+(1-8)^2+.....(4-5)^2)
...
sum((14-13)^2+(18-14)^2+(15-18)^2+.....(17-14)^2)
What I currently have is to use two for-loops (see below), which I don't think can handle large data sets:
sumPD<-function(vector1,vector2){
sumPD1<-sum((vector1-vector2)^2)
return(sumPD1)
}
loc<-matrix(NA,nrow=dim(xtrain)[1],ncol=dim(xtest)[1])
for(j in 1:dim(xtest)[1]){
for(i in 1:dim(xtrain)[1]){
loc[i,j]<-sumPD(xtrain[i,],xtest[j,])
}
}
I'd like to ask for suggestions on how to modify the code to make it efficient. Thank you in advance! Hope to have a good discussion!
The rdist package has functions for quickly calculating these kinds of pairwise distances:
rdist::cdist(xtrain, xtest)^2
Output:
[,1] [,2]
[1,] 65 1029
[2,] 94 1324
[3,] 165 1103
[4,] 1189 213
[5,] 1271 191