Subtract all entries in one file from corresponding entries in another file without considering the missing value

Question

I have two files with equal number of rows and columns. I would like to subtract the all entries in one file from the correspondng entries in another file without considering the missing values. e.g.

ifile1.txt    
3  5  2  2
1  ?  2  1
4  6  5  2
5  5  7  1

ifile2.txt
1  2  1  3
1  3  0  2
2  ?  5  1
0  0  1  1

Here "?" is the missing value and should not be considered in computation.

ofile.txt i.e. [(ifile1.txt) - (ifile2.txt)]
2.00  3.00  1.00 -1.00 
0.00  ?     2.00 -1.00 
2.00  ?     0.00  1.00 
5.00  5.00  6.00  0.00 

I could able to do it without any missing values in following way. But can't able to succeed with a missing value like here "?".

paste ifile1.txt ifile2.txt > ifile3.txt
awk '{n=NF/2; for (i=1;i<=n;i++) printf "%5.2f ", $i-$(i+n); print ""}' ifile3.txt > ofile.txt

Answer 1

didn't test, but something like this should work

$ paste ifile1.txt ifile2.txt |
  awk -v q='?' '{n=NF/2; 
                 for(i=1;i<=n;i++) 
                   printf "%5s ", (($i==q||$(i+n)==q)?q:$i-$(i+n)); 
                 print ""}' > ofile.txt