Detect incorrect points in a homogeneous surface

Question

In my project i have hige surfaces of 20.000 points computed by a algorithm. This algorithm, sometimes, has an error, computing 1 or more points in an small area incorrectly.

This error can not be solved in the algorithm, but needs to be detected afterwards.

The error can be seen in the next figure:

As you can see, there is a point wrongly computed that not only breaks the full homogeneous surface, but also destroys the aestetics of the plot (wich is also important in the project.)

Sometimes it can be more than a point, in general no more than 5 or 6. The error is allways the Z axis, so no need to check X and Y

I have been squeezing my mind to find a bit "generic" algorithm to detect this poitns. I thougth that maybe taking patches of surface and meaning the Z, then detecting the points out of the variance... but I dont think it will work allways.

Any ideas?

NOTE: I dont want someone to write code for me, just an idea.

PD: relevant code for the avobe image:

[x,y] = meshgrid([-2:.07:2]);
Z = x.*exp(-x.^2-y.^2);
subplot(1,2,1)
surf(x,y,Z,gradient(Z))
subplot(1,2,2)
Z(35,35)=Z(35,35)+0.3;
surf(x,y,Z,gradient(Z))

Answer 1

Since your functions seems to vary smoothly these abrupt changes can be detected by looking into the derivatives. You can

1. Take the derivative in one direction
2. Calculate mean and standard deviation of derivative
3. Find the points by looking for points that are further from mean by certain multiple of standard deviation.

Here is the code

U=diff(Z);
V=(U-mean(U(:)))/std(U(:));
surf(x(2:end,:),y(2:end,:),V)
V=[zeros(1,size(V,2)); V];
V(abs(V)<10)=0;
V=sign(V);
W=cumsum(V);
[I,J]=find(W);
outliers = [I, J];

For your example you get this plot for V with a peak at around 21.7 while second peak is at around 1.9528, so maybe a threshold of 10 is ok.

and running the code returns

outliers =

    35    35

The need for cumsum is for the cases that you have a patch of points next to each other that are incorrect.