Search code examples
cneural-networkfann

MLP training: how to handle unknown feature value

Suppose we have a MLP to be trained with a set of feature vectors, such that some of those vectors contain unknown values. How should I handle that? Is the MLP capable of this?

Suppose the training vectors are :

(1.0, 3.4, unknown, 2.0), (3.1, unknown, 1.2, 0.1), (2.1,3.4,1.2,4.5), ...

I'm using FANN.

Solution

  • Missing data

    You are referring to the missing data problem (Little. and Rubin 1987). This is not something the neural network classifier can handle well on its own. You should preprocess your data and try to fill in the missing data either by a simple statistically estimated value based conditionally on the known instance variable values (1) or a more advanced algorithm (2).

    (1) By example:

    instance1 = 0, 0, 1, 0, 1
instance2 = 0, 0, 1, 0, 1
instance3 = 1, 1, 1, 0, 0
instanceX = 1, 1, 1, 0, ?

# The statistical approach
We can see that instanceX shares a lot of instance3's features,
thus we will set the unknown variable accoring to instance3's defined value: 0
# The mean
We could calculate the dataset's mean value for this variable and
use the calculated value: 1

    (2) The EM algorithm

    This is a more sophisticated algorithm for finding approximate estimates for the missing data. Read a introduction to the algorithm here.