Tensorflow CustomEstimator DNNRegressor label_dimensions

Question

I am currently fairly new to tensorflow and I'm trying to predict the position of a particle based on it's position in past time steps.

my features looks like this:

    X_0    X_1    X_2    X_3    X_4    Y_0    Y_1    Y_2    Y_3    Y_4
19  650.0  651.0  652.0  653.0  654.0  128.3  135.9  143.5  151.1  158.7
16  647.0  648.0  649.0  650.0  651.0  105.5  113.1  120.7  128.3  135.9
...

and my labeles look like this:

     LabelX  LabelY
19   655.0   166.3
16   652.0   143.5
...

as you can see my Labels are 2-Dimensional. My first attempt uses tensorflows premade estimator tf.estimator.DNNRegressor and by giving it the argument label_dimension=2 during creation this works well.

Now I'd like to do the same with a custom estimator. Sadly the tutorials on the tensorflow site are all using Classifier instead of Regressors and the only example I've been able to find online is this, but they only use a one-dimensional output.

I've been experimenting quite a bit, but I wasn't able to get any progress. I'm fairly certain that I have to change line 41

output_layer = tf.layers.dense(inputs=top, units=1)

But if I do that, I can't get the rest of the file to function.

Answer 1

I got it to work.

you can find my code here.

import tensorflow as tf


def myCustomEstimator(features, labels, mode, params):
    """Modell funktion für Custom Estimator (DNN Regression)"""

    # Input Layer
    top = tf.feature_column.input_layer(features, params["feature_columns"])

    # basierend auf hidden Units wird die Netztopologie aufgebaut
    for units in params.get("hidden_units", [20]):
        top = tf.layers.dense(inputs=top, units=units, activation=tf.nn.relu)
        if "dropout" in params.keys() and params["dropout"] != 0:
            top = tf.layers.dropout(inputs=top, rate=params["dropout"], training=mode == tf.estimator.ModeKeys.TRAIN)

    # lineares output layer mit 2 Neuronen für die 2 Koordinaten
    output_layer = tf.layers.dense(inputs=top, units=2)

    if mode == tf.estimator.ModeKeys.PREDICT:
        # In `PREDICT` mode we only need to return predictions.
        return tf.estimator.EstimatorSpec(
            mode=mode, predictions={"predictions": output_layer})

    average_loss = tf.losses.mean_squared_error(tf.cast(labels, tf.float32), output_layer)
    tf.summary.scalar("average_loss", average_loss)

    MSE = tf.metrics.mean_squared_error(tf.cast(labels, tf.float32), output_layer)
    tf.summary.scalar('error', MSE[1])

    # Pre-made estimators use the total_loss instead of the average,
    # so report total_loss for compatibility.
    batch_size = tf.shape(labels)[0]
    total_loss = tf.to_float(batch_size) * average_loss

    if mode == tf.estimator.ModeKeys.TRAIN:
        optimizer = params.get("optimizer", tf.train.AdamOptimizer)
        optimizer = optimizer(params.get("learning_rate", None))
        train_op = optimizer.minimize(
            loss=average_loss, global_step=tf.train.get_global_step())

        return tf.estimator.EstimatorSpec(
            mode=mode, loss=total_loss, train_op=train_op)


    # In evaluation mode we will calculate evaluation metrics.
    assert mode == tf.estimator.ModeKeys.EVAL

    # Calculate root mean squared error
    rmse = tf.metrics.root_mean_squared_error(tf.cast(labels, tf.float32), output_layer)

    # Add the rmse to the collection of evaluation metrics.
    eval_metrics = {"rmse": rmse, "average_loss": MSE}

    return tf.estimator.EstimatorSpec(
        mode=mode,
        # Report sum of error for compatibility with pre-made estimators
        loss=total_loss,
        eval_metric_ops=eval_metrics)