Last output layer with multiple classes. Keras backed by Tensorflow

Question

Following this example and this article on reinforcement learning. I finally manage to create a similar Q-learning that learns to play another game environment. The only problem i have is with the last output layer of the neural network which represents the input actions of the game.

The mechanics/logic of the game environment are not relevant to this question, but the game environment require 2 types of inputs at the same time for each given action:

1. Input number 1 represents a single key press between 3 possible keys. So basically the layer should output 3 classes where the probabilities sum is 1. I will then pick the class with the highest probability out of these three.
2. Input number 2 represents a percentage ranging from 0 to 1. And should be independent of the first three classes.

I really do not see how can i create this last output layer so that it has a total of 4 output classes. The first 3 classes should give probabilities between each other with a total sum of 1. And the last class should be independent of the first three and should ranging from 0 to 1.

Can somebody point me in the right direction on how to achieve this? How do i structure such a layer?

I am thinking of something like this for the first input:

model.add(Dense(output_dim=3))
model.add(Activation("softmax"))
model.compile(loss='categorical_crossentropy', optimizer="adam")

and then for the second input something like this

model.add(Dense(output_dim=1))
model.add(Activation("sigmoid"))
model.compile(loss='binary_crossentropy', optimizer='adam')

but how do i combine them into one output layer?

Maybe you have another structure in mind?

Answer 1

You'll need to use the functional API. You can then have multiple outputs and use a different loss function for each output, as the loss argument to compile() accepts a dictionary. Here's a small example. I made up num_samples and num_features but have otherwise followed your specifications:

import numpy as np

from keras.layers import Input, Dense
from keras.models import Model
from keras.utils import to_categorical

num_samples = 10
num_features = 5
num_keys = 3

inputs = Input(shape=(num_features,))
hidden = Dense(30)(inputs)
key_press = Dense(num_keys, name='key_press', activation='softmax')(hidden)
percentage = Dense(1, name='percentage', activation='sigmoid')(hidden)

model = Model(inputs=inputs, outputs=[key_press, percentage])

loss_map = {'key_press': 'categorical_crossentropy', 'percentage': 'mse'}

model.compile(loss=loss_map, optimizer='sgd')

You'll then need to pass in a dictionary for y when you fit().

y_key_press = np.random.randint(num_keys, size=num_samples)
y_key_press = to_categorical(y_key_press)  # needs to be one-hot encoded

y_percentage = np.random.uniform(size=num_samples)

y = {'key_press': y_key_press, 'percentage': y_percentage}
x = np.random.normal(size=(num_samples, num_features))

model.fit(x=x, y=y, epochs=5)