How to train and update weights of 3D filters

Question

I have some problems with training CNN :(

For example, 3D data (width, height, depth):

Input 6x6x3, 1 core 3x3x3, output = 4x4x1 => pool: 2x2x1

CONVOLUTION EXAMPLE IMAGE

This and other tutors are explain to calc deltas for weights and Input only for 2D (without depth):

input * output=deltas for 2D weights

filter * out = input delta

But how I can to calc weights deltas for 3D filters?

I must to multiply each input by output as below?

FilterLayer1Delta = OutputDelta * InputLayer1

FilterLayer2Delta = OutputDelta * InputLayer2

FilterLayer3Delta = OutputDelta * InputLayer3

Where:

FilterLayerNDelta is delta for layer of the current filter

InputLayerN is input image for layer of the current filter

Answer 1

This is not a 3d filter, this is a standard 2d filter. A 3d filter looks like this: https://i.sstatic.net/IOgF4.png (orange box). This is a common misconception. A 2D convolution is actually more of a collection of filters (represented by depth) that each have their own depth equal to the depth of the input layer (aka channels).

However, I think what you're trying to ask is how to calculate the final value for each filter. I have already answered question like this on ai.stackexhange, see here for the link: https://ai.stackexchange.com/a/13591/26726

Essentially, you multiply each channel of the filter with the corresponding input channel at a specific location and add these to get a single value, then you sum each value at the output location for all the outputs for every channel to get the final value at that location for that specific filter in that specific convolutional layer.

EDIT:

Here is an image for a forward pass. I am working on creating the backward pass (backprop) visualisation at the moment, just using this as a placeholder.

EDIT:

Here is an image of the backward pass, with random gradients assigned for the output, and back propagated to find the gradient for each weight in filter 1 channel 1: