How to create a blurred label background with SVG filters?

I would like to have a good working approach for the following problem:

On a bar chart with bars filled with different but unknown colors, I would like to put labels with a fixed black color such that they are readable.

The approach I am searching for should look such as in the title bar of the window in the image. The label is black, it resides on top of a blurred white background which sits on top of the bar of any color.

Is there a way to do it with SVG filters?

Below are some approaches I tried:

    .black {
      fill: black;
    }
    .blue {
      fill: blue;
    }
    .red {
      fill: red;
    }    
    .yellow {
      fill: yellow;
    }
    .label{
      fill: black;
      font-size: 14px;
    }
    .label-background{
      fill: white;
      font-size: 16px;
    }
    .text-background{
      paint-order: stroke;
      stroke: #fff;
      fill: black;
      stroke-width: 5px;
    }
    .header{
      font-size: 20px;
    }
    .background{
      fill: white;
      opacity: 0.6;
    }

  <svg width="500" height="500">
    <defs>
      <filter x="-0.05" y="-0.1" width="1.08" height="1.2" id="solid">
        <feFlood flood-color="lightgrey"/>
        <feComposite in="SourceGraphic" operator="xor" />
      </filter>
    </defs>

    <g class="row-1" transform="translate(0,30)">
      <text class="header" x="20" y="-10">text label on top of bar </text>
      <g>
        <rect class="black" width=100 height=30></rect>
        <text class="label" x="20" y="20">my text label</text>
      </g>

      <g transform="translate(120,0)">
        <rect class="blue" width=100 height=30></rect>
        <text class="label" x="20" y="20">my text label</text>
      </g>

      <g transform="translate(240,0)">
        <rect class="red" width=100 height=30></rect>
        <text class="label" x="20" y="20">my text label</text>
      </g>

      <g transform="translate(360,0)">
        <rect class="yellow" width=100 height=30></rect>
        <text class="label" x="20" y="20">my text label</text>
      </g>
    </g>  
    
		<g class="row-2" transform="translate(0,100)">
      <text class="header" x="20" y="-10">text label on top of rect.background on top of bar </text>
      <g>
        <rect class="black" width=100 height=30></rect>
        <rect class="background" x=17 y=6 width=78 height=19></rect>
        <text class="label" x="20" y="20">my text label</text>
      </g>

      <g transform="translate(120,0)">
        <rect class="blue" width=100 height=30></rect>
        <rect class="background" x=17 y=6 width=78 height=19></rect>
        <text class="label" x="20" y="20">my text label</text>
      </g>

      <g transform="translate(240,0)">
        <rect class="red" width=100 height=30></rect>
        <rect class="background" x=17 y=6 width=78 height=19></rect>
        <text class="label" x="20" y="20">my text label</text>
      </g>

      <g transform="translate(360,0)">
        <rect class="yellow" width=100 height=30></rect>
        <rect class="background" x=17 y=6 width=78 height=19></rect>
        <text class="label" x="20" y="20">my text label</text>
      </g>
  	</g>
    
    <g class="row-3" transform="translate(0, 170)">
      <text class="header" x="20" y="-10">text label with filter #solid on top of bar </text>
      <g>
        <rect class="black" width=100 height=30></rect>
        <text class="label" filter="url(#solid)" x="20" y="20">my text label</text>
      </g>

      <g transform="translate(120,0)">
        <rect class="blue" width=100 height=30></rect>
        <text class="label" filter="url(#solid)" x="20" y="20">my text label</text>
      </g>

      <g transform="translate(240,0)">
        <rect class="red" width=100 height=30></rect>
        <text class="label" filter="url(#solid)" x="20" y="20">my text label</text>
      </g>

      <g transform="translate(360,0)">
        <rect class="yellow" width=100 height=30></rect>
        <text class="label" filter="url(#solid)" x="20" y="20">my text label</text>
      </g>
    </g> 

    <g class="row-4" transform="translate(0, 240)">
      <text class="header" x="20" y="-10">text label on top of larger text label with background color on top of bar </text>
      <g>
        <rect class="black" width=100 height=30></rect>
        <text class="label-background" x="18" y="22">my text label</text>
        <text class="label" x="20" y="20">my text label</text>
      </g>

      <g transform="translate(120,0)">
        <rect class="blue" width=100 height=30></rect>
        <text class="label-background" x="18" y="22">my text label</text>
        <text class="label" x="20" y="20">my text label</text>
      </g>

      <g transform="translate(240,0)">
        <rect class="red" width=100 height=30></rect>
        <text class="label-background" x="18" y="22">my text label</text>
        <text class="label" x="20" y="20">my text label</text>
      </g>

      <g transform="translate(360,0)">
        <rect class="yellow" width=100 height=30></rect>
        <text class="label-background" x="18" y="22">my text label</text>
        <text class="label" x="20" y="20">my text label</text>
      </g>
    </g> 
    
    <g class="row-5" transform="translate(0, 310)">
      <text class="header" x="20" y="-10">text label with paint-order: stroke on top of bar </text>
      <g>
        <rect class="black" width=100 height=30></rect>
        <text class="label text-background" x="20" y="20">my text label</text>
      </g>

      <g transform="translate(120,0)">
        <rect class="blue" width=100 height=30></rect>
        <text class="label text-background" x="20" y="20">my text label</text>
      </g>

      <g transform="translate(240,0)">
        <rect class="red" width=100 height=30></rect>
        <text class="label text-background" x="20" y="20">my text label</text>
      </g>

      <g transform="translate(360,0)">
        <rect class="yellow" width=100 height=30></rect>
        <text class="label text-background" x="20" y="20">my text label</text>
      </g>
    </g> 
  </svg>

Solution

Since you're asking for an SVG filter solution:

Start with SourceAlpha, which gives you a black silhouette of the element (i.e. the text), and then blur it with feGaussianBlur to create a soft shadow.
This shadow will be black, so use feColorMatrix to make it white.
Finally, put the original text on top of the shadow using feBlend.

References:

SVG Drop Shadows (see Example 3)
Finessing feColorMatrix

.black {
  fill: black;
}
.blue {
  fill: blue;
}
.red {
  fill: red;
}    
.yellow {
  fill: yellow;
}
.label {
  fill: black;
  font-size: 14px;
}

<svg width="500" height="80" color-interpolation-filters="sRGB">
  <defs>
    <filter id="shadow">
      <feGaussianBlur in="SourceAlpha" stdDeviation="3" result="blur" />
      <feColorMatrix in="blur" type="matrix" values="0  0  0  0  1
                                                     0  0  0  0  1
                                                     0  0  0  0  1
                                                     0  0  0  8  0" result="white" />
      <feColorMatrix in="white" type="matrix" values="1  0  0  0  0
                                                      0  1  0  0  0
                                                      0  0  1  0  0
                                                      0  0  0 .8  0" result="dim" />
      <feBlend in="SourceGraphic" in2="dim" mode="normal" />
    </filter>
  </defs>

  <g class="row-1" transform="translate(10,30)">
    <text class="header" x="0" y="-10">feGaussianBlur on SourceAlpha</text>
    <g>
      <rect class="black" width=100 height=30></rect>
      <text class="label" x="20" y="20" filter="url(#shadow)">my text label</text>
    </g>
    <g transform="translate(120,0)">
      <rect class="blue" width=100 height=30></rect>
      <text class="label" x="20" y="20" filter="url(#shadow)">my text label</text>
    </g>
    <g transform="translate(240,0)">
      <rect class="red" width=100 height=30></rect>
      <text class="label" x="20" y="20" filter="url(#shadow)">my text label</text>
    </g>
    <g transform="translate(360,0)">
      <rect class="yellow" width=100 height=30></rect>
      <text class="label" x="20" y="20" filter="url(#shadow)">my text label</text>
    </g>
  </g>  
</svg>