Remove white-space caused by wrap

I have a fieldset containing multiple buttons with the same fixed width. In a row/line it should fit as many buttons as possible. However during the linebreak it will cause a big white-space.

I tried display: flex; in combination with flex-wrap. However the white-space remains or the box overflows

Also I tried css-grid by using grid-template-columns: repeat(auto-fit, minmax(10em, 1fr)); which doesn't work either.

Anybody else with an idea how to size either size the box to the content to not leave a white-space or to size the buttons to fill up all the space without breaking the box max-width and fit as many as possible?

I know why the white space there, but I can't figure out how to remove it. I'm aware that it possibly requires Javascript. However related questions on SO mentioned this aswell but where unable to eleborate a Javascript solution to be understandable/reproduciable.

div {
  max-width: 80%;
}

button{
  width: 10em;
}

<div>
  <fieldset>
    <legend>Color</legend>
    <button>1</button>
    <button>2</button>
    <button>3</button>
    <button>4</button>
    <button>5</button>
    <button>6</button>
    <button>7</button>
    <button>8</button>
    <button>9</button>
  </fieldset>
</div>

Solution

If the sizes are fixed here is an idea using CSS grid. The trick is that we define a grid using a repetition of the button width and we make the fieldset take all the columns. This will for it to have a width equal to a N*width of button where N is dynamic and depend on the screen size

div.box {
  display:grid;
  font-size:14px;
  /*0.75em is the default padding of fieldset */
  grid-template-columns:0.75em repeat(auto-fit,10em) calc(0.75em + 4px);
  grid-gap:4px;
}
fieldset {
  grid-column:1/-1;
}

button{
  width: 10em;
  font-size:14px; /* define the font-size to avoid the default one*/
  margin-right:4px; /* same as gap */
}

<div class="box">
  <fieldset>
    <legend>Color</legend>
    <button>1</button><button>2</button><button>3</button><button>4</button><button>5</button><button>6</button><button>7</button><button>8</button><button>9</button>
  </fieldset>
</div>