Label that wraps an input - how to make focus style visible only when focused using the keyboard, not mouse click?

Question

In the following example, the button's focus style is visible only when the button is focused using a keyboard (not mouse click).

How could I achieve the same effect on a label that wraps a hidden input?

In the code below, the label's focus style is visible when the label is focused using a keyboard or using a mouse click. I'd like it to be visible only when focused using a keyboard.

.visually-hidden {
  position: absolute;
  border: 0;
  width: 1;
  height: 1;
  padding: 0;
  margin: -1;
  overflow: hidden;
  clip: rect(0, 0, 0, 0);
  white-space: nowrap;
  word-wrap: normal;
}

body {
  margin: 40px;
}

div {
  display: flex;
  gap: 20px;
}

input:focus {
  outline: 4px solid red;
}

button:focus-visible {
  outline: 4px solid red;
}

label:focus-within {
  outline: 4px solid red;
}

<div>
  <input type="text" />
  <button>Submit</button>
  <label>
    <input class="visually-hidden" type="checkbox" />
    Label
  </label>
</div>

Answer 1

Update the styles of the input inside it so it can style get the focus while being visually hidden:

label {
  position:relative;
}
.visually-hidden {
  position: absolute;
  inset: 0;
  margin: 0;
  padding: 0;
  -webkit-appearance: none;
  -moz-appearance: none;
  appearance: none;
  overflow: hidden;
}


button:focus-visible,
input:focus-visible{
  outline: 4px solid red;
}


body {
  margin: 40px;
}

div {
  display: flex;
  gap: 20px;
}

<div>
  <input type="text" />
  <button>Submit</button>
  <label>
    <input class="visually-hidden" type="checkbox" />
    Label
  </label>
</div>