WaitForSeconds Gradually become out of Sync... Unity

Question

I am creating a rythm VR game for google cardboard, inspired by Beat Saber on PSVR and Oculus Rift.

---

The concept is that blocks with different directions come at you, following the rythm of the music, just like in Beat Saber. For every music I have created a ScriptableObject called music, with every parameter it should contain like the difficulty of the song, the name, etc. but also an array of Vector3 called notes : the first float of the Vector corresponds to the time the note should be beaten in the rythm of the music starting from 0: ScriptableObject

Then in a script called SlashSpawnManagement, where every thing is based on WaitForSeconds(), I spawn the blocks to smash. It's realy hard for me to explain wih words the order I do it in, so here is an image : Explanation

In theory, what this script does, it waits for some time, spawns a block, waits for some time, spawn a block, etc. The logic seems okay, but here's the weird part. As you play the song the distance between each block gradually becomes bigger and bigger, meaning the blocks become more and more out of sync with the music. It starts very well, but at the end of the music there is at least a 5s gap.

---

I figured it has something to do with the frame rate drop, so I tried to set the frameRate to something low with :

QualitySettings.vSyncCount = 0;  // VSync must be disabled
Application.targetFrameRate = 30;

But it doesn't solve the problem. I tried with WaitForSecondsRealtime instead of WaitForSeconds, but nothing changes. I have read somewhere that WaitForSeconds depends on the frame rate... Where I calculated the time, I tried sustractng h divided by a number to even out the gradual gap. This works for some blocks but not every block: Notes[j][0] - h / 500

---

So here's my question, how do I make the WaitForSeconds, or any other method consistent with the seconds provided ?

Thank you In advance,

PS : For more clarifications, please ask and please forgive my typos and my english :)

Answer 1

Trying to use WaitForSeconds the timing of audio and hope for the best is hoping a big hope.

Have a list of Vector3s prepared in advance. If you want to prepare the list using the rythm - it will work. Use AudioSource's time to check every Update whether the timing is right and spawn a block at that moment.

void Update () {
    SpawnIfTimingIsRight();
}

void SpawnIfTimingIsRight() {
    float nextTiming = timingsArray[nextIndex].x;
    // Time For Block To Get To Position
    float blockTime = this.blockDistanceToTravel / this.blockSpeed;
    float timeToSpawnBlock = nextTiming - blockTime;
    if (this.audioSource.time >= nextTiming && (this.audioSource.time - Time.deltaTime) < nextTiming) {
        // Spawn block
    }
}