Prevent infinite loop when updating state via React useEffect Hook

Question

I always seem to run into this problem, and I've never been able to wrap my head around it well enough to deal with it effectively.

I'll use an example based on the code block below, which represents a React functional component that uses useState and useEffect.

Setup

1. There are some mp3 files located in an AWS S3 bucket. These have been processed so that the file names are in the format of "artist ||| title.mp3".

2. Metadata for those songs has been stored in a DyanamoDB table with a partition key of "artist" and a sort key of "title".

3. There is a function, getS3songs, that asynchronously gets a list of all songs in the form of an array of objects, with a key, "key", that holds the file name from #2 above.

4. That same function runs a forEach on the file list and parses out "artist" and "title" from each file, then does a separate asynchronous API call to get the metadata for each song from a DynamoDB table via an API Gateway.

5. An array, "songs", is created in state, using a React useState hook.

What I'm trying to do What I'm ultimately trying to do is use a useEffect hook to populate the "songs" array with the metadata returned for each song from step #4.

The problem The following code block results in an infinite loop being run, as [songs] is set as second parameter of the useEffect hook.

I've tried several variations, but I believe the below represents the crux of the problem I'm trying to work through.

Note The tricky part here isn't the initial get of the "s3Songs". This could be put straight into state as a single object. The tricky part is that there are multiple asynchronous calls to an API to get the metadata for each file and getting each one of these objects into the "songs" array. And this is what's doing my head in.

Question What is the best or recommended pattern for this problem?

import React, { useEffect, useState } from "react";
import Amplify, { API, Storage } from "aws-amplify";
import awsconfig from "./aws-exports";

Amplify.configure(awsconfig);
const StackOverflowExample = () => {
  const [songs, setSongs] = useState([]);
  useEffect(() => {
    const getS3Songs = async () => {
      const s3Songs = await Storage.list("", { level: "public" });
      s3Songs.forEach(async song => {
        const artist = song.key.split(" ||| ")[0];
        const title = song.key.split(" ||| ")[1].slice(0, -4);
        const metadata = await API.get("SongList", `/songs/${artist}/${title}`);
        // setSongs([...songs, metadata]); <= causes loop
      });
    };
    getS3Songs();
  }, [songs]); // <= "songs" auto added by linter in create-react-app in vsCode.  Removing "songs" and disabling linter on that line doesn't help.

const renderSongs = () => {
  return songs.map(song => {
    return <li>{song.title}</li>;
  });
};

return (
  <div>
    <ul>{renderSongs()}</ul>
   </div>
  );
};

export default StackOverflowExample;

Update Based on Will's comment regarding separating into two useEffect hooks. I have tried the below, and I'm thinking if I can work a Promise.all into the mix so as to return the songMetadata array in the second hook, after it's populated when all metadata promises have resolved, I'd be getting close.

  useEffect(() => {
    console.log("Effect!");
    const getS3Files = async () => {
      try {
        const response = await Storage.list("", { level: "public" });
        setS3FileList(response);
      } catch (e) {
        console.log(e);
      }
    };
    getS3Files();
  }, []);

  useEffect(() => {
    console.log("Effect!");
    const songMetadata = [];
    s3FileList.forEach(async song => {
      const artist = song.key.split(" ||| ")[0];
      const title = song.key.split(" ||| ")[1].slice(0, -4);
      const metadata = await API.get(
        "SongList",
        `/songs/object/${artist}/${title}`
      );
      console.log(metadata);
      songMetadata.push(metadata);
    });
    setSongs(songMetadata);
  }, [s3FileList]);

Answer 1

It looks like you need to separate the concerns of your two fetches by adding another useEffect. Then you can use Promise.all to wait for the responses from your second api call to complete before updating your songs.

const StackOverflowExample = () => {
  const [songs, setSongs] = useState([]);
  const [s3Songs, setS3Songs] = useState([]);
  const getSong = async song => {
    const artist = song.key.split(" ||| ")[0];
    const title = song.key.split(" ||| ")[1].slice(0, -4);
    const metadata = await API.get("SongList", `/songs/${artist}/${title}`);
    return metadata
  }
  useEffect(() => {
    const getS3Songs = async () => {
      const s3s = await Storage.list("", { level: "public" })n
       setS3Songs(s3s);
    };
    getS3Songs();
  }, []);
  useEffect(()=>{
     const pending = s3Songs.map(song=>getSong(song))
     Promise.all(pending).then(songs=>setSongs(songs));
  }, [s3Songs])
  const renderSongs = () => {
     return songs.map(song => {
       return <li>{song.title}</li>;
     });
};
return (
  <div>
    <ul>{renderSongs()}</ul>
   </div>
  );
};