I am in the process of developing a software emulation of a theremin in React. I am looking to use the user's webcam to track hands, which can then be used to control the synth (WebAudio API). I would just like some clarification on the best way to implement my components.
I have a video component, and a synthesizer component. Of course, the constant data stream output from the video will be used to consistently change the states in the synthesizer ... I have [frequency, setFrequency] and [volume, setVolume] as useStates() in a 'synthesizer' functional react components.
What is the best way to structure my components to achieve this? Should the data be passed into the synthesizer as props? Or is there a more appropriate method for achieving this? Is it better to simply combine them into one constantly-rerendering component?
Thanks!
//Perhaps data passed in as props?
export const Synthesizer = ({ freq, vol }) => {
//Changed with a slider
const [wavetype, setWavetype] = useState(0);
//CHANGED BY VIDEO DATA
const [volume, setVolume] = useState(1);
//CHANGED BY VIDEO DATA
const [frequency, setFrequency] = useState(100);
useEffect(() => {
osc.type = waveforms[wavetype];
}, [wavetype]);
useEffect(() => {
osc.frequency.value = frequency;
}, [frequency]);
useEffect(() => {
gain.gain.value = volume;
}, [volume]);
return( //return code here... );
There are a few ways to do that. You can pass the props into the synthesizer component like your code. Or you can use a state management tool like Redux or the Context API, so that you can share the state all over the app and access it in any components.
Generally speaking, it is not recommended to combine two components and have one component that constantly re-renders, this will reduce the performace.
Here is how you can implement it with different approaches:
We have a parent component that contains both the Video and Synthesizer components and it passes the props to the Synthesizer component, which uses the data to update its state variables.
import React, { useState } from 'react';
import Video from './Video';
import Synthesizer from './Synthesizer';
function App() {
const [videoData, setVideoData] = useState({
volume: 1,
frequency: 100,
});
// Update videoData state when video data changes
function handleVideoDataChange(newVideoData) {
setVideoData(newVideoData);
}
return (
<div>
<Video onVideoDataChange={handleVideoDataChange} />
<Synthesizer volume={videoData.volume} frequency={videoData.frequency} />
</div>
);
}
import React, { useState, useEffect } from 'react';
function Synthesizer(props) {
const [wavetype, setWavetype] = useState(0);
const [volume, setVolume] = useState(props.volume);
const [frequency, setFrequency] = useState(props.frequency);
useEffect(() => {
osc.type = waveforms[wavetype];
}, [wavetype]);
useEffect(() => {
osc.frequency.value = frequency;
}, [frequency]);
useEffect(() => {
gain.gain.value = volume;
}, [volume]);
return (
// Synthesizer UI code here
);
}
The video component updates the shared state object with new video data, which triggers a re-render of the Synthesizer component with the updated state.
import React, { useState, useContext } from 'react';
import { VideoDataContext } from './VideoDataContext';
function Video() {
const [videoData, setVideoData] = useContext(VideoDataContext);
// Update shared state object with new video data
function handleVideoDataChange(newVideoData) {
setVideoData(newVideoData);
}
return (
// Video component code here
);
}
The Synthesizer component can access the shared state object via the useContext hook and use it to update its state variables:
import { VideoDataContext } from './VideoDataContext';
function Synthesizer() {
const [wavetype, setWavetype] = useState(0);
const [volume, setVolume] = useState(1);
const [frequency, setFrequency] = useState(100);
const [videoData] = useContext(VideoDataContext);
// Update state variables with video data from shared state object
useEffect(() => {
setVolume(videoData.volume);
setFrequency(videoData.frequency);
}, [videoData]);
useEffect(() => {
osc.type = waveforms[wavetype];
}, [wavetype]);
useEffect(() => {
osc.frequency.value = frequency;
}, [frequency]);
useEffect(() => {
gain.gain.value = volume;
}, [volume]);
return (
// Synthesizer UI code here
);
}
There will be more boilerplates with Redux and it is usually not recommended to use for small apps. First, you need to create an action type and action creator for updating the video data:
export const UPDATE_VIDEO_DATA = 'UPDATE_VIDEO_DATA';
export function updateVideoData(newData) {
return { type: UPDATE_VIDEO_DATA, payload: newData };
}
Then you should create a reducer function to handle the UPDATE_VIDEO_DATA action and update the video data in the state:
const initialState = {
videoData: {
volume: 1,
frequency: 100,
},
};
function videoReducer(state = initialState, action) {
switch (action.type) {
case UPDATE_VIDEO_DATA:
return { ...state, videoData: action.payload };
default:
return state;
}
}
You also need to create a store with the videoReducer:
import { createStore } from 'redux';
import videoReducer from './reducers/videoReducer';
const store = createStore(videoReducer);
In the Video component, you can dispatch the updateVideoData action to update the video data in the state:
import React from 'react';
import { useDispatch } from 'react-redux';
import { updateVideoData } from '../actions/videoActions';
function Video() {
const dispatch = useDispatch();
// Update video data in the state when it changes
function handleVideoDataChange(newVideoData) {
dispatch(updateVideoData(newVideoData));
}
return (
// Video component code here
);
}
In the Synthesizer component, you can use the useSelector hook to access the video data from the state and update the component's state variables:
import React, { useState, useEffect } from 'react';
import { useSelector } from 'react-redux';
function Synthesizer() {
const [wavetype, setWavetype] = useState(0);
const [volume, setVolume] = useState(1);
const [frequency, setFrequency] = useState(100);
const videoData = useSelector(state => state.videoData);
useEffect(() => {
setVolume(videoData.volume);
setFrequency(videoData.frequency);
}, [videoData]);
useEffect(() => {
osc.type = waveforms[wavetype];
}, [wavetype]);
useEffect(() => {
osc.frequency.value = frequency;
}, [frequency]);
useEffect(() => {
gain.gain.value = volume;
}, [volume]);
return (
// Synthesizer UI code here
);
}
With this setup, the Video component dispatches an action to update the video data in the state, which triggers a re-render of the Synthesizer component with the updated state data. As you can see this will not be the approach if your app is going to be a big one. The best approach will depend on the specific needs of your app and the complexity of the data flow between the video component and synthesizer component.