Choppy frame rate using a SourceDataLine

Question

I've been programming sound with simple Swing graphics for a little while, but my frame rates are choppy for some reason.

Generally I'm doing something like the following on a background thread:

for(;;) {
    // do some drawing
    aPanel.updateABufferedImage();
    // ask for asynchronous repaint
    aPanel.repaint();

    // write the sound
    aSourceDataLine.write(bytes, 0, bytes.length);
}

Through debugging, I think I've already traced the problem to the blocking behavior of SourceDataLine#write. Its doc states the following:

If the caller attempts to write more data than can currently be written [...], this method blocks until the requested amount of data has been written.

So, what this seems to mean is SourceDataLine actually has its own buffer that it is filling when we pass our buffer to write. It only blocks when its own buffer is full. This seems to be the holdup: getting it to block predictably.

To demonstrate the issue, here's a minimal example which:

• writes 0's to a SourceDataLine (no audible sound) and times it.
• draws an arbitrary graphic (flips each pixel color) and times the repaint cycle.

import javax.swing.*;
import java.awt.*;
import java.awt.event.*;
import java.awt.image.*;
import javax.sound.sampled.*;

class FrameRateWithSound implements Runnable {
    public static void main(String[] args) {
        SwingUtilities.invokeLater(new FrameRateWithSound());
    }

    volatile boolean soundOn = true;
    PaintPanel panel;

    @Override
    public void run() {
        JFrame frame = new JFrame();
        JPanel content = new JPanel(new BorderLayout());

        final JCheckBox soundCheck = new JCheckBox("Sound", soundOn);
        soundCheck.addActionListener(new ActionListener() {
            @Override
            public void actionPerformed(ActionEvent e) {
                soundOn = soundCheck.isSelected();
            }
        });

        panel = new PaintPanel();

        content.add(soundCheck, BorderLayout.NORTH);
        content.add(panel, BorderLayout.CENTER);

        frame.setContentPane(content);
        frame.pack();
        frame.setLocationRelativeTo(null);
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        frame.setVisible(true);

        new Thread(new Worker()).start();
    }

    class Worker implements Runnable {
        @Override
        public void run() {
            AudioFormat fmt = new AudioFormat(
                AudioFormat.Encoding.PCM_SIGNED,
                44100f, 8, 1, 1, 44100f, true
            );

            // just 0's
            byte[] buffer = new byte[1000];

            SourceDataLine line = null;
            try {
                line = AudioSystem.getSourceDataLine(fmt);
                line.open(fmt);
                line.start();

                for(;;) {
                    panel.drawNextPixel();
                    panel.repaint();

                    if(soundOn) {
                        // time the write
                        long t = System.currentTimeMillis();

                        line.write(buffer, 0, buffer.length);

                        t = ( System.currentTimeMillis() - t );
                        System.out.println("sound:\t" + t);
                    }

                    // just so it doesn't fly off the handle
                    Thread.sleep(2);
                }
            } catch(Exception e) {
                // lazy...
                throw new RuntimeException(e);
            } finally {
                if(line != null) {
                    line.close();
                }
            }
        }
    }

    class PaintPanel extends JPanel {
        Dimension size = new Dimension(200, 100);

        BufferedImage img = new BufferedImage(
            size.width, size.height, BufferedImage.TYPE_INT_RGB);

        int x, y;

        int repaints;
        long begin, prev;
        String fps = "0";

        PaintPanel() {
            setPreferredSize(size);
            setOpaque(false);

            Graphics2D g = img.createGraphics();
            g.setColor(Color.LIGHT_GRAY);
            g.fillRect(0, 0, size.width, size.height);
            g.dispose();
        }

        synchronized void drawNextPixel() {
            img.setRGB(x, y, img.getRGB(x, y) ^ 0xFFFFFF); // flip

            if( ( ++x ) == size.width ) {
                x = 0;
                if( ( ++y ) == size.height ) {
                    y = 0;
                }
            }
        }

        @Override
        protected synchronized void paintComponent(Graphics g) {
            super.paintComponent(g);
            g.drawImage(img, 0, 0, size.width, size.height, null);

            long curr = System.currentTimeMillis();
            // time this cycle
            long cycle = ( curr - prev );
            System.out.println("paint:\t" + cycle);

            ++repaints;
            // time FPS every 1 second
            if(curr - begin >= 1000) {
                begin = curr;
                fps = String.valueOf(repaints);
                repaints = 0;
            }

            prev = curr;

            g.setColor(Color.RED);
            g.drawString(fps, 12, size.height - 12);
        }
    }
}

I recommend actually running the example if you are curious about this.

A typical System.out feed during "playback" is something like the following:

sound:  0
paint:  2
sound:  0
paint:  2
sound:  0
paint:  3
sound:  0
paint:  2
paint:  2
sound:  325 // <- 'write' seems to be blocking here
sound:  0
paint:  328
sound:  0
paint:  2

This shows the behavior of write pretty clearly: it spins a majority of the time, then blocks for an extended period, at which point the repaints chug as well. The FPS meter typically displays ~45 during playback, but the animation is obviously choppy.

When sound is turned off, FPS climbs and the animation is smooth.

So is there a way to fix it? What am I doing wrong? How can I get write to block at a regular interval?

This behavior is apparent both on Windows and OSX environments.

One thing I've tried is using Thread.sleep to regulate it, but it's not very good. It's still choppy.

Answer 1

The solution seems to be to use open(AudioFormat, int) to open the line with a specified buffer size.

line.open(fmt, buffer.length);

Timing it again, we can see that write blocks much more consistently:

sound:  22
paint:  24
sound:  21
paint:  24
sound:  20
paint:  22
sound:  21
paint:  23
sound:  20
paint:  23

And the animation is smooth.