I have a webcam (actually 3) and java program to capture the frames. However, I have yet to find a way to set the FPS. For example, when I'm setting the FPS using a new VideoFormat and I can see that it is correctly applied by camFormat.intersects(newFormat). However, when I get frames from the camera (either by the BufferTransferHandler or by manually fetching), I get more frames than the selected FPS.
Is there perhaps a way to poll if a new frame is ready?
I found a little more information, so I'm editing this.
It seems the source of the problems is the Microsoft WDM Image Capture driver. In VirtualDub capture mode, I can select each separate webcam and I can select WDM. When I select WDM, I get the same options as in the Custom Format dialog in JFM, and it doesn't capture at full FPS either. However, CaptureDeviceManager.getDeviceList seems to only return the WDM driver and not the individual webcams (and two microphones inside webcams but not the sound card microphone). Why is this?
It strikes me odd that I can specify the framerate, but I can't find a way to actually enforce it, even after days of searching the internet and trying different example code. Every example I find will just either request frames at its own pace or not care about fps at all.
My test class:
Run, use the top button to (re)open the Format Control window (internal to JMF), and try to change the fps. It's also odd that besides the non-functional fps field, other fields like the resolution field are also non-functional (as it is overridden by the resolution field in the "Custom Format..." window.
import java.awt.BorderLayout;
import java.awt.Component;
import java.awt.Dimension;
import java.awt.EventQueue;
import java.awt.Graphics;
import java.awt.Image;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.awt.event.WindowEvent;
import java.awt.event.WindowListener;
import java.awt.image.BufferedImage;
import java.util.Vector;
import javax.media.Buffer;
import javax.media.CaptureDeviceInfo;
import javax.media.CaptureDeviceManager;
import javax.media.ControllerAdapter;
import javax.media.ControllerClosedEvent;
import javax.media.ControllerErrorEvent;
import javax.media.Manager;
import javax.media.MediaLocator;
import javax.media.Player;
import javax.media.RealizeCompleteEvent;
import javax.media.StartEvent;
import javax.media.StopEvent;
import javax.media.control.FormatControl;
import javax.media.control.FrameGrabbingControl;
import javax.media.format.FormatChangeEvent;
import javax.media.format.VideoFormat;
import javax.media.format.YUVFormat;
import javax.media.util.BufferToImage;
import javax.swing.JButton;
import javax.swing.JComponent;
import javax.swing.JDialog;
import javax.swing.JFrame;
import javax.swing.JOptionPane;
public class CaptureTest extends JComponent implements ActionListener, WindowListener, Runnable
{
public static void main(String[] args)
{
Runnable r = new Runnable()
{
@Override
public void run()
{
new CaptureTest();
}
};
EventQueue.invokeLater(r);
}
Player player;
FrameGrabbingControl grabber;
FormatControl formatControl;
final JFrame frame;
public Image image;
public CaptureTest()
{
setLayout(new BorderLayout());
Vector<?> deviceVector = CaptureDeviceManager.getDeviceList(new YUVFormat());
final CaptureDeviceInfo[] deviceList = new CaptureDeviceInfo[deviceVector.size()];
for (int i = 0; i < deviceList.length; i++)
{
deviceList[i] = (CaptureDeviceInfo)deviceVector.get(i);
System.out.println("Capture Device found: " + deviceList[i].getName());
}
frame = new JFrame("Capture test");
frame.addWindowListener(this);
{
JButton b = new JButton("Format Control");
b.setEnabled(true);
b.addActionListener(this);
frame.add(b, BorderLayout.NORTH);
}
{
setPreferredSize(new Dimension(640, 480));
frame.add(this, BorderLayout.CENTER);
}
ControllerAdapter cl = new ControllerAdapter()
{
@Override
public void realizeComplete(RealizeCompleteEvent rce)
{
formatControl = (FormatControl)player.getControl("javax.media.control.FormatControl");
grabber = (FrameGrabbingControl)player.getControl("javax.media.control.FrameGrabbingControl");
Component co = formatControl.getControlComponent();
if (co != null)
{
player.stop();
JDialog d = new JDialog(frame, "Format Control", true);
d.add(co);
d.pack();
d.setLocationRelativeTo(frame);
d.setVisible(true);
d.dispose();
player.start();
}
System.out.println("realizeComplete: " + formatControl.getFormat());
}
@Override
public void formatChange(FormatChangeEvent fce)
{}
@Override
public void start(StartEvent se)
{}
@Override
public void stop(StopEvent se)
{}
@Override
public void controllerError(ControllerErrorEvent cee)
{
System.err.println(cee.getMessage());
System.err.println("Caused by: " + cee.getSource());
System.exit(0);
}
@Override
public void controllerClosed(ControllerClosedEvent cce)
{}
};
MediaLocator ml = deviceList[0].getLocator();
try
{
Manager.setHint(Manager.PLUGIN_PLAYER, Boolean.TRUE);
player = Manager.createPlayer(ml);
player.addControllerListener(cl);
player.start();
}
catch (Exception e)
{
e.printStackTrace();
JOptionPane.showMessageDialog(frame, "Could Not Create Player", "ERROR", JOptionPane.ERROR_MESSAGE);
}
frame.pack();
frame.setVisible(true);
new Thread(this).start();
}
public void grab()
{
if (grabber == null)
return;
Buffer buf = grabber.grabFrame();
// System.out.println(fps.getFPS() + "\t" + buf.getFormat());
BufferToImage b2i = new BufferToImage((VideoFormat)buf.getFormat());
BufferedImage bi = (BufferedImage)b2i.createImage(buf);
if (bi != null)
{
setImage(bi);
repaint();
}
}
public void setImage(Image im)
{
image = im;
Dimension size = new Dimension(im.getWidth(null), im.getHeight(null));
setPreferredSize(size);
setSize(size);
}
@Override
public void paintComponent(Graphics g)
{
if (image != null)
g.drawImage(image, 0, 0, this);
}
@Override
public void run()
{
while (true)
{
grab();
Thread.yield();
}
}
@Override
public void actionPerformed(ActionEvent e)
{
Component co = formatControl.getControlComponent();
if (co != null)
{
player.stop();
JDialog d = new JDialog(frame, "Format Control", true);
d.add(co);
d.pack();
d.setLocationRelativeTo(frame);
d.setVisible(true);
d.dispose();
player.start();
}
}
@Override
public void windowOpened(WindowEvent e)
{}
@Override
public void windowClosing(WindowEvent we)
{
if (player != null)
{
player.stop();
player.close();
}
System.exit(0);
}
@Override
public void windowClosed(WindowEvent e)
{}
@Override
public void windowIconified(WindowEvent e)
{}
@Override
public void windowDeiconified(WindowEvent e)
{}
@Override
public void windowActivated(WindowEvent e)
{}
@Override
public void windowDeactivated(WindowEvent e)
{}
}
I found a DirectShow to Java wrapper replacement for JMF (http://www.humatic.de/htools/dsj.htm) which is a lot slower, but does the job.