I wanted to capture only certain window using Imagemagick or Ffmpeg, but I heard that the x11 id needed for that is not supported on Mac.
The purpose is to capture the application window so that all areas are displayed even if some areas are out of the display and are not visible.
In general, applications on macOS use the native Cocoa framework to generate their GUI rather than X11. I'll write my answer to address each possibility in term, separating the two with a horizontal line.
You can install an X11 server called XQuartz on macOS using homebrew with:
brew cask install xquartz
However, only applications written to the X11 interface will create XQuartz windows and you can capture those windows in an X11 way, using xwininfo and passing the id to ImageMagick. However, relatively few apps use X11.
Let's just show that quickly with an example:
xeyes & # start "xeyes" which is an X11 app and get our prompt back
xwininfo -name xeyes # so we can get its id
...
...
xwininfo: Window id: 0xa0000a "xeyes"
...
...
# Tell ImageMagick to grab the "xeyes: window by its id and save as "xeyes.png"
magick import -window 0xa0000a xeyes.png
Another issue is that the homebrew version of ImageMagick doesn't support X11... so you'll have to either edit the homebrew formula, or run ./configure yourself and include X11 support - which is non-trivial.
If your app uses the native Cocoa interface, you can get its "Cocoa id" using a script I shared here. Slightly more simply, you can run some AppleScript to get the window id, e.g. to get the id of a window belonging to the Terminal app:
osascript -e 'tell app "Terminal" to id of window 1'
You can then use that id with the screencapture command supplied with macOS to hopefully do what you want with whatever application you are using. For example:
/usr/sbin/screencapture -l <WINDOWID> image.png
On macOS, ffmpeg uses AVFoundation under the covers, so first you need to get the index that AVFoundation assigns to your screen, like this:
ffmpeg -hide_banner -f avfoundation -list_devices true -i ""
[AVFoundation indev @ 0x131e05df0] AVFoundation video devices:
[AVFoundation indev @ 0x131e05df0] [0] FaceTime HD Camera
[AVFoundation indev @ 0x131e05df0] [1] Capture screen 0 <-- THIS LINE
[AVFoundation indev @ 0x131e05df0] AVFoundation audio devices:
[AVFoundation indev @ 0x131e05df0] [0] MacBook Pro Microphone
Look at the listing above and you can see that I must use device [1] if I want to record the screen. As I don't want sound recorded, I use none for the sound channel, so the basic ffmpeg command to record the screen on my Mac at 30fps will be:
ffmpeg -r 30 -f avfoundation -i "1:none" ...
Now, you want to record a specific window, but ffmpeg doesn't know about windows, it only knows coordinates, so we need to find the coordinates of our window. Imagine I want to record Safari main window, first I get its location with:
osascript -e 'tell application "Safari" to get the bounds of the front window'
87, 43, 1290, 538 # top-left x, top-left y, width, height
Now I tell ffmpeg to record that:
ffmpeg -y -r 30 -f avfoundation -i "1:none" -vf "crop=1290:538:87:43" screen.mp4
and it appears to work, but doesn't! Apparently there's no atom and ffplay can't play stuff without atoms. It transpires that atoms get written at the end of the video - unless you Control-C out of them. So, you can now make sure you don't need to Control-C by adding a duration of 5 seconds:
ffmpeg -y -r 30 -f avfoundation -i "1:none" -t 5 -vf "crop=1290:538:87:43" screen.mp4
That also appears to work, but doesn't actually work either. You can play it with ffplay but not Apple's QuickTime which only likes yuv420p format, so you can actually do what you want with:
ffmpeg -y -r 30 -f avfoundation -i "1:none" -t 5 -vf "crop=1290:538:87:43,format=yuv420p" screen.mp4