Is there a way in any language for Windows, Linux or Mac (I'll use whatever OS is easiest) to programatically screen cap a different program's window?
Also programatically recording a specific box on the screen would be acceptable if necessary.
Yes, it's generally possible on all of those OSs; the details are OS-specific, of course, and it's not exactly trivial on any of them. There are commercial or freeware tools for all OSs that do it. On Linux, you could just use, for example, Xvidcap.