My friend has to edit a lot of video at work and he's been talking about how they have to leave the computers to render sometimes for more than one day at a time. Last job was a 40+ hour render on a 12 core Mac Pro.
Anyway, I read a little about network rendering with After Effects, and it reminded me of when I was wanting to get some raspberry pi's to learn programming for distributed computing.
My question is this: are there any projects/frameworks that abstract the "distributed" aspect so that a program designed with single-machine use in mind, like Premiere Pro, can run on a distributed system? For example, maybe the library, or whatever it is, makes individual nodes appear to said program as threads, so that it doesn't know it's running on a distributed system...?
It's a new concept to me and I don't really want to dive into this stuff right now if what I'm imagining isn't possible.
What you are referring to is called Single System Image (SSI) computing. The term describes a computing architecture for high-performance computing (not necessarily distributed computing) where a single operating system is running and thus an application programmer would be able to run multiple threads within this single operating system. This has the big advantage for the application developer that parallel computing paradigms can be applied and the developers do not have to deal with the complexities that come from distributed computing i.e. inter-process computing between several computers in a cluster and the lack of a shared memory.
Clustered computing gained more interest in the past. One of the drivers behind that trend is the fact that clusters can easily be implemented on top of general purpose cloud services and in general the cluster approach allows to scale to larger sizes which is usually a requirement in web-scale applications.
However, you specifically ask for SSI systems. From the high-performance computing arena we know many such systems. One of these which is quite popular is the Altix line https://en.wikipedia.org/wiki/Altix. These are very large scale computers which often have thousands of processors and terra-bytes of RAM and run a SSI Linux kernel. From a hardware point of view individual processor and memory boards are interconnected and support what is called cache-coherent non-uniform memory access (cc-NUMA). Which is basically a mechanism that gives the operating system the impression that it is working with one large linear memory block where in fact memory is scattered across many compute nodes. Obviously this means that there are memory regions which are faster and others are slower. Application developers need to keep that in mind. Altix uses a proprietary, high-bandwith interconnect architecture which allows them to move memory blocks in the cluster very quickly. As you may assume these machines are very expensive and usually only available in research facilities.
There are approaches to achieve something similar on top of of the shelf commodity hardware. In the past the Beowulf (https://en.wikipedia.org/wiki/Beowulf_cluster) project was very popular and used in practice in quite a lot installations. For this project I am not too sure whether or not you are able to roll out an installation with modern Linux kernels. Similarly the OpenSSI (https://en.wikipedia.org/wiki/OpenSSI) and OpenMosix (https://en.wikipedia.org/wiki/OpenMosix) projects seems to not provide any recent releases. Perhaps Kerrighed (http://www.kerrighed.org/wiki/index.php/Main_Page) is worth a shot at least you should be able to get an installation on top x86_64 architecture running.
You are mentioning that you want to roll out something like this on Pi's which use ARM processor architecture. To the best of my knowledge none of the approaches mentioned above support ARM. However, in the cluster arena MPI is a standard which is quite widely used. There are tutorials as of how to get MPI running on top of a set of Pi's (http://thenewstack.io/installing-mpi-python-raspberry-pi-cluster-runs-docker/) and those clusters are used in teaching distributed computing. Their performance can in no way compete with commercial cluster systems. However, the architecture is the same and thus these installations serve the teaching purpose very well. Just imagine a cluster system which you can bring with you in a suitcase!