Search code examples
c#language-extension

Implementing C# language extensions

Using systems such as Parallel Linq, it's possible to split up execution of anonymous functions, queries, etc across multiple cores and threads within a single machine. I'd like the ability to extend this to run across multiple machines using standard language constructs such as for loops (like Parallel.For()), value types like ints, structs, etc., and keep the application source modifications to a minimum. Ideally, this would allow me to open a project, add an attribute to a method, and recompile to gain access to the enhanced functionality.

It seems that I'd need something along the lines of:

  1. The ability to capture a compiled block of code (such as a lambda) and pass it to a worker process running on another node, along with any data that is required, or

  2. Provide a preprocessor that would capture the code in question, compile it in a sort of template project that would replace variable references, etc, with references to a class that would handle network communication, caching and access to any other required assets, and send the resulting DLL to any available worker nodes running on other machines.

Roslyn appears to provide some tools that would be useful here. Is there way to hook into the current compilation pipeline to allow this?

Edit

Okay, I know this is possible, because these guys did it. The question is, how?

Solution

  • You don't have to extend the language persay to do what Brahma does. He just implemented a custom query provider that parses expression trees and emits GPGPU code (LINQ to SQL does the same thing but with SQL).

    I linked a basic guide on MSDN here that can get you up and running implementing an IQueryable provider.

    The hard part will be traversing the expression trees and generating OpenCL code. Once you can do that you just pass it off to Cloo and you should be running.

    Edit

    You linked a tool that compiles standard .NET code to GPU code with an attribute [Kernel]. They do this by having a post-build tool look for the attributes in the compiled IL and they perform IL rewriting to generate GPU calls. This is similar to PostSharp, an AOP solution.

    IL rewriting is time consuming and hard work but you could also go this route.