Is it possible to speed up scripts using multi-threading?

Question

I have to perform CPU intensive analysis on data in DigitalMicrographs's DM-script language. I noticed that only a single CPU core maxes out during processing. Is there a way to achieve better performance by multi-threading in DM scripting? Simple examples would be appreciated.

Answer 1

Multi-threading is certainly possible in DM-scripting, and it is documented in the F1 help here:

Whether or not speed improvement can be achieved depends on various things, most importantly whether or not the individual threads need access to the same resource (like the same data, or some GMS resource which is only available via the main-thread - f.e. the UI).

Also, a lot of data-processing is already multi-threaded internally when you use commands on image-expressions. You might achieve a lot more speedup by rephrasing your analytical processing in a way that doesn't requires for-loops in the scripting language but uses image-expressions instead.

Finally, doing things multi-threaded is a great way of introducing bugs and unexpected behavior which can be really hard to debug. Don't be frustrated if you run into those things while learning stuff.

That said, the below example demonstrates (at least on my machine) a speed improvement by "chunking" some data-analysis over multiple parallel background threads.

// Example showing the explicit use of multi-threading in DM scripting 

class CMultiThreadtest
{
    image data, keep
    number sx,sy,sz 
    number nChunks, chunksize, lastChunk, doneChunk 
    
    object SetData(object self, image img, number nChunks_) 
    { 
        if ( img.imagegetNumdimensions() != 3 ) throw( "only 3D data for testing please")
        img.ImageGetDimensionSizes(sx,sy,sz)
        nChunks = nChunks_
        if ( sz % nChunks != 0 ) Throw( "Z-size needs to be integer multiple of nChunks for this test.")
        chunksize = sz / nChunks
        
        data:=img
        keep = data
        
        return self
    }
    
    void CompareResult(object self){
        image dif := keep*2-data
        number ok = 0==sum(dif)
        Result("\n\t Result is " + (ok?"correct":"wrong"))
    }
    
    void RunOnData(object self){
        
        // For extra-caution of thread safety, the two lines below shoud be guarded with critical sections
        // but given the near-atomic nature of the call, this is omitted here.
        number chunkIndex = lastChunk
        lastChunk++
        
        image work := data.slice3(0,0,chunkIndex*chunksize, 0,sx,1, 1,sy,1, 2,chunksize,1)
        number startp = GetHighresTickCount()       
        for( number z=0;z<chunksize;z++)
            for( number y=0;y<sy;y++ )
                for( number x=0;x<sx;x++ ){
                        work[x,y,z] *= 2
        }
        number endp = GetHighresTickCount()     
        Result("\n\t\t Process (chunk "+chunkIndex+") done with " + sx*sy*chunksize + " steps in " + (endp-startp)/GetHighResTicksPerSecond())
        
        // For extra-caution of thread safety, the line below shoud be guarded with critical sections
        // but given the near-atomic nature of the call, this is omitted here.
        doneChunk++
    }
    
    void RunWithSubsets(object self, image src, number nChunks_, number inbackground){
        self.SetData(src, nChunks_)
        lastChunk = 0
        doneChunk = 0
        Result("\n.....\n Running with "+nChunks+" chunks of size "+chunksize+ " on " + (inbackground?" multiple background threads":" single main thread") +":")
        number startp = GetHighresTickCount()   
        for( number i=0; i<nChunks; i++){
            if ( inbackground )
                self.StartThread("runondata")
            else
                self.RunOnData()
        }   
        
        while( doneChunk != nChunks ){
            if ( ShiftDown() ){
                Throw("abort")
            doEvents()
            }
        }
        number endp = GetHighresTickCount()     
        Result("\n Total duration:" + (endp-startp)/GetHighResTicksPerSecond())
        self.CompareResult();
        Result("\n.....")
        
    }
};

void Test(){
    image img := RealImage("test cub",4,50,50,10)
    img = random()
    clearresults()
    object tester = Alloc(CMultiThreadtest)
    tester.RunWithSubsets(img, 1, 0)
    tester.RunWithSubsets(img, 1, 1)
    tester.RunWithSubsets(img, 5, 0)
    tester.RunWithSubsets(img, 5, 1)
}
test()