How to abort the loading of a file in Maya started via cmds.file in python

I have code that runs a batch operation on files in Maya. The actual implementation isn't important, just know that it gets fed a list of filepaths, maya opens the files, then an operation is performed on each file in turn.

In cases where a reference isn't valid for whatever reason (like an invalid path) I want to abort loading of that whole scene and skip to the next in the batch list.

Looking at other questions on this site and elsewhere I've only been able to see users asking how to query references. I already have a reasonable function for doing so, it's what to do after that function returns an invalid reference path that stumps me.

Resolving invalid reference paths has historically been done manually via the popup, however in large batches constantly babysitting the maya instance isn't feasible. Nor is suppressing the popup window itself, as I believe that will still open the file, and run the batch operation on the scene while it's in an invalid state.

I have been through the maya cmds python module to try and not load references, however when using the loadReferenceDepth flag along with cmds.file I'm still getting a popup:

cmds.file(r'c:\path\to\file.ma', o=1, f=1, lrd='none')  #invalid ref popup during file cmd

The second approach was to look into the maya open api and register a callback on the before open event. The code below is functionally the same as how the batch is set up:

import maya.OpenMaya as OpenM

batchFileList = [r"c:\path\to\file.ma", r"c:\path\to\file2.ma"]

def _test_raise_exception(arg0, arg1, arg2):
    #pretending that a file ref failed below. Ref path validation code would go here.
    print '\n'*5+'Test Logging'+'\n'*5
    return False

cId = OpenM.MSceneMessage.addCheckFileCallback(OpenM.MSceneMessage.kBeforeOpenCheck, _test_raise_exception)

for file_ in batchFileList:
    try:
        rv = cmds.file(file_, o=1)
        #do stuff to the maya scene here
    except:
        #something didn't validate on load, except and skip, logging the occurrence.
        print 'load cancelled'

OpenM.MSceneMessage.removeCallback(cId)

However, even though the addCheckFileCallback indicates that if the callback function returns False the operation is aborted, the file is loaded anyway.

Likewise, replacing the return False for a raise RuntimeError doesn't let me catch the exception. Instead the cmds.file completes and only prints out a small message in the log saying "python callback failed". The python open api docs say that the bindings prefer exceptions over MStatus return codes though, so I would've expected that to work.

We have removed the MStatus class. Python exceptions must be used instead of MStatus.

Am I missing something here? There must be a way to do this. Building up a very crude .ma parser is another option but that would mean dropping support for .mb files, which I don't want to do.

Thanks for your time!

Solution

I asked this on the Maya forums and got a very helpful hint from a poster by the handle of JoeAlter-Inc:

The documentation you're referring to is for the Maya Python API 2, but the classes you're using are from the Maya Python API 1.

In API 1 most methods take exactly the same parameters as in C++. This means that a CheckFileCallback will be passed three parameters, the first of which is a reference to a C++ bool variable. To abort the file load you must set that variable to false, which requires using either ctypes or MScriptUtil.

In API 2 the CheckFileCallback returns True or False to indicate whether the file load should proceed. So in your example change 'import maya.OpenMaya' to 'import maya.api.OpenMaya' and remove one parameter from _test_raise_exception's parameter list and you should be good to go.

I went on to test it and found that if I raised an exception it was caught internally and the process continued. The answer was to add some outData, set an exception in there, and then check if it had been hit afterwards.

import maya.api.OpenMaya as OpenM
batchFileList = [r"c:\path\to\file.ma", r"c:\path\to\file2.ma"]

def _test_raise_exception(fileObject, clientData):
    #pretending that a file ref failed below. Ref path validation code would go here.
    clientData['exception'] = RuntimeError('bad ref {}'.format(fileObject.expandedFullName()))
    return False

for file_ in batchFileList:
    try:
        outData = {} 
        cId = OpenM.MSceneMessage.addCheckFileCallback(OpenM.MSceneMessage.kBeforeCreateReferenceCheck, _test_raise_exception, clientData=outData)
        rv = cmds.file(file_, o=1)
        OpenM.MSceneMessage.removeCallback(cId)
        if 'exception' in outData: #check if an exception was set, if so, raise it
            raise outData['exception']
    except:
        #handle the exception here
        print 'load cancelled'

This way I can load my files in a batch process, and handle any exceptions from a failed load in a pythonic way.