Azure ML ops task to write a file in git repo

Question

I am trying to implement ml ops in azure. I am running a python script through azure cli task in devops. Though I can read files from the git folder but the py script is not able to generate the output csv in git. Strangely its also not giving any error.

I think the file is getting generated in the compute instance directory. How to instead write it to a git folder or any folder which I can see in the compute engine.

Answer 1

I had a similar situation where python couldn't find the files that were supposed to exist in the root of the Azure ML project folder after deploying. After investigation, I realized that Azure ML invokes your scripting code from a different root folder.

Here is an example of an operation that reads from the relative path where your code exists:

    SCRIPT_DIRECTORY = os.path.dirname(os.path.realpath(__file__))
    with open(SCRIPT_DIRECTORY+'filename.json', 'w') as outfile:
        json.dump(dict_object, outfile)

You can then join to SCRIPT_DIRECTORY the relative path of your git folder, before your output.

Alternatively, as per your comment "./output is not getting created", you can force it with:

os.makedirs("./outputs", exist_ok=True)

exist_ok (optional) : A default value False is used for this parameter. If the target directory already exists an OSError is raised if its value is False otherwise not. For value True leaves directory unaltered.