I am learning to use luigi and I would like to run this task I wrote which opens a file, modifies it, and re-saves the output:
class WindDirection(luigi.Task):
uas_filepath = luigi.parameter.Parameter()
def output(self):
return luigi.LocalTarget("testing/wdir.nc")
def run(self):
# My task
with nc.Dataset(self.uas_filepath, 'r') as uas_reader:
wdir = NCdata.wind_dir_from_component(uas_reader)
# Writing out the output
wdir.write(self.output())
The task runs fine, but it is saving the file as <luigi.local_target.LocalTarget object at 0x7f86ed64ee48> in the same directory as the task instead of in the testing folder with the name wdir.nc (the file contents are as they should be). Perhaps because of the naming issue, when I re-run the task it creates a new copy of the file instead of seeing that the ouptut file of this task has already been created. I have tried both relative and absolute filepaths as input to the LocalTarget object. The .write method that I am calling needs a string input that is a filepath, and I am wondering if that is causing issues.
What do I need to do so that the output file will be saved with the name I gave it?
In case it is important this is the command I am using to run the task:
PYTHONPATH='.' luigi --module basic_luigi WindDirection --uas-filepath /aboslute/path/to/my/file.nc --local-scheduler
It should be
class WindDirection(luigi.Task):
uas_filepath = luigi.parameter.Parameter()
def output(self):
return luigi.LocalTarget("testing/wdir.nc")
def run(self):
# My task
with nc.Dataset(self.uas_filepath, 'r') as uas_reader:
wdir = NCdata.wind_dir_from_component(uas_reader)
# Writing out the output
wdir.write(self.output().path)
When you call self.output() it will return luigi.LocalTarget("testing/wdir.nc") and not the path for the output. If you want the path you should call self.output().path