I am using doit to automate my data analysis pipeline as described in https://blogs.aalto.fi/marijn/2016/02/25/doit-a-python-alternative-to-make/. Unfortunately, doit always wants to re-run the entire pipeline from step1 although the file_deps for the later steps are already satisfied. How can I find out what caused doit to re-run the entire pipeline? E.g. a misconfigured dependency, changes in dodo.py?
Sample code would be helpful. With out knowing exactly whats going on it maybe hard to help. But in hopes of leading you in the right direction, here are some pointers;
doit has strace functionality, which has some caveats but may be of help, http://pydoit.org/cmd_other.html?highlight=strace#strace .
By default, doit will not rerun tasks if your dodo.py file has changed. I would check your code to see if you have added your dodo.py file as a file_deps someplace; http://pydoit.org/faq.html#dodo-py-file-itself-should-be-a-file-dep-for-all-tasks
If you are still having issues, check out the users group and post there. Edward seems to be quick to respond.