Module not found error while python file is in same folder

Question

I am currently working in Databricks and trying to run code from https://github.com/google/automl/tree/master/efficientdet#7-eval-on-coco-2017-val-or-test-dev%5Befficientdet%5D(https://stackoverflow.com)

I am trying to create a tfrecord from coco data.

Now this is what the directory looks like where I am working in:

.
├── Det-AdvProp.md
├── README.md
├── __init__.py
├── aug
│   ├── __init__.py
│   ├── autoaugment.py
│   ├── autoaugment_test.py
│   ├── gridmask.py
│   ├── gridmask_test.py
│   ├── mosaic.py
│   └── mosaic_test.py
├── backbone
│   ├── __init__.py
│   ├── backbone_factory.py
│   ├── efficientnet_builder.py
│   ├── efficientnet_builder_test.py
│   ├── efficientnet_lite_builder.py
│   ├── efficientnet_lite_builder_test.py
│   ├── efficientnet_model.py
│   └── efficientnet_model_test.py
├── coco_metric.py
├── coco_metric_test.py
├── dataloader.py
├── dataloader_test.py
├── dataset
│   ├── README.md
│   ├── __init__.py
│   ├── create_coco_tfrecord.py  <-- I am running this file
│   ├── create_coco_tfrecord_test.py  
│   ├── create_pascal_tfrecord.py
│   ├── create_pascal_tfrecord_test.py
│   ├── inspect_tfrecords.py
│   ├── label_map_util.py   <-- And the file above is trying to import this file
│   └── tfrecord_util.py

I am running this command.

!PYTHONPATH=".:$PYTHONPATH"  python dataset/create_coco_tfrecord.py \
    --image_dir=val2017 \
    --object_annotations_file=annotations/instances_val2017.json \
    --output_file_prefix=tfrecord/val \
    --num_shards=32

And I am getting this error:

Traceback (most recent call last):
  File "/databricks/driver/automl/efficientdet/dataset/create_coco_tfrecord.py", line 42, in <module>
    from dataset import label_map_util
ModuleNotFoundError: No module named 'dataset'

Try 1

I have tried to append the parent directory to sys.path.

sys.path.append("/databricks/driver/automl/efficientdet/dataset/..")

This didn't help at all. Still got the same error.

Try 2

from .dataset import label_map_util

This resulted in this error

Traceback (most recent call last):
  File "/databricks/driver/automl/efficientdet/dataset/create_coco_tfrecord.py", line 42, in <module>
    from .dataset import label_map_util
ImportError: attempted relative import with no known parent package

Try 3

I've checked that I am running the command in the "dataset" directory

That doesn't change anything as well

Try 4

I have also checked with os.getcwd() what the working directory is.

/databricks/driver/automl/

Any other solutions?

If someone would know other things one could try, i'd appreciate it!

Answer 1

Alright, it had to do with the working directory of create_coco_tfrecord.py file. Even though I changed my working directory to /databricks/driver/automl/efficientdet/dataset/ in databricks itself, it didn't change it for the create_coco_tfrecord.py file.

So when checking the current working directory in create_coco_tfrecord.py using os.getcwd(). I got /databricks/driver/automl/.

So when python tries to run from dataset import label_map_util it couldn't find the module because there isn't a "dataset" directory in /databricks/driver/automl/

The dataset directory is in /databricks/driver/automl/efficientdet directory

So the solution the my problem is to change it to from efficient.dataset import label_map_util.