PyCharm "Cannot find reference" error with nested packages in a Vagrant VM

Question

I'm working on a Python project that utilizes a unique nested package structure and is running within a Vagrant VM. Everything works fine when I run the code within the VM, but PyCharm IDE is throwing an error when trying to import these packages, even though the interpreter is set to the same Python interpreter as the VM.

The project structure looks something like this:

- root_folder
-- package1
--- src
---- app
----- package1
------ subpackage1
-- package2
--- src
---- app
----- package2
------ subpackage2
-- service
--- Vagrantfile
--- src
---- app
----- service
------ service1

Each src directory is marked as a Source Root in PyCharm. The Python interpreter for the project is set to the interpreter inside the Vagrant VM. The Vagrantfile is syncing the appropriate root directory.

Here is the VagrantFile content

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure("2") do |config|

  config.vm.define "my-service" do |my_service|
    my_service.vm.box = "ubuntu/focal64"
    my_service.vm.hostname = "my-service"
    my_service.vm.network "private_network", ip: "192.168.79.10"
    
    # Syncs the root folder of the project with the /vagrant directory in the VM
    my_service.vm.synced_folder "../..", "/vagrant"

    my_service.vm.provider "virtualbox" do |vb|
      vb.name = "My Service"
      vb.gui = false
      vb.memory = 2048
      vb.cpus = 2
    end

    my_service.vm.provision "ansible-base", type: "ansible" do |ansible|
      ansible.playbook = "vagrant-ansible/my-service.yml"

      ansible.extra_vars = {
          ansible_python_interpreter: "/usr/bin/python3",
      }
    end
  end

end

The issue arises when trying to import a package. For instance, when trying to import app.package1.subpackage1, I get an error in PyCharm: "Cannot find reference 'package1' in 'init.py'". However, this works perfectly when I run the code inside the VM.

Note that I cannot restructure the project or change how the imports are done. They need to stay the same.

I have tried checking the project interpreter, invalidating PyCharm's caches, verifying the PYTHONPATH, and confirming the package structure, all of which appear to be correct.

I am looking for a solution to prevent these erroneous import errors from appearing in PyCharm, while keeping the project structure and import methodology intact. Any suggestions would be greatly appreciated!

Answer 1

After further investigation, I discovered that the issue was due to the presence of an __init__.py file in only one of the packages.

My project structure was like this:

- root_folder
  -- package1
     --- src
       ---- app
         ----- package1
           ------ subpackage1
  -- package2
     --- src
       ---- app
         ----- package2
           ------ subpackage2
  -- service
     --- Vagrantfile
       --- src
         ---- app
         ---- __init__.py
           ----- service
             ------ __init__.py
               ------- service1

As you can see, there was an __init__.py file present only in the service package. __init__.py files are used by Python to recognize a directory as a Python package, which is a way of organizing related modules. However, PyCharm also uses these files to determine the boundaries of your project and its modules. Having __init__.py in only one package may have caused PyCharm to interpret your project structure incorrectly.

After I removed the __init__.py file from the service package, PyCharm was able to correctly resolve the package references and the issue was resolved.

This experience underlines the importance of consistency in your project structure, particularly when it comes to the use of __init__.py files. If your project uses them, they should be present in all packages; if it does not use them, they should not be present at all.

I hope this helps anyone else encountering a similar issue!