GAnalytics : script was working, now multiple module not found (GAnalytics lib)

Question

I made some modification on a script that was perfectly working for 2 months. Some code factorisation, add a bit of logging here and there... nothing big.

Now when I try my script I got several error that some google modules (I get data from Google Analytics here) are missing ?

I tried to update all the packages (I'm using conda) but I keep having the same errors :

WARNING:googleapiclient.discovery_cache:file_cache is unavailable when using oauth2client >= 4.0.0 or google-auth
Traceback (most recent call last):
  File "/Users/gil/anaconda3/envs/GA2DBenv/lib/python3.7/site-packages/googleapiclient/discovery_cache/__init__.py", line 36, in autodetect
    from google.appengine.api import memcache
ModuleNotFoundError: No module named 'google.appengine'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/gil/anaconda3/envs/GA2DBenv/lib/python3.7/site-packages/googleapiclient/discovery_cache/file_cache.py", line 33, in <module>
    from oauth2client.contrib.locked_file import LockedFile
ModuleNotFoundError: No module named 'oauth2client.contrib.locked_file'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/gil/anaconda3/envs/GA2DBenv/lib/python3.7/site-packages/googleapiclient/discovery_cache/file_cache.py", line 37, in <module>
    from oauth2client.locked_file import LockedFile
ModuleNotFoundError: No module named 'oauth2client.locked_file'

Answer 1

The issue in general lays within the way Google packages and maintains their modules.

The solutions, that are working in my environment(s), for the issues are as follows:

• ModuleNotFoundError: No module named 'google.appengine': https://github.com/GoogleCloudPlatform/getting-started-python/issues/157#issuecomment-577467711
• ModuleNotFoundError: No module named 'oauth2client.contrib.locked_file' and ModuleNotFoundError: No module named 'oauth2client.locked_file': https://github.com/googleapis/google-api-python-client/issues/325#issuecomment-274349841
  • Note that the file-cache fix is silencing/removing the locked_file warnings becuase the default/built-in caching is either disabled or replaced with custom Class (https://github.com/googleapis/google-api-python-client/issues/325#issuecomment-419387788). For your conviniance, here is actual code pieces:

File Cache disabled (bad solution):

discovery.build('drive', 'v3', http=http, cache_discovery=False)

File Cache workaround-ed (better solution):

import os.path
import hashlib
import tempfile

class DiscoveryCache:
    def filename(self, url):
        return os.path.join(
            tempfile.gettempdir(),
            'google_api_discovery_' + hashlib.md5(url.encode()).hexdigest())

    def get(self, url):
        try:
            with open(self.filename(url), 'rb') as f:
                return f.read().decode()
        except FileNotFoundError:
            return None

    def set(self, url, content):
        with tempfile.NamedTemporaryFile(delete=False) as f:
            f.write(content.encode())
            f.flush()
            os.fsync(f)
        os.rename(f.name, self.filename(url))

discovery.build('drive', 'v3', http=http, cache=DiscoveryCache())

I hope this helps.