Search code examples
pythondjangovagrantpuppetvagrant-provision

Puppet Dependency file failing in local Vagrant Environmnet


Below is my Puppet code for a Python Django App.

class ltp ($project_dir = '/vagrant') {

    exec {
        "install_project_dependencies":
            cwd => $project_dir,
            command => "/usr/bin/python setup.py install > /tmp/setuplog 2>&1",
            timeout => 0,
    }

    exec {
        "install_nltk_data":
            cwd => $project_dir,
            command => "su - vagrant -c \"cd /opt/vagrant; /usr/bin/python -m nltk.downloader stopwords > /tmp/nltklog 2>&1\"",
            require => Exec['install_project_dependencies']
    }

    exec {
        "install_django_db":
            cwd => $project_dir,
            user => 'vagrant',
            environment => ["ENV=DEV"],
            command => "/usr/bin/python manage.py syncdb --noinput > /tmp/syncdblog 2>&1",
            require => Exec['create_ba_database']
    }

    exec {
        "create_ba_database":
            cwd => $project_dir,
            command => "echo \"psql -c 'create database burson_clips;'\" | sudo su postgres",
            require => Exec['change_pg_password']
    }

    exec {
        "change_pg_password":
            cwd => $project_dir,
            command => "sudo -u postgres psql -c \"alter user postgres with password 'postgres';\""
    }

} 

Below is the error Vagrant gives when I run vagrant provision

clips: Error: /usr/bin/python setup.py install > /tmp/setuplog 2>&1 returned 1 instead of one of [0]
==> clips: Error: /Stage[main]/Ltp/Exec[install_project_dependencies]/returns: change from notrun to 0 failed: /usr/bin/python setup.py install > /tmp/setuplog 2>&1 returned 1 instead of one of [0]
==> clips: Warning: /Stage[main]/Ltp/Exec[install_nltk_data]: Skipping because of failed dependencies
==> clips: Notice: /Stage[main]/Ltp/Exec[change_pg_password]/returns: executed successfully
==> clips: Notice: /Stage[main]/Ltp/Exec[create_ba_database]/returns: ERROR:  database "burson_clips" already exists
==> clips: Notice: /Stage[main]/Ltp/Exec[install_django_db]: Dependency Exec[create_ba_database] has failures: true
==> clips: Error: echo "psql -c 'create database burson_clips;'" | sudo su postgres returned 1 instead of one of [0]
==> clips: Error: /Stage[main]/Ltp/Exec[create_ba_database]/returns: change from notrun to 0 failed: echo "psql -c 'create database burson_clips;'" | sudo su postgres returned 1 instead of one of [0]
==> clips: Warning: /Stage[main]/Ltp/Exec[install_django_db]: Skipping because of failed dependencies

Did any body received this error previously? Can any body let me know if any area of code I should look into. This happens on my local vagrant setup.

Update

So I tried Vagrant destroy and recreate the dependency now I am getting error.

==> clips: Error: /usr/bin/python setup.py install > /tmp/setuplog 2>&1 returned 1 instead of one of [0]
==> clips: Error: /Stage[main]/Ltp/Exec[install_project_dependencies]/returns: change from notrun to 0 failed: /usr/bin/python setup.py install > /tmp/setuplog 2>&1 returned 1 instead of one of [0]
==> clips: Warning: /Stage[main]/Ltp/Exec[install_nltk_data]: Skipping because of failed dependencies
==> clips: Notice: /Stage[main]/Ltp/Exec[change_pg_password]/returns: executed successfully
==> clips: Notice: /Stage[main]/Ltp/Exec[create_ba_database]/returns: executed successfully
==> clips: Error: /usr/bin/python manage.py syncdb --noinput > /tmp/syncdblog 2>&1 returned 1 instead of one of [0]
==> clips: Error: /Stage[main]/Ltp/Exec[install_django_db]/returns: change from notrun to 0 failed: /usr/bin/python manage.py syncdb --noinput > /tmp/syncdblog 2>&1 returned 1 instead of one of [0]

Adding setup.py

#!/usr/bin/env python
from setuptools import setup

# fix setuptools:
# wget http://bootstrap.pypa.io/ez_setup.py -O - | sudo python

install_requires = [
    'Django==1.8',
    'django-celery',
    'celery==3.1.23',
    'ipdb',
    # 'ipython',
    'django-kombu',
    'kombu',
    'billiard',
    'eventlet',
    #    'uwsgi',
    'xlutils',
    'pip-tools',
    'djangorestframework',
    'markdown',
    'django-filter',
    'django-cors-headers',
    'pyyaml',
    'sh',
    'pika==0.9.12',
    'pyparsing',
    'pyrabbit',
    'lxml==3.3.5',
    'xlrd',
    'raven',
    'six',
    'requests',
    'uritemplate',
    'twitter-text-python',  # for the ttp

    'python-dateutil==2.4.0',
    'redis',
    'python-docx',  ##for opening word documents
    'sqlparse',
    'httplib2',
    'simplejson',
    'requests==2.5.1',
    'six>=1.7.3',
    'django-organizations',
    'django_extensions',
    'feedparser==5.1.3',

    # The entry for this on pypi is f'd up.
    # pip install https://github.com/timothycrosley/RedisStore/blob/master/dist/RedisStore-0.1.tar.gz?raw=true
    # 'RedisStore',


    'xlsxwriter',
    'django-grappelli',

    # MDM: setuptools/pip can't install these
    # 'gensim',
    # 'numpy',
    # 'scikit-learn',
    # 'matplotlib',

    # MDM: only pip can install this one!
    # 'CairoSVG',

    'xmltodict',
    'twitter',
    'pygal',
    'tinycss',
    'cssselect',
    'dashboard_common==1.3.3',
    'pytz',
    'greenlet',
    'xlsxwriter',
    'pattern',
    'oauth2',

    'wordcloud',

    # MDM this is a dep-lib cluser-f
    # pip install it last so it can f up dependencies how ever it likes
    # 'python-social-auth',
    # 'newspaper', ##for article extraction

    'python-saml',
    'python-openid',
    'xmlsec',
    'paramiko',

    # has apt-get/brew deps
    'psycopg2'
]


# sudo -H pip install CairoSVG

dependency_links = [
    # fails unless vagrant box has a .ssh identity w/ git hub & trust github
    'workinggithublink',

]

Solution

  • The error is this

    ==> clips: Notice: /Stage[main]/Ltp/Exec[create_ba_database]/returns: ERROR:  database "burson_clips" already exists
    
    ==> clips: Error: echo "psql -c 'create database burson_clips;'" | sudo su postgres returned 1 instead of one of [0]
    

    your db already exists as burson_clips so the command fails and the rest of the provisioning fails.

    You should be able to leverage some answers from Check if database exists in PostgreSQL using shell to check if the db exists first and then create only if needed; something like

    exec {
        "create_ba_database":
            cwd => $project_dir,
            command => "echo \"psql -c 'create database burson_clips;'\" | sudo su postgres",
            require => Exec['change_pg_password'],
            unless => "psql -lqt | cut -d \| -f 1 | grep -qw 'burson_clips'"
    }