We are working on an opensource project which we have hosted on github. The project is written in Django and so we have our settings in a settings.py file. We have it open source not because we are too cheap for a subscription, but because we intentionally want it to be open source.
Anyways, we also want to run this code ourselves and so we have a site_settings.py which contains all sensitive data like the db password etc. Since we don't want this on github for everyone to see, this is in the .gitignore file.
Normally we clone the repo and add this file locally and on the server, this way we all have our personal settings and sensitive data isn't stored on git.
The problem is that we now want to run our code on Heroku, which doesn't support the creation of files on the server, since heroku doesn't work like a normal server ( it works with a distributed file system).
Of course I have thought about it myself, but I am not sure if this is the right approach. Please keep in mind that I am both new to Heroku and Git. The following is my current solution:
There are two remotes:
leaving out all the development and feature branches, we only have both the master branches (on Heroku and Github) to worry about.
My idea was to have the branches set up like this locally:
On the master branch we put the site_settings.py in the .gitignore so git will ignore it when committing to Github, the same goes for all the dev branches etc. We collect all our changes in the master branch until we have a new release.
On the heroku branch we edited the .gitignore file to stop ignoring the site_settings.py, so it will be committed to Heroku once we push to the heroku branch.
Once we have a new release we switch to heroku branch and merge master into heroku like so:
git checkout heroku
git pull
git merge master
Once we have sorted out merge conflicts from the merge, we commit to Heroku and we have a new release up and running.. =D
Is my idea acceptable to solve this (will it even work with the .gitignore like this?) and if not how do we solve this in the cleanest/best way possible? If you need more info don't hesitate to ask :)
For starters, storing your config in files is the wrong way to go about this, so don't do it.
From the 12factor app:
An app’s config is everything that is likely to vary between deploys (staging, production, developer environments, etc). This includes:
- Resource handles to the database, Memcached, and other backing services
- Credentials to external services such as Amazon S3 or Twitter
- Per-deploy values such as the canonical hostname for the deploy
Apps sometimes store config as constants in the code. This is a violation of twelve-factor, which requires strict separation of config from code. Config varies substantially across deploys, code does not.
A litmus test for whether an app has all config correctly factored out of the code is whether the codebase could be made open source at any moment, without compromising any credentials.
Note that this definition of “config” does not include internal application config, such as config/routes.rb in Rails, or how code modules are connected in Spring. This type of config does not vary between deploys, and so is best done in the code.
Another approach to config is the use of config files which are not checked into revision control, such as config/database.yml in Rails. This is a huge improvement over using constants which are checked into the code repo, but still has weaknesses: it’s easy to mistakenly check in a config file to the repo; there is a tendency for config files to be scattered about in different places and different formats, making it hard to see and manage all the config in one place. Further, these formats tend to be language- or framework-specific.
The twelve-factor app stores config in environment variables (often shortened to env vars or env). Env vars are easy to change between deploys without changing any code; unlike config files, there is little chance of them being checked into the code repo accidentally; and unlike custom config files, or other config mechanisms such as Java System Properties, they are a language- and OS-agnostic standard.
Another aspect of config management is grouping. Sometimes apps batch config into named groups (often called “environments”) named after specific deploys, such as the development, test, and production environments in Rails. This method does not scale cleanly: as more deploys of the app are created, new environment names are necessary, such as staging or qa. As the project grows further, developers may add their own special environments like joes-staging, resulting in a combinatorial explosion of config which makes managing deploys of the app very brittle.
In a twelve-factor app, env vars are granular controls, each fully orthogonal to other env vars. They are never grouped together as “environments,” but instead are independently managed for each deploy. This is a model that scales up smoothly as the app naturally expands into more deploys over its lifetime.
For Python, your config can be found in os.environ. For specific config, particularly the database you want to be using something like:
import os
import sys
import urlparse
# Register database schemes in URLs.
urlparse.uses_netloc.append('postgres')
urlparse.uses_netloc.append('mysql')
try:
# Check to make sure DATABASES is set in settings.py file.
# If not default to {}
if 'DATABASES' not in locals():
DATABASES = {}
if 'DATABASE_URL' in os.environ:
url = urlparse.urlparse(os.environ['DATABASE_URL'])
# Ensure default database exists.
DATABASES['default'] = DATABASES.get('default', {})
# Update with environment configuration.
DATABASES['default'].update({
'NAME': url.path[1:],
'USER': url.username,
'PASSWORD': url.password,
'HOST': url.hostname,
'PORT': url.port,
})
if url.scheme == 'postgres':
DATABASES['default']['ENGINE'] = 'django.db.backends.postgresql_psycopg2'
if url.scheme == 'mysql':
DATABASES['default']['ENGINE'] = 'django.db.backends.mysql'
except Exception:
print 'Unexpected error:', sys.exc_info()
For more info on config vars, see here: https://devcenter.heroku.com/articles/config-vars