Search code examples
cloud-foundrybosh-deployer

How to use BOSH lite as a developer?

I've been tasked with updating some BOSH scripts/jobs/what have you, and developing them is costing me a heck of a lot of time.

I finally was keyed into using BOSH lite, but I only really see how to deploy CloudFoundry to the BOSH lite environment.

However, I'm a bit lost as to what I need to put into my BOSH lite release/manifest/what goes here?

Can someone describe their workflow with BOSH lite, and what types of information I need to put in the release manifest to deploy my release and test out my jobs and errands in BOSH lite? I have been having a difficult time finding good resources in this area, and just BOSH in general.

Solution

  • The high level workflow is:

    • on your workstation, you have a repo for your BOSH release
    • you have a BOSH director somewhere
    • you work on your release, build it, and upload it to the director
    • you create/modify your deployment manifest that references the uploaded release
    • you run bosh deploy with your manifest so that the Director can create "VMs" in a "Cloud" and put the bits of software in your release on those VMs (and run the software) in the topology described in your manifest

    The three main things you need to tell a Director are the stemcell(s), release(s), and deployment manifest. By now, you have some idea what a release is, it's basically all the software that gets run.

    The stemcell is the base OS image that will be common to all your deployed VMs (you can have different stemcells within a deployment, but the most common thing is to have them all the same); this is a special image that has some stuff pre-baked into it to facilitate working with BOSH. Primarily, it has a BOSH agent, this is how the Director communicates with the VMs to tell it "download this package", "download this job", "start this process", etc.

    The deployment manifest is a YAML file where you specify several things:

    • The name of your deployment.
    • A list of the releases, along with specific versions, that you will be deploying as part of this deployment.
    • A description of the networks that you want to associate with the deployed VMs. If you're using an IaaS like AWS for example, you might be deploying into a VPC, and here is where you would specify some of your Subnet IDs.
    • A description of jobs, basically a list of several homogeneous clusters to be deployed, along with how many instances of VMs/nodes you want for each cluster. Say your release consists of a frontend service, a backend service, and a database service. Then you may want to deploy a frontend cluster which just runs the frontend job, and have there be 5 instances of that. And you may want 10 instances of the backend cluster, and probably just 1 instance of the database. Each job in the manifest can reference multiple jobs from multiple releases (yes, it's an unfortunate historical accident that these two things are named the same thing).
    • Configuration properties, e.g. your jobs might need a bunch of parameters and credentials configured, and any properties that need to be shared globally can be put in the properties section.

    BOSH-Lite is a Vagrant VM which is essentially running two things you care about:

    • The BOSH Director
    • Garden, a Linux container manager (if you've heard of Docker, Garden is similar but has been around longer and is better suited for production use cases). Garden acts like "the cloud" here, when the Director needs to create a VM, it delegates to its "Cloud Provider Interface" which in turn just asks Garden to create a container.

    The advantage of BOSH-Lite is that it's much cheaper and faster to launch a container within a VM on your laptop than it is to launch a real VM in AWS, vSphere, OpenStack, or other real datacenter.

    First-time workflow (after starting and targetting BOSH-Lite):

    $ git clone YOUR_RELEASE_REPO
$ cd YOUR_RELEASE_REPO
$ bosh create release && bosh upload release
$ # create manifest, call it manifest.yml
$ bosh -d manifest.yml deploy

    Iterating:

    $ # modify the code in your repo
$ bosh create release --force && bosh upload release
$ # modify your manifest if necessary
$ bosh -d manifest.yml deploy

    Creating the manifest from scratch can be hard if you're not familiar with BOSH manifests. One things you may want to consider doing is following the instructions you've found for creating the BOSH-Lite manifest for Cloud Foundry. Then modify that to suit your project.

    Here is the full documentation on the schema of a deployment manifest: https://bosh.io/docs/deployment-manifest.html.

    If you generate a manifest and have trouble with it, you can turn to GitHub issues or the mailing list which may be better suited for back-and-forth help on getting your manifest working.