Search code examples
yamlstatesalt-projectjinja2

Improved way of scaling in saltstack

I have a problem about the Jinja2 template and that problem is breaking a one line string over multiple lines when it comes to writing a state or anything in salt [my exact case refers to trying to write a list of machines one after the other,in a list,instead of just in a really long line].

What I am trying to say is that I want to achieve this:

nodegroups:
  - group: 'L@adsdasdadas' +
            'dasdasdasdas'
             .............->imagine 10.000 names coming here
             'adsasdasddsa'

Compared to the approach that I have to do now:

nodegroups:
  - group: 'L@adsdasdadas,dasdsadasdsa,dasdsadasdsa,......,asdqwe'

Is there a better way to do it?Is there a better way to handle thousands of machines?

You could say grains,and I thought about it but I was wondering if there's a better and elegant way of doing it.

Any thoughts or opinions would help me a lot

[Edit1]: I wrote a script that takes a list of hostnames and adds them to the master config file in the nodegroups section.For now it might work

Solution

  • Choice of data source

    I would recommend targeting with pillars because they are managed centrally from Master = convenient, rather than static custom grains (which are configured distributively on each Minion) = inconvenient - see comparison summary here.

    Limitations of configuration files

    The nodegroups are specified in Salt configuration file /etc/salt/master which is not a Jinja template (it has pure YAML format). So, you don't have an option to use Jinja to join external input with list of strings.

    Possible solution

    Why joining is even mentioned? You can turn the problem of "breaking a one line string over multiple lines" into solution of using lists right away - no need to break (and if you need "one line string" somewhere, joining list items is easy).

    In other words, you could define nodegroups via pillar (avoiding long strings as in your example). Pillars, in turn, are rendered by Jinja. Therefore, using the same list of Minions defined somewhere, you could generate derived product in pillars through Jinja (be it joined string of them or list as is). There is a trick which allows reusing the same external data in multiple pillars files.