Salt file server "lazy copy" to minions behaviour / testing a file/directory exists

Question

I'm trying to have salt automatically deploy files to users' home directory (after creating them). The directory structure (under file_roots) is as follows:

users/
  init.sls
  user_list.jinja
  files/
      userX/
        some_dir_I_want_deployed_to_userX_homedir/
          some_script_I_want_deployed.sh
          etc

user_list.jinja has a list of users. The idea is that if there's a directory for userX under files/, that directory subtree should be deployed to userX's home. However, I'd rather not have to create empty directories for userY etc if they don't have anything to deploy, so I'm trying to test existence of the directory to avoid an error.

Here's the relevant excerpt from users/init.sls:

{% from "users/user_list.jinja" import users with context %}
{% for name, user in users.items() %}
   {% set files_path = '{0}/files/{1}'.format(salt['file.dirname'](tplpath), name) %}

   {% if salt['file.directory_exists'](files_path) %}
     {{ user.home }}:
       file.recurse:
         - source: salt://users/files/{{ name }}
         - user: {{ name }}
         - group: {{ name }}
   {% endif %}
{% endfor %}

With a fair bit of debugging (which was rather necessary to figure out the above), I've worked out that this is a chicken-and-egg situation, namely:

1. The file.directory_exists test is run on the minion (that's fair)
2. The salt file-server seems to have an optimization whereby it only deploys to minion (to their local cache under /var/cache/salt/minion/file) items which are referenced in states (more likely, the minions only request stuff which they see referenced).
3. So unless the directory subtree for users/files/userX already exists on the minion, file.directory_exists returns False, which means the entire portion which starts with {{ user.home }} is suppressed during Jinja rendering; hence it's not referenced and the copy (to the minion's local cache) never occurs.

If I manually create an empty directory structure for users/files/userX on the minion, everything starts to work. This tells me my theory is at least partially correct.

I can "feel" I'm doing smth wrong here (the whole thing feels too procedural). What's the best approach to achieve this? The requirement itself doesn't seem too far-fetched.

Answer 1

The more salt-ish way to do this is to have some data in pillar data and check for the existence of that key. Something like user.enabled. But, that would require you to keep settings in 2 places, in pillar and in the file_roots.

You don't want to check for the existence of the directory on the minion server, you want to check for the existence of the file in your file roots.

Unfortunately, I don't think it's possible to check for the existence of a file under salt:// scheme. If I'm wrong, then all you have to do is replace your check for directory existence with the syntax to check for file_root file existence.

The more salt-ish approach is to define which users are enabled/disabled on each machine in pillar data, and use the user module to add them to the system.

https://github.com/saltstack-formulas/users-formula

You can add to the standard pillar that's given with the standard users formula and put a key that says to sync files

#pillar
users:
  ausername:
    fullname: A User
    syncfiles: True
  busername:
    fullname:  B User
    syncfiles: False



#state
{% for name, user in pillar.get('users', {}).items() if user.absent is not defined or not user.absent %}
   {% if user.syncfiles %}
     /home/{{ user.username }}:
       file.recurse:
         - source: salt://users/files/{{ user.username }}
         - user: {{ user.username }}
         {% if user.prime_group %}
         - group: {{ user.prime_group.name }}
         {% endif %}
   {% endif %}
{% endfor %}

Actually, the standard users-formula already handles pre-populating with files. I would just use that formula. I know it's keeping track of data in 2 places, but you get the bonus of leveraging an already built state file.