I have essentially a barebones saltstack "cluster" running. In fact, it's actually a master and a minion on the same host machine.
This is really a 2 part question.
1) The salt-master takes a really long time to boot. Like 2+ minutes. The debug logs look like it's got several threads going in a loop loading/reloading configs:
[DEBUG ] Reading configuration from /etc/salt/minion.d/master.conf
[DEBUG ] Reading configuration from /etc/salt/minion.d/master.conf
[DEBUG ] Using cached minion ID from /etc/salt/minion_id: host.my.domain
[DEBUG ] Using cached minion ID from /etc/salt/minion_id: host.my.domain
[DEBUG ] Reading configuration from /etc/salt/master
[DEBUG ] Reading configuration from /etc/salt/master
[DEBUG ] Including configuration from '/etc/salt/minion.d/master.conf'
[DEBUG ] Reading configuration from /etc/salt/minion.d/master.conf
[DEBUG ] Using cached minion ID from /etc/salt/minion_id: host.my.domain
[DEBUG ] Reading configuration from /etc/salt/master
[DEBUG ] Including configuration from '/etc/salt/minion.d/master.conf'
[DEBUG ] Reading configuration from /etc/salt/minion.d/master.conf
[DEBUG ] Using cached minion ID from /etc/salt/minion_id: host.my.domain
[DEBUG ] Including configuration from '/etc/salt/minion.d/master.conf'
[DEBUG ] Reading configuration from /etc/salt/minion.d/master.conf
[DEBUG ] Reading configuration from /etc/salt/master
[DEBUG ] Using cached minion ID from /etc/salt/minion_id: host.my.domain
[DEBUG ] Reading configuration from /etc/salt/master
[DEBUG ] LazyLoaded local_cache.clean_old_jobs
[DEBUG ] This salt-master instance has accepted 1 minion keys.
[DEBUG ] The `dmidecode` binary is not available on the system. GPU grains will not be available.
[DEBUG ] The `dmidecode` binary is not available on the system. GPU grains will not be available.
[DEBUG ] The `dmidecode` binary is not available on the system. GPU grains will not be available.
[DEBUG ] The `dmidecode` binary is not available on the system. GPU grains will not be available.
[DEBUG ] The `dmidecode` binary is not available on the system. GPU grains will not be available.
[DEBUG ] MasterEvent PUB socket URI: ipc:///var/run/salt/master/master_event_pub.ipc
[DEBUG ] MasterEvent PULL socket URI: ipc:///var/run/salt/master/master_event_pull.ipc
[DEBUG ] MasterEvent PUB socket URI: ipc:///var/run/salt/master/master_event_pub.ipc
[DEBUG ] MasterEvent PULL socket URI: ipc:///var/run/salt/master/master_event_pull.ipc
[DEBUG ] MasterEvent PUB socket URI: ipc:///var/run/salt/master/master_event_pub.ipc
[DEBUG ] MasterEvent PULL socket URI: ipc:///var/run/salt/master/master_event_pull.ipc
And 2) Once it finally finishes and I can auth the minion to it, just a simple test.ping is taking 30+ seconds:
time salt '*' test.ping
real 0m34.556s
user 0m12.230s
sys 0m0.500s
From the minion debug logs:
[DEBUG ] Loaded minion key: /etc/salt/pki/minion/minion.pem
[DEBUG ] Decrypting the current master AES key
[DEBUG ] Loaded minion key: /etc/salt/pki/minion/minion.pem
[DEBUG ] Decrypting the current master AES key
[DEBUG ] Loaded minion key: /etc/salt/pki/minion/minion.pem
[DEBUG ] Decrypting the current master AES key
[DEBUG ] Loaded minion key: /etc/salt/pki/minion/minion.pem
This is essentially an out of the box setup with not states or anything setup. Can anyone help me address these performance issues?
Running Arch Linux on a raspberry pi, installed via the official arch repo(https://www.archlinux.org/packages/community/any/salt-zmq/), version 7.2014. These are the logs from running it manually in debug mode. There is no specific pause, it just runs in what appears to be several loops for a few minutes spewing out the above(among other things, but all generally related to loading configuration files) until it finally settles down
If you are running the master and minion on Raspberry Pi's - that would be your problem.
I'm on real hardware for my salt master and salt minions - I get results from the test.ping module in seconds.
The master starts up in seconds as well.