Debugging Salt and Salt States
Some tips and tricks to debug salt (states).
Use salt-call on minion
- Trigger the deployment with direct and verbose feedback on terminal
- Enables identification of slow state functions, i.e. repository refreshs
Log into client machine, as root / sudo:
salt-call -l debug state.highstate
To apply a single state file (not recommended on production):
salt-call -l debug state.sls elasticsearch
Also useful to apply special housekeeping states (not in highstate), here a state that purges unnecessary kernels which is found in env stable:
salt-call state.sls kernelpurge stable
When fixing sth. on a machine an adapting the configuration management, one often want’s to double check that it is applied correctly before applying a state or a highstate again. This can be easily accomplished by running with test set to True:
salt-call state.sls elasticsearch test=True
salt-call state.highstate test=True
A diff of what would have been change is shown at the end but not applied, which can be checked for differences, i.e. in config files.
Use vagrant for development of states
- Get quick feedback on salt state editing
- Create Vagrantfile according to vagrant salt provisioner documentation.
- Setup master and minion as close to your production setup
Map salt sources into vm (/vagrant)
vagrant up salt-master vagrant up salt-minion
salt-callin conjunction with
state.slsas described above.
Start master / minion in debug mode
Useful to debug:
- Connection problems
- Salt State / Pillar rendering
service salt-minion stop
salt-minion -l debug
service salt-master stop
salt-master -l debug
In both cases, the log information is written to the terminal.
Further Debugging with pdb:
pdb $(which salt-master)
Check your YAML state files
Too many Functions...
/usr/local: file.directory: - recurse - mode: 755
recurseis not formulated as dict.
nginx: pkg.installed nginx: service.running
Solution 1: Use one state id
nginx: pkg.installed:  service.running: 
Solution 2: Use dedicated names for the state (e.g. if distributed across multiple files)
nginx_package: pkg.installed: - name: nginx nginx_service: service.running: - name: nginx