Why does including var files using vars_files not work in Ansible?

I want to setup a server using Ansible. This is my file structure:

group_vars/
  all.yml
  development.yml
  production.yml
  vault/
    all.yml
    development.yml
    production.yml
playbooks/
  development.yml
  production.yml
roles/
  common/
    tasks/
      main.yml
    vars/
      main.yml
ansible.cfg
hosts

This is my ansible.cfg:

[defaults]
vault_password_file = ./vault_pass.txt
host_key_checking = False
inventory = ./hosts

The development.yml playbook:

- hosts: all
  name: Development Playbook
  become: true
  roles:
    - ../roles/common
  vars_files:
    - ../group_vars/development.yml
    - ../group_vars/all.yml
    - ../group_vars/vault/development.yml
    - ../group_vars/vault/all.yml

And the tasks/main.yml file of the common role:

# Set hostame
- name: Set hostname
  become: true
  ansible.builtin.hostname:
    name: "{{ server.hostname }}"

# Set timezone
- name: Set timezone
  become: true
  community.general.timezone:
    name: "{{ server.timezone }}"

# Update all packages
- name: Update all packages
  become: true
  ansible.builtin.apt:
    upgrade: dist
    update_cache: true

The group_vars/all.yml file looks like this:

server:
  hostname: "myhostname"
  timezone: "Europe/Berlin"

When running the playbook using ansible-playbook playbooks/development.yml, I get this error:

fatal: [default]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'hostname'. 'dict object' has no attribute 'hostname'\n\nThe error appears to be in '/ansible/roles/common/tasks/main.yml': line 6, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n# Set hostame\n- name: Set hostname\n  ^ here\n"}

Can someone explain to me why the vars_files does not work and how to fix this?

Solution

Ansible automatically imports files and directories in group_vars that match the name of an active group. That is, if you are targeting the production group, and you have a file group_vars/production.yaml, the variables defined in this file will be imported automatically.

If instead of a file you have the directory group_vars/production/, then all files in that directory will be imported for hosts in the production group.

So your files in group_vars/vault/ will only be imported automatically for hosts in the vault hostgroup, which isn't the behavior you want.

Without knowing all the details about your deployment, I would suggest:

Create directories group_vars/{all,production/development}.
Rename group_vars/all.yml inventory files to group_vars/all/common.yml, and similarly for development.yml and production.yml (the name common.yml isn't special, you can use whatever name you want).
Rename group_vars/vault/all.yml to group_vars/all/vault.yaml, and similarly for the other files.

This will give you the following layout:

group_vars/
├── all
│   ├── common.yml
│   └── vault.yml
├── development
│   ├── common.yml
│   └── vault.yml
└── production
    ├── common.yml
    └── vault.yml