I have an Ansible role that uses a rather complex configuration in form of a dictionary. The old config format wasn't beneficial in many ways so I rewrote the role to be a lot easier to configure.

As the role is already in broad use I'd like to stay compatible with existing inventories and playbooks that use the old config format. In order to achieve this I thought I could migrate the old config into the new format (all data is there and migratable, so technically it's possible) before running the role.

Sample data

• NOTE: content makes no sense, but it shows the structural differences between formats
• NOTE2: this question is about a general approach to this problem (use set_facts? use templates? use custom Python script?) and not about the actual migration of the sample data

Old config format:

petshop_old:
  petshop_data:
    name: awesome_petshop
  employee_data:
    - name_of_employee: sara
      date_of_birth: 2000
    - name_of_employee: john
      date_of_birth: 1995
  task_data:
    - name: clean_dogs
      time: morning
      by: john
    - name: clean_cats
      time: evening
      by: john
    - name: feed_birds
      time: evening
      by: sara

New config format:

petshop_new:
  name: awesome_petshop
  employees:
    - name: sara
      birth_year: 2000
      tasks:
        - name: feed_birds
          time: evening
    - name: john
      birth_year: 1995
      tasks:
        - name: clean_dogs
          time: morning
        - name: clean_cats
          time: evening

tasks/main.yml:

- name: Migrate the old config data into the new format
  when: (petshop_old is defined) and (petshop_new is not defined)
  block:
    [...] Here come the migration tasks [...]

- name: This tasks prints the config, even when originally petshop_old was given to the role
  ansible.builtin.debug:
    var: petshop_new
[...] do more stuff with the data [...]

What would be the best approach to migrate the old config into the new format before running the actual tasks that perform the role?

I have an Ansible role that uses a rather complex configuration in form of a dictionary. The old config format wasn't beneficial in many ways so I rewrote the role to be a lot easier to configure.

As the role is already in broad use I'd like to stay compatible with existing inventories and playbooks that use the old config format. In order to achieve this I thought I could migrate the old config into the new format (all data is there and migratable, so technically it's possible) before running the role.

Sample data

• NOTE: content makes no sense, but it shows the structural differences between formats
• NOTE2: this question is about a general approach to this problem (use set_facts? use templates? use custom Python script?) and not about the actual migration of the sample data

Old config format:

petshop_old:
  petshop_data:
    name: awesome_petshop
  employee_data:
    - name_of_employee: sara
      date_of_birth: 2000
    - name_of_employee: john
      date_of_birth: 1995
  task_data:
    - name: clean_dogs
      time: morning
      by: john
    - name: clean_cats
      time: evening
      by: john
    - name: feed_birds
      time: evening
      by: sara

New config format:

petshop_new:
  name: awesome_petshop
  employees:
    - name: sara
      birth_year: 2000
      tasks:
        - name: feed_birds
          time: evening
    - name: john
      birth_year: 1995
      tasks:
        - name: clean_dogs
          time: morning
        - name: clean_cats
          time: evening

tasks/main.yml:

- name: Migrate the old config data into the new format
  when: (petshop_old is defined) and (petshop_new is not defined)
  block:
    [...] Here come the migration tasks [...]

- name: This tasks prints the config, even when originally petshop_old was given to the role
  ansible.builtin.debug:
    var: petshop_new
[...] do more stuff with the data [...]

What would be the best approach to migrate the old config into the new format before running the actual tasks that perform the role?

• ansible
• yaml

• Read old Yaml with some programming language, convert to new format using ordinarry array/object functions, write new Yaml config file – Justinas Commented Jan 23 at 7:21
• I thought there would be a good way to do this in Ansible as part of the role instead of doing this on the side... Looks like I approached this from the wrong angle :/ – MOnsDaR Commented Jan 23 at 7:33
• Why to convert it every time, if you can do it only once? – Justinas Commented Jan 23 at 7:50
• 1 Because I do not have control over all the old inventories and playbooks using the role. – MOnsDaR Commented Jan 23 at 8:55
• 1 Please refrain from adding an answer to the question itself. – β.εηοιτ.βε Commented Feb 5 at 12:43

2 Answers 2

For the given old.conf YAML file, using a quick and lazy loop approach with Jinja2 Templating, a minimal example playbook

---
- hosts: localhost
  become: false
  gather_facts: false

  tasks:

    - include_vars:
        file: old.conf
        name: old

    - debug:
        var: old

    - debug:
        msg: |
          petshop_new:
            name: {{ old.petshop_old.petshop_data.name }}
            employees:
          {% for employee in old.petshop_old.employee_data %}
              - name: {{ employee.name_of_employee }}
                birth_year: {{ employee.date_of_birth }}
                tasks:
          {% for task in old.petshop_old.task_data %}
          {% if employee.name_of_employee == task.by %}
                  - name: {{ task.name }}
                    time: {{ task.time }}
          {% endif %}
          {% endfor %}
          {% endfor %}

will result into an output of

TASK [debug] *****************
ok: [localhost] =>
  old:
    petshop_old:
      employee_data:
      - date_of_birth: 2000
        name_of_employee: sara
      - date_of_birth: 1995
        name_of_employee: john
      petshop_data:
        name: awesome_petshop
      task_data:
      - by: john
        name: clean_dogs
        time: morning
      - by: john
        name: clean_cats
        time: evening
      - by: sara
        name: feed_birds
        time: evening

TASK [debug] *****************
ok: [localhost] =>
  msg: |-
    petshop_new:
      name: awesome_petshop
      employees:
        - name: sara
          birth_year: 2000
          tasks:
            - name: feed_birds
              time: evening
        - name: john
          birth_year: 1995
          tasks:
            - name: clean_dogs
              time: morning
            - name: clean_cats
              time: evening

It can become easily written out via

    - copy:
        dest: new.conf
        content: |
          pet...

I solved it using a template and the ansible.builtin.template filter.

config_migration.j2:

petshop_new:
  name: {{ petshop_old['petshop_data']['name'] }}
  [...] further jinja-templating to create a YAML with the new cfg format [...]

Migration tasks:

- name: Migrate the old config data into the new format
  when: (petshop_old is defined) and (petshop_new is not defined)
  ansible.builtin.set_fact:
    petshop_new: '{{ lookup("ansible.builtin.template", "./config_migration.j2") | from_yaml }}'

- name: This tasks prints the config, even when originally petshop_old was given to the role
  ansible.builtin.debug:
    var: petshop_new
[...] do more stuff with the data [...]