Automation without testing is fragile. To ensure that playbooks, roles, and collections behave reliably across different environments, the Ansible ecosystem provides the Molecule testing tool.

Molecule defines a clear pipeline with several phases that guide the testing of Ansible content:

  1. create: Spins up the test instances, e.g., Podman containers as defined in the document.

  2. prepare: Prepares the instances, which can include installing dependencies or configuring the environment before applying the role.

  3. converge: Applies the role or playbook to the test instances, configuring them as intended.

  4. idempotence: Runs the playbook again to verify that no changes are made on a second run, ensuring idempotency.

  5. verify: Executes any assertions or tests defined in verify.yml to confirm the expected state and functionality.

  6. destroy: Tears down the test instances, cleaning up the environment.

Note

Idempotency is checked automatically by Molecule during the idempotence phase, helping to catch unintended changes or side effects in your playbook.

This document is supported by the https://github.com/rstyczynski/ansible-collection-howto repository, where all the code presented here is available for use.

Contents

1. Preparing the Environment

To avoid polluting the system Python, all the dependencies are installed in a virtual environment.

python3 -m venv .venv # create a virtual environment for Ansible/Molecule
source .venv/bin/activate # activate it
pip install --upgrade pip # upgrade pip
pip install ansible molecule molecule-plugins # install Ansible and Molecule

Check installation:

ansible --version
molecule --version

This repository provides a molecule_init.sh script located in the bin directory. To quickly activate the environment, source this script.

source bin/molecule_init.sh

1.1 Podman installation

Containerization makes testing much faster and easier because you do not need external dependencies. Not all tests can be performed in a container, but it’s a good starting point that will cover a large number of tests before executing the rest on real target systems. To run the tests in containers, you need to install a container manager. Podman is selected for execution under macOS.

brew install podman

Once Podman is installed, initialize and start the Podman machine, then check its status:

podman machine init
podman machine start
podman info

2. Test in the project structure

Any playbook, role, or collection can include a molecule/ directory with test scenarios. The molecule/ directory can also live outside the project, but it is recommended to keep it next to the tested component.

This repository contains Molecule test scenarios for playbooks, roles, and collections. Playbooks and roles are tested by Molecule scenarios stored at the playbook level, while the collection’s tests are embedded in the roles directory.

To get started quickly, you can scaffold a new scenario automatically using molecule init with a scenario and test name. If you omit the name, Molecule will create a default scenario:

molecule init

This generates the following default structure:

molecule/
  default/
    create.yml
    destroy.yml
    molecule.yml
    converge.yml
    verify.yml

The most critical file is molecule.yml, which defines the test scenario (driver, platforms, provisioner, etc.). Your actual verifications, i.e., assertions, are written in verify.yml; this file is the real center of the test. The target system is configured by the converge.yml file. Note that for playbook tests, this file may be omitted, having the playbook call encoded directly in the Molecule definition.

Note

The default scenario included in this repository is only to suppress errors and keep the execution logs clean during your learning path. It is not intended for regular testing and should be used only as a helper. In real tests, you will use the default scenario to better handle container life-cycle.

As mentioned before, for a regular test, the molecule.yml file may be used to carry all necessary elements including the converge stage to handle a call to a standalone playbook. Verify is critical and must be written as it’s the actual test assert code in verify.yml. A minimal example might look like this:

molecule/
  test1/
    molecule.yml
    verify.yml

3. Apache Playbook test

This repository includes example Apache configuration code that configures and starts the httpd service in three variants: Debian, RedHat, and multi-OS. Looking at the repo you see the molecule directory with specific test scenarios and the default one; at this stage you are executing such content. Now let’s take a look into details.

I will use the Debian example for the first test. Let’s look at the Debian test layout, which is still minimalistic with the molecule and verify files, along with an additional Dockerfile.

molecule/
  apache1_debian/
    Dockerfile
    molecule.yml
    verify.yml
    requirements.yml

Let’s look inside molecule.yml.

# molecule.yml
---
ansible:
  cfg:
    defaults:
      deprecation_warnings: false

driver:
  name: podman

platforms:
  - name: ubuntu
    image: ubuntu:22.04
    pre_build_image: false
    dockerfile: Dockerfile
    groups: [webservers]

provisioner:
  name: ansible
  playbooks:
    converge: ../../apache1_debian.yml

Notice the Podman driver, as the test will run on a Podman instance. The platforms section describes the infrastructure layer. Debian code is straightforward; however, the apache2_redhat platforms section comes with additional complexity due to Podman ignoring systemd; additional configurations configure systemd.

The provisioner section contains a link to the converge playbook. Because the goal was to test the playbook directly, it was natural to configure it here rather than in an external file. The inventory section is also defined in the same place. Finally, the verifier section uses Ansible, which points to the verify.yml file containing the actual test assertions.

Note

Apart from core functional arguments, you spot a few of them like test_scenario with commented lines and deprecation_warnings. I added them to make Molecule progress console log free of errors, which makes the learning path easier.

4. Executing playbook test

Running a test is super simple, and means just invoking molecule with test and the name of the test scenario. As the test is Podman-based, it’s assumed that the Podman machine is available; in case of errors, verify Podman with podman info.

molecule test -s apache1_debian

Running the test can take some time and produces long log output, as Molecule executes a series of stages: dependency, cleanup, destroy, syntax, create, prepare, converge, idempotence, verify, cleanup, and finally destroy.

Note

To reduce unnecessary error messages in the logs, I explicitly disabled the cleanup and prepare steps in molecule.yml.

4.1 Preparation stages

Let’s group these stages into practical categories. The first group is preparation, which sets up the test environment. In this phase, the Podman instance is prepared and started, dependencies are installed according to the test’s requirements.yml, and the converge play syntax is checked.

molecule dependency -s apache1_debian
molecule destroy -s apache1_debian
molecule syntax -s apache1_debian
molecule create -s apache1_debian

4.2 Configure and verify stages

The second phase is the main test execution. The converge step runs your playbook, applying all intended changes. Next, the idempotence step reruns the playbook to ensure that no further changes are made - verifying that your playbook is truly idempotent. If any changes are detected during this step, the idempotence test will fail, however the test pipeline will not be stopped. Finally, the verify step runs assertions to confirm that the system is in the desired state. Note that during regular repetitive tests supporting play development, you will use these three steps.

molecule converge -s apache1_debian
molecule idempotence -s apache1_debian
molecule verify -s apache1_debian

4.3 Cleanup stage

Finally, when the test is done, the Podman instance needs to be removed. The destroy step takes care of this.

molecule destroy -s apache1_debian

5. Writing the converge play

The converge play is a regular playbook, and you will specify in the molecule.yml a reference to your playbook when it’s a test target.

# molecule.yml (fragment)
provisioner:
  name: ansible
  playbooks:
    converge: ../../apache1_debian.yml

If you prefer to write your own play, create a converge.yml file in the test scenario directory. Example of such configuration is provided in the apache4_with_role test scenario.

# converge.yml
---
- name: Install Apache on RedHat and Debian systems (role)
  hosts: webservers
  become: true
  roles:
    - apache

Notice that in case of writing a converge.yml play you need to take care of roles to be available for Ansible. One technique to configure the right directory is to set ENV in the provisioner stage configuration. MOLECULE_PROJECT_DIRECTORY contains the path level for the tested component. In case of playbooks, it’s the repo root directory; it will be a little different for role components.

# molecule.yml (fragment)
provisioner:
  name: ansible
  env:
    ANSIBLE_ROLES_PATH: "${MOLECULE_PROJECT_DIRECTORY}/roles"

For clarity, I’ll show the converge.yml for a play using collections. It’s the same as a role with a change in fully qualified role name, which now is explicitly taken from the myorg.unix namespace.

# converge.yml
---
- name: Install Apache on RedHat and Debian systems (collection)
  hosts: webservers
  become: true
  roles:
    - myorg.unix.apache

Notice requirements.yml in the test scenario directory. This file is processed by the dependency stage to install all required collections.

# requirements.yml
---
collections:
  - name: collections/ansible_collections/myorg/unix/
    type: dir

The dependency stage is configured to use the requirements.yml file by the molecule.yml directive.

# molecule.yml (fragment)
dependency:
  name: galaxy
  options:
    requirements-file: requirements.yml

At this stage, you understand how to prepare Molecule tests for a standalone play, play with role, and a play using a collection executing in a Podman-controlled environment. Let’s take a closer look at the assertion play.

6. Writing the verification play

Verification code is a regular playbook that asserts the elements configured by the converge play. The main tool is the ansible.builtin.assert module, which evaluates Jinja2 tests and filters against Ansible variables - including facts, registered results, and user-defined variables. Combine assertions with other modules such as package_facts, service_facts, or wait_for (for port checks), etc., to verify that the converge play produced the expected results.

Note

The verify play is not intended to check idempotency. That aspect is handled by running the converge step twice, which is performed automatically during the idempotency phase.

 
# verify.yml
---
- name: Verify
  hosts: webservers
  become: true
  tasks:
    - name: Check if Apache is installed
      ansible.builtin.package_facts:
        manager: auto

    - name: Verify Apache package is installed
      ansible.builtin.assert:
        that:
          - "'apache2' in ansible_facts.packages"
        fail_msg: "Apache (apache2) package is not installed"

    # === Service Block ===
    - name: Gather service facts
      ansible.builtin.service_facts:

    - name: Assert apache2 service is running on Debian
      ansible.builtin.assert:
        that:
          - "'apache2' in ansible_facts.services"
          - "ansible_facts.services['apache2'].state == 'running'"
        fail_msg: "Apache (apache2) service is not running on Debian system"
        success_msg: "Apache (apache2) service is running on Debian system"

    # === TCP Block ===
    - name: Check if port 80 is open (Apache)
      ansible.builtin.wait_for:
        port: 80
        host: "{{ ansible_default_ipv4.address | default('127.0.0.1') }}"
        state: started
        timeout: 5
      register: apache_port_check

    - name: Assert port 80 is accessible
      ansible.builtin.assert:
        that:
          - apache_port_check.state == "started"
        fail_msg: "Port 80 is not accessible"
        success_msg: "Port 80 is accessible"

7. Molecule test at role level

It’s good practice to always keep test code next to the components. In the case of a role, this means placing it in the role’s directory.

roles/
  apache/
    meta/
      main.yml
    molecule/
      apache4_with_role/
        converge.yml
        Dockerfile.centos
        Dockerfile.ubuntu
        molecule.yml
        verify.yml
    tasks/
      main.yml

The test file layout is identical; everything is the same except for one difference inside molecule.yml/provisioner/env, where you configure ANSIBLE_ROLES_PATH to point to the repository root where the roles directory is located. I will again use MOLECULE_PROJECT_DIRECTORY, which conveniently contains the path to the tested component. In the case of a role, this is the role’s root directory, which is two levels below the repository root where the roles directory is located. This difference is reflected in the configuration, and it is the only change.

# molecule.yml (fragment)
provisioner:
  name: ansible
  env:
    ANSIBLE_ROLES_PATH: "${MOLECULE_PROJECT_DIRECTORY}/../../roles"

You can go to the role’s home and invoke the test.

cd roles/apache
molecule test -s apache4_with_role

8. Molecule test at collection’s role level

It’s good practice to always keep test code next to the components. In the case of a collection’s role, this means placing it in the role’s directory.

roles/
 apache/
   molecule/
     apache5_with_collection/
       converge.yml
       Dockerfile.centos
       Dockerfile.ubuntu
       molecule.yml
       verify.yml
   tasks/
     main.yml

The test file layout is identical; everything is the same. It is not necessary to configure any role or collection paths, as Molecule is aware of the collection context and automatically installs the collection in the dependency stage. The collection’s role molecule.yml is super easy. The only complexity we see now is related to the CentOS platform due to systemd default behavior. I kept suppression of deprecation warnings to keep the log clear.

# molecule.yml
---
ansible:
  cfg:
    defaults:
      deprecation_warnings: false

driver:
  name: podman

platforms:
  - name: centos
    image: quay.io/centos/centos:stream9
    pre_build_image: false
    dockerfile: Dockerfile.centos
    cgroupns_mode: host
    command: ["/usr/sbin/init"]
    volumes:
      - /sys/fs/cgroup:/sys/fs/cgroup:rw
    tmpfs:
      /run: rw
      /run/lock: rw
    env:
      container: docker
    groups: [webservers]

  - name: ubuntu
    image: ubuntu:22.04
    pre_build_image: false
    dockerfile: Dockerfile.ubuntu
    groups: [webservers]

You can go to the role’s home and invoke the test. This time I will invoke all the tests:

Use the default scenario to test both Debian and RedHat.

cd roles/apache
molecule test

The Debian alone:

molecule test -s debian

And the RedHat alone:

molecule test -s redhat

9. JUnit integration

Molecule supports test reporting through a regular Ansible ansible.builtin.junit callback. Configure the callback in the provisioner section of molecule.yml by setting environment variables. Having this, each task prefixed with TEST_CASE will be reported to the junit report file located in the reports directory relative to the test home.

# molecule.yml (fragment)
provisioner:
  name: ansible
  playbooks:
    converge: ../../apache1_debian.yml

  env:
    ANSIBLE_CALLBACKS_ENABLED: ansible.builtin.junit
    JUNIT_TEST_CASE_PREFIX: "TEST_CASE"
    JUNIT_OUTPUT_DIR: "reports"

The verify.yml file looks as before, with the only change: TEST_CASE prefixes for assertion tasks.

# verify.yml
---
- name: Verify
  hosts: webservers
  become: true
  tasks:
    - name: Check if Apache is installed
      ansible.builtin.package_facts:
        manager: auto

    - name: "TEST_CASE: Verify Apache package is installed"
      ansible.builtin.assert:
        that:
          - "'apache2' in ansible_facts.packages"
        fail_msg: "Apache (apache2) package is not installed"

    # === Service Block ===
    - name: Gather service facts
      ansible.builtin.service_facts:

    - name: "TEST_CASE: Assert apache2 service is running on Debian"
      ansible.builtin.assert:
        that:
          - "'apache2' in ansible_facts.services"
          - "ansible_facts.services['apache2'].state == 'running'"
        fail_msg: "Apache (apache2) service is not running on Debian system"
        success_msg: "Apache (apache2) service is running on Debian system"

    # === TCP Block ===
    - name: "TEST_CASE: Check if port 80 is open (Apache)"
      ansible.builtin.wait_for:
        port: 80
        host: "{{ ansible_default_ipv4.address | default('127.0.0.1') }}"
        state: started
        timeout: 5
      register: apache_port_check

    - name: "TEST_CASE: Assert port 80 is accessible"
      ansible.builtin.assert:
        that:
          - apache_port_check.state == "started"
        fail_msg: "Port 80 is not accessible"
        success_msg: "Port 80 is accessible"

During execution of such test, the junit report file is being populated with data in the report directory.

molecule test -s junit

After the test, the report must be converted from native xml format using any regular junit tool. Instead of looking for an available tool, I asked ChatGPT to create a simple script that converts the report to a dynamic HTML report.

A converter script is available in the bin directory generating a dynamic HTML report. Below code processes the latest report from junit scenario.

verify_junit=$(ls -t molecule/junit/reports/verify*.xml | head -n 1)
bin/convert_junit_report.sh $verify_junit

Exemplary test report is available here: JUnit HTML Report

Appendix. References

  1. Ansible Molecule (Note: use hamburger menu to go to other chapters), https://ansible.readthedocs.io/projects/molecule/

  2. Developing and Testing Ansible Roles with Molecule and Podman - Part 1, https://www.redhat.com/en/blog/developing-and-testing-ansible-roles-with-molecule-and-podman-part-1

  3. Developing and Testing Ansible Roles with Molecule and Podman - Part 2, https://origin-www.ansible.com/blog/developing-and-testing-ansible-roles-with-molecule-and-podman-part-2/

  4. Ansible variable validation with ansible.utils.assert, https://www.puppeteers.net/blog/ansible-quality-assurance-part-1-ansible-variable-validation-with-assert