Tests

The filosofy to test is:

In Travis CI these combinations are called a matrix. You can consider this overview per role:

Distribution Ansible 2.7 Ansible 2.8 Ansible devel
Alpine latest yes yes yes
Alpine edge yes * yes * yes
Archlinux (base) yes yes yes
CentOS 7 yes yes yes
CentOS latest yes yes yes
Debian stable yes yes yes
Debian latest yes yes yes
Debian unstable yes * yes * yes
Fedora latest yes yes yes
Fedora rawhide yes * yes yes
OpenSuse Leap yes yes yes
OpenSuse Tumbleweed yes yes yes
Ubuntu Artful (17) yes yes yes
Ubuntu latest yes yes yes
Ubuntu devel yes * yes * yes

Distributions or Ansible versions marked with an astriks are allowed to fail. This combination is built to prepare for future distributions or releases of Ansible.

Read [this page to understand the tools (Travis, Molecule and Tox)](tox-molecule-travis.html] better.

There are multiple tests configured, here is how they relate.

Unit tests

To test an Ansible role, Travis CI runs molecule on a commit. This verifies that the role does it’s job, but does not ensure that it works in combination with other roles.

An example for the unit test for the Ansible role java.

Time based unit tests

Because distriutions, molecule, ansible and goss change over time, a monthly test is done to all roles using this schedule:

Day of month Ansible Role
1 alternatives
1 anaconda
1 ansible
1 ansible_lint
1 apt_autostart
1 ara
1 artifactory
1 at
1 atom
1 awx
1 auditd
2 backup
2 bios_update
2 bootstrap
2 buildtools
3 ca
3 ca_certificates
3 cargo
3 cntlm
3 common
3 container_docs
3 core_dependencies
3 cron
3 clamav
3 cloud9
4 **debug
4 dhcpd
4 digitalocean_agent
4 dns
4 docker
4 docker_ce
4 dovecot
4 dsvpn
5 earlyoom
5 environment
5 epel
5 etherpad
6 fail2ban
6 firewall
7 git
7 gitlab_runner
7 glusterfs
7 go
7 gotop
8 haproxy
8 haveged
8 httpd
8 hostname
9 investigate
9 irslackd
10 java
10 jenkins
11 kernel
11 kubectl
12 logrotate
12 logwatch
12 locale
12 lynis
13 memcached
13 mediawiki
13 mitogen
13 minikube
13 molecule
13 mssql
13 mysql
14 natrouter
14 nginx
14 npm
14 ntp
15 obsproject
15 omsagent
15 openssh
15 openvas
15 owncloud
16 packer
16 php
16 php_fpm
16 phpmyadmin
16 postfix
16 postgres
16 powertop
16 python_pip
18 reboot
18 redis
18 release
18 remi
18 restore
18 revealmd
18 roundcubemail
18 rpmfusion
18 rsyslog
18 ruby
18 rundeck
19 scl
19 selinux
19 service
19 snort
19 sosreport
19 spamassassin
19 squid
19 storage
19 stratis
19 subversion
19 sudo-pair
19 sysctl
19 sysstat
20 terraform
20 tftpd
20 tomcat
20 travis
21 update
21 update_package_cache
21 ulimit
21 users
22 vagrant
24 xinetd
25 y
26 zabbix
26 zabbix_agent
26 zabbix_proxy
26 zabbix_repository
26 zabbix_server
26 zabbix_web

Integration

To test a combination of Ansible roles, Travis CI runs terraform and a complex playbook.

There is currently one integration test. A report is saved every run.