class: center, middle # Keeping track of CentOS infrastructure deployments ## (with Ansible and ARA) ### David Moreau-Simard / Fabian Arrotin May 2021 @ CentOS Dojo ??? These are some notes presented in presenter mode and not on screen --- class: center, middle, inverse ## /whois dmsimard ##### dmsimard@redhat.com = ['ansible', 'ops', 'community', 'tooling', 'ci/cd'] ## /whois arrfab ##### arrfab@centos.org = ['ops', 'infra', 'floor sweeper'] --- # Agenda ## ARA Records Ansible: * What it is (and isn't) * What it does * How it works * Getting started --- # Agenda ## CentOS infrastructure * Why ARA * How we consume it * ConfigManagement SIG work ---  ## What it is - 5 year old 🎂 - Another recursive acronym - A free and open source Ansible community project (GPLv3) - Packaged on PyPi, for Fedora, CentOS, Debian and OpenSUSE - Available on [DockerHub](https://hub.docker.com/r/recordsansible/ara-api) and [Quay.io](https://quay.io/repository/recordsansible/ara-api) --- ## What it isn't - A tool for running playbooks - A replacement for AWX (or Tower) - A parser for your Ansible console output --- ## What it does - Makes Ansible easier to understand and troubleshoot - Records playbook results to a sqlite, mysql or postgresql database - Provides an API for playbook, task and host results - Provides a web reporting interface and a CLI to browse results --- ## How it works  --- class: center, middle  --- ## How it works About Ansible callback plugins... ```python # https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/callback/__init__.py # [...] def v2_on_any(self, *args, **kwargs): def v2_runner_on_failed(self, result, ignore_errors=False): def v2_runner_on_ok(self, result): def v2_runner_on_skipped(self, result): def v2_runner_on_unreachable(self, result): def v2_playbook_on_start(self, playbook): def v2_playbook_on_task_start(self, task, is_conditional): def v2_playbook_on_play_start(self, play): def v2_playbook_on_stats(self, stats): def v2_runner_item_on_ok(self, result): def v2_runner_item_on_failed(self, result): def v2_runner_item_on_skipped(self, result): # [...] ``` --- # Getting started With PyPi: ```bash python3 -m pip install --user ansible ara[server] export ANSIBLE_CALLBACK_PLUGINS=$(python3 -m ara.setup.callback_plugins) ansible-playbook playbook.yml ara playbook list ara-manage runserver # -> http://127.0.0.1:8000 ``` --- # Getting started With CentOS 8 repositories: ```bash dnf install -y centos-release-ansible-29 dnf install -y ansible ara-server export ANSIBLE_CALLBACK_PLUGINS=$(python3 -m ara.setup.callback_plugins) ansible-playbook playbook.yml ara playbook list ara-manage runserver # -> http://127.0.0.1:8000 ``` ---  # Links and community Live demo: https://demo.recordsansible.org More reading: - **Website & Blog**: https://ara.recordsansible.org - **Code**: https://github.com/ansible-community/ara - **Documentation**: https://ara.readthedocs.io/ Stay up to date or come chat: - **Twitter**: https://twitter.com/RecordsAnsible - **IRC**: #ara on freenode - **Slack**: Invite in the README (*bridged to IRC!*) --- # CentOS Infra ## How it started ### Background/context * Coming from puppet * Using Foreman as ENC and dashboard * .... --- # CentOS Infra ## Ansible migration, the beginning * inventories are git repos (different environments/teams/projects) * git-crypt * ansible-vault * playbooks and requirements.yml files for all inventories is a git repo * one requirement.yml per inventory/project => "cherry-picking" * each role is a git repo More info on https://github.com/centos/ansible-infra-playbooks --- # CentOS Infra ## What about reporting ? * CentOS infra *was* migrated to Ansible from puppet * Need to have central ansible station (regular desired state check) * Good to have reports (// foreman dashboard for puppet execution) --- # CentOS Infra ## KiSS ! Non "invasive" add-ons (like mitogen as an example): ```bash {% if item.use_mitogen %} strategy_plugins = {{ item.base_path }}/mitogen/ansible_mitogen/plugins/strategy strategy = mitogen_linear {% else %} strategy = linear {% endif %} ``` --- # CentOS Infra ## ARA *is* non invasive ```bash {% if ansible_use_ara %} callback_plugins=/usr/lib/python3.6/site-packages/ara/plugins/callback action_plugins=/usr/lib/python3.6/site-packages/ara/plugins/action lookup_plugins=/usr/lib/python3.6/site-packages/ara/plugins/lookup {% endif %} {% if ansible_use_ara %} [ara] api_client = http api_server = http://127.0.0.1:8000 default_labels = {{ item.name }} {% endif %} ``` --- # CentOS ConfigManagement SIG work * rebuild ara (client part) -- * rebuild needed pkgs non in EPEL8 -- * rebuild ara with server support --- # CentOS Infra ## We started with container https://github.com/ansible-community/ara/blob/master/contrib/container-images/centos8-pypi.sh ```bash podman pull quay.io/recordsansible/ara-api:centos8-pypi-latest ``` More info: https://github.com/CentOS/ansible-role-ansible-host/blob/master/defaults/main.yml#L62 --- ``` ara python3-ara-server python3-django : python-django-2.2.9-1.el8 : tag-build DONE python3-django-cors-headers : built in common : DONE python3-django-health-check DONE python3-django-rest-framework : built in common : DONE python3-dynaconf' (dynaconf) : DONE python3-box DONE python3-dotenv DONE python-sh DONE (fixed with carl's patch) python-ipython temporary DONE python3-backcall DONE python3-ipykernel' IP python3-jedi DONE python3-jupyter-client' DONE but needs to be done again after python-ipython-doc (bypassed for now) python-ipykernel old 4.5.2, so then bump python3-matplotlib' tag-build DONE python3-nbformat' DONE python-jupiter-core DONE python-sphinxcontrib-github-alt DONE python-flit-1.3-4.el8 tag-build DONE python3-pickleshare DONE python3-prompt-toolkit DONE python3-wcwidth tag build DONE python3-simplegeneric' tag-build DONE python3-testpath' tag-build DONE python3-tornado tag build DONE python3-traitlets DONE python3-ipython_genutils DONE python3-zmq python3-zmq-tests : tag build DONE python3-toml DONE python3-factory-boy : built in common : DONE python3-faker DONE pytest-runner : python-pytest-runner-4.0-8.el8 : tag-build DONE python3-pbr python-pbr-5.4.3-2.el8 tag-build DONE python3-sphinxcontrib-programoutput python-sphinxcontrib-programoutput-0.14-4.el8 tag-build DONE python3-tzlocal : python-tzlocal-1.5.1-9.el8 tag-build DONE python3-whitenoise DONE python-brotli brotli-1.0.6-1.el8 : tag-build DONE ``` --- # RPM build in progress .... .left[] --- # CentOS Infra ## rpm variant (now that we have it) ```bash sudo dnf install -y centos-release-ansible-29 sudo dnf install -y ansible ara python3-ara-server python3-gunicorn ``` --- # CentOS Infra ## systemd unit file (not provided by ara pkg) ```bash [Unit] Description=ARA api service DefaultDependencies=no After=network.target [Install] WantedBy=multi-user.target [Service] Type=simple User=ara Group=ara WorkingDirectory=/home/ara/.ara ExecStart=/usr/bin/gunicorn --workers={{ ansible_ara_workers }} ara.server.wsgi ExecStop=/bin/kill -s TERM $MAINPID ``` --- # CentOS infra ## KiSS (sqlite) ```bash -rw-r--r--. 1 ara ara 5.2G May 10 12:55 ara.sqlite ``` --- class: left, top # CentOS Infra .left[] --- # CentOS Infra ## cli output ```bash ara playbook list --status failed --limit 5 --format value -c id -c path -c started 180208 ...tobisna-centos/playbooks/adhoc-refresh-facts.yml 2021-05-14T07:15:02.023003Z 180194 ...var/lib/tobisna-centos/playbooks/role-kvm-host.yml 2021-05-14T06:14:02.814713Z 180142 ...tobisna-centos/playbooks/adhoc-refresh-facts.yml 2021-05-14T05:15:02.049694Z 180120 ...tobisna-centos/playbooks/adhoc-refresh-facts.yml 2021-05-14T04:15:02.104483Z 180119 ...tobisna-centos/playbooks/adhoc-refresh-facts.yml 2021-05-14T03:15:02.524431Z ``` -- ```bash echo tobisna|rev ansibot ``` --- class: center, middle # Thank you ! ## Any questions ?