A couple weekends ago I had some time to try new things and I decided it’s high time to make some Kubernetes operators using Operator SDK with one of its core feature – Ansible operator support.

But… what is operator-sdk?

Although you can write Kubernetes operator using pure Go with Kubernetes client libraries, it takes some time to write basic constructs. Operator SDK provides ability to generate most of the code using nice CLI tool :)

And the nicest thing is that it can generate not only Go-based operators, but also Helm and Ansible ones. I won’t cover Go or Helm operators here, but as the title of the post suggest – focus on Ansible instead.

Why you may want to create Ansible operator?

Go operators are undoubtedly the most powerful ones as you can use any code you can imagine. But on the other hand you have to create that code and often there’s more code required (compared to Ansible) and sometimes we also reinvent the wheel. The wheel that Ansible already has in the form of a module :) This may save you a lot of time as you don’t have to know details of every tool you’re integrating with.

The other thing is that for the teams where most of the people have sysadmin background coding operators in Go may not be the best idea, it’s much easier for the people to learn Ansible than a full-blown programming language.

How does Ansible Operator work?

Ansible operator is like every other Kubernetes operator – it watches Custom Resources of specific kind and reacts to changes, in this case by executing Ansible playbooks and roles.

Every operator is built from a special container that implements Custom Resource monitoring logic, a tool called ansible-runner, a proxy for accessing Kubernetes (used for tracking creation of resources and adding owner references) and… Ansible code to be executed as a result of CR changes.

Now, there’s a special file called watches.yaml in which we define mappings between CRD and playbooks/roles we want to run:

---
- version: v1
  group: mysql.operator.luktom.net
  kind: MysqlDatabase
  playbook: /opt/ansible/playbooks/database/main.yaml
  finalizer:
    name: finalizer.mysql.operator.luktom.net
    vars:
      state: absent
  watchDependentResources: false

- version: v1
  group: mysql.operator.luktom.net
  kind: MysqlUser
  playbook: /opt/ansible/playbooks/user/main.yaml
  finalizer:
    name: finalizer.mysql.operator.luktom.net
    vars:
      state: absent
  watchDependentResources: false

A binary from parent image parses the file and – in above example – starts to watch for changes in MysqlDatabase and MysqlUser. When there’s a change detected it runs the playbook for the operation. And basically that’s all the “magic” of Ansible operator. It’s as simple as that :)

Instead of “hello world” – let me introduce mysql-operator

I don’t like writing “hello world” things… I prefer to write something useful instead :) That’s why I decided to replace one of my Ansible playbooks that created MySQL databases and users with operator-based approach.

Here’s what I wanted to accomplish:

• We define three CRD: MysqlCluster, MysqlDatabase and MysqlUser.
• MysqlCluster is a configuration with credentials to the target database cluster.
• MysqlDatabase is a database object that should be created (when Custom Resource is created in Kubernetes) or dropped (when CR is deleted in Kubernetes).
• The same for MysqlUser – but creates user in Mysql of course.

You can check the code of operator on my GitHub, where I described that operator in details. I won’t explain it line-by-line here as there’re hundreds of tutorials in the Internet and above sample operator is quite easy to read :) Instead, I want to focus on my experiences using Ansible Operator.

Some ugly things

Unfortunately not everything in Ansible Operator is nice and shiny.

The first thing is docker image size – it’s about 150 MB, not so big, but Go apps build from scratch image are usually much smaller than that. Hopefully they are built from base image and in operator’s Dockerfile we only add our Ansible code, so size doesn’t hurt us a lot, especially when we have multiple operators.

The worse things is that – for my example mysql operator – I needed to allocate at least 0.5 GB of RAM – too much for such a simple code, I think.
I tested it with lower values, but in that case the operator was unable to handle more than one CRD in watches.yaml – the processes executed by Ansible were killed and operator was unable to finish the playbook.

The running operator is also a little hard to debug – especially in cases related to failure of underlying modules due to memory limits :)

And maybe I missed something but it looks like watches.yaml file requires to specify absolute path to playbooks – it bites you when you want to test your operator on local machine as you have to change that lines or symlink to have similar directory structure like the one in docker image of operator.

But apart from above Ansible Operators are really great and I think they can revolutionize the way we approach automation using Ansible also enabling new levels of productivity :)

I already have some idea on how to use them in everyday work, so… subscribe my blog and watch my GitHub :)