Replacing Foreman with Waitron and pixiecore

From my experience provisioning of bare metal hardware is almost always a PITA, the system is either extremely fragile or too complex. The previous setup at my job had all of the above until we sat down and looked at the parts we actually needed. Our setup was based on Foreman which is a good tool but it tries to solve so many problems and we just cared about the base installation which can be solved by simpler and smaller tools. Foreman was basically used as a massive templating engine. So we simplified!

V2 Installation using Waitron and pixiecore

We replaced Foreman with two small applications written in Go, both running in a Docker contanier

  • Waitron Templates preseed files and provide pixiecore with boot instructions based on machine definition
  • pixiecore “PXE booting for people in a hurry”.

Machine definitions and preseed files moved git(Version control ftw!) instead of a database. The Machine definition is written in YAML.

machines/my-service.example.com.yaml

hostname: my-service.example.com
operatingsystem: "14.04"
preseed: preseed.j2
finish: finish.j2
image_url: http://archive.ubuntu.com/ubuntu/dists/trusty-updates/main/installer-amd64/current/images/netboot/ubuntu-installer/amd64/
kernel: linux
initrd: initrd.gz
#BaseURL is this waitron instance url, it is read from the config file
#Hostname is the hostname for this very host, it is read from this file
#Token is generated at runtime by waitron
cmdline: "interface=auto url=//preseed/ ramdisk_size=10800 root=/dev/rd/0 rw auto hostname= console-setup/ask_detect=false console-setup/layout=USA console-setup/variant=USA keyboard-configuration/layoutcode=us localechooser/translation/warn-light=true localechooser/translation/warn-severe=true locale=en_US"
network:
  - name: eth0
    ipaddress: 10.20.30.40
    macaddress: 00:11:44:24:50
    netmask: 10.120.255.254
    gateway: 10.120.255.254
params:
    foo: False
    bar: "Hello world"

The machine definition is then used to template the preseed file and the boot file used by pixiecore

Result

  1. Machine is set build mode using the Waitron API curl -X POST http://waitron/<hostname>/build
  2. Tell machine to reboot and boot on PXE
  3. Machine boot using pixiecore which will fetch the information from http://waitron/boot/{macAddr}
  4. Machine downloads the preseed/finish scripts from Waitron and notify waitron using the API when installation is complete.
  5. Machine is ready to be used by configuration management tool of choice