Deploying your NixOS configurations with Colmena

Table of contents

What is Colmena

Colmena is one of many NixOS configuration deployment software. Others exist such as NixOps and Morph.

Colmena has these functionalities:

  • Deploy to remote servers by tag, so you can - for example - deploy all web servers at once.
  • Secrets management. Secrets are uploaded to the remote server during configuration deployment. This removes the presence of secrets in the Nix store, which is readable by any user!
  • Extracting data from the configuration. For example you could extract all of the domain names of your servers and their IPs. Can be used to populate reports or configuration files.
  • Parallel deployment of several servers.
  • Builds and downloads configurations on the remote servers. This saves up bandwidth compared to downloading on your local machine, then pushing to the remote server.

Why use Colmena

In the previous article, you installed NixOS on your OVH remote server. Once finished, you could change its configuration and deploy it again with the nixos-rebuild command:

nixos-rebuild switch --flake .#mamachine --use-remote-sudo --target-host utilisateur@mamachine.example.com --build-host utilisateur@mamachine.example.com

But for everyday use, there are several problems:

  1. It's a long command line.
  2. If you set up several servers in your flake.nix, then you need to be very careful not to deploy one onto another by mistake. For example deploying .#machine1 on user@other-server.example.com.
  3. You cannot do mass deployments.

With Colmena the previous command becomes:

colmena apply --on nommachine --impure
The command in details.
apply
to work on a remote server. apply-local would be used instead for working locally.
--on nommachine
Select which server configurations to deploy. You can list several servers or use tags like "@web" or "@db".
--impure
Tells Colmena it should use your files even if they have uncommitted changes in Git. Useful when you're working on your configurations.

The server name appears only once, its connection address is read from the Colmena configuration in the flake. Much safer.

Prerequisites

You need a NixOS configuration in the form of a flake. For example the one built in the previous article.

Customizing code examples

Throughout the article, you can enter the values corresponding to your situation in the form boxes. The article code examples will use your values to be used directly. The change is made locally with Javascript, no data is sent anywhere.

Otherwise you can simply adapt the examples according to your case.

Changes to your flake

To deploy with Colmena, you must add your server in it's configuration. Your server will appear in the nixosConfigurations output of flake (so that nixos-anywhere can be still be used) as well as in the colmena output. We must also add the colmena tool to the shell environment.

The configurations files for your server will also be moved to a subdirectory so that more servers can be added later. The file structure will become:

|- flake.nix
   |- hosts
      |- nommachine
      |  |- configuration.nix
      |  |- disk.nix
      |  |- hardware.nix
      |- anotherserver 
      |  |- configuration.nix
      |  |- disk.nix
      |  |- hardware.nix

You can note the name of your server, its address and a SSH user below. The user needs to be able to run commands via Sudo without password, so protect its private SSH key well:

Make the following changes to your files:

  • Create the hosts/nommachine directory and move configuration.nix, disk.nix and hardware.nix into it. (Don't forget to use git mv instead if your files are in a Git repository.)

    mkdir -p hosts/nommachine
mv configuration.nix disk.nix hardware.nix hosts/nommachine

  • The flake.nix file has to be changed. See details below.

    flake.nix 
    {
  description = "My NixOS servers";

  inputs = {
    # NixOS and its packages.
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-25.11";

    # Disko, the disk partitioning module.
    disko = {
      url = "github:nix-community/disko";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };

  # Flake's output. A shell with needed tools, nixosConfigurations so nixos-anywhere can run and
  # the Colmena configuration.
  outputs =
    {
      self,
      nixpkgs,
      disko,
      ...
    }:
    let
      # Get x86_64-linux packages from nixpkgs.
      system = "x86_64-linux";
      pkgs = import nixpkgs {
        inherit system;
      };
      lib = nixpkgs.lib;

      # Define a nixosSystem from the servers attrset.
      customNixosSystem =
        name:
        let
          # The same as modules = servers."${name}".modules.
          inherit (servers."${name}") modules;
        in
        lib.nixosSystem {
          inherit system;

          # The same as modules = modules.
	  # Modules (configuration.nix, etc..) making up the server's configuration.
          inherit modules;
        };

      # Define a Colmena node from the servers attrset.
      colmenaNode =
        name:
        let
          # The same as colmena = servers."${name}".colmena, etc.
          inherit (servers."${name}") colmena modules;
        in
        { ... }:

        # Get colmena attributes for the server and add an "import" attribute with the server modules.
        # "//" merges two attrsets. Attributes from the right override attributes from the left.
	# Don't use it for deep merges, but in our case it is fine.
        colmena
        // {
          # In Colmena, the attribute must be named "imports" instead of "modules".
          imports = modules;
        };

      # Put the list of your servers here.
      servers = {
        nommachine = {

          # Attributes used by Colmena.
          colmena = {
            # The server address.
            deployment.targetHost = "addrmachine";

            # SSH username to use. That user needs to have password-less Sudo rights, or be "root".
            deployment.targetUser = "nomutilisateur";

            # Work and download on the remote server. Better than wasting bandwidth downloading on your
	    # computer, then pushing to the server.
            deployment.buildOnTarget = true;

            # Colmena tags.
	    deployment.tags = [ "web" "ovh" ];
          };

          # The module making up the configuration of the server.
          modules = [
            # Make Disko options available.
            disko.nixosModules.disko

            # Hardware configuration (built by nixos-anywhere when you install the server with it)
	    ./hosts/nommachine/hardware.nix

            # Disk partitionning and file systems to build and mount.
            ./hosts/nommachine/disk.nix

            # The server's NixOS configuration.
            ./hosts/nommachine/configuration.nix
          ];
        };
      };
    in
    {
      # A shell environment with tools available. We add Colmena to nixos-anywhere.
      devShells.x86_64-linux.default = pkgs.mkShell {
        packages = with pkgs; [
          colmena
          nixos-anywhere
        ];
      };

      # Your servers configurations in nixosConfigurations.
      # We need them here because this is what nixos-anywhere consumes to be able to install
      # servers.
      nixosConfigurations.nommachine = customNixosSystem "nommachine";

      # Colmena configuration and nodes (servers).
      colmena = {
        # The meta attribute will apply to all nodes (servers).
        meta = {
          nixpkgs = import nixpkgs { inherit system; };
        };

        # Your servers here.
        nommachine = colmenaNode "nommachine";
      };
    };
}

Changes made to the flake.nix file

Instead of placing the configuration of our server into nixosSystem or colmena, we place it in a common set of attributes (attrset) named ‘servers'. It will contain, for each server, two sets named "colmena" and "modules" (see below for explanation). In the example below we also add two tags to illustrate the use of this feature. You can name these tags however you want.

  servers = {
    nommachine = {
        colmena = {
          deployment.targetHost = "addrmachine";
          deployment.targetUser = "nomutilisateur";
          deployment.buildOnTarget = true;
		  deployment.tags = [ "web" "ovh" ];
        };
        modules = [
          disko.nixosModules.disko
          ./hosts/nommachine/hardware.nix
          ./hosts/nommachine/disk.nix
          ./hosts/nommachine/configuration.nix
        ];
      };
    };
  • colmena
    Colmena attributes, in our case the value for deployment.targetHost which indicates the domain or IP of the machine, the SSH user in deployment.targetUser (you can remove the line if you use root), and a configuration to make the build happen on the machine. Other possible configurations are described in Colmena documentation.
  • modules
    List of modules making up the server configuration de la machine.

A function is defined that transforms these values into a nixosSystem. The values for modules' are brought into the local scope of the function, then it applies the nixosSystemfunction to get a result that can be used in thenixosConfiguration` output of the flake.

  customNixosSystem = name:
    let
      inherit (servers."${name}") modules;
    in lib.nixosSystem {
      inherit system;
      inherit modules;
    };

here's an example of what this function returns:

 lib.nixosSystem {
    inherit system;

    modules = [
      disko.nixosModules.disko
      ./hosts/nommachine/hardware.nix
      etc.
    ];
  };

Another function is defined that transforms these values into a Colmena node. A colmena node is a module (it has arguments: "{...}"). The difference with the set of attributes in nixosSystem is that modules becomes imports. The set of colmena attributes is merged to the rest so that they all end up at the same level.

  colmenaNode = name:
    let
      inherit (servers."${name}") colmena modules;
    in {...}:
      colmena // {
        imports = modules;
      };

here's an example of what this function returns:

  {...}: {
    deployment.targetHost = "addrmachine";
    deployment.buildOnTarget = true;

    imports = [
      disko.nixosModules.disko
      ./hosts/nommachine/hardware.nix
      etc.
    ];
  };

The rest of the flake file defines a shell environment with the colmena command available. Then it defines the nixosConfiguration and colmena outputs, using the two functions previously seen to populate them.

  nixosConfigurations.nommachine = customNixosSystem "nommachine";

  colmena = {
    meta = {
      nixpkgs = import nixpkgs { inherit system; };
    };

    nommachine = colmenaNode "nommachine";
  };

Deploying the configuration with Colmena

First, enter the flake's shell environment. You need it to run the colmena command.

nix develop

Then you can deploy your configuration to your remote server:

colmena apply --on nommachine --impure switch
The command in details.
apply

work on a remote server. apply-local would be used instead for working locally.

--on nommachine

Tells Colmena which servers to deploy. You can give a list of names separated with commas, tag names and use wildcards too. Ex: --on web-*,sql-* or '--on '@web'.

--impure --impure

Tells Colmena it should use your files even if they have uncommitted changes in Git. Useful when you're working on your configurations.

switch

Pushes the configuration onto the server and sets up boot to start on it next time the server restarts. Some other possible commands:

  • build: only builds the configuration.
  • push: push the configuration, but don't activates it. All needed derivations will be installed onto the remote server, ready to go.
  • boot: Do not activate the configuration, but sets up boot so that the server starts on it next time it boots.
  • test: activate the configuration, do not sets up boot. Great if you're working on the network configuration: if you lose access to the server a reboot is enough to go back to a good configuration.
  • dry-activate: Only shows what would be done.
Consider testing before deploying to boot.

When colmena, like nixos-rebuild, deploys the configuration with the switch or boot option, it makes it start when the server starts. So when working on the network or firewall configuration of your machines, always use "test" instead of "switch". If you lose access to your sever, just restart it and it will start on the old configuration with working network. Otherwise you will have to run a KVM window to choose the previous configuration in the Grub meny at startup during reboot.

Adding more servers to your flake

To add another server using the previous article, you can create its .nix files (except flake.nix) in a new directory in hosts/ and then add it to the flake configuration, for example:

      servers = {
        nommachine = {
          colmena = {
            deployment.targetHost = "addrmachine";
            deployment.targetUser = "nomutilisateur";
            deployment.buildOnTarget = true;
            deployment.tags = [ "web" "ovh" ];
          };
          modules = [
            disko.nixosModules.disko
            ./hosts/nommachine/hardware.nix
            ./hosts/nommachine/disk.nix
            ./hosts/nommachine/configuration.nix
          ];
        };
		
        anotherserver = {
          colmena = {
            deployment.targetHost = "anotherserver.example.com";
            deployment.targetUser = "nomutilisateur";
            deployment.buildOnTarget = true;
            deployment.tags = [ "database" "ovh" ];
          };
          modules = [
            disko.nixosModules.disko
            ./hosts/anotherserver/hardware.nix
            ./hosts/anotherserver/disk.nix
            ./hosts/anotherserver/configuration.nix
          ];
        };		
      };

      nixosConfigurations.nommachine = customNixosSystem "nommachine";
      nixosConfigurations.anotherserver = customNixosSystem "anotherserver";

      colmena = {
        meta = {
          nixpkgs = import nixpkgs { inherit system; };
        };

        nommachine = colmenaNode "nommachine";
        anotherserver = colmenaNode "anotherserver";
      };

The nixos-anywhere command will work just as well. You just need to update the path for hardware.nix in it, the file is now in the server's directory.

nixos-anywhere --flake .#nommachine --generate-hardware-config nixos-generate-config ./hosts/nommachine/hardware.nix --target-host root@addrmachine --build-on-remote

Conclusion

You can now manage several remote NixOS server with no risk of error from mixing up names and addresses. You can also deploy to several servers at once.