Luis/clan-master-thesis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

add treefmt and jupyter lab

Browse Source
This commit is contained in:
Qubasa
2024-11-24 23:38:01 +01:00
parent 6b95d8632c
commit 5b0ae1e598
46 changed files with 6836 additions and 279 deletions
Download Patch File Download Diff File Expand all files Collapse all files

149
AI_Data/Clan/getting-started/backups.md Normal file
View File

@@ -0,0 +1,149 @@
# Backups
## Introduction to Backups
When you're managing your own services, creating regular backups is crucial to ensure your data's safety.
This guide introduces you to Clan's built-in backup functionalities.
Clan supports backing up your data to both local storage devices (like USB drives) and remote servers, using well-known tools like borgbackup and rsnapshot.
We might add more options in the future, but for now, let's dive into how you can secure your data.
## Backing Up Locally with Localbackup
### What is Localbackup?
Localbackup lets you backup your data onto physical storage devices connected to your computer,
such as USB hard drives or network-attached storage. It uses a tool called rsnapshot for this purpose.
### Setting Up Localbackup
1. **Identify Your Backup Device:**
First, figure out which device you'll use for backups. You can see all connected devices by running this command in your terminal:
```bash
lsblk --output NAME,PTUUID,FSTYPE,SIZE,MOUNTPOINT
```
Look for the device you intend to use for backups and note its details.
2. **Configure Your Backup Device:**
Once you've identified your device, you'll need to add it to your configuration.
Here's an example NixOS configuration for a device located at `/dev/sda2` with an `ext4` filesystem:
```nix
{
fileSystems."/mnt/hdd" = {
device = "/dev/sda2";
fsType = "ext4";
options = [ "defaults" "noauto" ];
};
}
```
Replace `/dev/sda2` with your device and `/mnt/hdd` with your preferred mount point.
3. **Set Backup Targets:** Next, define where on your device you'd like the backups to be stored:
```nix
{
clan.localbackup.targets.hdd = {
directory = "/mnt/hdd/backup";
mountpoint = "/mnt/hdd";
};
}
```
Change `/mnt/hdd` to the actual mount point you're using.
4. **Create Backups:** To create a backup, run:
```bash
clan backups create mymachine
```
This command saves snapshots of your data onto the backup device.
5. **Listing Backups:** To see available backups, run:
```bash
clan backups list mymachine
```
## Remote Backups with Borgbackup
### Overview of Borgbackup
Borgbackup splits the backup process into two parts: a backup client that sends data to a backup server.
The server stores the backups.
### Setting Up the Borgbackup Client
1. **Specify Backup Server:**
Start by indicating where your backup data should be sent. Replace `hostname` with your server's address:
```nix
{
clan.borgbackup.destinations = {
myhostname = {
repo = "borg@backuphost:/var/lib/borgbackup/myhostname";
};
};
}
```
2. **Select Folders to Backup:**
Decide which folders you want to back up. For example, to backup your home and root directories:
```nix
{ clan.core.state.userdata.folders = [ "/home" "/root" ]; }
```
3. **Generate Backup Credentials:**
Run `clan facts generate <yourmachine>` to prepare your machine for backup, creating necessary SSH keys and credentials.
### Setting Up the Borgbackup Server
1. **Configure Backup Repository:**
On the server where backups will be stored, enable the SSH daemon and set up a repository for each client:
```nix
{
services.borgbackup.repos.myhostname = {
path = "/var/lib/borgbackup/myhostname";
authorizedKeys = [
(builtins.readFile (config.clan.core.clanDir + "/machines/myhostname/facts/borgbackup.ssh.pub"))
];
};
}
```
Ensure the path to the public key is correct.
2. **Update Your Systems:** Apply your changes by running `clan machines update` to both the server and your client
### Managing Backups
- **Scheduled Backups:**
Backups are automatically performed nightly. To check the next scheduled backup, use:
```bash
systemctl list-timers | grep -E 'NEXT|borg'
```
- **Listing Backups:** To see available backups, run:
```bash
clan backups list mymachine
```
- **Manual Backups:** You can also initiate a backup manually:
```bash
clan backups create mymachine
```

223
AI_Data/Clan/getting-started/configure.md Normal file
View File

@@ -0,0 +1,223 @@
# Configuration - How to configure clan with your own machines
Managing machine configurations can be done in the following ways:
- writing `nix` expressions in a `flake.nix` file,
- placing `autoincluded` files into your machine directory,
- configuring everything in a simple UI (upcoming).
Clan currently offers the following methods to configure machines:
!!! Success "Recommended for nix people"
- flake.nix (i.e. via `buildClan`)
- `machine` argument
- `inventory` argument
- machines/`machine_name`/configuration.nix (`autoincluded` if it exists)
???+ Note "Used by CLI & UI"
- inventory.json
- machines/`machine_name`/hardware-configuration.nix (`autoincluded` if it exists)
!!! Warning "Deprecated"
machines/`machine_name`/settings.json
## Global configuration
In the `flake.nix` file:
- [x] set a unique `name`.
=== "**normal flake template**"
```nix title="flake.nix" hl_lines="3"
buildClan {
# Set a unique name
meta.name = "Lobsters";
# Should usually point to the directory of flake.nix
directory = ./.;
machines = {
jon = {
# ...
};
# ...
}
}
```
=== "**template using flake-parts**"
!!! info "See [Clan with flake-parts](../manual/flake-parts.md) for help migrating to flake-parts."
```nix title="flake.nix" hl_lines="3"
clan = {
# Set a unique name
meta.name = "Lobsters";
machines = {
jon = {
# ...
};
# ...
}
};
```
## Machine configuration
Adding or configuring a new machine requires two simple steps:
### Step 1. Identify Target Disk-ID
1. Find the remote disk id by executing:
```bash title="setup computer"
ssh root@<IP> lsblk --output NAME,ID-LINK,FSTYPE,SIZE,MOUNTPOINT
```
!!! Note
Replace `<IP>` with the IP address of the machine if you don't have the avahi service running which resolves mDNS local domains.
Which should show something like:
```{.shellSession hl_lines="6" .no-copy}
NAME ID-LINK FSTYPE SIZE MOUNTPOINT
sda usb-ST_16GB_AA6271026J1000000509-0:0 14.9G
├─sda1 usb-ST_16GB_AA6271026J1000000509-0:0-part1 1M
├─sda2 usb-ST_16GB_AA6271026J1000000509-0:0-part2 vfat 100M /boot
└─sda3 usb-ST_16GB_AA6271026J1000000509-0:0-part3 ext4 2.9G /
nvme0n1 nvme-eui.e8238fa6bf530001001b448b4aec2929 476.9G
├─nvme0n1p1 nvme-eui.e8238fa6bf530001001b448b4aec2929-part1 vfat 512M
├─nvme0n1p2 nvme-eui.e8238fa6bf530001001b448b4aec2929-part2 ext4 459.6G
└─nvme0n1p3 nvme-eui.e8238fa6bf530001001b448b4aec2929-part3 swap 16.8G
```
1. Edit the following fields inside the `./machines/jon/configuration.nix` and/or `./machines/sara/configuration.nix`
```nix title="./machines/<machine>/configuration.nix" hl_lines="13 18 23 27"
{
imports = [
./hardware-configuration.nix
# contains your disk format and partitioning configuration.
../../modules/disko.nix
# this file is shared among all machines
../../modules/shared.nix
# enables GNOME desktop (optional)
../../modules/gnome.nix
];
# Put your username here for login
users.users.user.username = "__YOUR_USERNAME__";
# Set this for clan commands use ssh i.e. `clan machines update`
# If you change the hostname, you need to update this line to root@<new-hostname>
# This only works however if you have avahi running on your admin machine else use IP
clan.core.networking.targetHost = "root@__IP__";
# You can get your disk id by running the following command on the installer:
# Replace <IP> with the IP of the installer printed on the screen or by running the `ip addr` command.
# ssh root@<IP> lsblk --output NAME,ID-LINK,FSTYPE,SIZE,MOUNTPOINT
disko.devices.disk.main.device = "/dev/disk/by-id/__CHANGE_ME__";
# IMPORTANT! Add your SSH key here
# e.g. > cat ~/.ssh/id_ed25519.pub
users.users.root.openssh.authorizedKeys.keys = [ "__YOUR_SSH_KEY__" ];
# ...
}
```
!!! Info "Replace `__YOUR_USERNAME__` with the ip of your machine, if you use avahi you can also use your hostname"
!!! Info "Replace `__IP__` with the ip of your machine, if you use avahi you can also use your hostname"
!!! Info "Replace `__CHANGE_ME__` with the appropriate identifier, such as `nvme-eui.e8238fa6bf530001001b448b4aec2929`"
!!! Info "Replace `__YOUR_SSH_KEY__` with your personal key, like `ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILoMI0NC5eT9pHlQExrvR5ASV3iW9+BXwhfchq0smXUJ jon@jon-desktop`"
These steps will allow you to update your machine later.
### Step 2: Detect Drivers
Generate the `hardware-configuration.nix` file for your machine by executing the following command:
```bash
clan machines update-hardware-config [MACHINE_NAME] [HOSTNAME]
```
replace `[MACHINE_NAME]` with the name of the machine i.e. `jon` and `[HOSTNAME]` with the `ip_address` or `hostname` of the machine within the network. i.e. `<IP>`
!!! Example
```bash
clan machines update-hardware-config jon
```
This command connects to the ip configured in the previous step, runs `nixos-generate-config` to detect hardware configurations (excluding filesystems), and writes them to `machines/jon/hardware-configuration.nix`.
### Step 3: Custom Disk Formatting
In `./modules/disko.nix`, a simple `ext4` disk partitioning scheme is defined for the Disko module. For more complex disk partitioning setups,
refer to the [Disko templates](https://github.com/nix-community/disko-templates) or [Disko examples](https://github.com/nix-community/disko/tree/master/example).
### Step 4: Custom Configuration
Modify `./machines/jon/configuration.nix` to personalize the system settings according to your requirements.
If you wish to name your machine to something else, do the following steps:
```
mv ./machines/jon/configuration.nix ./machines/newname/configuration.nix
```
Than rename `jon` to your preferred name in `machines` in `flake.nix` as well as the import line:
```diff
- imports = [ ./machines/jon/configuration.nix ];
+ imports = [ ./machines/__NEW_NAME__/configuration.nix ];
```
!!! Info "Replace `__NEW_NAME__` with the name of the machine"
Note that our clan lives inside a git repository.
Only files that have been added with `git add` are recognized by `nix`.
So for every file that you add or rename you also need to run:
```
git add ./path/to/my/file
```
For renaming jon to your own machine name, you can use the following command:
```
git mv ./machines/jon ./machines/newname
```
If you only want to setup a single machine at this point, you can delete `sara` from flake.nix as well as from the machines directory:
```
git rm ./machines/sara
```
### Step 5: Check Configuration
Validate your configuration by running:
```bash
nix flake check
```
This command helps ensure that your system configuration is correct and free from errors.
!!! Tip
You can integrate this step into your [Continuous Integration](https://en.wikipedia.org/wiki/Continuous_integration) workflow to ensure that only valid Nix configurations are merged into your codebase.
---
## Whats next?
- [Secrets & Facts](secrets.md): Setting up secrets with nix-sops
---

226
AI_Data/Clan/getting-started/deploy.md Normal file
View File

@@ -0,0 +1,226 @@
# Deploy Machine
Integrating a new machine into your Clan environment is an easy yet flexible process, allowing for a straight forward management of multiple NixOS configurations.
We'll walk you through adding a new computer to your Clan.
## Installing a New Machine
Clan CLI, in conjunction with [nixos-anywhere](https://github.com/nix-community/nixos-anywhere), provides a seamless method for installing NixOS on various machines.
This process involves preparing a suitable hardware and disk partitioning configuration and ensuring the target machine is accessible via SSH.
### Step 0. Prerequisites
=== "**Physical Hardware**"
- [x] **Two Computers**: You need one computer that you're getting ready (we'll call this the Target Computer) and another one to set it up from (we'll call this the Setup Computer). Make sure both can talk to each other over the network using SSH.
- [x] **Machine configuration**: See our basic [configuration guide](./configure.md)
- [x] **Initialized secrets**: See [secrets](secrets.md) for how to initialize your secrets.
- [x] **USB Flash Drive**: See [Clan Installer](installer.md)
!!! Steps
1. Create a NixOS installer image and transfer it to a bootable USB drive as described in the [installer](./installer.md).
2. Boot the target machine and connect it to a network that makes it reachable from your setup computer.
=== "**Remote Machines**"
- [x] **Two Computers**: You need one computer that you're getting ready (we'll call this the Target Computer) and another one to set it up from (we'll call this the Setup Computer). Make sure both can talk to each other over the network using SSH.
- [x] **Machine configuration**: See our basic [configuration guide](./configure.md)
- [x] **Initialized secrets**: See [secrets](secrets.md) for how to initialize your secrets.
!!! Steps
- Any cloud machine if it is reachable via SSH and supports `kexec`.
### Step 1. Deploy the machine
**Finally deployment time!** Use the following command to build and deploy the image via SSH onto your machine.
=== "**Image Installer**"
This method makes use of the image installers of [nixos-images](https://github.com/nix-community/nixos-images).
See how to prepare the installer for use [here](./installer.md).
The installer will randomly generate a password and local addresses on boot, then run ssh with these preconfigured.
The installer shows it's deployment relevant information in two formats, a text form, as well as a QR code.
This is an example of the booted installer.
```{ .bash .annotate .no-copy .nohighlight}
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ ┌───────────────────────────┐ │
│ │███████████████████████████│ # This is the QR Code (1) │
│ │██ ▄▄▄▄▄ █▀▄█▀█▀▄█ ▄▄▄▄▄ ██│ │
│ │██ █ █ █▀▄▄▄█ ▀█ █ █ ██│ │
│ │██ █▄▄▄█ █▀▄ ▀▄▄▄█ █▄▄▄█ ██│ │
│ │██▄▄▄▄▄▄▄█▄▀ ▀▄▀▄█▄▄▄▄▄▄▄██│ │
│ │███▀▀▀ █▄▄█ ▀▄ ▄▀▄█ ███│ │
│ │██▄██▄▄█▄▄▀▀██▄▀ ▄▄▄ ▄▀█▀██│ │
│ │██ ▄▄▄▄▄ █▄▄▄▄ █ █▄█ █▀ ███│ │
│ │██ █ █ █ █ █ ▄▄▄ ▄▀▀ ██│ │
│ │██ █▄▄▄█ █ ▄ ▄ ▄ ▀█ ▄███│ │
│ │██▄▄▄▄▄▄▄█▄▄▄▄▄▄█▄▄▄▄▄█▄███│ │
│ │███████████████████████████│ │
│ └───────────────────────────┘ │
│ ┌─────────────────────────────────────────────────────────────────────────────────┐ │
│ │Root password: cheesy-capital-unwell # password (2) │ │
│ │Local network addresses: │ │
│ │enp1s0 UP 192.168.178.169/24 metric 1024 fe80::21e:6ff:fe45:3c92/64 │ │
│ │enp2s0 DOWN │ │
│ │wlan0 DOWN # connect to wlan (3) │ │
│ │Onion address: 6evxy5yhzytwpnhc2vpscrbti3iktxdhpnf6yim6bbs25p4v6beemzyd.onion │ │
│ │Multicast DNS: nixos-installer.local │ │
│ └─────────────────────────────────────────────────────────────────────────────────┘ │
│ Press 'Ctrl-C' for console access │
│ │
└─────────────────────────────────────────────────────────────────────────────────────┘
```
1. This is not an actual QR code, because it is displayed rather poorly on text sites.
This would be the actual content of this specific QR code prettified:
```json
{
"pass": "cheesy-capital-unwell",
"tor": "6evxy5yhzytwpnhc2vpscrbti3iktxdhpnf6yim6bbs25p4v6beemzyd.onion",
"addrs": [
"2001:9e8:347:ca00:21e:6ff:fe45:3c92"
]
}
```
To generate the actual QR code, that would be displayed use:
```shellSession
echo '{"pass":"cheesy-capital-unwell","tor":"6evxy5yhzytwpnhc2vpscrbti3iktxdhpnf6yim6bbs25p4v6beemzyd.onion","addrs":["2001:9e8:347:ca00:21e:6ff:fe45:3c92"]}' | nix run nixpkgs#qrencode -- -s 2 -m 2 -t utf8
```
2. The root password for the installer medium.
This password is autogenerated and meant to be easily typeable.
3. See how to connect the installer medium to wlan [here](./installer.md#optional-connect-to-wifi-manually).
4. :man_raising_hand: I'm a code annotation! I can contain `code`, __formatted
text__, images, ... basically anything that can be written in Markdown.
!!!tip
For easy sharing of deployment information via QR code, we highly recommend using [KDE Connect](https://apps.kde.org/de/kdeconnect/).
There are two ways to deploy your machine:
1. **SSH with Password Authentication**
Run the following command to install using SSH:
```bash
clan machines install [MACHINE] --target-host <IP>
```
2. **Scanning a QR Code for Installation Details**
You can input the information by following one of these methods:
- **Using a JSON String or File Path:**
Provide the path to a JSON string or input the string directly:
```terminal
clan machines install [MACHINE] --json [JSON]
```
- **Using an Image Containing the QR Code:**
Provide the path to an image file containing the relevant QR code:
```terminal
clan machines install [MACHINE] --png [PATH]
```
=== "**SSH access**"
Replace `<target_host>` with the **target computers' ip address**:
```bash
clan machines install [MACHINE] --target-host <target_host>
```
If you are using our template `[MACHINE]` would be `jon`
!!! success
Your machine is all set up. 🎉 🚀
## Update Your Machines
Clan CLI enables you to remotely update your machines over SSH. This requires setting up a target address for each target machine.
### Setting the Target Host
Replace `root@jon` with the actual hostname or IP address of your target machine in the `configuration.nix` of the machine:
```{.nix hl_lines="9" .no-copy}
{
# ...
# Set this for clan commands use ssh i.e. `clan machines update`
# If you change the hostname, you need to update this line to root@<new-hostname>
# This only works however if you have avahi running on your admin machine else use IP
clan.core.networking.targetHost = "root@jon";
};
```
!!! warning
The use of `root@` in the target address implies SSH access as the `root` user.
Ensure that the root login is secured and only used when necessary.
### Updating Machine Configurations
Execute the following command to update the specified machine:
```bash
clan machines update jon
```
You can also update all configured machines simultaneously by omitting the machine name:
```bash
clan machines update
```
### Setting a Build Host
If the machine does not have enough resources to run the NixOS evaluation or build itself,
it is also possible to specify a build host instead.
During an update, the cli will ssh into the build host and run `nixos-rebuild` from there.
```{.nix hl_lines="5" .no-copy}
buildClan {
# ...
machines = {
"jon" = {
clan.core.networking.buildHost = "root@<host_or_ip>";
};
};
};
```
### Excluding a machine from `clan machine update`
To exclude machines from being updated when running `clan machines update` without any machines specified,
one can set the `clan.deployment.requireExplicitUpdate` option to true:
```{.nix hl_lines="5" .no-copy}
buildClan {
# ...
machines = {
"jon" = {
clan.deployment.requireExplicitUpdate = true;
};
};
};
```
This is useful for machines that are not always online or are not part of the regular update cycle.
---
## What's next ?
- [**Disk Encryption**](./disk-encryption.md): Configure disk encryption with remote decryption
- [**Mesh VPN**](./mesh-vpn.md): Configuring a secure mesh network.
---

180
AI_Data/Clan/getting-started/disk-encryption.md Normal file
View File

@@ -0,0 +1,180 @@
This guide provides an example setup for a single-disk ZFS system with native encryption, accessible for decryption remotely.
!!! Warning
This configuration only applies to `systemd-boot` enabled systems and **requires** UEFI booting.
Replace the highlighted lines with your own disk-id.
You can find our your disk-id by executing:
```bash
lsblk --output NAME,ID-LINK,FSTYPE,SIZE,MOUNTPOINT
```
=== "**Single Disk**"
Below is the configuration for `disko.nix`
```nix hl_lines="17 48"
--8<-- "docs/code-examples/disko-single-disk.nix"
```
=== "**Raid 1**"
Below is the configuration for `disko.nix`
```nix hl_lines="17 48 49"
--8<-- "docs/code-examples/disko-raid.nix"
```
Below is the configuration for `initrd.nix`.
Replace `<yourkey>` with your ssh public key.
Replace `kernelModules` with the ethernet module loaded one on your target machine.
```nix hl_lines="18 29"
{config, pkgs, ...}:
{
boot.initrd.systemd = {
enable = true;
};
# uncomment this if you want to be asked for the decryption password on login
#users.root.shell = "/bin/systemd-tty-ask-password-agent";
boot.initrd.network = {
enable = true;
ssh = {
enable = true;
port = 7172;
authorizedKeys = [ "<yourkey>" ];
hostKeys = [
"/var/lib/initrd-ssh-key"
];
};
};
boot.initrd.availableKernelModules = [
"xhci_pci"
];
# Find out the required network card driver by running `lspci -k` on the target machine
boot.initrd.kernelModules = [ "r8169" ];
}
```
### Step 1: Copying SSH Public Key
Before starting the installation process, ensure that the SSH public key is copied to the NixOS installer.
1. Copy your public SSH key to the installer, if it has not been copied already:
```bash
ssh-copy-id -o PreferredAuthentications=password -o PubkeyAuthentication=no root@nixos-installer.local
```
### Step 1.5: Prepare Secret Key and Clear Disk Data
1. Access the installer using SSH:
```bash
ssh root@nixos-installer.local
```
2. Create a `secret.key` file in `/tmp` using `nano` or another text editor:
```bash
nano /tmp/secret.key
```
3. Discard the old disk partition data:
```bash
blkdiscard /dev/disk/by-id/nvme-eui.002538b931b59865
```
4. Run the `clan` machine installation with the following command:
```bash
clan machines install gchq-local --target-host root@nixos-installer --yes --no-reboot
```
### Step 2: ZFS Pool Import and System Installation
1. SSH into the installer once again:
```bash
ssh root@nixos-installer.local
```
2. Perform the following commands on the remote installation environment:
```bash
zpool import zroot
zfs set keylocation=prompt zroot/root
zfs load-key zroot/root
zfs set mountpoint=/mnt zroot/root/nixos
mount /dev/nvme0n1p2 /mnt/boot
```
3. Disconnect from the SSH session:
```bash
CTRL+D
```
4. Securely copy your local `initrd_rsa_key` to the installer's `/mnt` directory:
```bash
scp ~/.ssh/initrd_rsa_key root@nixos-installer.local:/mnt/var/lib/initrd-ssh-key
```
5. SSH back into the installer:
```bash
ssh root@nixos-installer.local
```
6. Navigate to the `/mnt` directory, enter the `nixos-enter` environment, and then exit:
```bash
cd /mnt
nixos-enter
realpath /run/current-system
exit
```
7. Run the `nixos-install` command with the appropriate system path `<SYS_PATH>`:
```bash
nixos-install --no-root-passwd --no-channel-copy --root /mnt --system <SYS_PATH>
```
8. After the installation process, unmount `/mnt/boot`, change the ZFS mountpoint, and reboot the system:
```bash
umount /mnt/boot
cd /
zfs set mountpoint=/ zroot/root/nixos
reboot
```
9. Perform a hard reboot of the machine and remove the USB stick.
### Step 3: Accessing the Initial Ramdisk (initrd) Environment
1. SSH into the initrd environment using the `initrd_rsa_key` and provided port:
```bash
ssh -p 7172 root@192.168.178.141
```
2. Run the `systemd-tty-ask-password-agent` utility to query a password:
```bash
systemd-tty-ask-password-agent --query
```
After completing these steps, your NixOS should be successfully installed and ready for use.
**Note:** Replace `root@nixos-installer.local` and `192.168.178.141` with the appropriate user and IP addresses for your setup. Also, adjust `<SYS_PATH>` to reflect the correct system path for your environment.

109
AI_Data/Clan/getting-started/index.md Normal file
View File

@@ -0,0 +1,109 @@
# :material-clock-fast: Getting Started
Create your own clan with these initial steps and manage a fleet of machines with one single testable git repository!
### Prerequisites
=== "**Linux**"
Clan depends on nix installed on your system. Run the following command to install nix.
```bash
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install
```
If you already have installed Nix, make sure you have the `nix-command` and `flakes` configuration enabled in your ~/.config/nix/nix.conf.
The determinate installer already comes with this configuration by default.
```bash
# /etc/nix/nix.conf or ~/.config/nix/nix.conf
experimental-features = nix-command flakes
```
=== "**NixOS**"
If you run NixOS the `nix` binary is already installed.
You will also need to enable the `flakes` and `nix-commands` experimental features in your configuration.nix:
```nix
{ nix.settings.experimental-features = [ "nix-command" "flakes" ]; }
```
=== "**Other**"
Clan doesn't offer dedicated support for other operating systems yet.
### Step 1: Add Clan CLI to Your Shell
Add the Clan CLI into your development workflow:
```bash
nix shell git+https://git.clan.lol/clan/clan-core#clan-cli
```
You can find reference documentation for the `clan` cli program [here](../reference/cli/index.md).
Alternatively you can check out the help pages directly:
```terminalSession
clan --help
```
### Step 2: Initialize Your Project
Set the foundation of your Clan project by initializing it as follows:
```bash
clan flakes create my-clan
```
This command creates the `flake.nix` and `.clan-flake` files for your project.
It will also generate files from a default template, to help show general clan usage patterns.
### Step 3: Verify the Project Structure
Ensure that all project files exist by running:
```bash
cd my-clan
tree
```
This should yield the following:
``` { .console .no-copy }
.
├── flake.nix
├── machines
│   ├── jon
│   │   ├── configuration.nix
│   │   └── hardware-configuration.nix
│   └── sara
│   ├── configuration.nix
│   └── hardware-configuration.nix
└── modules
└── shared.nix
5 directories, 9 files
```
```bash
clan machines list
```
``` { .console .no-copy }
jon
sara
```
!!! success
You just successfully bootstrapped your first clan directory.
---
### What's Next?
- [**Installer**](./installer.md): Setting up new computers remotely is easy with an USB stick.
---

174
AI_Data/Clan/getting-started/installer.md Normal file
View File

@@ -0,0 +1,174 @@
# Installer
Our installer image simplifies the process of performing remote installations.
Follow our step-by-step guide to create and transfer this image onto a bootable USB drive.
!!! info
If you already have a NixOS machine you can ssh into (in the cloud for example) you can skip this chapter and go directly to [Configure Machines](configure.md).
### Step 0. Prerequisites
- [x] A free USB Drive with at least 1.5GB (All data on it will be lost)
- [x] Linux/NixOS Machine with Internet
### Step 1. Identify the USB Flash Drive
1. Insert your USB flash drive into your computer.
2. Identify your flash drive with `lsblk`:
```shellSession
lsblk
```
```{.shellSession hl_lines="2" .no-copy}
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sdb 8:0 1 117,2G 0 disk
└─sdb1 8:1 1 117,2G 0 part /run/media/qubasa/INTENSO
nvme0n1 259:0 0 1,8T 0 disk
├─nvme0n1p1 259:1 0 512M 0 part /boot
└─nvme0n1p2 259:2 0 1,8T 0 part
└─luks-f7600028-9d83-4967-84bc-dd2f498bc486 254:0 0 1,8T 0 crypt /nix/store
```
!!! Info "In this case the USB device is `sdb`"
3. Ensure all partitions on the drive are unmounted. Replace `sdb1` in the command below with your device identifier (like `sdc1`, etc.):
```shellSession
sudo umount /dev/sdb1
```
=== "**Linux OS**"
### Step 2. Flash Custom Installer
Using clan flash enables the inclusion of ssh public keys and wifi access points.
It also allows to set language and keymap in the installer image.
```bash
clan flash write --flake git+https://git.clan.lol/clan/clan-core \
--ssh-pubkey $HOME/.ssh/id_ed25519.pub \
--keymap us \
--language en_US.UTF-8 \
--disk main /dev/sd<X> \
flash-installer
```
!!! Note
Replace `$HOME/.ssh/id_ed25519.pub` with a path to your SSH public key.
Replace `/dev/sd<X>` with the drive path you want to flash
!!! Danger "Specifying the wrong device can lead to unrecoverable data loss."
The `clan flash` utility will erase the disk. Make sure to specify the correct device
- **SSH-Pubkey Option**:
To add an ssh public key into the installer image append the option:
```
--ssh-pubkey <pubkey_path>
```
If you do not have an ssh key yet, you can generate one with `ssh-keygen -t ed25519` command.
- **Wifi Option**:
To add wifi credentials into the installer image append the option:
```
--wifi <ssid> <password>
```
- **List Keymaps**:
You can get a list of all keymaps with the following command:
```
clan flash list keymaps
```
- **List Languages**:
You can get a list of all languages with the following command:
```
clan flash list languages
```
=== "**Other OS**"
### Step 2. Download Generic Installer
```shellSession
wget https://github.com/nix-community/nixos-images/releases/download/nixos-unstable/nixos-installer-x86_64-linux.iso
```
### Step 2.5 Flash the Installer to the USB Drive
!!! Danger "Specifying the wrong device can lead to unrecoverable data loss."
The `dd` utility will erase the disk. Make sure to specify the correct device (`of=...`)
For example if the USB device is `sdb` use `of=/dev/sdb`.
Use the `dd` utility to write the NixOS installer image to your USB drive:
```shellSession
sudo dd bs=4M conv=fsync oflag=direct status=progress if=./nixos-installer-x86_64-linux.iso of=/dev/sd<X>
```
### Step 3: Boot From USB Stick
- To use, boot from the Clan USB drive with **secure boot turned off**. For step by step instructions go to [Disabling Secure Boot](../manual/secure-boot.md)
## (Optional) Connect to Wifi Manually
If you don't have access via LAN the Installer offers support for connecting via Wifi.
```shellSession
iwctl
```
This will enter `iwd`
```{.console, .no-copy}
[iwd]#
```
Now run the following command to connect to your Wifi:
```{.shellSession .no-copy}
# Identify your network device.
device list
# Replace 'wlan0' with your wireless device name
# Find your Wifi SSID.
station wlan0 scan
station wlan0 get-networks
# Replace your_ssid with the Wifi SSID
# Connect to your network.
station wlan0 connect your_ssid
# Verify you are connected
station wlan0 show
```
If the connection was successful you should see something like this:
```{.console, .no-copy}
State connected
Connected network FRITZ!Box (Your router device)
IPv4 address 192.168.188.50 (Your new local ip)
```
Press ++ctrl+d++ to exit `IWD`.
!!! Important
Press ++ctrl+d++ **again** to update the displayed QR code and connection information.
You're all set up
---
## Whats next?
- [Configure Machines](configure.md): Customize machine configuration
---

117
AI_Data/Clan/getting-started/mesh-vpn.md Normal file
View File

@@ -0,0 +1,117 @@
# Mesh VPN
This guide provides detailed instructions for configuring
[ZeroTier VPN](https://zerotier.com) within Clan. Follow the
outlined steps to set up a machine as a VPN controller (`<CONTROLLER>`) and to
include a new machine into the VPN.
## Concept
By default all machines within one clan are connected via a chosen network technology.
```{.no-copy}
Clan
Node A
<-> (zerotier / mycelium / ...)
Node B
```
If you select multiple network technologies at the same time. e.g. (zerotier + yggdrassil)
You must choose one of them as primary network and the machines are always connected via the primary network.
## 1. Set-Up the VPN Controller
The VPN controller is initially essential for providing configuration to new
peers. Once addresses are allocated, the controller's continuous operation is not essential.
1. **Designate a Machine**: Label a machine as the VPN controller in the clan,
referred to as `<CONTROLLER>` henceforth in this guide.
2. **Add Configuration**: Input the following configuration to the NixOS
configuration of the controller machine:
```nix
clan.core.networking.zerotier.controller = {
enable = true;
public = true;
};
```
3. **Update the Controller Machine**: Execute the following:
```bash
clan machines update <CONTROLLER>
```
Your machine is now operational as the VPN controller.
## 2. Add Machines to the VPN
To introduce a new machine to the VPN, adhere to the following steps:
1. **Update Configuration**: On the new machine, incorporate the following to its
configuration, substituting `<CONTROLLER>` with the controller machine name:
```nix
{ config, ... }: {
clan.core.networking.zerotier.networkId = builtins.readFile (config.clan.core.clanDir + "/machines/<CONTROLLER>/facts/zerotier-network-id");
}
```
1. **Update the New Machine**: Execute:
```bash
$ clan machines update <NEW_MACHINE>
```
Replace `<NEW_MACHINE>` with the designated new machine name.
!!! Note "For Private Networks"
1. **Retrieve Zerotier Metadata**
=== "From the repo"
**Retrieve the ZeroTier IP**: In the clan repo, execute:
```console
$ clan facts list <NEW_MACHINE> | jq -r '.["zerotier-ip"]'
```
The returned address is the Zerotier IP address of the machine.
=== "On the new machine"
**Retrieve the ZeroTier ID**: On the `new_machine`, execute:
```bash
$ sudo zerotier-cli info
```
Example Output:
```{.console, .no-copy}
200 info d2c71971db 1.12.1 OFFLINE
```
, where `d2c71971db` is the ZeroTier ID.
2. **Authorize the New Machine on the Controller**: On the controller machine,
execute:
=== "with ZerotierIP"
```bash
$ sudo zerotier-members allow --member-ip <IP>
```
Substitute `<IP>` with the ZeroTier IP obtained previously.
=== "with ZerotierID"
```bash
$ sudo zerotier-members allow <ID>
```
Substitute `<ID>` with the ZeroTier ID obtained previously.
2. **Verify Connection**: On the `new_machine`, re-execute:
```bash
$ sudo zerotier-cli info
```
The status should now be "ONLINE":
```{.console, .no-copy}
200 info d2c71971db 1.12.1 ONLINE
```
!!! success "Congratulations!"
The new machine is now part of the VPN, and the ZeroTier
configuration on NixOS within the Clan project is complete.
## Further
Currently you can only use **Zerotier** as networking technology because this is the first network stack we aim to support.
In the future we plan to add additional network technologies like tinc, head/tailscale, yggdrassil and mycelium.
We chose zerotier because in our tests it was a straight forwards solution to bootstrap.
It allows you to selfhost a controller and the controller doesn't need to be globally reachable.
Which made it a good fit for starting the project.

63
AI_Data/Clan/getting-started/secrets.md Normal file
View File

@@ -0,0 +1,63 @@
# Secrets / Facts
Clan enables encryption of secrets (such as passwords & keys) ensuring security and ease-of-use among users.
Clan utilizes the [sops](https://github.com/getsops/sops) format and integrates with [sops-nix](https://github.com/Mic92/sops-nix) on NixOS machines.
This guide will walk you through:
- **Creating a Keypair for Your User**: Learn how to generate a keypair for $USER to securely control all secrets.
- **Creating Your First Secret**: Step-by-step instructions on creating your initial secret.
- **Assigning Machine Access to the Secret**: Understand how to grant a machine access to the newly created secret.
## Create Your Admin Keypair
To get started, you'll need to create **Your admin keypair**.
!!! info
Don't worry — if you've already made one before, this step won't change or overwrite it.
```bash
clan secrets key generate
```
**Output**:
```{.console, .no-copy}
Public key: age1wkth7uhpkl555g40t8hjsysr20drq286netu8zptw50lmqz7j95sw2t3l7
Generated age private key at '/home/joerg/.config/sops/age/keys.txt' for your user. Please back it up on a secure location or you will lose access to your secrets.
Also add your age public key to the repository with 'clan secrets users add YOUR_USER age1wkth7uhpkl555g40t8hjsysr20drq286netu8zptw50lmqz7j95sw2t3l7' (replace YOUR_USER with your actual username)
```
!!! warning
Make sure to keep a safe backup of the private key you've just created.
If it's lost, you won't be able to get to your secrets anymore because they all need the admin key to be unlocked.
!!! note
It's safe to add any secrets created by the clan CLI and placed in your repository to version control systems like `git`.
### Add Your Public Key
```bash
clan secrets users add $USER <your_public_key>
```
It's best to choose the same username as on your Setup/Admin Machine that you use to control the deployment with.
Once run this will create the following files:
```{.console, .no-copy}
sops/
└── users/
└── <your_username>/
└── key.json
```
If you followed the quickstart tutorial all necessary secrets are initialized at this point.
---
## Whats next?
- [Deployment](deploy.md): How to remotely deploy your machine
- Full [Secrets](../manual/secrets.md) guide If you want to know more about how to save and share passwords in your clan