Luis/clan-master-thesis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

add treefmt and jupyter lab

Browse Source
This commit is contained in:
Qubasa
2024-11-24 23:38:01 +01:00
parent 6b95d8632c
commit 5b0ae1e598
46 changed files with 6836 additions and 279 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
.envrc Normal file
View File

@@ -0,0 +1,3 @@
# shellcheck shell=bash
use flake

2
.gitattributes vendored Normal file
View File

@@ -0,0 +1,2 @@
Figures filter=lfs diff=lfs merge=lfs -text
AI_Data filter=lfs diff=lfs merge=lfs -text

5
.gitignore vendored
View File

@@ -1,5 +1,6 @@
# LaTeX intermediate and output files # LaTeX intermediate and output files
*.aux *.aux
**/*.bak.*
*.bbl *.bbl
*.blg *.blg
*.brf *.brf
@@ -32,6 +33,8 @@
*.swp *.swp
*.swo *.swo
*.DS_Store *.DS_Store
.direnv
.jupyter
# Directory for auxiliary files created by latexmk # Directory for auxiliary files created by latexmk
latexmk/ latexmk/

1270
AI_Data/Clan/Clan.html Normal file
View File

File diff suppressed because one or more lines are too long

8
AI_Data/Clan/_index.md Normal file
View File

@@ -0,0 +1,8 @@
+++
title = 'Blog'
date = 2023-01-01T08:30:00-07:00
menuInverted = true
draft = false
+++
Tempor est exercitation ad qui pariatur quis adipisicing aliquip nisi ea consequat ipsum occaecat. Nostrud consequat ullamco laboris fugiat esse esse adipisicing velit laborum ipsum incididunt ut enim. Dolor pariatur nulla quis fugiat dolore excepteur. Aliquip ad quis aliqua enim do consequat.

392
AI_Data/Clan/data-mesher/Data-Mesher Specification.md Normal file
View File

@@ -0,0 +1,392 @@
# Data-Mesher Specification
The Data-Mesher is an application that runs on every node in a peer-to-peer network. It functions as a database that eventually synchronizes across all nodes, using a special data structure called a **CRDT (Conflict-Free Replicated Data Type)** to resolve conflicts. What sets it apart from other databases is that even untrusted nodes can add data without compromising data added by others. Its primary use is for announcing hostnames and application settings, though it is flexible enough to support other use cases as well.
Below is a more detailed explanation of how it works:
---
### **What is a "dm-network"?**
A **dm-network** is a group of hosts (computers/nodes) and settings that are grouped under a single key in a JSON file. This key is a public **ed25519 key**.
- **Settings:**
- The settings in a dm-network are protected by a digital signature to prevent tampering.
- The admin users are the only ones with the private key needed to change these settings.
- However, any node in the network can update the settings as long as they sign the change correctly (with the admin's private key). This ensures there isn’t just one "admin" node, allowing settings to be changed on any local node and then pushed to other nodes in the network.
### **Hosts in the dm-network**
A dm-network also includes a **hosts** attribute that stores information about hostnames for DNS lookups.
- Each node can add its own hostnames to this list.
- Every node has its own pair of private and public keys to sign the hostnames it adds.
- In case multiple nodes try to use the same hostname, the one with the **oldest signed entry** (earliest timestamp) will be chosen.
#### **Preventing clock manipulation:**
To avoid cheating with time (for example, backdating a hostname entry), the dm-network relies on **trusted timestamp attestation servers**. We can use the [Time-Stamp Protocol RFC3161](https://datatracker.ietf.org/doc/html/rfc3161)
which allows sending a payload and getting a signed sha256 with a timestamp back. We can use the [FreeTSA](https://www.freetsa.org/index_de.php) servers for that.
Or we could use the [Opentimestamp](https://opentimestamps.org/) specification and their free to use servers.
---
### **Data Synchronization Between Nodes**
When two nodes communicate, they exchange their entire set of data.
After ensuring the data is merged, both nodes should end up with the same `data.json` file. The **merge function** ensures that both nodes arrive at the same result, no matter the order or timing.
*currently the merge function is quite primite: for settings it checks if the signature is valid and afterwards the bigger* last_update timestamp wins.
---
### **Handling Invalid or Missing Timestamps**
- If a hostname entry doesn't have a valid timestamp, it will still be shared with other nodes, but it won’t be active or used yet.
- The entry stays inactive until it reaches a trusted dm-node that also acts as a **timestamp attestation node** (TSP), which will add a timestamp and sign the entry. From that point, the hostname becomes valid and can be used in the network. The **timestamp attestation nodes** are listed under the "settings" key in the JSON file, and only the dm-network's admin can modify this list.
- This means there needs to be **one or many trusted dm-nodes**, which attest that the timestamps are correct. If one of the trusted dm-nodes is compromised, hostnames can be malicously claimed and redirected to attacker controlled nodes.
- This also means that **hostnames are not to be trusted**, and instead a Certificate Authority should be used to verify the authenticity of endpoints.
- This design has been chosen because:
- It enables having completely off-grid nodes, that are only inside the mesh VPN
- A node can start claiming it's hostname offline and just sync it into the VPN network once it's online
- No timing attacks: An attacker cannot pre-fetch timestamps to then use them to override hostnames
### **Security: Invalid Signatures**
- If a hostname or timestamp has an invalid signature, it won’t be shared with other nodes.
- An alert will be triggered for further action.
- Additionally, hosts must go through a verification step to ensure that the IPs and ports are reachable and that the machine behind them holds the correct private keys that signed the entry. Before accepting a new host into the configuration, the node will attempt to contact the host’s IP and port, and a challenge-response protocol will be used to verify that it is the correct machine.
---
Below is an example `data.json`:
```json
{
"22excOG1Q7hlNMyRPWz4eZNeTqsH18p0+r0KGPUqVR8=": {
"hosts": {
"7BZSfLVyoTc12xgpvMUSWGTNsjjP4iqv/JSgpYbHQC4=": {
"hostnames": {
"green": {
"hostname": "green"
}
},
"ip": "fdcc:c5da:5295:c853:d499:937c:31a2:1e86",
"last_seen": 1731199277,
"port": 7331,
"signature": "RUZEqQoH1E2TuuB0rcQeaEuyxLTB70xgcj2VvRpvDwRtxvbaXegErJ7ei5obS46x3ApjgVP+3Di7OTXBSxqUCnsiaG9zdG5hbWVzIjogeyJncmVlbiI6IHsiaG9zdG5hbWUiOiAiZ3JlZW4ifX0sICJpcCI6ICJmZGNjOmM1ZGE6NTI5NTpjODUzOmQ0OTk6OTM3YzozMWEyOjFlODYiLCAibGFzdF9zZWVuIjogMTczMTE5OTI3NywgInBvcnQiOiA3MzMxfQ=="
},
"D9mq63wEznl4kHhsoQbq8hpncvGZeWC0vEOekcB8Nko=": {
"hostnames": {
"mors": {
"hostname": "mors"
}
},
"ip": "fdcc:c5da:5295:c853:d499:93e9:c5fc:c8b5",
"last_seen": 1731180076,
"port": 7331,
"signature": "/9o91MnAmSQTnbJCOK29zc2NcoAg8jI3SHbJ1NLiVQfCWafZ9MRqakkT/yLbgOTaepTCy2VFmu2HXalqnnUyC3siaG9zdG5hbWVzIjogeyJtb3JzIjogeyJob3N0bmFtZSI6ICJtb3JzIn19LCAiaXAiOiAiZmRjYzpjNWRhOjUyOTU6Yzg1MzpkNDk5OjkzZTk6YzVmYzpjOGI1IiwgImxhc3Rfc2VlbiI6IDE3MzExODAwNzYsICJwb3J0IjogNzMzMX0="
}
},
"settings": {
"banned_keys": [],
"host_signing_keys": [],
"hostname_overrides": {},
"last_update": 1724161701,
"public": true,
"signature": "K3UIjSbQkjKRM2yBlj8PIoeIZq4PyvImJss6SWremYBzggzibjnx8A5mifh0GF0xHig0J4gVhDmsYqogovRuDA==",
"tld": "nether"
}
}
}
```
### Examples for merging
for settings bigger last_update always wins
```
a:
{ hosts: { ... },
settings: {
last_update: 2,
tld: "test",
signature: "sig2"
}
}
b:
{ hosts: { ... },
settings: {
last_update: 3,
tld: "test2",
signature: "sig3"
}
}
result:
{ hosts: { ... },
settings: {
last_update: 3,
tld: "test2",
signature: "sig3"
}
}
```
hosts with bigger last_update win,
new hosts will be added
```
a:
{
hosts: {
pub1: {
ip: "42::1",
port: 7331,
last_update 3,
signature "sig_pub1_3"
}
},
settings: { ... },
}
b:
{
hosts: {
pub1: {
ip: "42::3",
port: 7331,
last_update 1,
signature "sig_pub1_1"
},
pub2: {
ip: "42::2",
port: 7331,
last_update 2,
signature "sig_pub2_2"
}
},
settings: { ... },
}
result:
{
hosts: {
pub1: {
ip: "42::1",
port: 7331,
last_update 3,
signature "sig_pub1_3"
},
pub2: {
ip: "42::2",
port: 7331,
last_update 2,
signature "sig_pub2_2"
}
},
settings: { ... },
}
```
### Time-to-Live (TTL) and Gossip Protocol
In the Data-Mesher, all information should "decay" over time, meaning it automatically expires after a set period (this feature is not yet implemented). The **settings** field should include a configurable **Time-to-Live (TTL)**, which removes old information, such as host entries, once they exceed the specified TTL.
- This makes attacks possible where Bobs Laptop is offline for TTL time (because Bob is on vacation) and another user claims their hostname.
- This means **hostnames** are only to be trusted if
A) no trusted dm-node has been compromised and
B) The hosts are never longer offline then the TTL
To prevent their data from expiring, hosts must regularly send updates to other peers. However, these updates don’t need to reach every peer directly. Since nodes share their entire data set during communication, information can relay through other nodes. As a result, a **gossip-style communication system** can be used, where information spreads gradually across the network through indirect connections, ensuring that all nodes stay up-to-date without overwhelming network traffic.
### Joining Multiple DM-Networks
Here’s an example of how you can configure Data-Mesher in NixOS to join two different DM-Networks. In this setup, the `.qubasa.clan` domain gets higher priority than the `.clan` domain.
What does this mean? If both networks have a `home` hostname (e.g., `home.qubasa.clan`), the one from `.qubasa.clan` will take precedence over the one from `.clan`. The network with the **lower priority number** wins in the case of conflicts (closer to 0).
Here's the Nix configuration:
```nix
services.data-mesher = {
enable = true; # Enable Data-Mesher service
interface = "<mesh_vpn>"; # The network interface Data-Mesher will use
openFirewall = true; # Ensure the firewall allows Data-Mesher traffic
# Define the DM-Networks to join
networks = {
"qubasa.clan" = {
pubkey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKTi2h4X56CzjeY4L1INl1d5JvYwh7HpaSuUlD33RhnY"; # Public key for the qubasa.clan network
priority = 1; # Higher priority (lower number = higher priority)
bootstrapPeers = [
"http://[fd27:bb88:dbef:737b:3799:9318:aa77:ec12]:7331" # A peer within this network to bootstrap from
];
};
"clan" = {
pubkey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJ1UM2Cza+GIRyuB9C3NqY0pSWnGC4DzmQOcWOa4SafV"; # Public key for the clan network
priority = 2; # Lower priority (higher number = lower priority)
bootstrapPeers = [
"http://[fd16:aa77:dbef:737b:3799:9316:aa77:dbef]:7331" # A peer within this network to bootstrap from
];
};
};
};
```
### Key Points:
- **Priority System**: `qubasa.clan` has a **priority** of `1.0`, while `clan` has `2.0`. So, if there's a conflicting hostname, `qubasa.clan` wins since it has a smaller priority number.
- **Bootstrap Peers**: Each network defines one or more **bootstrap peers**—these are known nodes that help your node join the network by sharing the network’s data.
- **Public Keys**: Each network you join requires its **public key** to verify the network’s integrity.
- Note: We use floats here because they are infinitely divisible. Which means one can add a higher priority node anywhere without having to change other nodes priorities.
This setup allows you to join and sync with multiple peer-to-peer DM-Networks and ensures the network with the defined priority takes precedence where necessary.
### DNS Output (`dns.json`)
The Data-Mesher generates a file called `dns.json`, which contains all valid and verified hostnames in a simple, flat JSON format. This file is designed to be easy to consume by other applications or services that need to reference known network hosts.
Here’s an example of a `dns.json` file:
```json
{"hostname": "mors.nether", "ip": "fdcc:c5da:5295:c853:d499:93e9:c5fc:c8b5"}
{"hostname": "green.nether", "ip": "fdcc:c5da:5295:c853:d499:937c:31a2:1e86"}
```
Each entry consists of:
- **hostname**: A valid, peer-reviewed hostname in the network.
- **ip**: The corresponding IP address (IPv6) of the hostname.
This file provides an up-to-date list of hostnames that are known to be good and usable within the network.
---
### signing timestaps by trusted nodes
It should probably look like this:
```json
{
"xxx": {
"hosts": {
"123": {
"hostnames": {
"123": {
"xyz": {
"signed_at": 128389182,
"signature": "signedbyxyz"
}
}
},
last_seen: 11,
signature "signedby123withtime11"
},
"456": {
"hostnames": {
"456": [
"xyz": {
"signed_at": 1231881298,
"signature": "signedbyxyz"
}
]
}
}
},
"settings": {
"host_signing_keys": [
"xyz"
]
}
}
}
```
### Bootstrapping
Currently, the Data-Mesher starts by using a list of URLs to pull bootstrap data from. These URLs can point to peers in the network or just a web server that makes `data.json` available.
If a node suddenly finds itself isolated because all other hosts have "decayed" or become unreachable, it should go back to the bootstrap peers to retrieve fresh network information.
It would also be useful to have a feature that allows you to add new peers to a running Data-Mesher instance. For example, you could say, "Hey, connect to this peer and get its data" while the system is still running, without needing to restart or manually reconfigure everything.
---
### Future Ideas
Ideas that are not currently on our roadmap but we want to see in the future.
#### **Host Schema**
It would be great if we could make the host schema more flexible, so it isn't tied specifically to hostnames. Hostnames could then just become one possible implementation of this schema. Here's an idea of how the settings could look:
```json
{
"settings": {
"hostSchema": {
"hostnames": {
"schema": $some JSON schema,
"merge": oldest_wins,
}
}
}
}
```
However, this doesn't cover the requirement of **signing hosts** (i.e., ensuring hostnames are signed by trusted peers). One possible solution could be to require signatures by trusted third-party hosts for all data in these schemas, since we're already using them for merging as well.
#### **Data Schema**
It would also be interesting to expand the current schema (which supports **hosts** and **settings**) with a third top-level key, like **data**. This new data section would have its own schema defined in the settings, along with a merge function. Specific hosts will be authorized to modify this data, meaning nodes can verify if the data has been changed by an allowed writer.
To accomplish this, we'd need a protocol to verify the **signatures** and the **origin** of the data (this part hasn’t been specified fully yet, but I’m happy to discuss ideas!).
Here’s a rough idea of how the structure could look:
```json
{
hosts: {
"123": { ... },
"456": { ... }
},
settings: {
dataSchema: {
x: {
schema: $some JSON schema,
merge: newest_wins,
allowed_writers: [
"123"
]
}
}
},
"data":{
x: {
value: "10.42.0.1",
signed_at: 1231212,
signature: "signedby123"
}
}
}
```
In this example:
- The **dataSchema** would define the rules for the data field, including how it should be merged (e.g., **newest-wins**) and which hosts are **allowed to write** the data (e.g., only host "123").
- The **data** section includes information such as the **value**, when it was **signed**, and the **signature** to verify who modified it.

160
AI_Data/Clan/data-mesher/index.md Normal file
View File

@@ -0,0 +1,160 @@
+++
title= "Introducing Data-Mesher: A CRDT-Based Peer-to-Peer Database"
subline= "DNS was designed for resilience, built to function even during catastrophic failures. But despite its distributed nature, it places control in the hands of..."
date = 2024-11-18T09:08:10+02:00
draft = false
authors = [ "Lassulus", "Qubasa" ]
tags = ['Dev Report']
+++
**one authority per domain**. This centralization limits flexibility in networks where multiple, independent nodes need to contribute and manage data collaboratively.
**Data-Mesher** changes this. Running on every node in a peer-to-peer network, Data-Mesher functions as a fully decentralized database using **CRDTs (Conflict-Free Replicated Data Types)** to resolve conflicts. What makes it unique is that **any node—trusted or untrusted—can add data** without compromising others'. Multiple authorities can coexist, allowing for decentralized control of key information, such as DNS-like hostnames and application settings.
In Data-Mesher, the network, not a single authority, resolves conflicts, making it ideal for systems that need true multi-party collaboration.
---
### How Does Data-Mesher Work?
At its core, Data-Mesher runs on every participating node (host) within a distributed system, allowing each node to independently store, update, and sync data across other nodes. It uses a **CRDT**-based approach to resolve conflicts, ensuring that all nodes eventually converge on the same dataset.
What makes Data-Mesher unique is its capability to allow untrusted nodes to contribute data without compromising the integrity of other nodes' contributions. This is particularly useful in peer-to-peer environments, where nodes might not have complete trust in each other but still require collaborative data sharing.
****
---
### The Basic Structure of a dm-network
The data in Data-Mesher is grouped into what we call a **dm-network**, which is primarily a key-value structure. In a very basic form, the dm-network is simply a group of hosts (nodes) and **settings** bundled under a shared public key (an ed25519 key). Nodes in the dm-network collaborate by announcing DNS hostnames and other settings relevant to application configuration.
Here are key elements of a dm-network:
- **Settings**
Protected by a digital signature only accessible to admins with the correct private key. Settings control policy, which could include anything from banning specific keys to establishing rules for hostname overrides.
- **Hosts**
Acts as the storage for **hostnames** that nodes contribute. Each hostname entry is signed, and the node uses its ed25519 key pair to generate this signature. In case two hosts try to register the same hostname, Data-Mesher selects the earliest signed entry.
---
### Key Mechanisms of Data Synchronization
1. **Node Communication and Data Sync**
When two nodes communicate, they exchange their entire dataset in the form of a `data.json` file. This file contains all known settings and hosts, where each entry is signed and timestamped. The **merge** function ensures that both nodes end up with identical data, regardless of the order in which changes are received. The merge strategy for configuration settings is simple—whichever entry has the most recent `last_updated` timestamp wins.
2. **Timestamp Attestation**
To avoid timestamp manipulation (such as backdating a record to give precedence to newer data), Data-Mesher relies on **timestamp attestation servers**. RFC3161-based services like [FreeTSA](https://www.freetsa.org/index_de.php) or [OpenTimestamps](https://opentimestamps.org/) are options for obtaining cryptographic timestamps. These services accept payloads (e.g., the CRDT entries) and return cryptographically signed timestamps, ensuring tamper-proofing.
3. **Handling Invalid Hostnames**
If a node submits a hostname without a valid timestamp, this hostname will be stored but marked as inactive until it can be verified by a timestamp attestation service. Only once validation is complete and the hostname carries a valid signed timestamp will it be activated within the network.
4. **Invalid Signatures**
Hostnames or timestamp entries backed by invalid signatures are rejected upfront and do not propagate through the network, adding a layer of security to prevent malicious data from infiltrating the network.
---
### Use Case: Distributed DNS and Hostname Management
One of the main applications implemented in Data-Mesher is in managing DNS-like functionality for decentralized networks. Nodes in the dm-network can announce DNS hostnames, and these are propagated across the peer-to-peer system without requiring centralized management.
- Each node registers its own hostnames.
- Host entries are shared across nodes in a **gossip-style** protocol to ensure they're up-to-date without overloading network throughput. Nodes communicate indirectly and share updates with the nodes they connect to, spreading the data gradually rather than all at once.
#### **Example `dns.json` Output**
As part of its functionality, Data-Mesher periodically generates a `dns.json` file, which contains hostname-to-IP mappings in a consumable format. Multiple services can utilize this information to route traffic within the network.
Example:
```json
{"hostname": "mors.nether", "ip": "fdcc:c5da:5295:c853:d499:93e9:c5fc:c8b5"}
{"hostname": "green.nether", "ip": "fdcc:c5da:5295:c853:d499:937c:31a2:1e86"}
```
---
### Security and Accuracy Mechanisms
One inherent challenge in any distributed network is ensuring accuracy and security without compromising on decentralization. Data-Mesher balances these demands with several meaningful mechanisms:
1. **Signature Validation**
Each data entry (whether it's a setting or hostname) must have a valid signature to be trusted and propagated across nodes. This allows nodes to validate the origin and authenticity of every piece of shared data.
2. **Reachability Checks**
When a node announces a new hostname, other nodes verify the reachability of the target IP and port before incorporating the hostname into their working configuration. The verification protocol ensures that the machine providing the hostname entry can correctly respond to a challenge tied to the relevant private key.
---
### Joining Multiple DM-Networks
Data-Mesher supports joining multiple DM-Networks, and nodes can prioritize which network has control in cases of conflicts. Here’s an example of how you can configure Data-Mesher in NixOS to join two different DM-Networks.
In this setup, the `.qubasa.clan` domain gets higher priority than the `.clan` domain.
What does this mean?
If both networks have colliding hostname (e.g., `home.qubasa.clan`), the one from `.qubasa.clan` will take precedence over the one from `.clan`. The network with the **lower priority number** wins in the case of conflicts (closer to 0).
Here’s a Nix configuration to demonstrate this:
```nix
services.data-mesher = {
enable = true; # Enable Data-Mesher service
interface = "<mesh_vpn>"; # The network interface Data-Mesher will use
openFirewall = true; # Ensure the firewall allows Data-Mesher traffic
# Define the DM-Networks to join
networks = {
"qubasa.clan" = {
pubkey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKTi2h4X56CzjeY4L1INl1d5JvYwh7HpaSuUlD33RhnY"; # Public key for the qubasa.clan network
priority = 1; # Higher priority (lower number = higher priority)
bootstrapPeers = [
"http://[fd27:bb88:dbef:737b:3799:9318:aa77:ec12]:7331" # A peer within this network to bootstrap from
];
};
"clan" = {
pubkey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJ1UM2Cza+GIRyuB9C3NqY0pSWnGC4DzmQOcWOa4SafV"; # Public key for the clan network
priority = 2; # Lower priority (higher number = lower priority)
bootstrapPeers = [
"http://[fd16:aa77:dbef:737b:3799:9316:aa77:dbef]:7331" # A peer within this network to bootstrap from
];
};
};
};
```
#### Key Points:
- **Priority System**: `qubasa.clan` has a priority of `1`, while `clan` has priority `2`. If there’s a conflicting hostname, Data-Mesher will resolve it in favor of the `qubasa.clan` network since it has a lower priority number.
- **Bootstrap Peers**: Each network is associated with one or more bootstrap peers, which help your node join that network by sharing an initial dataset. In this example, two peers are provided, one from the `qubasa.clan` and another from the `clan` domain.
- **Public Keys**: Each network you join requires a valid public key. The key is used to verify that the data received belongs to that specific network and has not been tampered with.
- **Floating Point Priorities**: Data-Mesher uses floating-point numbers for priorities. This allows you to insert new networks at any level of priority without reworking the entire priority hierarchy. For example, you could add a network with priority `1.5` between `qubasa.clan` (priority `1`) and `clan` (priority `2`).
This setup allows nodes to coordinate and sync across multiple DM-Networks seamlessly, while ensuring that conflicts are handled predictably based on each network’s priority.
---
### Bootstrapping and Network Resurrection
In case a node finds itself isolated from the rest of the dm-network (perhaps due to network partitions or decayed peers), it can bootstrap itself by pulling static data from a list of pre-configured URLs. These bootstrap URLs can point either to other nodes in the dm-network or a simple web server serving a static `data.json` file.
---
### What’s Next? Future Enhancements
#### **Customizable Data Schemas**
While the current focus is on hostnames and settings, we foresee a broader application case by allowing more diverse use cases like network configurations, or even custom application settings. This would be achieved by extending the current schema definitions to allow broader categories beyond just "hosts" and "settings".
This could involve creating more flexible merge criteria (newer-wins, append-only, etc.) and defining writers and permissions for specific data keys.
#### **Flexible Host Object Definitions**
At the moment, the "host" object in the schema is somewhat inflexible, focusing primarily on DNS information. A future enhancement might include expanding this definition into a more abstract schema, allowing for various other types of node-related information beyond DNS settings.
---
### From Theory to Practice
Data-Mesher isn't just a theoretical construct—it can be a practical solution for networks that require decentralized peer coordination, conflict-free updates, and scalable distribution of discrete configurations like DNS names.
For those interested in deploying decentralized networks with configurations that automatically self-consume and degrade over time (via TTL), Data-Mesher offers a foundational approach that prioritizes security, consistency, and flexibility in equal measure.

BIN
AI_Data/Clan/data-mesher/post-hero-image.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.7 MiB

132
AI_Data/Clan/declarative-backups-and-restore/index.md Normal file
View File

@@ -0,0 +1,132 @@
+++
title = 'Declarative Backups and Restore'
subline = 'Our goal with Clan is to give users control over their data. However, with great power comes great responsibility, and owning your data means you also need to take care of backups yourself.'
date = 2024-06-24T09:08:15+02:00
draft = false
author = "Mic92"
tags = ['Dev Report']
+++
Our goal with [Clan](https://clan.lol/) is to give users control over their data.
However, with great power comes great responsibility, and owning your data means you also need to take care of backups yourself.
In our experience, setting up automatic backups is often a tedious process as it requires custom integration of the backup software and
the services that produce the state. More important than the backup is the restore.
Restores are often not well tested or documented, and if not working correctly, they can render the backup useless.
In Clan, we want to make backup and restore a first-class citizen.
Every service should describe what state it produces and how it can be backed up and restored.
In this article, we will discuss how our backup interface in Clan works.
The interface allows different backup software to be used interchangeably and
allows module authors to define custom backup and restore logic for their services.
## First Comes the State
Our services are built from Clan modules. Clan modules are essentially [NixOS modules](https://wiki.nixos.org/wiki/NixOS_modules), the basic configuration components of NixOS.
However, we have enhanced them with additional features provided by Clan and restricted certain option types to enable configuration through a [graphical interface](https://docs.clan.lol/blog/2024/05/25/jsonschema-converter/).
In a simple case, this can be just a bunch of directories, such as what we define for our [ZeroTier](https://www.zerotier.com/) VPN service.
```nix
{
clan.core.state.zerotier.folders = [ "/var/lib/zerotier-one" ];
}
```
For other systems, we need more complex backup and restore logic.
For each state, we can provide custom command hooks for backing up and restoring.
In our PostgreSQL module, for example, we define `preBackupCommand` and `postRestoreCommand` to use `pg_dump` and `pg_restore` to backup and restore individual databases:
```nix
preBackupCommand = ''
# ...
runuser -u postgres -- pg_dump ${compression} --dbname=${db.name} -Fc -c > "${current}.tmp"
# ...
'';
postRestoreCommand = ''
# ...
runuser -u postgres -- dropdb "${db.name}"
runuser -u postgres -- pg_restore -C -d postgres "${current}"
# ...
'';
```
## Then the Backup
Our CLI unifies the different backup providers in one [interface](https://docs.clan.lol/reference/cli/backups/).
As of now, we support backups using [BorgBackup](https://www.borgbackup.org/) and
a backup module called "localbackup" based on [rsnapshot](https://rsnapshot.org/), optimized for backup on locally attached storage media.
To use different backup software, a module needs to set the options provided by our backup interface.
The following Nix code is a toy example that uses the `tar` program to perform backup and restore to illustrate how the backup interface works:
```nix
clan.core.backups.providers.tar = {
list = ''
echo /var/lib/system-back.tar
'';
create = let
uniqueFolders = lib.unique (
lib.flatten (lib.mapAttrsToList (_name: state: state.folders) config.clan.core.state)
);
in ''
# FIXME: a proper implementation should also run `state.preBackupCommand` of each state
if [ -f /var/lib/system-back.tar ]; then
tar -uvpf /var/lib/system-back.tar ${builtins.toString uniqueFolders}
else
tar -cvpf /var/lib/system-back.tar ${builtins.toString uniqueFolders}
fi
'';
restore = ''
IFS=':' read -ra FOLDER <<< "''$FOLDERS"
echo "${FOLDER[@]}" > /run/folders-to-restore.txt
tar -xvpf /var/lib/system-back.tar -C / -T /run/folders-to-restore.txt
'';
};
```
For better real-world implementations, check out the implementations for [BorgBackup](https://git.clan.lol/clan/clan-core/src/branch/main/clanModules/borgbackup/default.nix)
and [localbackup](https://git.clan.lol/clan/clan-core/src/branch/main/clanModules/localbackup/default.nix).
## What It Looks Like to the End User
After following the guide for configuring a [backup](https://docs.clan.lol/getting-started/backups/),
users can use the CLI to create backups, list them, and restore them.
Backups can be created through the CLI like this:
```
clan backups create web01
```
BorgBackup will also create backups itself every day by default.
Completed backups can be listed like this:
```
clan backups list web01
...
web01::u366395@u366395.your-storagebox.de:/./borgbackup::web01-web01-2024-06-18T01:00:00
web03::u366395@u366395.your-storagebox.de:/./borgbackup::web01-web01-2024-06-19T01:00:00
```
One cool feature of our backup system is that it is aware of individual services/applications.
Let's say we want to restore the state of our [Matrix](https://matrix.org/) chat server; we can just specify it like this:
```
clan backups restore --service matrix-synapse web01 borgbackup web03::u366395@u366395.your-storagebox.de:/./borgbackup::web01-web01-2024-06-19T01:00:00
```
In this case, it will first stop the matrix-synapse systemd service, then delete the [PostgreSQL](https://www.postgresql.org/) database, restore the database from the backup, and then start the matrix-synapse service again.
## Future work
As of now we implemented our backup and restore for a handful of services and we expect to refine the interface as we test the interface for more applications.
Currently, our backup implementation backs up filesystem state from running services.
This can lead to inconsistencies if applications change the state while the backup is running.
In the future, we hope to make backups more atomic by backing up a filesystem snapshot instead of normal directories.
This, however, requires the use of modern filesystems that support these features.

BIN
AI_Data/Clan/declarative-backups-and-restore/post-hero-image.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.2 MiB

149
AI_Data/Clan/getting-started/backups.md Normal file
View File

@@ -0,0 +1,149 @@
# Backups
## Introduction to Backups
When you're managing your own services, creating regular backups is crucial to ensure your data's safety.
This guide introduces you to Clan's built-in backup functionalities.
Clan supports backing up your data to both local storage devices (like USB drives) and remote servers, using well-known tools like borgbackup and rsnapshot.
We might add more options in the future, but for now, let's dive into how you can secure your data.
## Backing Up Locally with Localbackup
### What is Localbackup?
Localbackup lets you backup your data onto physical storage devices connected to your computer,
such as USB hard drives or network-attached storage. It uses a tool called rsnapshot for this purpose.
### Setting Up Localbackup
1. **Identify Your Backup Device:**
First, figure out which device you'll use for backups. You can see all connected devices by running this command in your terminal:
```bash
lsblk --output NAME,PTUUID,FSTYPE,SIZE,MOUNTPOINT
```
Look for the device you intend to use for backups and note its details.
2. **Configure Your Backup Device:**
Once you've identified your device, you'll need to add it to your configuration.
Here's an example NixOS configuration for a device located at `/dev/sda2` with an `ext4` filesystem:
```nix
{
fileSystems."/mnt/hdd" = {
device = "/dev/sda2";
fsType = "ext4";
options = [ "defaults" "noauto" ];
};
}
```
Replace `/dev/sda2` with your device and `/mnt/hdd` with your preferred mount point.
3. **Set Backup Targets:** Next, define where on your device you'd like the backups to be stored:
```nix
{
clan.localbackup.targets.hdd = {
directory = "/mnt/hdd/backup";
mountpoint = "/mnt/hdd";
};
}
```
Change `/mnt/hdd` to the actual mount point you're using.
4. **Create Backups:** To create a backup, run:
```bash
clan backups create mymachine
```
This command saves snapshots of your data onto the backup device.
5. **Listing Backups:** To see available backups, run:
```bash
clan backups list mymachine
```
## Remote Backups with Borgbackup
### Overview of Borgbackup
Borgbackup splits the backup process into two parts: a backup client that sends data to a backup server.
The server stores the backups.
### Setting Up the Borgbackup Client
1. **Specify Backup Server:**
Start by indicating where your backup data should be sent. Replace `hostname` with your server's address:
```nix
{
clan.borgbackup.destinations = {
myhostname = {
repo = "borg@backuphost:/var/lib/borgbackup/myhostname";
};
};
}
```
2. **Select Folders to Backup:**
Decide which folders you want to back up. For example, to backup your home and root directories:
```nix
{ clan.core.state.userdata.folders = [ "/home" "/root" ]; }
```
3. **Generate Backup Credentials:**
Run `clan facts generate <yourmachine>` to prepare your machine for backup, creating necessary SSH keys and credentials.
### Setting Up the Borgbackup Server
1. **Configure Backup Repository:**
On the server where backups will be stored, enable the SSH daemon and set up a repository for each client:
```nix
{
services.borgbackup.repos.myhostname = {
path = "/var/lib/borgbackup/myhostname";
authorizedKeys = [
(builtins.readFile (config.clan.core.clanDir + "/machines/myhostname/facts/borgbackup.ssh.pub"))
];
};
}
```
Ensure the path to the public key is correct.
2. **Update Your Systems:** Apply your changes by running `clan machines update` to both the server and your client
### Managing Backups
- **Scheduled Backups:**
Backups are automatically performed nightly. To check the next scheduled backup, use:
```bash
systemctl list-timers | grep -E 'NEXT|borg'
```
- **Listing Backups:** To see available backups, run:
```bash
clan backups list mymachine
```
- **Manual Backups:** You can also initiate a backup manually:
```bash
clan backups create mymachine
```

223
AI_Data/Clan/getting-started/configure.md Normal file
View File

@@ -0,0 +1,223 @@
# Configuration - How to configure clan with your own machines
Managing machine configurations can be done in the following ways:
- writing `nix` expressions in a `flake.nix` file,
- placing `autoincluded` files into your machine directory,
- configuring everything in a simple UI (upcoming).
Clan currently offers the following methods to configure machines:
!!! Success "Recommended for nix people"
- flake.nix (i.e. via `buildClan`)
- `machine` argument
- `inventory` argument
- machines/`machine_name`/configuration.nix (`autoincluded` if it exists)
???+ Note "Used by CLI & UI"
- inventory.json
- machines/`machine_name`/hardware-configuration.nix (`autoincluded` if it exists)
!!! Warning "Deprecated"
machines/`machine_name`/settings.json
## Global configuration
In the `flake.nix` file:
- [x] set a unique `name`.
=== "**normal flake template**"
```nix title="flake.nix" hl_lines="3"
buildClan {
# Set a unique name
meta.name = "Lobsters";
# Should usually point to the directory of flake.nix
directory = ./.;
machines = {
jon = {
# ...
};
# ...
}
}
```
=== "**template using flake-parts**"
!!! info "See [Clan with flake-parts](../manual/flake-parts.md) for help migrating to flake-parts."
```nix title="flake.nix" hl_lines="3"
clan = {
# Set a unique name
meta.name = "Lobsters";
machines = {
jon = {
# ...
};
# ...
}
};
```
## Machine configuration
Adding or configuring a new machine requires two simple steps:
### Step 1. Identify Target Disk-ID
1. Find the remote disk id by executing:
```bash title="setup computer"
ssh root@<IP> lsblk --output NAME,ID-LINK,FSTYPE,SIZE,MOUNTPOINT
```
!!! Note
Replace `<IP>` with the IP address of the machine if you don't have the avahi service running which resolves mDNS local domains.
Which should show something like:
```{.shellSession hl_lines="6" .no-copy}
NAME ID-LINK FSTYPE SIZE MOUNTPOINT
sda usb-ST_16GB_AA6271026J1000000509-0:0 14.9G
├─sda1 usb-ST_16GB_AA6271026J1000000509-0:0-part1 1M
├─sda2 usb-ST_16GB_AA6271026J1000000509-0:0-part2 vfat 100M /boot
└─sda3 usb-ST_16GB_AA6271026J1000000509-0:0-part3 ext4 2.9G /
nvme0n1 nvme-eui.e8238fa6bf530001001b448b4aec2929 476.9G
├─nvme0n1p1 nvme-eui.e8238fa6bf530001001b448b4aec2929-part1 vfat 512M
├─nvme0n1p2 nvme-eui.e8238fa6bf530001001b448b4aec2929-part2 ext4 459.6G
└─nvme0n1p3 nvme-eui.e8238fa6bf530001001b448b4aec2929-part3 swap 16.8G
```
1. Edit the following fields inside the `./machines/jon/configuration.nix` and/or `./machines/sara/configuration.nix`
```nix title="./machines/<machine>/configuration.nix" hl_lines="13 18 23 27"
{
imports = [
./hardware-configuration.nix
# contains your disk format and partitioning configuration.
../../modules/disko.nix
# this file is shared among all machines
../../modules/shared.nix
# enables GNOME desktop (optional)
../../modules/gnome.nix
];
# Put your username here for login
users.users.user.username = "__YOUR_USERNAME__";
# Set this for clan commands use ssh i.e. `clan machines update`
# If you change the hostname, you need to update this line to root@<new-hostname>
# This only works however if you have avahi running on your admin machine else use IP
clan.core.networking.targetHost = "root@__IP__";
# You can get your disk id by running the following command on the installer:
# Replace <IP> with the IP of the installer printed on the screen or by running the `ip addr` command.
# ssh root@<IP> lsblk --output NAME,ID-LINK,FSTYPE,SIZE,MOUNTPOINT
disko.devices.disk.main.device = "/dev/disk/by-id/__CHANGE_ME__";
# IMPORTANT! Add your SSH key here
# e.g. > cat ~/.ssh/id_ed25519.pub
users.users.root.openssh.authorizedKeys.keys = [ "__YOUR_SSH_KEY__" ];
# ...
}
```
!!! Info "Replace `__YOUR_USERNAME__` with the ip of your machine, if you use avahi you can also use your hostname"
!!! Info "Replace `__IP__` with the ip of your machine, if you use avahi you can also use your hostname"
!!! Info "Replace `__CHANGE_ME__` with the appropriate identifier, such as `nvme-eui.e8238fa6bf530001001b448b4aec2929`"
!!! Info "Replace `__YOUR_SSH_KEY__` with your personal key, like `ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILoMI0NC5eT9pHlQExrvR5ASV3iW9+BXwhfchq0smXUJ jon@jon-desktop`"
These steps will allow you to update your machine later.
### Step 2: Detect Drivers
Generate the `hardware-configuration.nix` file for your machine by executing the following command:
```bash
clan machines update-hardware-config [MACHINE_NAME] [HOSTNAME]
```
replace `[MACHINE_NAME]` with the name of the machine i.e. `jon` and `[HOSTNAME]` with the `ip_address` or `hostname` of the machine within the network. i.e. `<IP>`
!!! Example
```bash
clan machines update-hardware-config jon
```
This command connects to the ip configured in the previous step, runs `nixos-generate-config` to detect hardware configurations (excluding filesystems), and writes them to `machines/jon/hardware-configuration.nix`.
### Step 3: Custom Disk Formatting
In `./modules/disko.nix`, a simple `ext4` disk partitioning scheme is defined for the Disko module. For more complex disk partitioning setups,
refer to the [Disko templates](https://github.com/nix-community/disko-templates) or [Disko examples](https://github.com/nix-community/disko/tree/master/example).
### Step 4: Custom Configuration
Modify `./machines/jon/configuration.nix` to personalize the system settings according to your requirements.
If you wish to name your machine to something else, do the following steps:
```
mv ./machines/jon/configuration.nix ./machines/newname/configuration.nix
```
Than rename `jon` to your preferred name in `machines` in `flake.nix` as well as the import line:
```diff
- imports = [ ./machines/jon/configuration.nix ];
+ imports = [ ./machines/__NEW_NAME__/configuration.nix ];
```
!!! Info "Replace `__NEW_NAME__` with the name of the machine"
Note that our clan lives inside a git repository.
Only files that have been added with `git add` are recognized by `nix`.
So for every file that you add or rename you also need to run:
```
git add ./path/to/my/file
```
For renaming jon to your own machine name, you can use the following command:
```
git mv ./machines/jon ./machines/newname
```
If you only want to setup a single machine at this point, you can delete `sara` from flake.nix as well as from the machines directory:
```
git rm ./machines/sara
```
### Step 5: Check Configuration
Validate your configuration by running:
```bash
nix flake check
```
This command helps ensure that your system configuration is correct and free from errors.
!!! Tip
You can integrate this step into your [Continuous Integration](https://en.wikipedia.org/wiki/Continuous_integration) workflow to ensure that only valid Nix configurations are merged into your codebase.
---
## Whats next?
- [Secrets & Facts](secrets.md): Setting up secrets with nix-sops
---

226
AI_Data/Clan/getting-started/deploy.md Normal file
View File

@@ -0,0 +1,226 @@
# Deploy Machine
Integrating a new machine into your Clan environment is an easy yet flexible process, allowing for a straight forward management of multiple NixOS configurations.
We'll walk you through adding a new computer to your Clan.
## Installing a New Machine
Clan CLI, in conjunction with [nixos-anywhere](https://github.com/nix-community/nixos-anywhere), provides a seamless method for installing NixOS on various machines.
This process involves preparing a suitable hardware and disk partitioning configuration and ensuring the target machine is accessible via SSH.
### Step 0. Prerequisites
=== "**Physical Hardware**"
- [x] **Two Computers**: You need one computer that you're getting ready (we'll call this the Target Computer) and another one to set it up from (we'll call this the Setup Computer). Make sure both can talk to each other over the network using SSH.
- [x] **Machine configuration**: See our basic [configuration guide](./configure.md)
- [x] **Initialized secrets**: See [secrets](secrets.md) for how to initialize your secrets.
- [x] **USB Flash Drive**: See [Clan Installer](installer.md)
!!! Steps
1. Create a NixOS installer image and transfer it to a bootable USB drive as described in the [installer](./installer.md).
2. Boot the target machine and connect it to a network that makes it reachable from your setup computer.
=== "**Remote Machines**"
- [x] **Two Computers**: You need one computer that you're getting ready (we'll call this the Target Computer) and another one to set it up from (we'll call this the Setup Computer). Make sure both can talk to each other over the network using SSH.
- [x] **Machine configuration**: See our basic [configuration guide](./configure.md)
- [x] **Initialized secrets**: See [secrets](secrets.md) for how to initialize your secrets.
!!! Steps
- Any cloud machine if it is reachable via SSH and supports `kexec`.
### Step 1. Deploy the machine
**Finally deployment time!** Use the following command to build and deploy the image via SSH onto your machine.
=== "**Image Installer**"
This method makes use of the image installers of [nixos-images](https://github.com/nix-community/nixos-images).
See how to prepare the installer for use [here](./installer.md).
The installer will randomly generate a password and local addresses on boot, then run ssh with these preconfigured.
The installer shows it's deployment relevant information in two formats, a text form, as well as a QR code.
This is an example of the booted installer.
```{ .bash .annotate .no-copy .nohighlight}
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ ┌───────────────────────────┐ │
│ │███████████████████████████│ # This is the QR Code (1) │
│ │██ ▄▄▄▄▄ █▀▄█▀█▀▄█ ▄▄▄▄▄ ██│ │
│ │██ █ █ █▀▄▄▄█ ▀█ █ █ ██│ │
│ │██ █▄▄▄█ █▀▄ ▀▄▄▄█ █▄▄▄█ ██│ │
│ │██▄▄▄▄▄▄▄█▄▀ ▀▄▀▄█▄▄▄▄▄▄▄██│ │
│ │███▀▀▀ █▄▄█ ▀▄ ▄▀▄█ ███│ │
│ │██▄██▄▄█▄▄▀▀██▄▀ ▄▄▄ ▄▀█▀██│ │
│ │██ ▄▄▄▄▄ █▄▄▄▄ █ █▄█ █▀ ███│ │
│ │██ █ █ █ █ █ ▄▄▄ ▄▀▀ ██│ │
│ │██ █▄▄▄█ █ ▄ ▄ ▄ ▀█ ▄███│ │
│ │██▄▄▄▄▄▄▄█▄▄▄▄▄▄█▄▄▄▄▄█▄███│ │
│ │███████████████████████████│ │
│ └───────────────────────────┘ │
│ ┌─────────────────────────────────────────────────────────────────────────────────┐ │
│ │Root password: cheesy-capital-unwell # password (2) │ │
│ │Local network addresses: │ │
│ │enp1s0 UP 192.168.178.169/24 metric 1024 fe80::21e:6ff:fe45:3c92/64 │ │
│ │enp2s0 DOWN │ │
│ │wlan0 DOWN # connect to wlan (3) │ │
│ │Onion address: 6evxy5yhzytwpnhc2vpscrbti3iktxdhpnf6yim6bbs25p4v6beemzyd.onion │ │
│ │Multicast DNS: nixos-installer.local │ │
│ └─────────────────────────────────────────────────────────────────────────────────┘ │
│ Press 'Ctrl-C' for console access │
│ │
└─────────────────────────────────────────────────────────────────────────────────────┘
```
1. This is not an actual QR code, because it is displayed rather poorly on text sites.
This would be the actual content of this specific QR code prettified:
```json
{
"pass": "cheesy-capital-unwell",
"tor": "6evxy5yhzytwpnhc2vpscrbti3iktxdhpnf6yim6bbs25p4v6beemzyd.onion",
"addrs": [
"2001:9e8:347:ca00:21e:6ff:fe45:3c92"
]
}
```
To generate the actual QR code, that would be displayed use:
```shellSession
echo '{"pass":"cheesy-capital-unwell","tor":"6evxy5yhzytwpnhc2vpscrbti3iktxdhpnf6yim6bbs25p4v6beemzyd.onion","addrs":["2001:9e8:347:ca00:21e:6ff:fe45:3c92"]}' | nix run nixpkgs#qrencode -- -s 2 -m 2 -t utf8
```
2. The root password for the installer medium.
This password is autogenerated and meant to be easily typeable.
3. See how to connect the installer medium to wlan [here](./installer.md#optional-connect-to-wifi-manually).
4. :man_raising_hand: I'm a code annotation! I can contain `code`, __formatted
text__, images, ... basically anything that can be written in Markdown.
!!!tip
For easy sharing of deployment information via QR code, we highly recommend using [KDE Connect](https://apps.kde.org/de/kdeconnect/).
There are two ways to deploy your machine:
1. **SSH with Password Authentication**
Run the following command to install using SSH:
```bash
clan machines install [MACHINE] --target-host <IP>
```
2. **Scanning a QR Code for Installation Details**
You can input the information by following one of these methods:
- **Using a JSON String or File Path:**
Provide the path to a JSON string or input the string directly:
```terminal
clan machines install [MACHINE] --json [JSON]
```
- **Using an Image Containing the QR Code:**
Provide the path to an image file containing the relevant QR code:
```terminal
clan machines install [MACHINE] --png [PATH]
```
=== "**SSH access**"
Replace `<target_host>` with the **target computers' ip address**:
```bash
clan machines install [MACHINE] --target-host <target_host>
```
If you are using our template `[MACHINE]` would be `jon`
!!! success
Your machine is all set up. 🎉 🚀
## Update Your Machines
Clan CLI enables you to remotely update your machines over SSH. This requires setting up a target address for each target machine.
### Setting the Target Host
Replace `root@jon` with the actual hostname or IP address of your target machine in the `configuration.nix` of the machine:
```{.nix hl_lines="9" .no-copy}
{
# ...
# Set this for clan commands use ssh i.e. `clan machines update`
# If you change the hostname, you need to update this line to root@<new-hostname>
# This only works however if you have avahi running on your admin machine else use IP
clan.core.networking.targetHost = "root@jon";
};
```
!!! warning
The use of `root@` in the target address implies SSH access as the `root` user.
Ensure that the root login is secured and only used when necessary.
### Updating Machine Configurations
Execute the following command to update the specified machine:
```bash
clan machines update jon
```
You can also update all configured machines simultaneously by omitting the machine name:
```bash
clan machines update
```
### Setting a Build Host
If the machine does not have enough resources to run the NixOS evaluation or build itself,
it is also possible to specify a build host instead.
During an update, the cli will ssh into the build host and run `nixos-rebuild` from there.
```{.nix hl_lines="5" .no-copy}
buildClan {
# ...
machines = {
"jon" = {
clan.core.networking.buildHost = "root@<host_or_ip>";
};
};
};
```
### Excluding a machine from `clan machine update`
To exclude machines from being updated when running `clan machines update` without any machines specified,
one can set the `clan.deployment.requireExplicitUpdate` option to true:
```{.nix hl_lines="5" .no-copy}
buildClan {
# ...
machines = {
"jon" = {
clan.deployment.requireExplicitUpdate = true;
};
};
};
```
This is useful for machines that are not always online or are not part of the regular update cycle.
---
## What's next ?
- [**Disk Encryption**](./disk-encryption.md): Configure disk encryption with remote decryption
- [**Mesh VPN**](./mesh-vpn.md): Configuring a secure mesh network.
---

180
AI_Data/Clan/getting-started/disk-encryption.md Normal file
View File

@@ -0,0 +1,180 @@
This guide provides an example setup for a single-disk ZFS system with native encryption, accessible for decryption remotely.
!!! Warning
This configuration only applies to `systemd-boot` enabled systems and **requires** UEFI booting.
Replace the highlighted lines with your own disk-id.
You can find our your disk-id by executing:
```bash
lsblk --output NAME,ID-LINK,FSTYPE,SIZE,MOUNTPOINT
```
=== "**Single Disk**"
Below is the configuration for `disko.nix`
```nix hl_lines="17 48"
--8<-- "docs/code-examples/disko-single-disk.nix"
```
=== "**Raid 1**"
Below is the configuration for `disko.nix`
```nix hl_lines="17 48 49"
--8<-- "docs/code-examples/disko-raid.nix"
```
Below is the configuration for `initrd.nix`.
Replace `<yourkey>` with your ssh public key.
Replace `kernelModules` with the ethernet module loaded one on your target machine.
```nix hl_lines="18 29"
{config, pkgs, ...}:
{
boot.initrd.systemd = {
enable = true;
};
# uncomment this if you want to be asked for the decryption password on login
#users.root.shell = "/bin/systemd-tty-ask-password-agent";
boot.initrd.network = {
enable = true;
ssh = {
enable = true;
port = 7172;
authorizedKeys = [ "<yourkey>" ];
hostKeys = [
"/var/lib/initrd-ssh-key"
];
};
};
boot.initrd.availableKernelModules = [
"xhci_pci"
];
# Find out the required network card driver by running `lspci -k` on the target machine
boot.initrd.kernelModules = [ "r8169" ];
}
```
### Step 1: Copying SSH Public Key
Before starting the installation process, ensure that the SSH public key is copied to the NixOS installer.
1. Copy your public SSH key to the installer, if it has not been copied already:
```bash
ssh-copy-id -o PreferredAuthentications=password -o PubkeyAuthentication=no root@nixos-installer.local
```
### Step 1.5: Prepare Secret Key and Clear Disk Data
1. Access the installer using SSH:
```bash
ssh root@nixos-installer.local
```
2. Create a `secret.key` file in `/tmp` using `nano` or another text editor:
```bash
nano /tmp/secret.key
```
3. Discard the old disk partition data:
```bash
blkdiscard /dev/disk/by-id/nvme-eui.002538b931b59865
```
4. Run the `clan` machine installation with the following command:
```bash
clan machines install gchq-local --target-host root@nixos-installer --yes --no-reboot
```
### Step 2: ZFS Pool Import and System Installation
1. SSH into the installer once again:
```bash
ssh root@nixos-installer.local
```
2. Perform the following commands on the remote installation environment:
```bash
zpool import zroot
zfs set keylocation=prompt zroot/root
zfs load-key zroot/root
zfs set mountpoint=/mnt zroot/root/nixos
mount /dev/nvme0n1p2 /mnt/boot
```
3. Disconnect from the SSH session:
```bash
CTRL+D
```
4. Securely copy your local `initrd_rsa_key` to the installer's `/mnt` directory:
```bash
scp ~/.ssh/initrd_rsa_key root@nixos-installer.local:/mnt/var/lib/initrd-ssh-key
```
5. SSH back into the installer:
```bash
ssh root@nixos-installer.local
```
6. Navigate to the `/mnt` directory, enter the `nixos-enter` environment, and then exit:
```bash
cd /mnt
nixos-enter
realpath /run/current-system
exit
```
7. Run the `nixos-install` command with the appropriate system path `<SYS_PATH>`:
```bash
nixos-install --no-root-passwd --no-channel-copy --root /mnt --system <SYS_PATH>
```
8. After the installation process, unmount `/mnt/boot`, change the ZFS mountpoint, and reboot the system:
```bash
umount /mnt/boot
cd /
zfs set mountpoint=/ zroot/root/nixos
reboot
```
9. Perform a hard reboot of the machine and remove the USB stick.
### Step 3: Accessing the Initial Ramdisk (initrd) Environment
1. SSH into the initrd environment using the `initrd_rsa_key` and provided port:
```bash
ssh -p 7172 root@192.168.178.141
```
2. Run the `systemd-tty-ask-password-agent` utility to query a password:
```bash
systemd-tty-ask-password-agent --query
```
After completing these steps, your NixOS should be successfully installed and ready for use.
**Note:** Replace `root@nixos-installer.local` and `192.168.178.141` with the appropriate user and IP addresses for your setup. Also, adjust `<SYS_PATH>` to reflect the correct system path for your environment.

109
AI_Data/Clan/getting-started/index.md Normal file
View File

@@ -0,0 +1,109 @@
# :material-clock-fast: Getting Started
Create your own clan with these initial steps and manage a fleet of machines with one single testable git repository!
### Prerequisites
=== "**Linux**"
Clan depends on nix installed on your system. Run the following command to install nix.
```bash
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install
```
If you already have installed Nix, make sure you have the `nix-command` and `flakes` configuration enabled in your ~/.config/nix/nix.conf.
The determinate installer already comes with this configuration by default.
```bash
# /etc/nix/nix.conf or ~/.config/nix/nix.conf
experimental-features = nix-command flakes
```
=== "**NixOS**"
If you run NixOS the `nix` binary is already installed.
You will also need to enable the `flakes` and `nix-commands` experimental features in your configuration.nix:
```nix
{ nix.settings.experimental-features = [ "nix-command" "flakes" ]; }
```
=== "**Other**"
Clan doesn't offer dedicated support for other operating systems yet.
### Step 1: Add Clan CLI to Your Shell
Add the Clan CLI into your development workflow:
```bash
nix shell git+https://git.clan.lol/clan/clan-core#clan-cli
```
You can find reference documentation for the `clan` cli program [here](../reference/cli/index.md).
Alternatively you can check out the help pages directly:
```terminalSession
clan --help
```
### Step 2: Initialize Your Project
Set the foundation of your Clan project by initializing it as follows:
```bash
clan flakes create my-clan
```
This command creates the `flake.nix` and `.clan-flake` files for your project.
It will also generate files from a default template, to help show general clan usage patterns.
### Step 3: Verify the Project Structure
Ensure that all project files exist by running:
```bash
cd my-clan
tree
```
This should yield the following:
``` { .console .no-copy }
.
├── flake.nix
├── machines
│   ├── jon
│   │   ├── configuration.nix
│   │   └── hardware-configuration.nix
│   └── sara
│   ├── configuration.nix
│   └── hardware-configuration.nix
└── modules
└── shared.nix
5 directories, 9 files
```
```bash
clan machines list
```
``` { .console .no-copy }
jon
sara
```
!!! success
You just successfully bootstrapped your first clan directory.
---
### What's Next?
- [**Installer**](./installer.md): Setting up new computers remotely is easy with an USB stick.
---

174
AI_Data/Clan/getting-started/installer.md Normal file
View File

@@ -0,0 +1,174 @@
# Installer
Our installer image simplifies the process of performing remote installations.
Follow our step-by-step guide to create and transfer this image onto a bootable USB drive.
!!! info
If you already have a NixOS machine you can ssh into (in the cloud for example) you can skip this chapter and go directly to [Configure Machines](configure.md).
### Step 0. Prerequisites
- [x] A free USB Drive with at least 1.5GB (All data on it will be lost)
- [x] Linux/NixOS Machine with Internet
### Step 1. Identify the USB Flash Drive
1. Insert your USB flash drive into your computer.
2. Identify your flash drive with `lsblk`:
```shellSession
lsblk
```
```{.shellSession hl_lines="2" .no-copy}
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sdb 8:0 1 117,2G 0 disk
└─sdb1 8:1 1 117,2G 0 part /run/media/qubasa/INTENSO
nvme0n1 259:0 0 1,8T 0 disk
├─nvme0n1p1 259:1 0 512M 0 part /boot
└─nvme0n1p2 259:2 0 1,8T 0 part
└─luks-f7600028-9d83-4967-84bc-dd2f498bc486 254:0 0 1,8T 0 crypt /nix/store
```
!!! Info "In this case the USB device is `sdb`"
3. Ensure all partitions on the drive are unmounted. Replace `sdb1` in the command below with your device identifier (like `sdc1`, etc.):
```shellSession
sudo umount /dev/sdb1
```
=== "**Linux OS**"
### Step 2. Flash Custom Installer
Using clan flash enables the inclusion of ssh public keys and wifi access points.
It also allows to set language and keymap in the installer image.
```bash
clan flash write --flake git+https://git.clan.lol/clan/clan-core \
--ssh-pubkey $HOME/.ssh/id_ed25519.pub \
--keymap us \
--language en_US.UTF-8 \
--disk main /dev/sd<X> \
flash-installer
```
!!! Note
Replace `$HOME/.ssh/id_ed25519.pub` with a path to your SSH public key.
Replace `/dev/sd<X>` with the drive path you want to flash
!!! Danger "Specifying the wrong device can lead to unrecoverable data loss."
The `clan flash` utility will erase the disk. Make sure to specify the correct device
- **SSH-Pubkey Option**:
To add an ssh public key into the installer image append the option:
```
--ssh-pubkey <pubkey_path>
```
If you do not have an ssh key yet, you can generate one with `ssh-keygen -t ed25519` command.
- **Wifi Option**:
To add wifi credentials into the installer image append the option:
```
--wifi <ssid> <password>
```
- **List Keymaps**:
You can get a list of all keymaps with the following command:
```
clan flash list keymaps
```
- **List Languages**:
You can get a list of all languages with the following command:
```
clan flash list languages
```
=== "**Other OS**"
### Step 2. Download Generic Installer
```shellSession
wget https://github.com/nix-community/nixos-images/releases/download/nixos-unstable/nixos-installer-x86_64-linux.iso
```
### Step 2.5 Flash the Installer to the USB Drive
!!! Danger "Specifying the wrong device can lead to unrecoverable data loss."
The `dd` utility will erase the disk. Make sure to specify the correct device (`of=...`)
For example if the USB device is `sdb` use `of=/dev/sdb`.
Use the `dd` utility to write the NixOS installer image to your USB drive:
```shellSession
sudo dd bs=4M conv=fsync oflag=direct status=progress if=./nixos-installer-x86_64-linux.iso of=/dev/sd<X>
```
### Step 3: Boot From USB Stick
- To use, boot from the Clan USB drive with **secure boot turned off**. For step by step instructions go to [Disabling Secure Boot](../manual/secure-boot.md)
## (Optional) Connect to Wifi Manually
If you don't have access via LAN the Installer offers support for connecting via Wifi.
```shellSession
iwctl
```
This will enter `iwd`
```{.console, .no-copy}
[iwd]#
```
Now run the following command to connect to your Wifi:
```{.shellSession .no-copy}
# Identify your network device.
device list
# Replace 'wlan0' with your wireless device name
# Find your Wifi SSID.
station wlan0 scan
station wlan0 get-networks
# Replace your_ssid with the Wifi SSID
# Connect to your network.
station wlan0 connect your_ssid
# Verify you are connected
station wlan0 show
```
If the connection was successful you should see something like this:
```{.console, .no-copy}
State connected
Connected network FRITZ!Box (Your router device)
IPv4 address 192.168.188.50 (Your new local ip)
```
Press ++ctrl+d++ to exit `IWD`.
!!! Important
Press ++ctrl+d++ **again** to update the displayed QR code and connection information.
You're all set up
---
## Whats next?
- [Configure Machines](configure.md): Customize machine configuration
---

117
AI_Data/Clan/getting-started/mesh-vpn.md Normal file
View File

@@ -0,0 +1,117 @@
# Mesh VPN
This guide provides detailed instructions for configuring
[ZeroTier VPN](https://zerotier.com) within Clan. Follow the
outlined steps to set up a machine as a VPN controller (`<CONTROLLER>`) and to
include a new machine into the VPN.
## Concept
By default all machines within one clan are connected via a chosen network technology.
```{.no-copy}
Clan
Node A
<-> (zerotier / mycelium / ...)
Node B
```
If you select multiple network technologies at the same time. e.g. (zerotier + yggdrassil)
You must choose one of them as primary network and the machines are always connected via the primary network.
## 1. Set-Up the VPN Controller
The VPN controller is initially essential for providing configuration to new
peers. Once addresses are allocated, the controller's continuous operation is not essential.
1. **Designate a Machine**: Label a machine as the VPN controller in the clan,
referred to as `<CONTROLLER>` henceforth in this guide.
2. **Add Configuration**: Input the following configuration to the NixOS
configuration of the controller machine:
```nix
clan.core.networking.zerotier.controller = {
enable = true;
public = true;
};
```
3. **Update the Controller Machine**: Execute the following:
```bash
clan machines update <CONTROLLER>
```
Your machine is now operational as the VPN controller.
## 2. Add Machines to the VPN
To introduce a new machine to the VPN, adhere to the following steps:
1. **Update Configuration**: On the new machine, incorporate the following to its
configuration, substituting `<CONTROLLER>` with the controller machine name:
```nix
{ config, ... }: {
clan.core.networking.zerotier.networkId = builtins.readFile (config.clan.core.clanDir + "/machines/<CONTROLLER>/facts/zerotier-network-id");
}
```
1. **Update the New Machine**: Execute:
```bash
$ clan machines update <NEW_MACHINE>
```
Replace `<NEW_MACHINE>` with the designated new machine name.
!!! Note "For Private Networks"
1. **Retrieve Zerotier Metadata**
=== "From the repo"
**Retrieve the ZeroTier IP**: In the clan repo, execute:
```console
$ clan facts list <NEW_MACHINE> | jq -r '.["zerotier-ip"]'
```
The returned address is the Zerotier IP address of the machine.
=== "On the new machine"
**Retrieve the ZeroTier ID**: On the `new_machine`, execute:
```bash
$ sudo zerotier-cli info
```
Example Output:
```{.console, .no-copy}
200 info d2c71971db 1.12.1 OFFLINE
```
, where `d2c71971db` is the ZeroTier ID.
2. **Authorize the New Machine on the Controller**: On the controller machine,
execute:
=== "with ZerotierIP"
```bash
$ sudo zerotier-members allow --member-ip <IP>
```
Substitute `<IP>` with the ZeroTier IP obtained previously.
=== "with ZerotierID"
```bash
$ sudo zerotier-members allow <ID>
```
Substitute `<ID>` with the ZeroTier ID obtained previously.
2. **Verify Connection**: On the `new_machine`, re-execute:
```bash
$ sudo zerotier-cli info
```
The status should now be "ONLINE":
```{.console, .no-copy}
200 info d2c71971db 1.12.1 ONLINE
```
!!! success "Congratulations!"
The new machine is now part of the VPN, and the ZeroTier
configuration on NixOS within the Clan project is complete.
## Further
Currently you can only use **Zerotier** as networking technology because this is the first network stack we aim to support.
In the future we plan to add additional network technologies like tinc, head/tailscale, yggdrassil and mycelium.
We chose zerotier because in our tests it was a straight forwards solution to bootstrap.
It allows you to selfhost a controller and the controller doesn't need to be globally reachable.
Which made it a good fit for starting the project.

63
AI_Data/Clan/getting-started/secrets.md Normal file
View File

@@ -0,0 +1,63 @@
# Secrets / Facts
Clan enables encryption of secrets (such as passwords & keys) ensuring security and ease-of-use among users.
Clan utilizes the [sops](https://github.com/getsops/sops) format and integrates with [sops-nix](https://github.com/Mic92/sops-nix) on NixOS machines.
This guide will walk you through:
- **Creating a Keypair for Your User**: Learn how to generate a keypair for $USER to securely control all secrets.
- **Creating Your First Secret**: Step-by-step instructions on creating your initial secret.
- **Assigning Machine Access to the Secret**: Understand how to grant a machine access to the newly created secret.
## Create Your Admin Keypair
To get started, you'll need to create **Your admin keypair**.
!!! info
Don't worry — if you've already made one before, this step won't change or overwrite it.
```bash
clan secrets key generate
```
**Output**:
```{.console, .no-copy}
Public key: age1wkth7uhpkl555g40t8hjsysr20drq286netu8zptw50lmqz7j95sw2t3l7
Generated age private key at '/home/joerg/.config/sops/age/keys.txt' for your user. Please back it up on a secure location or you will lose access to your secrets.
Also add your age public key to the repository with 'clan secrets users add YOUR_USER age1wkth7uhpkl555g40t8hjsysr20drq286netu8zptw50lmqz7j95sw2t3l7' (replace YOUR_USER with your actual username)
```
!!! warning
Make sure to keep a safe backup of the private key you've just created.
If it's lost, you won't be able to get to your secrets anymore because they all need the admin key to be unlocked.
!!! note
It's safe to add any secrets created by the clan CLI and placed in your repository to version control systems like `git`.
### Add Your Public Key
```bash
clan secrets users add $USER <your_public_key>
```
It's best to choose the same username as on your Setup/Admin Machine that you use to control the deployment with.
Once run this will create the following files:
```{.console, .no-copy}
sops/
└── users/
└── <your_username>/
└── key.json
```
If you followed the quickstart tutorial all necessary secrets are initialized at this point.
---
## Whats next?
- [Deployment](deploy.md): How to remotely deploy your machine
- Full [Secrets](../manual/secrets.md) guide If you want to know more about how to save and share passwords in your clan

173
AI_Data/Clan/interfaces/index.md Normal file
View File

@@ -0,0 +1,173 @@
+++
title= "Providing Type-safe interfaces between NixOS and other languages"
subline= "One method for creating type-safe interfaces in a nix based software stack"
date = 2024-09-11T09:08:10+02:00
draft = false
authors = ["Johannes Kirshbauer" ]
tags = ['Dev Report']
+++
When building a consumer-facing project on top of NixOS, one crucial question arises:
How can we provide type-safe interfaces within a polyglot software stack?
This blogpost discusses one method for creating type-safe interfaces in a software stack by using JSON-schema to maintain consistent models across layers of an application.
---
Within the [clan](https://clan.lol) project, we explored one possible solution to this challenge. Our tech stack is composed of three main components:
- Nix: Handles the core business logic.
- Python: Acts as a thin wrapper, exposing the business logic through a convenient to use CLI and API.
- TypeScript: Manages the presentation and GUI layer, communicating with Python via an API.
This architecture is a product of our guiding principles: We aim to encapsulate as much business logic as possible in pure Nix, ensuring that anyone familiar with Nix can utilize it.
### The Challenge of Polyglot Architectures
Throughout the lifecycle of an application, architectural models, relationships, and fields are typically refined and improved over time.
By this, I refer to constructs such as classes, structs, or enums.
These elements are often required across multiple layers of the application
and must remain consistent to avoid discrepancies.
Logically, maintaining these models in a single location is crucial to prevent discrepancies and
eliminate a common source of errors — interface inconsistencies between software components.
This approach can save significant time during development cycles,
particularly in dynamically typed environments like Nix and Python,
where errors are often caught through extensive unit testing or at runtime when issues arise.
The Nix language presents a significant challenge due to its untyped and dynamic nature. Combined with NixOS, a constantly evolving collection of modules, it becomes incredibly difficult to build stable interfaces. As we develop more complex applications, a crucial question emerges: "How can models (such as classes or structs) be shared between multiple foreign languages and Nix?"
One potential solution is to define the model once in a chosen language and then generate the necessary code for all other languages. This approach ensures consistency and reduces the likelihood of errors caused by manual translations between languages.
Well-defined, statically typed models would provide build-time checks for correctness and prevent many issues and regressions that could have been avoided with robust interfaces.
Building on the earlier blog post about the [NixOS modules to JSON-schema converter](https://docs.clan.lol/blog/2024/05/25/jsonschema-converter/), a further exploration could involve using JSON-schema as an intermediate format. While not explicitly mentioned in the blog post, the JSON-schema converter operates solely on the interface declaration. It can also populate example values and other metadata that may become important later.
In our case, we decided to use NixOS module interface declarations as the source of truth, as all our models are Nix-first citizens. We will use JSON-schema as an interoperable format that can further be utilized to generate Python classes and TypeScript types.
For example, the desired Python code output could be a `TypedDict` or a `dataclass`. Since our input data might contain Nix attribute names that are invalid identifiers in Python, and vice versa, it is preferable to choose dataclasses. This allows us to store more metadata about the mapping relationships within the field properties.
```nix title="in.nix"
{lib, ...}:
let
types = lib.types;
option = t: lib.mkOption {
type = t;
};
in
{
options = {
submodule = option (types.submodule {
options = {
string = option types.str;
list-str = option (types.listOf types.str);
attrs-str = option (types.attrsOf types.str);
};
});
};
}
```
With the following nix code this can be converted into python
```nix title="convert.nix"
let
# Import clan-core flake
clan-core = builtins.getFlake "git+https://git.clan.lol/clan/clan-core";
pkgs = import clan-core.inputs.nixpkgs {};
# Step 1: Convert NixOS module expression to JSON schema
serialize = expr: builtins.toFile "in.json" (builtins.toJSON expr);
schema = serialize ((clan-core.lib.jsonschema {}).parseModule ./in.nix);
# classgenerator
inherit (clan-core.packages.x86_64-linux) classgen;
in
{
inherit schema;
python-classes = pkgs.runCommand "py-cls" {}
''
${classgen}/bin/classgen ${schema} $out
'';
}
```
Now execute the following:
```shellSession
nix build -f convert.nix python-classes
```
The final Python code ensures that the Python component is always in sync with the Nix code.
```python title="out.py"
@dataclass
class Submodule:
string: str
attrs_str: dict[str, str] = field(default_factory = dict, metadata = {"alias": "attrs-str"})
list_str: list[str] = field(default_factory = list, metadata = {"alias": "list-str"})
@dataclass
class Module:
submodule: Submodule
```
---
However, this approach comes with some constraints for both the interface itself and the tools surrounding it:
- All types used in the interface must be JSON-serializable (e.g., Number, String, AttrSet, List, etc.).
- We identified certain constraints that work best for dataclasses, while also enhancing the final user experience:
- Top-level options should specify a default value or be nullable.
- Ideally, option identifiers should use names that don't require a "field-alias," although this might not always be feasible.
- Neutral values for lists or dictionaries, such as an empty list or empty dictionary, must be supported.
The Python generator adds default constructors for dictionary and list types because the absence of a value would violate our type constraints.
It is also important to note that we control both the JSON schema converter and the class generator, which is crucial. This control allows us to limit their scope to a subset of JSON schema features and ensure interoperability between the two generators.
Another consideration is serialization and deserialization. In Python, Pydantic is typically a great choice, as it also offers [custom serializers](https://docs.pydantic.dev/latest/concepts/serialization/#custom-serializers). However, when working with NixOS modules, we chose not to emit unset or null values because they create merge conflicts in the underlying NixOS modules. We also wanted to use field-aliases for names that are invalid identifiers in Python or TypeScript and wanted validation to catch errors early (in the deserializer) between our frontend and Nix, allowing us to present well-formatted errors instead of Nix evaluation error stack traces. Nevertheless, we ultimately did not use Pydantic because it did not meet our needs.
---
### Catching errors
Interface violations or regressions can be detected during the development cycle at build time.
```python
Submodule(string=1)
```
```sh
> Argument of type "Literal[1]" cannot be assigned to parameter "string" of type "str" in function "__init__"
"Literal[1]" is incompatible with "str"
```
Since all our layers communicate through JSON interfaces, any potential runtime type errors are caught in Python during deserialization before they can trigger any Nix stack traces. This allows for errors to be neatly formatted for the consumer.
```python
data = {"submodule": { "string": 1 } }
checked(Model, data)
>>> Traceback (most recent call last):
>>> ...
>>> Expected string, got 1
```
---
### Future
By adopting this approach, we aim to provide a stable and secure interface for polyglot software stacks built on top of Nixpkgs,
ultimately enhancing the reliability and maintainability of complex applications.
Additionally, we will improve the tooling and develop a library, making this methodology applicable to other projects as well.
### Links
- https://docs.clan.lol/blog/2024/05/25/jsonschema-converter/
---

BIN
AI_Data/Clan/interfaces/post-hero-image.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.7 MiB

69
AI_Data/Clan/introduction-clan/index.md Normal file
View File

@@ -0,0 +1,69 @@
+++
title= "Introducing Clan: Full-Stack Computing Redefined"
subline= "Introducing Clan, a new model for a decentralized network, designed to provide families, smaller groups, and small businesses a platform that’s private, secure, and user-friendly. „Clan reclaims computing and networking from the ground up.”"
date = 2024-03-19T09:08:10+02:00
draft = false
authors = [ "W", "Qubasa" ]
tags = ['Dev Report']
asciinema = true
+++
In a digital age where users are guided increasingly toward submission and dependence, Clan reclaims computing and networking from the ground up.
Clan enables users to build any system from a git repository, automate secret handling, and join devices in a secure darknet. This control extends beyond applications to communication protocols and the operating system itself, putting you fully in charge of your own digital environment.
## Why We're Building Clan
Our mission is simple: to restore fun, freedom, and functionality to computing as an open source project. We believe in building tools that empower users, foster innovation, and challenge the limitations imposed by outdated paradigms. Clan, in its essence, is an open source endeavor; it's our contribution to a future where technology serves humanity, not the other way around.
## How Clan Changes the Game
Clan embodies a new philosophy in system, application, and network design. It enables seamless, secure communication across devices, simplifies software distribution and updates, and offers both public and private network configurations. Here are some of the ways it accomplishes this:
**Nix as a Foundation:** Imagine a safety net for your computer's operating system, one that lets you make changes or updates without the fear of causing a crash or losing data. Nix simplifies the complexities of system design, ensuring that updates are safe and systems are more reliable.
**Simplified System Deployment:** Building and managing a computer system, from the operating system to the software you use, often feels like putting together a complex puzzle. With Clan, the puzzle pieces are replaced by a set of building blocks. Leveraging the power of Nix and Clan's innovative toolkit, anyone from tech-savvy administrators to everyday users can create and maintain what we call "full-stack systems" (everything your computer needs to run smoothly).
**A Leap in Connectivity:** Imagine if you could create private, secure pathways between your devices, bypassing the noisy and often insecure internet. Clan makes this possible through something called "overlay networks." These networks are like private tunnels, allowing your devices to talk to each other securely and directly. With Clan's built-in overlay networks and automatically configured services, connecting your devices becomes seamless, secure, and hassle-free.
**Security Through Separation:** Clan employs sandboxing and virtual machines, a technology that runs code in isolated environments - so even if you explore new Clans, your system remains protected from potential threats.
**Reliable:** With Clan, your data and services are preserved for the long haul. We focus on self-hosted backups and integration with the [Fediverse](https://de.wikipedia.org/wiki/Fediverse), a network of interconnected, independent online communities, so your digital life remains uninterrupted and under your control.
## A Glimpse at Clan's Features
**Social Scaling:** Choose between creating a private sanctuary for your closest contacts, a dynamic space for a self-contained community, or embracing the open web with public Clans anyone can join.
{{< video key="show_join" >}}
**Seamless VM Integration:** Applications running in virtual machines can appear and behave as if they're part of your main operating system — a blend of power and simplicity.
{{< video key="show_run" >}}
**Robust Backup Management:** Keep your data safe _forever_ - never worry about cloud services disappearing in 10 years.
{{< asciinema key="backups" >}}
**Intuitive Secret Management:** Clan simplifies digital security by automating the creation and management of encryption keys and passwords for your services.
{{< asciinema key="secrets" >}}
**Remote Install:** Set up and manage Clan systems anywhere in the world with just a QR scan or SSH access, making remote installations as easy as snapping a photo or sharing a link.
{{< asciinema key="nixos-install" >}}
## Who Stands to Benefit?
Clan is for anyone and everyone who believes in the power of open source technology to connect, empower, and protect. From system administrators to less tech-savvy individuals, small business owners to privacy-conscious users, Clan offers something for everyone — a way to reclaim control and redefine how we interact with technology.
## Join the Revolution
Ready to control your digital world? Clan is more than a tool—it's a movement. Secure your data, manage your systems easily, or connect with others how you like. Start with Clan for a better digital future.
Connect with us on our [Matrix channel at clan.lol](https://matrix.to/#/#clan:clan.lol) or through our IRC bridges (coming soon).
Want to see the code? Check it out [on our Gitea](https://git.clan.lol/clan/clan-core) or [on GitHub](https://github.com/clan-lol/clan-core).
Or follow our [RSS feed](https://docs.clan.lol/feed_rss_created.xml)!
Join us and be part of changing technology for the better, together.

BIN
AI_Data/Clan/introduction-clan/post-hero-image.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 565 KiB

200
AI_Data/Clan/json-schema-converter/index.md Normal file
View File

@@ -0,0 +1,200 @@
+++
title= "Introducing the NixOS to JSON Schema Converter"
subline= "Discover our new library designed to extract JSON schema interfaces from NixOS modules, streamlining frontend development"
date = 2024-05-25T09:08:10+02:00
draft = false
author = "DavHau"
tags = ['Dev Report']
+++
## Overview
We’ve developed a new library designed to extract interfaces from NixOS modules and convert them into JSON schemas, paving the way for effortless GUI generation. This blog post outlines the motivations behind this development, demonstrates the capabilities of the library, and guides you through leveraging it to create GUIs seamlessly.
## Motivation
In recent months, our team has been exploring various graphical user interfaces (GUIs) to streamline NixOS machine configuration. While our opinionated Clan modules simplify NixOS configurations, there's a need to configure these modules from diverse frontends, such as:
- Command-line interfaces (CLIs)
- Web-based UIs
- Desktop applications
- Mobile applications
- Large Language Models (LLMs)
Given this need, a universal format like JSON is a natural choice. It is already possible as of now, to import json based NixOS configurations, as illustrated below:
`configuration.json`:
```json
{ "networking": { "hostName": "my-machine" } }
```
This configuration can be then imported inside a classic NixOS config:
```nix
{config, lib, pkgs, ...}: {
imports = [
(lib.importJSON ./configuration.json)
];
}
```
This straightforward approach allows us to build a frontend that generates JSON, enabling the configuration of NixOS machines. But, two critical questions arise:
1. How does the frontend learn about existing configuration options?
2. How can it verify user input without running Nix?
Introducing [JSON schema](https://json-schema.org/), a widely supported standard that defines interfaces in JSON and validates input against them.
Example schema for `networking.hostName`:
```json
{
"type": "object",
"properties": {
"networking": {
"type": "object",
"properties": {
"hostName": {
"type": "string",
"pattern": "^$|^[a-z0-9]([a-z0-9_-]{0,61}[a-z0-9])?$"
}
}
}
}
}
```
## Client-Side Input Validation
Validating input against JSON schemas is both efficient and well-supported across numerous programming languages. Using JSON schema validators, you can accurately check configurations like our `configuration.json`.
Validation example:
```shell
$ nix-shell -p check-jsonschema
$ jsonschema -o pretty ./schema.json -i ./configuration.json
===[SUCCESS]===(./configuration.json)===
```
In case of invalid input, schema validators provide explicit error messages:
```shell
$ echo '{ "networking": { "hostName": "my/machine" } }' > configuration.json
$ jsonschema -o pretty ./schema.json -i ./configuration.json
===[ValidationError]===(./configuration.json)===
'my/machine' does not match '^$|^[a-z0-9]([a-z0-9_-]{0,61}[a-z0-9])?$'
Failed validating 'pattern' in schema['properties']['networking']['properties']['hostName']:
{'pattern': '^$|^[a-z0-9]([a-z0-9_-]{0,61}[a-z0-9])?$',
'type': 'string'}
On instance['networking']['hostName']:
'my/machine'
```
## Automatic GUI Generation
Certain libraries facilitate straightforward GUI generation from JSON schemas. For instance, the [react-jsonschema-form playground](https://rjsf-team.github.io/react-jsonschema-form/) auto-generates a form for any given schema.
## NixOS Module to JSON Schema Converter
To enable the development of responsive frontends, our library allows the extraction of interfaces from NixOS modules to JSON schemas. Open-sourced for community collaboration, this library supports building sophisticated user interfaces for NixOS.
Here’s a preview of our library's functions exposed through the [clan-core](https://git.clan.lol/clan/clan-core) flake:
- `lib.jsonschema.parseModule` - Generates a schema for a NixOS module.
- `lib.jsonschema.parseOption` - Generates a schema for a single NixOS option.
- `lib.jsonschema.parseOptions` - Generates a schema from an attrset of NixOS options.
Example:
`module.nix`:
```nix
{lib, config, pkgs, ...}: {
# a simple service with two options
options.services.example-web-service = {
enable = lib.mkEnableOption "Example web service";
port = lib.mkOption {
type = lib.types.int;
description = "Port used to serve the content";
};
};
}
```
Converted, using the `parseModule` function:
```shell
$ cd clan-core
$ nix eval --json --impure --expr \
'(import ./lib/jsonschema {}).parseModule ./module.nix' | jq | head
{
"properties": {
"services": {
"properties": {
"example-web-service": {
"properties": {
"enable": {
"default": false,
"description": "Whether to enable Example web service.",
"examples": [
...
```
This utility can also generate interfaces for existing NixOS modules or options.
## GUI for NGINX in Under a Minute
Creating a prototype GUI for the NGINX module using our library and [react-jsonschema-form playground](https://rjsf-team.github.io/react-jsonschema-form/) can be done quickly:
1. Export all NGINX options into a JSON schema using a Nix expression:
```nix
# export.nix
let
pkgs = import <nixpkgs> {};
clan-core = builtins.getFlake "git+https://git.clan.lol/clan/clan-core";
options = (pkgs.nixos {}).options.services.nginx;
in
clan-core.lib.jsonschema.parseOption options
```
2. Write the schema into a file:
```shell
$ nix eval --json -f ./export.nix | jq > nginx.json
```
3. Open the [react-jsonschema-form playground](https://rjsf-team.github.io/react-jsonschema-form/), select `Blank` and paste the `nginx.json` contents.
This provides a quick look at a potential GUI (screenshot is cropped).
![Image title](https://clan.lol/static/blog-post-jsonschema/nginx-gui.jpg)
## Limitations
### Laziness
JSON schema mandates the declaration of all required fields upfront, which might be configured implicitly or remain unused. For instance, `services.nginx.virtualHosts.<name>.sslCertificate` must be specified even if SSL isn’t enabled.
### Limited Types
Certain NixOS module types, like `types.functionTo` and `types.package`, do not map straightforwardly to JSON. For full compatibility, adjustments to NixOS modules might be necessary, such as substituting `listOf package` with `listOf str`.
### Parsing NixOS Modules
Currently, our converter relies on the `options` attribute of evaluated NixOS modules, extracting information from the `type.name` attribute, which is suboptimal. Enhanced introspection capabilities within the NixOS module system would be beneficial.
## Future Prospects
We hope these experiments inspire the community, encourage contributions and further development in this space. Share your ideas and contributions through our issue tracker or matrix channel!
## Links
- [Comments on NixOS Discourse](https://discourse.nixos.org/t/introducing-the-nixos-to-json-schema-converter/45948)
- [Source Code of the JSON Schema Library](https://git.clan.lol/clan/clan-core/src/branch/main/lib/jsonschema)
- [Our Issue Tracker](https://git.clan.lol/clan/clan-core/issues)
- [Our Matrix Channel](https://matrix.to/#/#clan:clan.lol)
- [react-jsonschema-form Playground](https://rjsf-team.github.io/react-jsonschema-form/)

50
AI_Data/Clan/manual/adding-machines.md Normal file
View File

@@ -0,0 +1,50 @@
# Adding Machines
Clan has two general methods of adding machines
- **Automatic**: Detects every folder in the `machines` folder.
- **Declarative**: Explicit declarations in nix.
## Automatic register
Every machine of the form `machines/{machineName}` will be registered automatically.
Automatically imported:
- [x] ``machines/{machineName}/configuration.nix`
- [x] ``machines/{machineName}/hardware-configuration.nix`
- [x] ``machines/{machineName}/facter.json` Automatically configured, for further information see [nixos-facter](https://clan.lol/blog/nixos-facter/)
## Manual declaration
Machines can also be added manually under `buildClan`, `clan.*` in flake-parts or via [`inventory`](../manual/inventory.md).
!!! Note
It is possible to use `inventory` and `buildClan` together at the same time.
=== "**Individual Machine Configuration**"
```{.nix}
buildClan {
machines = {
"jon" = {
# Any valid nixos config
};
};
}
```
=== "**Inventory Configuration**"
```{.nix}
buildClan {
inventory = {
machines = {
"jon" = {
# Inventory machines can set tags
tags = [ "zone1" ];
};
};
};
}
```

1
AI_Data/Clan/manual/contribute.md Symbolic link
View File

@@ -0,0 +1 @@
../../CONTRIBUTING.md

98
AI_Data/Clan/manual/flake-parts.md Normal file
View File

@@ -0,0 +1,98 @@
# Clan with `flake-parts`
Clan supports integration with [flake.parts](https://flake.parts/) a tool which allows composing nixos modules in a modular way.
Here's how to set up Clan using `nix flakes` and `flake-parts`.
## 1. Update Your Flake Inputs
To begin, you'll need to add `flake-parts` as a new dependency in your flake's inputs. This is alongside the already existing dependencies, such as `clan-core` and `nixpkgs`. Here's how you can update your `flake.nix` file:
```nix
# flake.nix
inputs = {
nixpkgs.url = "github:nixos/nixpkgs?ref=nixos-unstable";
# New flake-parts input
flake-parts.url = "github:hercules-ci/flake-parts";
flake-parts.inputs.nixpkgs-lib.follows = "nixpkgs";
clan-core = {
url = "git+https://git.clan.lol/clan/clan-core";
inputs.nixpkgs.follows = "nixpkgs"; # Needed if your configuration uses nixpkgs unstable.
# New
inputs.flake-parts.follows = "flake-parts";
};
}
```
## 2. Import Clan-Core Flake Module
After updating your flake inputs, the next step is to import the `clan-core` flake module. This will make the [clan options](../reference/nix-api/buildclan.md) available within `mkFlake`.
```nix
outputs =
inputs@{ flake-parts, ... }:
flake-parts.lib.mkFlake { inherit inputs; } (
{
#
imports = [
inputs.clan-core.flakeModules.default
];
}
);
```
### 3. Configure Clan Settings and Define Machines
Configure your clan settings and define machine configurations.
Below is a guide on how to structure this in your flake.nix:
```nix
outputs = inputs@{ flake-parts, clan-core, ... }:
flake-parts.lib.mkFlake { inherit inputs; } ({self, pkgs, ...}: {
# We define our own systems below. you can still use this to add system specific outputs to your flake.
# See: https://flake.parts/getting-started
systems = [];
# import clan-core modules
imports = [
clan-core.flakeModules.default
];
# Define your clan
# See: https://docs.clan.lol/reference/nix-api/buildclan/
clan = {
# Clan wide settings. (Required)
meta.name = ""; # Ensure to choose a unique name.
machines = {
jon = {
imports = [
./machines/jon/configuration.nix
./modules/disko.nix
# ... more modules
];
nixpkgs.hostPlatform = "x86_64-linux";
# Set this for clan commands use ssh i.e. `clan machines update`
clan.core.networking.targetHost = pkgs.lib.mkDefault "root@jon";
# remote> lsblk --output NAME,ID-LINK,FSTYPE,SIZE,MOUNTPOINT
disko.devices.disk.main = {
device = "/dev/disk/by-id/nvme-eui.e8238fa6bf530001001b448b4aec2929";
};
# There needs to be exactly one controller per clan
clan.core.networking.zerotier.controller.enable = true;
};
};
};
});
```
For detailed information about configuring `flake-parts` and the available options within Clan,
refer to the Clan module documentation located [here](https://git.clan.lol/clan/clan-core/src/branch/main/flakeModules/clan.nix).
---

66
AI_Data/Clan/manual/index.md Normal file
View File

@@ -0,0 +1,66 @@
# :material-book: Guides
Instructions and explanations for practical Implementations ordered by Topics.
## Tutorials
**Learning-oriented adventures with a hands-on experience.**
<div class="grid cards" markdown>
- :material-clock-fast:{ .lg .middle } __Set up in 15 minutes__
---
Create your own clan and get everything
running in minutes
[:octicons-arrow-right-24: Getting started](../getting-started/index.md)
- :fontawesome-solid-user-group:{ .lg .middle } __Authoring Modules__
---
Create clanModules that can be reused by the community.
[:octicons-arrow-right-24: Authoring clanModules](../clanmodules/index.md)
</div>
## Guides
**How-to Guides for achieving a certain goal or solving a specific issue.**
<div class="grid cards" markdown>
- [Machines](./adding-machines.md)
---
Learn how Clan automatically includes machines and Nix files.
- [Secrets](./secrets.md)
---
Learn how to manage secrets.
- [Inventory](./inventory.md)
---
Clan's declaration format for running **services** on one or multiple **machines**.
- [Flake-parts](./flake-parts.md)
---
Use clan with [https://flake-parts.dev]()
- [Contribute](./contribute.md)
---
Discover how to set up a development environment to contribute to Clan!
</div>

135
AI_Data/Clan/manual/inventory.md Normal file
View File

@@ -0,0 +1,135 @@
# Inventory
`Inventory` is an abstract service layer for consistently configuring distributed services across machine boundaries.
See [Inventory API Documentation](../reference/nix-api/inventory.md)
This guide will walk you through setting up a backup service, where the inventory becomes useful.
!!! example "Experimental status"
The inventory implementation is not considered stable yet.
We are actively soliciting feedback from users.
Stabilizing the API is a priority.
## Prerequisites
- [x] [Add machines](./adding-machines.md) to your clan.
## Services
The inventory defines `services`. Membership of `machines` is defined via roles exclusively.
See the each [module documentation](../reference/clanModules/index.md) for available roles.
!!! Note
It is possible to use any [clanModule](../reference/clanModules/index.md) in the inventory and add machines via
`roles.default.*`
### Adding services to machines
A module can be added to one or multiple machines via `Roles`. clan's `Role` interface provide sane defaults for a module this allows the module author to reduce the configuration overhead to a minimum.
Each service can still be customized and configured according to the modules options.
- Per instance configuration via `services.<serviceName>.<instanceName>.config`
- Per role configuration via `services.<serviceName>.<instanceName>.roles.<roleName>.config`
- Per machine configuration via `services.<serviceName>.<instanceName>.machines.<machineName>.config`
### Setting up the Backup Service
!!! Example "Borgbackup Example"
To configure a service it needs to be added to the machine.
It is required to assign the service (`borgbackup`) an arbitrary instance name. (`instance_1`)
See also: [Multiple Service Instances](#multiple-service-instances)
```{.nix hl_lines="6-7"}
buildClan {
inventory = {
services = {
borgbackup.instance_1 = {
# Machines can be added here.
roles.client.machines = [ "jon" ];
roles.server.machines = [ "backup_server" ];
};
};
};
}
```
### Scaling the Backup
The inventory allows machines to set Tags
It is possible to add services to multiple machines via tags as shown
!!! Example "Tags Example"
```{.nix hl_lines="5 8 14"}
buildClan {
inventory = {
machines = {
"jon" = {
tags = [ "backup" ];
};
"sara" = {
tags = [ "backup" ];
};
# ...
};
services = {
borgbackup.instance_1 = {
roles.client.tags = [ "backup" ];
roles.server.machines = [ "backup_server" ];
};
};
};
}
```
### Multiple Service Instances
!!! danger "Important"
Not all modules implement support for multiple instances yet.
Multiple instance usage could create complexity, refer to each modules documentation, for intended usage.
!!! Example
In this example `backup_server` has role `client` and `server` in different instances.
```{.nix hl_lines="11 14"}
buildClan {
inventory = {
machines = {
"jon" = {};
"backup_server" = {};
"backup_backup_server" = {}
};
services = {
borgbackup.instance_1 = {
roles.client.machines = [ "jon" ];
roles.server.machines = [ "backup_server" ];
};
borgbackup.instance_2 = {
roles.client.machines = [ "backup_server" ];
roles.server.machines = [ "backup_backup_server" ];
};
};
};
}
```
### API specification
**The complete schema specification is available [here](../reference/nix-api/inventory.md)**
Or it can build anytime via:
```sh
nix build git+https://git.clan.lol/clan/clan-core#schemas.inventory
> result
> ├── schema.cue
> └── schema.json
```

251
AI_Data/Clan/manual/secrets.md Normal file
View File

@@ -0,0 +1,251 @@
If you want to know more about how to save and share passwords in your clan read further!
### Adding a Secret
```shellSession
clan secrets set mysecret
Paste your secret:
```
### Retrieving a Stored Secret
```bash
clan secrets get mysecret
```
### List all Secrets
```bash
clan secrets list
```
### NixOS integration
A NixOS machine will automatically import all secrets that are encrypted for the
current machine. At runtime it will use the host key to decrypt all secrets into
an in-memory, non-persistent filesystem using [sops-nix](https://github.com/Mic92/sops-nix).
In your nixos configuration you can get a path to secrets like this `config.sops.secrets.<name>.path`. For example:
```nix
{ config, ...}: {
sops.secrets.my-password.neededForUsers = true;
users.users.mic92 = {
isNormalUser = true;
passwordFile = config.sops.secrets.my-password.path;
};
}
```
### Assigning Access
When using `clan secrets set <secret>` without arguments, secrets are encrypted for the key of the user named like your current $USER.
To add machines/users to an existing secret use:
```bash
clan secrets machines add-secret <machine_name> <secret_name>
```
Alternatively specify users and machines while creating a secret:
```bash
clan secrets set --machine <machine1> --machine <machine2> --user <user1> --user <user2> <secret_name>
```
## Advanced
In this section we go into more advanced secret management topics.
### Groups
Clan CLI makes it easy to manage access by allowing you to create groups.
All users within a group inherit access to all secrets of the group.
This feature eases the process of handling permissions for multiple users.
Here's how to get started:
1. **Creating Groups**:
Assign users to a new group, e.g., `admins`:
```bash
clan secrets groups add admins <username>
```
2. **Listing Groups**:
```bash
clan secrets groups list
```
3. **Assigning Secrets to Groups**:
```bash
clan secrets groups add-secret <group_name> <secret_name>
```
### Adding Machine Keys
New machines in Clan come with age keys stored in `./sops/machines/<machine_name>`. To list these machines:
```bash
clan secrets machines list
```
For existing machines, add their keys:
```bash
clan secrets machines add <machine_name> <age_key>
```
To fetch an age key from an SSH host key:
```bash
ssh-keyscan <domain_name> | nix shell nixpkgs#ssh-to-age -c ssh-to-age
```
### Migration: Importing existing sops-based keys / sops-nix
`clan secrets` stores each secret in a single file, whereas [sops](https://github.com/Mic92/sops-nix) commonly allows to put all secrets in a yaml or json document.
If you already happened to use sops-nix, you can migrate by using the `clan secrets import-sops` command by importing these files:
```bash
% clan secrets import-sops --prefix matchbox- --group admins --machine matchbox nixos/matchbox/secrets/secrets.yaml
```
This will create secrets for each secret found in `nixos/matchbox/secrets/secrets.yaml` in a `./sops` folder of your repository.
Each member of the group `admins` in this case will be able to decrypt the secrets with their respective key.
Since our clan secret module will auto-import secrets that are encrypted for a particular nixos machine,
you can now remove `sops.secrets.<secrets> = { };` unless you need to specify more options for the secret like owner/group of the secret file.
## Indepth Explanation
The secrets system conceptually knows two different entities:
- **Machine**: consumes secrets
- **User**: manages access to secrets
**A Users** Can add or revoke machines' access to secrets.
**A machine** Can decrypt secrets that where encrypted specifically for that machine.
!!! Danger
**Always make sure at least one _User_ has access to a secret**. Otherwise you could lock yourself out from accessing the secret.
### Inherited implications
By default clan uses [sops](https://github.com/getsops/sops) through [sops-nix](https://github.com/Mic92/sops-nix) for managing its secrets which inherits some implications that are important to understand:
- **Public/Private keys**: Entities are identified via their public keys. Each Entity can use their respective private key to decrypt a secret.
- **Public keys are stored**: All Public keys are stored inside the repository
- **Secrets are stored Encrypted**: secrets are stored inside the repository encrypted with the respective public keys
- **Secrets are deployed encrypted**: Fully encrypted secrets are deployed to machines at deployment time.
- **Secrets are decrypted by sops on-demand**: Each machine decrypts its secrets at runtime and stores them at an ephemeral location.
- **Machine key-pairs are auto-generated**: When a machine is created **no user-interaction is required** to setup public/private key-pairs.
- **secrets are re-encrypted**: In case machines, users or groups are modified secrets get re-encrypted on demand.
!!! Important
After revoking access to a secret you should also change the underlying secret. i.e. change the API key, or the password.
---
### Machine and user keys
The following diagrams illustrates how a user can provide a secret (i.e. a Password).
- By using the **Clan CLI** a user encrypts the password with both the **User public-key** and the **machine's public-key**
- The *Machine* can decrypt the password with its private-key on demand.
- The *User* is able to decrypt the password to make changes to it.
```plantuml
@startuml
!include C4_Container.puml
Person(user, "User", "Someone who manages secrets")
ContainerDb(secret, "Secret")
Container(machine, "Machine", "A Machine. i.e. Needs the Secret for a given Service." )
Rel_R(user, secret, "Encrypt", "", "Pubkeys: User, Machine")
Rel_L(secret, user, "Decrypt", "", "user privkey")
Rel_R(secret, machine, "Decrypt", "", "machine privkey" )
@enduml
```
#### User groups
Here we illustrate how machine groups work.
Common use cases:
- **Shared Management**: Access among multiple users. I.e. a subset of secrets/machines that have two admins
```plantuml
@startuml
!include C4_Container.puml
System_Boundary(c1, "Group") {
Person(user1, "User A", "has access")
Person(user2, "User B", "has access")
}
ContainerDb(secret, "Secret")
Container(machine, "Machine", "A Machine. i.e. Needs the Secret for a given Service." )
Rel_R(c1, secret, "Encrypt", "", "Pubkeys: User A, User B, Machine")
Rel_R(secret, machine, "Decrypt", "", "machine privkey" )
@enduml
```
<!-- TODO: See also [Groups Reference](#groups-reference) -->
---
#### Machine groups
Here we illustrate how machine groups work.
Common use cases:
- **Shared secrets**: Among multiple machines such as Wifi passwords
```plantuml
@startuml
!include C4_Container.puml
!include C4_Deployment.puml
Person(user, "User", "Someone who manages secrets")
ContainerDb(secret, "Secret")
System_Boundary(c1, "Group") {
Container(machine1, "Machine A", "Both machines need the same secret" )
Container(machine2, "Machine B", "Both machines need the same secret" )
}
Rel_R(user, secret, "Encrypt", "", "Pubkeys: machine A, machine B, User")
Rel(secret, c1, "Decrypt", "", "Both machine A or B can decrypt using their private key" )
@enduml
```
<!-- TODO: See also [Groups Reference](#groups-reference) -->
See the [readme](https://github.com/Mic92/sops-nix) of sops-nix for more
examples.

49
AI_Data/Clan/manual/secure-boot.md Normal file
View File

@@ -0,0 +1,49 @@
At the moment, NixOS/Clan does not support [Secure Boot](https://wiki.gentoo.org/wiki/Secure_Boot). Therefore, you need to disable it in the BIOS. You can watch this [video guide](https://www.youtube.com/watch?v=BKVShiMUePc) or follow the instructions below:
### Step 1: Insert the USB Stick
- Begin by inserting the USB stick into a USB port on your computer.
### Step 2: Access the UEFI/BIOS Menu
- Restart your computer.
- As your computer restarts, press the appropriate key to enter the UEFI/BIOS settings.
??? tip "The key depends on your laptop or motherboard manufacturer. Click to see a reference list:"
| Manufacturer | UEFI/BIOS Key(s) |
|--------------------|---------------------------|
| ASUS | `Del`, `F2` |
| MSI | `Del`, `F2` |
| Gigabyte | `Del`, `F2` |
| ASRock | `Del`, `F2` |
| Lenovo | `F1`, `F2`, `Enter` (alternatively `Fn + F2`) |
| HP | `Esc`, `F10` |
| Dell | `F2`, `Fn + F2`, `Esc` |
| Acer | `F2`, `Del` |
| Samsung | `F2`, `F10` |
| Toshiba | `F2`, `Esc` |
| Sony | `F2`, `Assist` button |
| Fujitsu | `F2` |
| Microsoft Surface | `Volume Up` + `Power` |
| IBM/Lenovo ThinkPad| `Enter`, `F1`, `F12` |
| Biostar | `Del` |
| Zotac | `Del`, `F2` |
| EVGA | `Del` |
| Origin PC | `F2`, `Delete` |
!!! Note
Pressing the key quickly and repeatedly is sometimes necessary to access the UEFI/BIOS menu, as the window to enter this mode is brief.
### Step 3: Access Advanced Mode (Optional)
- If your UEFI/BIOS has a `Simple` or `Easy` mode interface, look for an option labeled `Advanced Mode` (often found in the lower right corner).
- Click on `Advanced Mode` to access more settings. This step is optional, as your boot settings might be available in the basic view.
### Step 4: Disable Secure Boot
- Locate the `Secure Boot` option in your UEFI/BIOS settings. This is typically found under a `Security` tab, `Boot` tab, or a similarly named section.
- Set the `Secure Boot` option to `Disabled`.
### Step 5: Change Boot Order
- Find the option to adjust the boot order—often labeled `Boot Order`, `Boot Sequence`, or `Boot Priority`.
- Ensure that your USB device is set as the first boot option. This allows your computer to boot from the USB stick.
### Step 6: Save and Exit
- Save your changes before exiting the UEFI/BIOS menu. Look for a `Save & Exit` option or press the corresponding function key (often `F10`).
- Your computer should now restart and boot from the USB stick.

12
AI_Data/Clan/new-docs/index.md Normal file
View File

@@ -0,0 +1,12 @@
+++
title= "New documentation site and weekly new meetup"
subline= "We are still working on improving the installation procedures, so stay tuned."
date= 2024-04-16T09:08:10+02:00
author= "Lassulus"
tags = ['Info']
+++
Last week, we added a new documentation hub for clan at [docs.clan.lol](https://docs.clan.lol).
We now have weekly office hours where people are invited to hangout and ask questions.
They are every Wednesday 15:30 UTC (17:30 CEST) in our [jitsi](https://jitsi.lassul.us/clan.lol).
Otherwise drop by in our [matrix channel](https://matrix.to/#/#clan:clan.lol).

99
AI_Data/Clan/nixos-facter/index.md Normal file
View File

@@ -0,0 +1,99 @@
+++
title= "Introducing NixOS Facter"
subline= "Declarative Hardware Configuration in NixOS"
date = 2024-07-19T09:08:10+02:00
draft = false
author = "Brian McGee"
tags = ['Dev Report']
+++
If you've ever installed [NixOS], you'll be familiar with a little Perl script called [nixos-generate-config]. Unsurprisingly, it generates a couple of NixOS modules based on available hardware, mounted filesystems, configured swap, etc.
It's a critical component of the install process, aiming to ensure you have a good starting point for your NixOS system, with necessary or recommended kernel modules, file system mounts, networking config and much more.
As solutions go, it's a solid one. It has helped many users take their first steps into this rabbit hole we call NixOS. However, it does suffer from one fundamental limitation.
## Static Generation
When a user generates a `hardware-configuration.nix` with `nixos-generate-config`, it makes choices based on the current state of the world as it sees it. By its very nature, then, it cannot account for changes in NixOS over time.
A recommended configuration option today might be different two NixOS releases from now.
To account for this, you could always run `nixos-generate-config` again. But that requires a working system, which may have broken due to the historical choices made last time, or worst-case, requiring you to fire up the installer again.
## A Layer of Indirection
What if, instead of generating some Nix code, we first describe the current hardware in an intermediate format? This hardware report would be _'pure'_, devoid of any reference to NixOS, and intended as a stable, longer-term representation of the system.
From here, we can create a series of NixOS modules designed to examine the report's contents and make the same kinds of decisions that `nixos-generate-config` does. The critical difference is that as NixOS evolves, so can these modules, and with a full hardware report available we can make more interesting config choices about things such as GPUs and other devices.
In a perfect world, we should not need to regenerate the underlying report as long as there are no hardware changes. We can take this one step further.
Provided that certain sensitive information, such as serial numbers and MAC addresses, is filtered out, there is no reason why these hardware reports could not be shared after they are generated for things like EC2 instance types, specific laptop models, and so on, much like [NixOS Hardware] currently shares Nix configs.
## Introducing NixOS Facter
Still in its early stages, [NixOS Facter] is intended to do what I've described above.
A user can generate a JSON-based hardware report using a (eventually static) Go program: `nixos-facter -o facter.json`. From there, they can include this report in their NixOS config and make use of our [NixOS modules](https://github.com/numtide/nixos-facter-modules) as follows:
=== "**flake.nix**"
```nix
{
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
nixos-facter-modules.url = "github:numtide/nixos-facter-modules";
};
outputs = inputs @ {
nixpkgs,
...
}: {
nixosConfigurations.basic = nixpkgs.lib.nixosSystem {
modules = [
inputs.nixos-facter-modules.nixosModules.facter
{ config.facter.reportPath = ./facter.json; }
# ...
];
};
};
}
```
=== "**without flakes**"
```nix
# configuration.nix
{
imports = [
"${(builtins.fetchTarball {
url = "https://github.com/numtide/nixos-facter-modules/";
})}/modules/nixos/facter.nix"
];
config.facter.reportPath = ./facter.json;
}
```
That's it.
> We assume that users will rely on [disko], so we have not implemented file system configuration yet (it's on the roadmap).
> In the meantime, if you don't use disko you have to specify that part of the configuration yourself or take it from `nixos-generate-config`.
## Early Days
Please be aware that [NixOS Facter] is still in early development and is still subject to significant changes especially the output json format as we flesh things out. Our initial goal is to reach feature parity with [nixos-generate-config].
From there, we want to continue building our NixOS modules, opening things up to the community, and beginning to capture shared hardware configurations for providers such as Hetzner, etc.
Over the coming weeks, we will also build up documentation and examples to make it easier to play with. For now, please be patient.
> Side note: if you are wondering why the repo is in the [Numtide] org, we started partnering with Clan! Both companies are looking to make self-hosting easier and we're excited to be working together on this. Expect more tools and features to come!
[NixOS Facter]: https://github.com/numtide/nixos-facter
[NixOS Hardware]: https://github.com/NixOS/nixos-hardware
[NixOS]: https://nixos.org "Declarative builds and deployments"
[Numtide]: https://numtide.com
[disko]: https://github.com/nix-community/disko
[nixos-generate-config]: https://github.com/NixOS/nixpkgs/blob/dac9cdf8c930c0af98a63cbfe8005546ba0125fb/nixos/modules/installer/tools/nixos-generate-config.pl

BIN
AI_Data/Clan/nixos-facter/post-hero-image.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.6 MiB

151
AI_Data/Zotero/files/20/sci-hub.usualwant.com.html Normal file
View File

File diff suppressed because one or more lines are too long

59
AI_Data/Zotero/files/43/S0167642312000639.html Normal file
View File

File diff suppressed because one or more lines are too long

419
AI_Data/Zotero/files/46/6607691.html Normal file
View File

File diff suppressed because one or more lines are too long

513
Chapters/Chapter1.tex
View File

@@ -2,11 +2,11 @@
\chapter{Chapter Title Here} % Main chapter title \chapter{Chapter Title Here} % Main chapter title
\label{Chapter1} % For referencing the chapter elsewhere, use \ref{Chapter1} \label{Chapter1} % For referencing the chapter elsewhere, use \ref{Chapter1}
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
% Define some commands to keep the formatting separated from the content % Define some commands to keep the formatting separated from the content
\newcommand{\keyword}[1]{\textbf{#1}} \newcommand{\keyword}[1]{\textbf{#1}}
\newcommand{\tabhead}[1]{\textbf{#1}} \newcommand{\tabhead}[1]{\textbf{#1}}
\newcommand{\code}[1]{\texttt{#1}} \newcommand{\code}[1]{\texttt{#1}}
@@ -16,66 +16,134 @@
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
\section{Welcome and Thank You} \section{Welcome and Thank You}
Welcome to this \LaTeX{} Thesis Template, a beautiful and easy to use template for writing a thesis using the \LaTeX{} typesetting system. Welcome to this \LaTeX{} Thesis Template, a beautiful and easy to use
template for writing a thesis using the \LaTeX{} typesetting system.
If you are writing a thesis (or will be in the future) and its subject is technical or mathematical (though it doesn't have to be), then creating it in \LaTeX{} is highly recommended as a way to make sure you can just get down to the essential writing without having to worry over formatting or wasting time arguing with your word processor. If you are writing a thesis (or will be in the future) and its
subject is technical or mathematical (though it doesn't have to be),
then creating it in \LaTeX{} is highly recommended as a way to make
sure you can just get down to the essential writing without having to
worry over formatting or wasting time arguing with your word processor.
\LaTeX{} is easily able to professionally typeset documents that run to hundreds or thousands of pages long. With simple mark-up commands, it automatically sets out the table of contents, margins, page headers and footers and keeps the formatting consistent and beautiful. One of its main strengths is the way it can easily typeset mathematics, even \emph{heavy} mathematics. Even if those equations are the most horribly twisted and most difficult mathematical problems that can only be solved on a super-computer, you can at least count on \LaTeX{} to make them look stunning. \LaTeX{} is easily able to professionally typeset documents that run
to hundreds or thousands of pages long. With simple mark-up commands,
it automatically sets out the table of contents, margins, page
headers and footers and keeps the formatting consistent and
beautiful. One of its main strengths is the way it can easily typeset
mathematics, even \emph{heavy} mathematics. Even if those equations
are the most horribly twisted and most difficult mathematical
problems that can only be solved on a super-computer, you can at
least count on \LaTeX{} to make them look stunning.
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
\section{Learning \LaTeX{}} \section{Learning \LaTeX{}}
\LaTeX{} is not a \textsc{wysiwyg} (What You See is What You Get) program, unlike word processors such as Microsoft Word or Apple's Pages. Instead, a document written for \LaTeX{} is actually a simple, plain text file that contains \emph{no formatting}. You tell \LaTeX{} how you want the formatting in the finished document by writing in simple commands amongst the text, for example, if I want to use \emph{italic text for emphasis}, I write the \verb|\emph{text}| command and put the text I want in italics in between the curly braces. This means that \LaTeX{} is a \enquote{mark-up} language, very much like HTML. \LaTeX{} is not a \textsc{wysiwyg} (What You See is What You Get)
program, unlike word processors such as Microsoft Word or Apple's
Pages. Instead, a document written for \LaTeX{} is actually a simple,
plain text file that contains \emph{no formatting}. You tell \LaTeX{}
how you want the formatting in the finished document by writing in
simple commands amongst the text, for example, if I want to use
\emph{italic text for emphasis}, I write the \verb|\emph{text}|
command and put the text I want in italics in between the curly
braces. This means that \LaTeX{} is a \enquote{mark-up} language,
very much like HTML.
\subsection{A (not so short) Introduction to \LaTeX{}} \subsection{A (not so short) Introduction to \LaTeX{}}
If you are new to \LaTeX{}, there is a very good eBook -- freely available online as a PDF file -- called, \enquote{The Not So Short Introduction to \LaTeX{}}. The book's title is typically shortened to just \emph{lshort}. You can download the latest version (as it is occasionally updated) from here: If you are new to \LaTeX{}, there is a very good eBook -- freely
available online as a PDF file -- called, \enquote{The Not So Short
Introduction to \LaTeX{}}. The book's title is typically shortened to
just \emph{lshort}. You can download the latest version (as it is
occasionally updated) from here:
\url{http://www.ctan.org/tex-archive/info/lshort/english/lshort.pdf} \url{http://www.ctan.org/tex-archive/info/lshort/english/lshort.pdf}
It is also available in several other languages. Find yours from the list on this page: \url{http://www.ctan.org/tex-archive/info/lshort/} It is also available in several other languages. Find yours from the
list on this page: \url{http://www.ctan.org/tex-archive/info/lshort/}
It is recommended to take a little time out to learn how to use \LaTeX{} by creating several, small `test' documents, or having a close look at several templates on:\\ It is recommended to take a little time out to learn how to use
\url{http://www.LaTeXTemplates.com}\\ \LaTeX{} by creating several, small `test' documents, or having a
Making the effort now means you're not stuck learning the system when what you \emph{really} need to be doing is writing your thesis. close look at several templates on:\\
\url{http://www.LaTeXTemplates.com}\\
Making the effort now means you're not stuck learning the system when
what you \emph{really} need to be doing is writing your thesis.
\subsection{A Short Math Guide for \LaTeX{}} \subsection{A Short Math Guide for \LaTeX{}}
If you are writing a technical or mathematical thesis, then you may want to read the document by the AMS (American Mathematical Society) called, \enquote{A Short Math Guide for \LaTeX{}}. It can be found online here: If you are writing a technical or mathematical thesis, then you may
want to read the document by the AMS (American Mathematical Society)
called, \enquote{A Short Math Guide for \LaTeX{}}. It can be found online here:
\url{http://www.ams.org/tex/amslatex.html} \url{http://www.ams.org/tex/amslatex.html}
under the \enquote{Additional Documentation} section towards the bottom of the page. under the \enquote{Additional Documentation} section towards the
bottom of the page.
\subsection{Common \LaTeX{} Math Symbols} \subsection{Common \LaTeX{} Math Symbols}
There are a multitude of mathematical symbols available for \LaTeX{} and it would take a great effort to learn the commands for them all. The most common ones you are likely to use are shown on this page: There are a multitude of mathematical symbols available for \LaTeX{}
and it would take a great effort to learn the commands for them all.
The most common ones you are likely to use are shown on this page:
\url{http://www.sunilpatel.co.uk/latex-type/latex-math-symbols/} \url{http://www.sunilpatel.co.uk/latex-type/latex-math-symbols/}
You can use this page as a reference or crib sheet, the symbols are rendered as large, high quality images so you can quickly find the \LaTeX{} command for the symbol you need. You can use this page as a reference or crib sheet, the symbols are
rendered as large, high quality images so you can quickly find the
\LaTeX{} command for the symbol you need.
\subsection{\LaTeX{} on a Mac} \subsection{\LaTeX{} on a Mac}
The \LaTeX{} distribution is available for many systems including Windows, Linux and Mac OS X. The package for OS X is called MacTeX and it contains all the applications you need -- bundled together and pre-customized -- for a fully working \LaTeX{} environment and work flow. The \LaTeX{} distribution is available for many systems including
Windows, Linux and Mac OS X. The package for OS X is called MacTeX
MacTeX includes a custom dedicated \LaTeX{} editor called TeXShop for writing your `\file{.tex}' files and BibDesk: a program to manage your references and create your bibliography section just as easily as managing songs and creating playlists in iTunes. and it contains all the applications you need -- bundled together and
pre-customized -- for a fully working \LaTeX{} environment and work flow.
MacTeX includes a custom dedicated \LaTeX{} editor called TeXShop for
writing your `\file{.tex}' files and BibDesk: a program to manage
your references and create your bibliography section just as easily
as managing songs and creating playlists in iTunes.
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
\section{Getting Started with this Template} \section{Getting Started with this Template}
If you are familiar with \LaTeX{}, then you should explore the directory structure of the template and then proceed to place your own information into the \emph{THESIS INFORMATION} block of the \file{main.tex} file. You can then modify the rest of this file to your unique specifications based on your degree/university. Section \ref{FillingFile} on page \pageref{FillingFile} will help you do this. Make sure you also read section \ref{ThesisConventions} about thesis conventions to get the most out of this template. If you are familiar with \LaTeX{}, then you should explore the
directory structure of the template and then proceed to place your
own information into the \emph{THESIS INFORMATION} block of the
\file{main.tex} file. You can then modify the rest of this file to
your unique specifications based on your degree/university. Section
\ref{FillingFile} on page \pageref{FillingFile} will help you do
this. Make sure you also read section \ref{ThesisConventions} about
thesis conventions to get the most out of this template.
If you are new to \LaTeX{} it is recommended that you carry on reading through the rest of the information in this document. If you are new to \LaTeX{} it is recommended that you carry on
reading through the rest of the information in this document.
Before you begin using this template you should ensure that its style complies with the thesis style guidelines imposed by your institution. In most cases this template style and layout will be suitable. If it is not, it may only require a small change to bring the template in line with your institution's recommendations. These modifications will need to be done on the \file{MastersDoctoralThesis.cls} file. Before you begin using this template you should ensure that its style
complies with the thesis style guidelines imposed by your
institution. In most cases this template style and layout will be
suitable. If it is not, it may only require a small change to bring
the template in line with your institution's recommendations. These
modifications will need to be done on the \file{MastersDoctoralThesis.cls} file.
\subsection{About this Template} \subsection{About this Template}
This \LaTeX{} Thesis Template is originally based and created around a \LaTeX{} style file created by Steve R.\ Gunn from the University of Southampton (UK), department of Electronics and Computer Science. You can find his original thesis style file at his site, here: This \LaTeX{} Thesis Template is originally based and created around
a \LaTeX{} style file created by Steve R.\ Gunn from the University
of Southampton (UK), department of Electronics and Computer Science.
You can find his original thesis style file at his site, here:
\url{http://www.ecs.soton.ac.uk/~srg/softwaretools/document/templates/} \url{http://www.ecs.soton.ac.uk/~srg/softwaretools/document/templates/}
Steve's \file{ecsthesis.cls} was then taken by Sunil Patel who modified it by creating a skeleton framework and folder structure to place the thesis files in. The resulting template can be found on Sunil's site here: Steve's \file{ecsthesis.cls} was then taken by Sunil Patel who
modified it by creating a skeleton framework and folder structure to
place the thesis files in. The resulting template can be found on
Sunil's site here:
\url{http://www.sunilpatel.co.uk/thesis-template} \url{http://www.sunilpatel.co.uk/thesis-template}
Sunil's template was made available through \url{http://www.LaTeXTemplates.com} where it was modified many times based on user requests and questions. Version 2.0 and onwards of this template represents a major modification to Sunil's template and is, in fact, hardly recognisable. The work to make version 2.0 possible was carried out by \href{mailto:vel@latextemplates.com}{Vel} and Johannes Böttcher. Sunil's template was made available through
\url{http://www.LaTeXTemplates.com} where it was modified many times
based on user requests and questions. Version 2.0 and onwards of this
template represents a major modification to Sunil's template and is,
in fact, hardly recognisable. The work to make version 2.0 possible
was carried out by \href{mailto:vel@latextemplates.com}{Vel} and
Johannes Böttcher.
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
@@ -83,130 +151,299 @@ Sunil's template was made available through \url{http://www.LaTeXTemplates.com}
\subsection{Folders} \subsection{Folders}
This template comes as a single zip file that expands out to several files and folders. The folder names are mostly self-explanatory: This template comes as a single zip file that expands out to several
files and folders. The folder names are mostly self-explanatory:
\keyword{Appendices} -- this is the folder where you put the appendices. Each appendix should go into its own separate \file{.tex} file. An example and template are included in the directory. \keyword{Appendices} -- this is the folder where you put the
appendices. Each appendix should go into its own separate \file{.tex}
file. An example and template are included in the directory.
\keyword{Chapters} -- this is the folder where you put the thesis chapters. A thesis usually has about six chapters, though there is no hard rule on this. Each chapter should go in its own separate \file{.tex} file and they can be split as: \keyword{Chapters} -- this is the folder where you put the thesis
chapters. A thesis usually has about six chapters, though there is no
hard rule on this. Each chapter should go in its own separate
\file{.tex} file and they can be split as:
\begin{itemize} \begin{itemize}
\item Chapter 1: Introduction to the thesis topic \item Chapter 1: Introduction to the thesis topic
\item Chapter 2: Background information and theory \item Chapter 2: Background information and theory
\item Chapter 3: (Laboratory) experimental setup \item Chapter 3: (Laboratory) experimental setup
\item Chapter 4: Details of experiment 1 \item Chapter 4: Details of experiment 1
\item Chapter 5: Details of experiment 2 \item Chapter 5: Details of experiment 2
\item Chapter 6: Discussion of the experimental results \item Chapter 6: Discussion of the experimental results
\item Chapter 7: Conclusion and future directions \item Chapter 7: Conclusion and future directions
\end{itemize} \end{itemize}
This chapter layout is specialised for the experimental sciences, your discipline may be different. This chapter layout is specialised for the experimental sciences,
your discipline may be different.
\keyword{Figures} -- this folder contains all figures for the thesis. These are the final images that will go into the thesis document. \keyword{Figures} -- this folder contains all figures for the thesis.
These are the final images that will go into the thesis document.
\subsection{Files} \subsection{Files}
Included are also several files, most of them are plain text and you can see their contents in a text editor. After initial compilation, you will see that more auxiliary files are created by \LaTeX{} or BibTeX and which you don't need to delete or worry about: Included are also several files, most of them are plain text and you
can see their contents in a text editor. After initial compilation,
you will see that more auxiliary files are created by \LaTeX{} or
BibTeX and which you don't need to delete or worry about:
\keyword{example.bib} -- this is an important file that contains all the bibliographic information and references that you will be citing in the thesis for use with BibTeX. You can write it manually, but there are reference manager programs available that will create and manage it for you. Bibliographies in \LaTeX{} are a large subject and you may need to read about BibTeX before starting with this. Many modern reference managers will allow you to export your references in BibTeX format which greatly eases the amount of work you have to do. \keyword{example.bib} -- this is an important file that contains all
the bibliographic information and references that you will be citing
in the thesis for use with BibTeX. You can write it manually, but
there are reference manager programs available that will create and
manage it for you. Bibliographies in \LaTeX{} are a large subject and
you may need to read about BibTeX before starting with this. Many
modern reference managers will allow you to export your references in
BibTeX format which greatly eases the amount of work you have to do.
\keyword{MastersDoctoralThesis.cls} -- this is an important file. It is the class file that tells \LaTeX{} how to format the thesis. \keyword{MastersDoctoralThesis.cls} -- this is an important file. It
is the class file that tells \LaTeX{} how to format the thesis.
\keyword{main.pdf} -- this is your beautifully typeset thesis (in the PDF file format) created by \LaTeX{}. It is supplied in the PDF with the template and after you compile the template you should get an identical version. \keyword{main.pdf} -- this is your beautifully typeset thesis (in the
PDF file format) created by \LaTeX{}. It is supplied in the PDF with
the template and after you compile the template you should get an
identical version.
\keyword{main.tex} -- this is an important file. This is the file that you tell \LaTeX{} to compile to produce your thesis as a PDF file. It contains the framework and constructs that tell \LaTeX{} how to layout the thesis. It is heavily commented so you can read exactly what each line of code does and why it is there. After you put your own information into the \emph{THESIS INFORMATION} block -- you have now started your thesis! \keyword{main.tex} -- this is an important file. This is the file
that you tell \LaTeX{} to compile to produce your thesis as a PDF
file. It contains the framework and constructs that tell \LaTeX{} how
to layout the thesis. It is heavily commented so you can read exactly
what each line of code does and why it is there. After you put your
own information into the \emph{THESIS INFORMATION} block -- you have
now started your thesis!
Files that are \emph{not} included, but are created by \LaTeX{} as auxiliary files include: Files that are \emph{not} included, but are created by \LaTeX{} as
auxiliary files include:
\keyword{main.aux} -- this is an auxiliary file generated by \LaTeX{}, if it is deleted \LaTeX{} simply regenerates it when you run the main \file{.tex} file. \keyword{main.aux} -- this is an auxiliary file generated by
\LaTeX{}, if it is deleted \LaTeX{} simply regenerates it when you
run the main \file{.tex} file.
\keyword{main.bbl} -- this is an auxiliary file generated by BibTeX, if it is deleted, BibTeX simply regenerates it when you run the \file{main.aux} file. Whereas the \file{.bib} file contains all the references you have, this \file{.bbl} file contains the references you have actually cited in the thesis and is used to build the bibliography section of the thesis. \keyword{main.bbl} -- this is an auxiliary file generated by BibTeX,
if it is deleted, BibTeX simply regenerates it when you run the
\file{main.aux} file. Whereas the \file{.bib} file contains all the
references you have, this \file{.bbl} file contains the references
you have actually cited in the thesis and is used to build the
bibliography section of the thesis.
\keyword{main.blg} -- this is an auxiliary file generated by BibTeX, if it is deleted BibTeX simply regenerates it when you run the main \file{.aux} file. \keyword{main.blg} -- this is an auxiliary file generated by BibTeX,
if it is deleted BibTeX simply regenerates it when you run the main
\file{.aux} file.
\keyword{main.lof} -- this is an auxiliary file generated by \LaTeX{}, if it is deleted \LaTeX{} simply regenerates it when you run the main \file{.tex} file. It tells \LaTeX{} how to build the \emph{List of Figures} section. \keyword{main.lof} -- this is an auxiliary file generated by
\LaTeX{}, if it is deleted \LaTeX{} simply regenerates it when you
run the main \file{.tex} file. It tells \LaTeX{} how to build the
\emph{List of Figures} section.
\keyword{main.log} -- this is an auxiliary file generated by \LaTeX{}, if it is deleted \LaTeX{} simply regenerates it when you run the main \file{.tex} file. It contains messages from \LaTeX{}, if you receive errors and warnings from \LaTeX{}, they will be in this \file{.log} file. \keyword{main.log} -- this is an auxiliary file generated by
\LaTeX{}, if it is deleted \LaTeX{} simply regenerates it when you
run the main \file{.tex} file. It contains messages from \LaTeX{}, if
you receive errors and warnings from \LaTeX{}, they will be in this
\file{.log} file.
\keyword{main.lot} -- this is an auxiliary file generated by \LaTeX{}, if it is deleted \LaTeX{} simply regenerates it when you run the main \file{.tex} file. It tells \LaTeX{} how to build the \emph{List of Tables} section. \keyword{main.lot} -- this is an auxiliary file generated by
\LaTeX{}, if it is deleted \LaTeX{} simply regenerates it when you
run the main \file{.tex} file. It tells \LaTeX{} how to build the
\emph{List of Tables} section.
\keyword{main.out} -- this is an auxiliary file generated by \LaTeX{}, if it is deleted \LaTeX{} simply regenerates it when you run the main \file{.tex} file. \keyword{main.out} -- this is an auxiliary file generated by
\LaTeX{}, if it is deleted \LaTeX{} simply regenerates it when you
run the main \file{.tex} file.
So from this long list, only the files with the \file{.bib}, \file{.cls} and \file{.tex} extensions are the most important ones. The other auxiliary files can be ignored or deleted as \LaTeX{} and BibTeX will regenerate them. So from this long list, only the files with the \file{.bib},
\file{.cls} and \file{.tex} extensions are the most important ones.
The other auxiliary files can be ignored or deleted as \LaTeX{} and
BibTeX will regenerate them.
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
\section{Filling in Your Information in the \file{main.tex} File}\label{FillingFile} \section{Filling in Your Information in the \file{main.tex}
File}\label{FillingFile}
You will need to personalise the thesis template and make it your own by filling in your own information. This is done by editing the \file{main.tex} file in a text editor or your favourite LaTeX environment. You will need to personalise the thesis template and make it your own
by filling in your own information. This is done by editing the
\file{main.tex} file in a text editor or your favourite LaTeX environment.
Open the file and scroll down to the third large block titled \emph{THESIS INFORMATION} where you can see the entries for \emph{University Name}, \emph{Department Name}, etc \ldots Open the file and scroll down to the third large block titled
\emph{THESIS INFORMATION} where you can see the entries for
\emph{University Name}, \emph{Department Name}, etc \ldots
Fill out the information about yourself, your group and institution. You can also insert web links, if you do, make sure you use the full URL, including the \code{http://} for this. If you don't want these to be linked, simply remove the \verb|\href{url}{name}| and only leave the name. Fill out the information about yourself, your group and institution.
You can also insert web links, if you do, make sure you use the full
URL, including the \code{http://} for this. If you don't want these
to be linked, simply remove the \verb|\href{url}{name}| and only leave the name.
When you have done this, save the file and recompile \code{main.tex}. All the information you filled in should now be in the PDF, complete with web links. You can now begin your thesis proper! When you have done this, save the file and recompile \code{main.tex}.
All the information you filled in should now be in the PDF, complete
with web links. You can now begin your thesis proper!
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
\section{The \code{main.tex} File Explained} \section{The \code{main.tex} File Explained}
The \file{main.tex} file contains the structure of the thesis. There are plenty of written comments that explain what pages, sections and formatting the \LaTeX{} code is creating. Each major document element is divided into commented blocks with titles in all capitals to make it obvious what the following bit of code is doing. Initially there seems to be a lot of \LaTeX{} code, but this is all formatting, and it has all been taken care of so you don't have to do it. The \file{main.tex} file contains the structure of the thesis. There
are plenty of written comments that explain what pages, sections and
formatting the \LaTeX{} code is creating. Each major document element
is divided into commented blocks with titles in all capitals to make
it obvious what the following bit of code is doing. Initially there
seems to be a lot of \LaTeX{} code, but this is all formatting, and
it has all been taken care of so you don't have to do it.
Begin by checking that your information on the title page is correct. For the thesis declaration, your institution may insist on something different than the text given. If this is the case, just replace what you see with what is required in the \emph{DECLARATION PAGE} block. Begin by checking that your information on the title page is correct.
For the thesis declaration, your institution may insist on something
different than the text given. If this is the case, just replace what
you see with what is required in the \emph{DECLARATION PAGE} block.
Then comes a page which contains a funny quote. You can put your own, or quote your favourite scientist, author, person, and so on. Make sure to put the name of the person who you took the quote from. Then comes a page which contains a funny quote. You can put your own,
or quote your favourite scientist, author, person, and so on. Make
sure to put the name of the person who you took the quote from.
Following this is the abstract page which summarises your work in a condensed way and can almost be used as a standalone document to describe what you have done. The text you write will cause the heading to move up so don't worry about running out of space. Following this is the abstract page which summarises your work in a
condensed way and can almost be used as a standalone document to
describe what you have done. The text you write will cause the
heading to move up so don't worry about running out of space.
Next come the acknowledgements. On this page, write about all the people who you wish to thank (not forgetting parents, partners and your advisor/supervisor). Next come the acknowledgements. On this page, write about all the
people who you wish to thank (not forgetting parents, partners and
your advisor/supervisor).
The contents pages, list of figures and tables are all taken care of for you and do not need to be manually created or edited. The next set of pages are more likely to be optional and can be deleted since they are for a more technical thesis: insert a list of abbreviations you have used in the thesis, then a list of the physical constants and numbers you refer to and finally, a list of mathematical symbols used in any formulae. Making the effort to fill these tables means the reader has a one-stop place to refer to instead of searching the internet and references to try and find out what you meant by certain abbreviations or symbols. The contents pages, list of figures and tables are all taken care of
for you and do not need to be manually created or edited. The next
set of pages are more likely to be optional and can be deleted since
they are for a more technical thesis: insert a list of abbreviations
you have used in the thesis, then a list of the physical constants
and numbers you refer to and finally, a list of mathematical symbols
used in any formulae. Making the effort to fill these tables means
the reader has a one-stop place to refer to instead of searching the
internet and references to try and find out what you meant by certain
abbreviations or symbols.
The list of symbols is split into the Roman and Greek alphabets. Whereas the abbreviations and symbols ought to be listed in alphabetical order (and this is \emph{not} done automatically for you) the list of physical constants should be grouped into similar themes. The list of symbols is split into the Roman and Greek alphabets.
Whereas the abbreviations and symbols ought to be listed in
alphabetical order (and this is \emph{not} done automatically for
you) the list of physical constants should be grouped into similar themes.
The next page contains a one line dedication. Who will you dedicate your thesis to? The next page contains a one line dedication. Who will you dedicate
your thesis to?
Finally, there is the block where the chapters are included. Uncomment the lines (delete the \code{\%} character) as you write the chapters. Each chapter should be written in its own file and put into the \emph{Chapters} folder and named \file{Chapter1}, \file{Chapter2}, etc\ldots Similarly for the appendices, uncomment the lines as you need them. Each appendix should go into its own file and placed in the \emph{Appendices} folder. Finally, there is the block where the chapters are included.
Uncomment the lines (delete the \code{\%} character) as you write the
chapters. Each chapter should be written in its own file and put into
the \emph{Chapters} folder and named \file{Chapter1},
\file{Chapter2}, etc\ldots Similarly for the appendices, uncomment
the lines as you need them. Each appendix should go into its own file
and placed in the \emph{Appendices} folder.
After the preamble, chapters and appendices finally comes the bibliography. The bibliography style (called \option{authoryear}) is used for the bibliography and is a fully featured style that will even include links to where the referenced paper can be found online. Do not underestimate how grateful your reader will be to find that a reference to a paper is just a click away. Of course, this relies on you putting the URL information into the BibTeX file in the first place. After the preamble, chapters and appendices finally comes the
bibliography. The bibliography style (called \option{authoryear}) is
used for the bibliography and is a fully featured style that will
even include links to where the referenced paper can be found online.
Do not underestimate how grateful your reader will be to find that a
reference to a paper is just a click away. Of course, this relies on
you putting the URL information into the BibTeX file in the first place.
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
\section{Thesis Features and Conventions}\label{ThesisConventions} \section{Thesis Features and Conventions}\label{ThesisConventions}
To get the best out of this template, there are a few conventions that you may want to follow. To get the best out of this template, there are a few conventions
that you may want to follow.
One of the most important (and most difficult) things to keep track of in such a long document as a thesis is consistency. Using certain conventions and ways of doing things (such as using a Todo list) makes the job easier. Of course, all of these are optional and you can adopt your own method. One of the most important (and most difficult) things to keep track
of in such a long document as a thesis is consistency. Using certain
conventions and ways of doing things (such as using a Todo list)
makes the job easier. Of course, all of these are optional and you
can adopt your own method.
\subsection{Printing Format} \subsection{Printing Format}
This thesis template is designed for double sided printing (i.e. content on the front and back of pages) as most theses are printed and bound this way. Switching to one sided printing is as simple as uncommenting the \option{oneside} option of the \code{documentclass} command at the top of the \file{main.tex} file. You may then wish to adjust the margins to suit specifications from your institution. This thesis template is designed for double sided printing (i.e.
content on the front and back of pages) as most theses are printed
and bound this way. Switching to one sided printing is as simple as
uncommenting the \option{oneside} option of the \code{documentclass}
command at the top of the \file{main.tex} file. You may then wish to
adjust the margins to suit specifications from your institution.
The headers for the pages contain the page number on the outer side (so it is easy to flick through to the page you want) and the chapter name on the inner side. The headers for the pages contain the page number on the outer side
(so it is easy to flick through to the page you want) and the chapter
name on the inner side.
The text is set to 11 point by default with single line spacing, again, you can tune the text size and spacing should you want or need to using the options at the very start of \file{main.tex}. The spacing can be changed similarly by replacing the \option{singlespacing} with \option{onehalfspacing} or \option{doublespacing}. The text is set to 11 point by default with single line spacing,
again, you can tune the text size and spacing should you want or need
to using the options at the very start of \file{main.tex}. The
spacing can be changed similarly by replacing the
\option{singlespacing} with \option{onehalfspacing} or \option{doublespacing}.
\subsection{Using US Letter Paper} \subsection{Using US Letter Paper}
The paper size used in the template is A4, which is the standard size in Europe. If you are using this thesis template elsewhere and particularly in the United States, then you may have to change the A4 paper size to the US Letter size. This can be done in the margins settings section in \file{main.tex}. The paper size used in the template is A4, which is the standard size
in Europe. If you are using this thesis template elsewhere and
particularly in the United States, then you may have to change the A4
paper size to the US Letter size. This can be done in the margins
settings section in \file{main.tex}.
Due to the differences in the paper size, the resulting margins may be different to what you like or require (as it is common for institutions to dictate certain margin sizes). If this is the case, then the margin sizes can be tweaked by modifying the values in the same block as where you set the paper size. Now your document should be set up for US Letter paper size with suitable margins. Due to the differences in the paper size, the resulting margins may
be different to what you like or require (as it is common for
institutions to dictate certain margin sizes). If this is the case,
then the margin sizes can be tweaked by modifying the values in the
same block as where you set the paper size. Now your document should
be set up for US Letter paper size with suitable margins.
\subsection{References} \subsection{References}
The \code{biblatex} package is used to format the bibliography and inserts references such as this one \parencite{Reference1}. The options used in the \file{main.tex} file mean that the in-text citations of references are formatted with the author(s) listed with the date of the publication. Multiple references are separated by semicolons (e.g. \parencite{Reference2, Reference1}) and references with more than three authors only show the first author with \emph{et al.} indicating there are more authors (e.g. \parencite{Reference3}). This is done automatically for you. To see how you use references, have a look at the \file{Chapter1.tex} source file. Many reference managers allow you to simply drag the reference into the document as you type. The \code{biblatex} package is used to format the bibliography and
inserts references such as this one \parencite{Reference1}. The
options used in the \file{main.tex} file mean that the in-text
citations of references are formatted with the author(s) listed with
the date of the publication. Multiple references are separated by
semicolons (e.g. \parencite{Reference2, Reference1}) and references
with more than three authors only show the first author with \emph{et
al.} indicating there are more authors (e.g. \parencite{Reference3}).
This is done automatically for you. To see how you use references,
have a look at the \file{Chapter1.tex} source file. Many reference
managers allow you to simply drag the reference into the document as you type.
Scientific references should come \emph{before} the punctuation mark if there is one (such as a comma or period). The same goes for footnotes\footnote{Such as this footnote, here down at the bottom of the page.}. You can change this but the most important thing is to keep the convention consistent throughout the thesis. Footnotes themselves should be full, descriptive sentences (beginning with a capital letter and ending with a full stop). The APA6 states: \enquote{Footnote numbers should be superscripted, [...], following any punctuation mark except a dash.} The Chicago manual of style states: \enquote{A note number should be placed at the end of a sentence or clause. The number follows any punctuation mark except the dash, which it precedes. It follows a closing parenthesis.} Scientific references should come \emph{before} the punctuation mark
if there is one (such as a comma or period). The same goes for
footnotes\footnote{Such as this footnote, here down at the bottom of
the page.}. You can change this but the most important thing is to
keep the convention consistent throughout the thesis. Footnotes
themselves should be full, descriptive sentences (beginning with a
capital letter and ending with a full stop). The APA6 states:
\enquote{Footnote numbers should be superscripted, [...], following
any punctuation mark except a dash.} The Chicago manual of style
states: \enquote{A note number should be placed at the end of a
sentence or clause. The number follows any punctuation mark except
the dash, which it precedes. It follows a closing parenthesis.}
The bibliography is typeset with references listed in alphabetical order by the first author's last name. This is similar to the APA referencing style. To see how \LaTeX{} typesets the bibliography, have a look at the very end of this document (or just click on the reference number links in in-text citations). The bibliography is typeset with references listed in alphabetical
order by the first author's last name. This is similar to the APA
referencing style. To see how \LaTeX{} typesets the bibliography,
have a look at the very end of this document (or just click on the
reference number links in in-text citations).
\subsubsection{A Note on bibtex} \subsubsection{A Note on bibtex}
The bibtex backend used in the template by default does not correctly handle unicode character encoding (i.e. "international" characters). You may see a warning about this in the compilation log and, if your references contain unicode characters, they may not show up correctly or at all. The solution to this is to use the biber backend instead of the outdated bibtex backend. This is done by finding this in \file{main.tex}: \option{backend=bibtex} and changing it to \option{backend=biber}. You will then need to delete all auxiliary BibTeX files and navigate to the template directory in your terminal (command prompt). Once there, simply type \code{biber main} and biber will compile your bibliography. You can then compile \file{main.tex} as normal and your bibliography will be updated. An alternative is to set up your LaTeX editor to compile with biber instead of bibtex, see \href{http://tex.stackexchange.com/questions/154751/biblatex-with-biber-configuring-my-editor-to-avoid-undefined-citations/}{here} for how to do this for various editors. The bibtex backend used in the template by default does not correctly
handle unicode character encoding (i.e. "international" characters).
You may see a warning about this in the compilation log and, if your
references contain unicode characters, they may not show up correctly
or at all. The solution to this is to use the biber backend instead
of the outdated bibtex backend. This is done by finding this in
\file{main.tex}: \option{backend=bibtex} and changing it to
\option{backend=biber}. You will then need to delete all auxiliary
BibTeX files and navigate to the template directory in your terminal
(command prompt). Once there, simply type \code{biber main} and biber
will compile your bibliography. You can then compile \file{main.tex}
as normal and your bibliography will be updated. An alternative is to
set up your LaTeX editor to compile with biber instead of bibtex, see
\href{http://tex.stackexchange.com/questions/154751/biblatex-with-biber-configuring-my-editor-to-avoid-undefined-citations/}{here}
for how to do this for various editors.
\subsection{Tables} \subsection{Tables}
Tables are an important way of displaying your results, below is an example table which was generated with this code: Tables are an important way of displaying your results, below is an
example table which was generated with this code:
{\small {\small
\begin{verbatim} \begin{verbatim}
\begin{table} \begin{table}
\caption{The effects of treatments X and Y on the four groups studied.} \caption{The effects of treatments X and Y on the four groups studied.}
\label{tab:treatments} \label{tab:treatments}
@@ -222,30 +459,34 @@ Tables are an important way of displaying your results, below is an example tabl
\bottomrule\\ \bottomrule\\
\end{tabular} \end{tabular}
\end{table} \end{table}
\end{verbatim} \end{verbatim}
} }
\begin{table} \begin{table}
\caption{The effects of treatments X and Y on the four groups studied.} \caption{The effects of treatments X and Y on the four groups studied.}
\label{tab:treatments} \label{tab:treatments}
\centering \centering
\begin{tabular}{l l l} \begin{tabular}{l l l}
\toprule \toprule
\tabhead{Groups} & \tabhead{Treatment X} & \tabhead{Treatment Y} \\ \tabhead{Groups} & \tabhead{Treatment X} & \tabhead{Treatment Y} \\
\midrule \midrule
1 & 0.2 & 0.8\\ 1 & 0.2 & 0.8\\
2 & 0.17 & 0.7\\ 2 & 0.17 & 0.7\\
3 & 0.24 & 0.75\\ 3 & 0.24 & 0.75\\
4 & 0.68 & 0.3\\ 4 & 0.68 & 0.3\\
\bottomrule\\ \bottomrule\\
\end{tabular} \end{tabular}
\end{table} \end{table}
You can reference tables with \verb|\ref{<label>}| where the label is defined within the table environment. See \file{Chapter1.tex} for an example of the label and citation (e.g. Table~\ref{tab:treatments}). You can reference tables with \verb|\ref{<label>}| where the label is
defined within the table environment. See \file{Chapter1.tex} for an
example of the label and citation (e.g. Table~\ref{tab:treatments}).
\subsection{Figures} \subsection{Figures}
There will hopefully be many figures in your thesis (that should be placed in the \emph{Figures} folder). The way to insert figures into your thesis is to use a code template like this: There will hopefully be many figures in your thesis (that should be
placed in the \emph{Figures} folder). The way to insert figures into
your thesis is to use a code template like this:
\begin{verbatim} \begin{verbatim}
\begin{figure} \begin{figure}
\centering \centering
@@ -255,34 +496,57 @@ There will hopefully be many figures in your thesis (that should be placed in th
\label{fig:Electron} \label{fig:Electron}
\end{figure} \end{figure}
\end{verbatim} \end{verbatim}
Also look in the source file. Putting this code into the source file produces the picture of the electron that you can see in the figure below. Also look in the source file. Putting this code into the source file
produces the picture of the electron that you can see in the figure below.
\begin{figure}[th] \begin{figure}[th]
\centering \centering
\includegraphics{Figures/Electron} \includegraphics{Figures/Electron}
\decoRule \decoRule
\caption[An Electron]{An electron (artist's impression).} \caption[An Electron]{An electron (artist's impression).}
\label{fig:Electron} \label{fig:Electron}
\end{figure} \end{figure}
Sometimes figures don't always appear where you write them in the source. The placement depends on how much space there is on the page for the figure. Sometimes there is not enough room to fit a figure directly where it should go (in relation to the text) and so \LaTeX{} puts it at the top of the next page. Positioning figures is the job of \LaTeX{} and so you should only worry about making them look good! Sometimes figures don't always appear where you write them in the
source. The placement depends on how much space there is on the page
for the figure. Sometimes there is not enough room to fit a figure
directly where it should go (in relation to the text) and so \LaTeX{}
puts it at the top of the next page. Positioning figures is the job
of \LaTeX{} and so you should only worry about making them look good!
Figures usually should have captions just in case you need to refer to them (such as in Figure~\ref{fig:Electron}). The \verb|\caption| command contains two parts, the first part, inside the square brackets is the title that will appear in the \emph{List of Figures}, and so should be short. The second part in the curly brackets should contain the longer and more descriptive caption text. Figures usually should have captions just in case you need to refer
to them (such as in Figure~\ref{fig:Electron}). The \verb|\caption|
command contains two parts, the first part, inside the square
brackets is the title that will appear in the \emph{List of Figures},
and so should be short. The second part in the curly brackets should
contain the longer and more descriptive caption text.
The \verb|\decoRule| command is optional and simply puts an aesthetic horizontal line below the image. If you do this for one image, do it for all of them. The \verb|\decoRule| command is optional and simply puts an aesthetic
horizontal line below the image. If you do this for one image, do it
for all of them.
\LaTeX{} is capable of using images in pdf, jpg and png format. \LaTeX{} is capable of using images in pdf, jpg and png format.
\subsection{Typesetting mathematics} \subsection{Typesetting mathematics}
If your thesis is going to contain heavy mathematical content, be sure that \LaTeX{} will make it look beautiful, even though it won't be able to solve the equations for you. If your thesis is going to contain heavy mathematical content, be
sure that \LaTeX{} will make it look beautiful, even though it won't
be able to solve the equations for you.
The \enquote{Not So Short Introduction to \LaTeX} (available on \href{http://www.ctan.org/tex-archive/info/lshort/english/lshort.pdf}{CTAN}) should tell you everything you need to know for most cases of typesetting mathematics. If you need more information, a much more thorough mathematical guide is available from the AMS called, \enquote{A Short Math Guide to \LaTeX} and can be downloaded from: The \enquote{Not So Short Introduction to \LaTeX} (available on
\href{http://www.ctan.org/tex-archive/info/lshort/english/lshort.pdf}{CTAN})
should tell you everything you need to know for most cases of
typesetting mathematics. If you need more information, a much more
thorough mathematical guide is available from the AMS called,
\enquote{A Short Math Guide to \LaTeX} and can be downloaded from:
\url{ftp://ftp.ams.org/pub/tex/doc/amsmath/short-math-guide.pdf} \url{ftp://ftp.ams.org/pub/tex/doc/amsmath/short-math-guide.pdf}
There are many different \LaTeX{} symbols to remember, luckily you can find the most common symbols in \href{http://ctan.org/pkg/comprehensive}{The Comprehensive \LaTeX~Symbol List}. There are many different \LaTeX{} symbols to remember, luckily you
can find the most common symbols in
\href{http://ctan.org/pkg/comprehensive}{The Comprehensive \LaTeX~Symbol List}.
You can write an equation, which is automatically given an equation number by \LaTeX{} like this: You can write an equation, which is automatically given an equation
number by \LaTeX{} like this:
\begin{verbatim} \begin{verbatim}
\begin{equation} \begin{equation}
E = mc^{2} E = mc^{2}
@@ -292,11 +556,13 @@ E = mc^{2}
This will produce Einstein's famous energy-matter equivalence equation: This will produce Einstein's famous energy-matter equivalence equation:
\begin{equation} \begin{equation}
E = mc^{2} E = mc^{2}
\label{eqn:Einstein} \label{eqn:Einstein}
\end{equation} \end{equation}
All equations you write (which are not in the middle of paragraph text) are automatically given equation numbers by \LaTeX{}. If you don't want a particular equation numbered, use the unnumbered form: All equations you write (which are not in the middle of paragraph
text) are automatically given equation numbers by \LaTeX{}. If you
don't want a particular equation numbered, use the unnumbered form:
\begin{verbatim} \begin{verbatim}
\[ a^{2}=4 \] \[ a^{2}=4 \]
\end{verbatim} \end{verbatim}
@@ -305,20 +571,33 @@ All equations you write (which are not in the middle of paragraph text) are auto
\section{Sectioning and Subsectioning} \section{Sectioning and Subsectioning}
You should break your thesis up into nice, bite-sized sections and subsections. \LaTeX{} automatically builds a table of Contents by looking at all the \verb|\chapter{}|, \verb|\section{}| and \verb|\subsection{}| commands you write in the source. You should break your thesis up into nice, bite-sized sections and
subsections. \LaTeX{} automatically builds a table of Contents by
looking at all the \verb|\chapter{}|, \verb|\section{}| and
\verb|\subsection{}| commands you write in the source.
The Table of Contents should only list the sections to three (3) levels. A \verb|chapter{}| is level zero (0). A \verb|\section{}| is level one (1) and so a \verb|\subsection{}| is level two (2). In your thesis it is likely that you will even use a \verb|subsubsection{}|, which is level three (3). The depth to which the Table of Contents is formatted is set within \file{MastersDoctoralThesis.cls}. If you need this changed, you can do it in \file{main.tex}. The Table of Contents should only list the sections to three (3)
levels. A \verb|chapter{}| is level zero (0). A \verb|\section{}| is
level one (1) and so a \verb|\subsection{}| is level two (2). In your
thesis it is likely that you will even use a \verb|subsubsection{}|,
which is level three (3). The depth to which the Table of Contents is
formatted is set within \file{MastersDoctoralThesis.cls}. If you need
this changed, you can do it in \file{main.tex}.
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
\section{In Closing} \section{In Closing}
You have reached the end of this mini-guide. You can now rename or overwrite this pdf file and begin writing your own \file{Chapter1.tex} and the rest of your thesis. The easy work of setting up the structure and framework has been taken care of for you. It's now your job to fill it out! You have reached the end of this mini-guide. You can now rename or
overwrite this pdf file and begin writing your own
\file{Chapter1.tex} and the rest of your thesis. The easy work of
setting up the structure and framework has been taken care of for
you. It's now your job to fill it out!
Good luck and have lots of fun! Good luck and have lots of fun!
\begin{flushright} \begin{flushright}
Guide written by ---\\ Guide written by ---\\
Sunil Patel: \href{http://www.sunilpatel.co.uk}{www.sunilpatel.co.uk}\\ Sunil Patel: \href{http://www.sunilpatel.co.uk}{www.sunilpatel.co.uk}\\
Vel: \href{http://www.LaTeXTemplates.com}{LaTeXTemplates.com} Vel: \href{http://www.LaTeXTemplates.com}{LaTeXTemplates.com}
\end{flushright} \end{flushright}

51
Chapters/ChapterTemplate.tex
View File

@@ -2,34 +2,67 @@
\chapter{Chapter Title Here} % Main chapter title \chapter{Chapter Title Here} % Main chapter title
\label{ChapterX} % Change X to a consecutive number; for referencing this chapter elsewhere, use \ref{ChapterX} \label{ChapterX} % Change X to a consecutive number; for referencing
% this chapter elsewhere, use \ref{ChapterX}
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
% SECTION 1 % SECTION 1
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
\section{Main Section 1} \section{Main Section 1}
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam ultricies lacinia euismod. Nam tempus risus in dolor rhoncus in interdum enim tincidunt. Donec vel nunc neque. In condimentum ullamcorper quam non consequat. Fusce sagittis tempor feugiat. Fusce magna erat, molestie eu convallis ut, tempus sed arcu. Quisque molestie, ante a tincidunt ullamcorper, sapien enim dignissim lacus, in semper nibh erat lobortis purus. Integer dapibus ligula ac risus convallis pellentesque. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam
ultricies lacinia euismod. Nam tempus risus in dolor rhoncus in
interdum enim tincidunt. Donec vel nunc neque. In condimentum
ullamcorper quam non consequat. Fusce sagittis tempor feugiat. Fusce
magna erat, molestie eu convallis ut, tempus sed arcu. Quisque
molestie, ante a tincidunt ullamcorper, sapien enim dignissim lacus,
in semper nibh erat lobortis purus. Integer dapibus ligula ac risus
convallis pellentesque.
%----------------------------------- %-----------------------------------
% SUBSECTION 1 % SUBSECTION 1
%----------------------------------- %-----------------------------------
\subsection{Subsection 1} \subsection{Subsection 1}
Nunc posuere quam at lectus tristique eu ultrices augue venenatis. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Aliquam erat volutpat. Vivamus sodales tortor eget quam adipiscing in vulputate ante ullamcorper. Sed eros ante, lacinia et sollicitudin et, aliquam sit amet augue. In hac habitasse platea dictumst. Nunc posuere quam at lectus tristique eu ultrices augue venenatis.
Vestibulum ante ipsum primis in faucibus orci luctus et ultrices
posuere cubilia Curae; Aliquam erat volutpat. Vivamus sodales tortor
eget quam adipiscing in vulputate ante ullamcorper. Sed eros ante,
lacinia et sollicitudin et, aliquam sit amet augue. In hac habitasse
platea dictumst.
%----------------------------------- %-----------------------------------
% SUBSECTION 2 % SUBSECTION 2
%----------------------------------- %-----------------------------------
\subsection{Subsection 2} \subsection{Subsection 2}
Morbi rutrum odio eget arcu adipiscing sodales. Aenean et purus a est pulvinar pellentesque. Cras in elit neque, quis varius elit. Phasellus fringilla, nibh eu tempus venenatis, dolor elit posuere quam, quis adipiscing urna leo nec orci. Sed nec nulla auctor odio aliquet consequat. Ut nec nulla in ante ullamcorper aliquam at sed dolor. Phasellus fermentum magna in augue gravida cursus. Cras sed pretium lorem. Pellentesque eget ornare odio. Proin accumsan, massa viverra cursus pharetra, ipsum nisi lobortis velit, a malesuada dolor lorem eu neque. Morbi rutrum odio eget arcu adipiscing sodales. Aenean et purus a est
pulvinar pellentesque. Cras in elit neque, quis varius elit.
Phasellus fringilla, nibh eu tempus venenatis, dolor elit posuere
quam, quis adipiscing urna leo nec orci. Sed nec nulla auctor odio
aliquet consequat. Ut nec nulla in ante ullamcorper aliquam at sed
dolor. Phasellus fermentum magna in augue gravida cursus. Cras sed
pretium lorem. Pellentesque eget ornare odio. Proin accumsan, massa
viverra cursus pharetra, ipsum nisi lobortis velit, a malesuada dolor
lorem eu neque.
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
% SECTION 2 % SECTION 2
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
\section{Main Section 2} \section{Main Section 2}
Sed ullamcorper quam eu nisl interdum at interdum enim egestas. Aliquam placerat justo sed lectus lobortis ut porta nisl porttitor. Vestibulum mi dolor, lacinia molestie gravida at, tempus vitae ligula. Donec eget quam sapien, in viverra eros. Donec pellentesque justo a massa fringilla non vestibulum metus vestibulum. Vestibulum in orci quis felis tempor lacinia. Vivamus ornare ultrices facilisis. Ut hendrerit volutpat vulputate. Morbi condimentum venenatis augue, id porta ipsum vulputate in. Curabitur luctus tempus justo. Vestibulum risus lectus, adipiscing nec condimentum quis, condimentum nec nisl. Aliquam dictum sagittis velit sed iaculis. Morbi tristique augue sit amet nulla pulvinar id facilisis ligula mollis. Nam elit libero, tincidunt ut aliquam at, molestie in quam. Aenean rhoncus vehicula hendrerit. Sed ullamcorper quam eu nisl interdum at interdum enim egestas.
Aliquam placerat justo sed lectus lobortis ut porta nisl porttitor.
Vestibulum mi dolor, lacinia molestie gravida at, tempus vitae
ligula. Donec eget quam sapien, in viverra eros. Donec pellentesque
justo a massa fringilla non vestibulum metus vestibulum. Vestibulum
in orci quis felis tempor lacinia. Vivamus ornare ultrices facilisis.
Ut hendrerit volutpat vulputate. Morbi condimentum venenatis augue,
id porta ipsum vulputate in. Curabitur luctus tempus justo.
Vestibulum risus lectus, adipiscing nec condimentum quis, condimentum
nec nisl. Aliquam dictum sagittis velit sed iaculis. Morbi tristique
augue sit amet nulla pulvinar id facilisis ligula mollis. Nam elit
libero, tincidunt ut aliquam at, molestie in quam. Aenean rhoncus
vehicula hendrerit.

127
Chapters/Introduction.tex
View File

@@ -2,20 +2,135 @@
\chapter{Introduction} % Main chapter title \chapter{Introduction} % Main chapter title
\label{Introduction} % Change X to a consecutive number; for referencing this chapter elsewhere, use \ref{ChapterX} \label{Introduction} % Change X to a consecutive number; for
% referencing this chapter elsewhere, use \ref{ChapterX}
\begin{figure}[h!]
\centering
\includesvg[width=\textwidth]{Figures/clan_thesis_argumentation_tree.drawio.svg}
\caption{Argumentation Tree for the Clan Thesis}
\label{fig:clan_thesis_argumentation_tree}
\end{figure}
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
% SECTION 1 % SECTION 1
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
\section{Methodology} \section{Methodology}
In this chapter, we will discuss the methodology used in this research. We will discuss the research design, the data collection methods, the data analysis methods, and the research limitations. We will also discuss the ethical considerations of this research. In this chapter, we will discuss the methodology used in this
research. We will discuss the research design, and the research limitations.
\section{Related Work}
The Clan framework operates within the realm of software deployment
and peer-to-peer networking,
necessitating a deep understanding of existing methodologies in these
areas to tackle contemporary challenges.
This section will discuss related works encompassing system
deployment, peer data management,
and low maintenance structured peer-to-peer overlays, which inform
the development and positioning of the Clan framework.
\section{Related Workd} \subsection{Nix: A Safe and Policy-Free System for Software Deployment}
In this chapter, we will discuss the related work in the field of software engineering. We will discuss the research that has been done in the field of software engineering, and how it relates to this research. We will also discuss the limitations of the existing research, and how this research will address those limitations. Nix addresses significant issues in software deployment by utilizing
a technique that employs cryptographic
hashes to ensure unique paths for component instances[1].
The system is distinguished by its features, such as concurrent
installation of multiple versions and variants,
atomic upgrades, and safe garbage collection.
These capabilities lead to a flexible deployment system that
harmonizes source and binary deployments.
Nix conceptualizes deployment without imposing rigid policies,
thereby offering adaptable strategies for component management[2].
This contrasts with many prevailing systems that are constrained by
policy-specific designs,
making Nix an agile choice for modern deployment needs[3]. The
insights gained from Nix's approach
to non-interference and de-risking upgrades through atomic
transactions highlight the potential
for deploying software components seamlessly in decentralized networks[4].
\subsection{NixOS: A Purely Functional Linux Distribution}
NixOS is an extension of the principles established by Nix,
presenting a Linux distribution that manages system configurations
using purely functional methods . This model ensures that system
configurations are reproducible and isolated
from stateful interactions typical in imperative models of package management.
Because NixOS configurations are built by pure functions, they can overcome the
challenges of easily rolling back changes, deploying multiple package versions
side-by-side, and achieving deterministic configuration reproduction .
The solution is particularly compelling in environments necessitating rigorous
reproducibility and minimal configuration drift—a valuable feature
for distributed networks .
\subsection{Disnix: A Toolset for Distributed Deployment}
The Disnix toolset extends the deployment capabilities of Nix into
distributed systems,
focusing on automating the deployment process across different network nodes .
By leveraging the modular approach of Nix, Disnix enables the
consistent deployment of
software environments, reducing the incidence of configuration errors
across heterogeneous systems.
This approach aligns with the needs of distributed systems like those
utilized in peer-to-peer networks,
where maintaining consistency across nodes is crucial for operational
integrity .
\subsection{The Piazza Peer Data Management Project}
The peer data management landscape is further enriched by the Piazza
project, which offers a
flexible integration framework for heterogeneous data sources[5].
Piazza's approach to routing and
indexing extends beyond traditional network boundaries, showcasing a
scalable method for handling
vast data spaces in peer-to-peer systems. By addressing challenges
related to schema mediation and
decentralized data queries, Piazza provides a foundational structure
from which Clan can envisage
robust peer data management underpinned by strong consistency and reach.
\subsection{Software-Defined Networking and Low Maintenance Overlays}
The transition towards software-defined networking (SDN) is
represented by systems that decouple
the control plane from the data plane, enabling flexible network
configurations[6]. In particular,
SDN introduces novel paradigms in managing network resources
dynamically, facilitating more adaptive
and responsive network overlays. The Clan framework can draw on SDN
principles to facilitate
low-maintenance structured overlays, ensuring robust connectivity and
efficient resource allocation
in peer-to-peer environments. These systems reduce the overhead
associated with managing network state,
thus aligning with the inherent value propositions of decentralized networks.
\subsection{Charon: Declarative Provisioning and Deployment}
Charon adds another dimension to deployment via its declarative
provisioning capabilities[7].
By emphasizing a policy-driven approach, Charon aligns closely with
the principles of
infrastructure-as-code, where deployment processes are consistently
repeatable and auditable.
Such characteristics are beneficial for frameworks like Clan,
ensuring transparency and accuracy
in deployment processes across peer-to-peer nodes[8].
In summation, the evolution of deployment systems and peer data
management frameworks underscores
the steps necessary for developing robust decentralized systems like
the Clan framework.
By integrating features from Nix, NixOS, Disnix, and leveraging
insights from SDN and Charon,
the Clan framework can offer a high degree of reliability,
flexibility, and efficiency across
distributed networks. This related work provides a foundational
understanding that supports the
enhancement of the Clan framework towards achieving these objectives.
\parencite{kjorveziroski_full-mesh_2024}

4
Figures/clan_thesis_argumentation_tree.drawio.svg Normal file
View File

File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 392 KiB

656
flake.lock generated Normal file
View File

@@ -0,0 +1,656 @@
{
"nodes": {
"flake-compat": {
"flake": false,
"locked": {
"lastModified": 1696426674,
"narHash": "sha256-kvjfFW7WAETZlt09AgDn1MrtKzP7t90Vf7vypd3OL1U=",
"owner": "edolstra",
"repo": "flake-compat",
"rev": "0f9255e01c2351cc7d116c072cb317785dd33b33",
"type": "github"
},
"original": {
"owner": "edolstra",
"repo": "flake-compat",
"type": "github"
}
},
"flake-compat_2": {
"flake": false,
"locked": {
"lastModified": 1627913399,
"narHash": "sha256-hY8g6H2KFL8ownSiFeMOjwPC8P0ueXpCVEbxgda3pko=",
"owner": "edolstra",
"repo": "flake-compat",
"rev": "12c64ca55c1014cdc1b16ed5a804aa8576601ff2",
"type": "github"
},
"original": {
"owner": "edolstra",
"repo": "flake-compat",
"type": "github"
}
},
"flake-parts": {
"inputs": {
"nixpkgs-lib": [
"nixpkgs"
]
},
"locked": {
"lastModified": 1730504689,
"narHash": "sha256-hgmguH29K2fvs9szpq2r3pz2/8cJd2LPS+b4tfNFCwE=",
"owner": "hercules-ci",
"repo": "flake-parts",
"rev": "506278e768c2a08bec68eb62932193e341f55c90",
"type": "github"
},
"original": {
"owner": "hercules-ci",
"repo": "flake-parts",
"type": "github"
}
},
"flake-utils": {
"inputs": {
"systems": "systems"
},
"locked": {
"lastModified": 1694529238,
"narHash": "sha256-zsNZZGTGnMOf9YpHKJqMSsa0dXbfmxeoJ7xHlrt+xmY=",
"owner": "numtide",
"repo": "flake-utils",
"rev": "ff7b65b44d01cf9ba6a71320833626af21126384",
"type": "github"
},
"original": {
"owner": "numtide",
"repo": "flake-utils",
"type": "github"
}
},
"flake-utils_2": {
"locked": {
"lastModified": 1629481132,
"narHash": "sha256-JHgasjPR0/J1J3DRm4KxM4zTyAj4IOJY8vIl75v/kPI=",
"owner": "numtide",
"repo": "flake-utils",
"rev": "997f7efcb746a9c140ce1f13c72263189225f482",
"type": "github"
},
"original": {
"owner": "numtide",
"repo": "flake-utils",
"type": "github"
}
},
"gitignore": {
"flake": false,
"locked": {
"lastModified": 1611672876,
"narHash": "sha256-qHu3uZ/o9jBHiA3MEKHJ06k7w4heOhA+4HCSIvflRxo=",
"owner": "hercules-ci",
"repo": "gitignore.nix",
"rev": "211907489e9f198594c0eb0ca9256a1949c9d412",
"type": "github"
},
"original": {
"owner": "hercules-ci",
"repo": "gitignore.nix",
"type": "github"
}
},
"gitignore_2": {
"inputs": {
"nixpkgs": [
"jupyenv",
"pre-commit-hooks",
"nixpkgs"
]
},
"locked": {
"lastModified": 1660459072,
"narHash": "sha256-8DFJjXG8zqoONA1vXtgeKXy68KdJL5UaXR8NtVMUbx8=",
"owner": "hercules-ci",
"repo": "gitignore.nix",
"rev": "a20de23b925fd8264fd7fad6454652e142fd7f73",
"type": "github"
},
"original": {
"owner": "hercules-ci",
"repo": "gitignore.nix",
"type": "github"
}
},
"hls": {
"inputs": {
"flake-compat": "flake-compat_2",
"flake-utils": "flake-utils_2",
"gitignore": "gitignore",
"nixpkgs": "nixpkgs",
"pre-commit-hooks": "pre-commit-hooks"
},
"locked": {
"lastModified": 1637213318,
"narHash": "sha256-ZgxPwV7t4DyGYP7aXoetq+JHtd73XlOV2fYSflQmOXw=",
"owner": "haskell",
"repo": "haskell-language-server",
"rev": "311107eabbf0537e0c192b2c377d282505b4eff1",
"type": "github"
},
"original": {
"owner": "haskell",
"repo": "haskell-language-server",
"type": "github"
}
},
"ihaskell": {
"inputs": {
"flake-compat": [
"jupyenv"
],
"flake-utils": [
"jupyenv",
"flake-utils"
],
"hls": "hls",
"nixpkgs": [
"jupyenv",
"nixpkgs"
]
},
"locked": {
"lastModified": 1692365546,
"narHash": "sha256-qn8TQezUbA66I7ASnfu0Ib5CWBFfYloKwFxn686VlMw=",
"owner": "ihaskell",
"repo": "ihaskell",
"rev": "d7ad5d57ceb852a19a98266dfb53ec3b3441dda4",
"type": "github"
},
"original": {
"owner": "ihaskell",
"repo": "ihaskell",
"type": "github"
}
},
"jupyenv": {
"inputs": {
"flake-compat": "flake-compat",
"flake-utils": "flake-utils",
"ihaskell": "ihaskell",
"nix-dart": "nix-dart",
"nixpkgs": "nixpkgs_2",
"nixpkgs-stable": "nixpkgs-stable",
"npmlock2nix": "npmlock2nix",
"opam-nix": "opam-nix",
"poetry2nix": "poetry2nix",
"pre-commit-hooks": "pre-commit-hooks_2",
"rust-overlay": "rust-overlay"
},
"locked": {
"lastModified": 1732481790,
"narHash": "sha256-xSwU4PAaDmBM1BsnWYWoZ+af+cHk2ry9cKxUjHVXpIQ=",
"owner": "tweag",
"repo": "jupyenv",
"rev": "59d97bf6715053502d8f13e5fb45342ca80efa09",
"type": "github"
},
"original": {
"owner": "tweag",
"repo": "jupyenv",
"type": "github"
}
},
"mirage-opam-overlays": {
"flake": false,
"locked": {
"lastModified": 1661959605,
"narHash": "sha256-CPTuhYML3F4J58flfp3ZbMNhkRkVFKmBEYBZY5tnQwA=",
"owner": "dune-universe",
"repo": "mirage-opam-overlays",
"rev": "05f1c1823d891ce4d8adab91f5db3ac51d86dc0b",
"type": "github"
},
"original": {
"owner": "dune-universe",
"repo": "mirage-opam-overlays",
"type": "github"
}
},
"nix-dart": {
"inputs": {
"flake-utils": [
"jupyenv",
"flake-utils"
],
"nixpkgs": [
"jupyenv",
"nixpkgs"
],
"pub2nix": "pub2nix"
},
"locked": {
"lastModified": 1673740150,
"narHash": "sha256-JiZrr75JILHW7IaNW3MwpYn+084Q6/gnXScPR7Pozhs=",
"owner": "djacu",
"repo": "nix-dart",
"rev": "8ee4e1a5ec0cc6c1e15860c4733f741485e8231e",
"type": "github"
},
"original": {
"owner": "djacu",
"repo": "nix-dart",
"type": "github"
}
},
"nix-github-actions": {
"inputs": {
"nixpkgs": [
"jupyenv",
"poetry2nix",
"nixpkgs"
]
},
"locked": {
"lastModified": 1698974481,
"narHash": "sha256-yPncV9Ohdz1zPZxYHQf47S8S0VrnhV7nNhCawY46hDA=",
"owner": "nix-community",
"repo": "nix-github-actions",
"rev": "4bb5e752616262457bc7ca5882192a564c0472d2",
"type": "github"
},
"original": {
"owner": "nix-community",
"repo": "nix-github-actions",
"type": "github"
}
},
"nixpkgs": {
"locked": {
"lastModified": 1630887066,
"narHash": "sha256-0ecIlrLsNIIa+zrNmzXXmbMBLZlmHU/aWFsa4bq99Hk=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "5e47a07e9f2d7ed999f2c7943b0896f5f7321ca3",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixpkgs-unstable",
"repo": "nixpkgs",
"type": "github"
}
},
"nixpkgs-stable": {
"locked": {
"lastModified": 1698288402,
"narHash": "sha256-jIIjApPdm+4yt8PglX8pUOexAdEiAax/DXW3S/Mb21E=",
"owner": "nixos",
"repo": "nixpkgs",
"rev": "60b9db998f71ea49e1a9c41824d09aa274be1344",
"type": "github"
},
"original": {
"owner": "nixos",
"ref": "nixos-23.05",
"repo": "nixpkgs",
"type": "github"
}
},
"nixpkgs-stable_2": {
"locked": {
"lastModified": 1685801374,
"narHash": "sha256-otaSUoFEMM+LjBI1XL/xGB5ao6IwnZOXc47qhIgJe8U=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "c37ca420157f4abc31e26f436c1145f8951ff373",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-23.05",
"repo": "nixpkgs",
"type": "github"
}
},
"nixpkgs_2": {
"locked": {
"lastModified": 1702938738,
"narHash": "sha256-O7Vb0xC9s4Dmgxj8APEpuuMj7HsLgPbpy1UKvNVJp7o=",
"owner": "nixos",
"repo": "nixpkgs",
"rev": "dd8e82f3b4017b8faa52c2b1897a38d53c3c26cb",
"type": "github"
},
"original": {
"owner": "nixos",
"ref": "nixpkgs-unstable",
"repo": "nixpkgs",
"type": "github"
}
},
"nixpkgs_3": {
"locked": {
"lastModified": 1732014248,
"narHash": "sha256-y/MEyuJ5oBWrWAic/14LaIr/u5E0wRVzyYsouYY3W6w=",
"owner": "nixos",
"repo": "nixpkgs",
"rev": "23e89b7da85c3640bbc2173fe04f4bd114342367",
"type": "github"
},
"original": {
"owner": "nixos",
"ref": "nixos-unstable",
"repo": "nixpkgs",
"type": "github"
}
},
"nixpkgs_4": {
"locked": {
"lastModified": 1731890469,
"narHash": "sha256-D1FNZ70NmQEwNxpSSdTXCSklBH1z2isPR84J6DQrJGs=",
"owner": "nixos",
"repo": "nixpkgs",
"rev": "5083ec887760adfe12af64830a66807423a859a7",
"type": "github"
},
"original": {
"owner": "nixos",
"ref": "nixpkgs-unstable",
"repo": "nixpkgs",
"type": "github"
}
},
"npmlock2nix": {
"flake": false,
"locked": {
"lastModified": 1668989938,
"narHash": "sha256-/IxdS0AiqSN0/VEOLnnfHyi4nP17yPrkhGf6KlXVwrc=",
"owner": "nix-community",
"repo": "npmlock2nix",
"rev": "0ba0746d62974403daf717cded3f24c617622bc7",
"type": "github"
},
"original": {
"owner": "nix-community",
"repo": "npmlock2nix",
"rev": "0ba0746d62974403daf717cded3f24c617622bc7",
"type": "github"
}
},
"opam-nix": {
"inputs": {
"flake-compat": [
"jupyenv"
],
"flake-utils": [
"jupyenv",
"flake-utils"
],
"mirage-opam-overlays": "mirage-opam-overlays",
"nixpkgs": [
"jupyenv",
"nixpkgs"
],
"opam-overlays": "opam-overlays",
"opam-repository": "opam-repository",
"opam2json": "opam2json"
},
"locked": {
"lastModified": 1695904678,
"narHash": "sha256-g4ZcyFsQdcDYgBOGKIwFTu050Yw2xvYHpKSLxlA9p2Q=",
"owner": "tweag",
"repo": "opam-nix",
"rev": "d4a42c5ce58d4f0b07ee158867d085def0de33ce",
"type": "github"
},
"original": {
"owner": "tweag",
"repo": "opam-nix",
"type": "github"
}
},
"opam-overlays": {
"flake": false,
"locked": {
"lastModified": 1654162756,
"narHash": "sha256-RV68fUK+O3zTx61iiHIoS0LvIk0E4voMp+0SwRg6G6c=",
"owner": "dune-universe",
"repo": "opam-overlays",
"rev": "c8f6ef0fc5272f254df4a971a47de7848cc1c8a4",
"type": "github"
},
"original": {
"owner": "dune-universe",
"repo": "opam-overlays",
"type": "github"
}
},
"opam-repository": {
"flake": false,
"locked": {
"lastModified": 1695894792,
"narHash": "sha256-7Llico807vq14AkqAaDIWogC50xLxU38nuNEH06YNPE=",
"owner": "ocaml",
"repo": "opam-repository",
"rev": "33fcf32f269ee5af70b31e27442397a0cdaf28b2",
"type": "github"
},
"original": {
"owner": "ocaml",
"repo": "opam-repository",
"type": "github"
}
},
"opam2json": {
"inputs": {
"nixpkgs": [
"jupyenv",
"opam-nix",
"nixpkgs"
]
},
"locked": {
"lastModified": 1671540003,
"narHash": "sha256-5pXfbUfpVABtKbii6aaI2EdAZTjHJ2QntEf0QD2O5AM=",
"owner": "tweag",
"repo": "opam2json",
"rev": "819d291ea95e271b0e6027679de6abb4d4f7f680",
"type": "github"
},
"original": {
"owner": "tweag",
"repo": "opam2json",
"type": "github"
}
},
"poetry2nix": {
"inputs": {
"flake-utils": [
"jupyenv",
"flake-utils"
],
"nix-github-actions": "nix-github-actions",
"nixpkgs": [
"jupyenv",
"nixpkgs"
],
"systems": "systems_2",
"treefmt-nix": [
"jupyenv"
]
},
"locked": {
"lastModified": 1703076010,
"narHash": "sha256-VaoDgib09zqtF0rREUvD2inmE9N4ECT1i02DVAhmOPk=",
"owner": "nix-community",
"repo": "poetry2nix",
"rev": "b76e91da74ad7af0bc397561a59a19fcbc2ad488",
"type": "github"
},
"original": {
"owner": "nix-community",
"repo": "poetry2nix",
"type": "github"
}
},
"pre-commit-hooks": {
"inputs": {
"flake-utils": [
"jupyenv",
"ihaskell",
"hls",
"flake-utils"
],
"nixpkgs": [
"jupyenv",
"ihaskell",
"hls",
"nixpkgs"
]
},
"locked": {
"lastModified": 1624971177,
"narHash": "sha256-Amf/nBj1E77RmbSSmV+hg6YOpR+rddCbbVgo5C7BS0I=",
"owner": "cachix",
"repo": "pre-commit-hooks.nix",
"rev": "397f0713d007250a2c7a745e555fa16c5dc8cadb",
"type": "github"
},
"original": {
"owner": "cachix",
"repo": "pre-commit-hooks.nix",
"type": "github"
}
},
"pre-commit-hooks_2": {
"inputs": {
"flake-compat": [
"jupyenv"
],
"flake-utils": [
"jupyenv",
"flake-utils"
],
"gitignore": "gitignore_2",
"nixpkgs": [
"jupyenv",
"nixpkgs"
],
"nixpkgs-stable": "nixpkgs-stable_2"
},
"locked": {
"lastModified": 1698227354,
"narHash": "sha256-Fi5H9jbaQLmLw9qBi/mkR33CoFjNbobo5xWdX4tKz1Q=",
"owner": "cachix",
"repo": "pre-commit-hooks.nix",
"rev": "bd38df3d508dfcdff52cd243d297f218ed2257bf",
"type": "github"
},
"original": {
"owner": "cachix",
"repo": "pre-commit-hooks.nix",
"type": "github"
}
},
"pub2nix": {
"flake": false,
"locked": {
"lastModified": 1594192744,
"narHash": "sha256-pDvcXSG1Mh2BpwkqAcNDJzcupV3pIAAtZJLfkiHMAz4=",
"owner": "paulyoung",
"repo": "pub2nix",
"rev": "0c7ecca590fcd1616db8c6468f799ffef36c85e9",
"type": "github"
},
"original": {
"owner": "paulyoung",
"repo": "pub2nix",
"type": "github"
}
},
"root": {
"inputs": {
"flake-parts": "flake-parts",
"jupyenv": "jupyenv",
"nixpkgs": "nixpkgs_3",
"treefmt-nix": "treefmt-nix"
}
},
"rust-overlay": {
"inputs": {
"flake-utils": [
"jupyenv",
"flake-utils"
],
"nixpkgs": [
"jupyenv",
"nixpkgs"
]
},
"locked": {
"lastModified": 1696126582,
"narHash": "sha256-uo4cn/d2rHPy/fpKZKFBOaVO531zs/Doxz43imrpqZM=",
"owner": "oxalica",
"repo": "rust-overlay",
"rev": "fc6fe50d9a4540a1111731baaa00f207301fdeb7",
"type": "github"
},
"original": {
"owner": "oxalica",
"repo": "rust-overlay",
"type": "github"
}
},
"systems": {
"locked": {
"lastModified": 1681028828,
"narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
"owner": "nix-systems",
"repo": "default",
"rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
"type": "github"
},
"original": {
"owner": "nix-systems",
"repo": "default",
"type": "github"
}
},
"systems_2": {
"locked": {
"lastModified": 1681028828,
"narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
"owner": "nix-systems",
"repo": "default",
"rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
"type": "github"
},
"original": {
"id": "systems",
"type": "indirect"
}
},
"treefmt-nix": {
"inputs": {
"nixpkgs": "nixpkgs_4"
},
"locked": {
"lastModified": 1732292307,
"narHash": "sha256-5WSng844vXt8uytT5djmqBCkopyle6ciFgteuA9bJpw=",
"owner": "numtide",
"repo": "treefmt-nix",
"rev": "705df92694af7093dfbb27109ce16d828a79155f",
"type": "github"
},
"original": {
"owner": "numtide",
"repo": "treefmt-nix",
"type": "github"
}
}
},
"root": "root",
"version": 7
}

58
flake.nix Normal file
View File

@@ -0,0 +1,58 @@
{
description = "Your jupyenv project";
nixConfig.extra-substituters = [
"https://tweag-jupyter.cachix.org"
];
nixConfig.extra-trusted-public-keys = [
"tweag-jupyter.cachix.org-1:UtNH4Zs6hVUFpFBTLaA4ejYavPo5EFFqgd7G7FxGW9g="
];
inputs = {
treefmt-nix.url = "github:numtide/treefmt-nix";
flake-parts.url = "github:hercules-ci/flake-parts";
flake-parts.inputs.nixpkgs-lib.follows = "nixpkgs";
nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
jupyenv.url = "github:tweag/jupyenv";
};
outputs =
inputs@{ flake-parts, ... }:
flake-parts.lib.mkFlake { inherit inputs; } {
imports = [ ./treefmt.nix ];
systems = [
"x86_64-linux"
"aarch64-linux"
];
perSystem =
{
lib,
system,
pkgs,
...
}:
let
inherit (inputs.jupyenv.lib.${system}) mkJupyterlabNew;
jupyterlab = mkJupyterlabNew (
{ ... }:
{
# nixpkgs = nixpkgs;
imports = [ (import ./kernels.nix) ];
}
);
in
{
devShells.default = pkgs.mkShell {
buildInputs = with pkgs; [
tex-fmt
];
};
packages = {
inherit jupyterlab;
};
packages.default = jupyterlab;
apps.default.program = "${jupyterlab}/bin/jupyter-lab";
apps.default.type = "app";
};
};
}

14
kernels.nix Normal file
View File

@@ -0,0 +1,14 @@
{ ... }:
{
kernel.python.minimal = {
enable = true;
};
kernel.nix.minimal = {
enable = true;
};
kernel.bash.minimal = {
enable = true;
};
}

339
main.tex
View File

@@ -1,5 +1,5 @@
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Masters/Doctoral Thesis % Masters/Doctoral Thesis
% LaTeX Template % LaTeX Template
% Version 2.5 (27/8/17) % Version 2.5 (27/8/17)
% %
@@ -10,7 +10,8 @@
% Vel (vel@latextemplates.com) % Vel (vel@latextemplates.com)
% %
% This template is based on a template by: % This template is based on a template by:
% Steve Gunn (http://users.ecs.soton.ac.uk/srg/softwaretools/document/templates/) % Steve Gunn
% (http://users.ecs.soton.ac.uk/srg/softwaretools/document/templates/)
% Sunil Patel (http://www.sunilpatel.co.uk/thesis-template/) % Sunil Patel (http://www.sunilpatel.co.uk/thesis-template/)
% %
% Template license: % Template license:
@@ -19,181 +20,243 @@
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
% PACKAGES AND OTHER DOCUMENT CONFIGURATIONS % PACKAGES AND OTHER DOCUMENT CONFIGURATIONS
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
\documentclass[ \documentclass[
11pt, % The default document font size, options: 10pt, 11pt, 12pt 11pt, % The default document font size, options: 10pt, 11pt, 12pt
oneside, % Two side (alternating margins) for binding by default, uncomment to switch to one side oneside, % Two side (alternating margins) for binding by default,
english, % ngerman for German % uncomment to switch to one side
singlespacing, % Single line spacing, alternatives: onehalfspacing or doublespacing english, % ngerman for German
%draft, % Uncomment to enable draft mode (no pictures, no links, overfull hboxes indicated) singlespacing, % Single line spacing, alternatives: onehalfspacing
%nolistspacing, % If the document is onehalfspacing or doublespacing, uncomment this to set spacing in lists to single % or doublespacing
%liststotoc, % Uncomment to add the list of figures/tables/etc to the table of contents %draft, % Uncomment to enable draft mode (no pictures, no links,
%toctotoc, % Uncomment to add the main table of contents to the table of contents % overfull hboxes indicated)
%parskip, % Uncomment to add space between paragraphs %nolistspacing, % If the document is onehalfspacing or
%nohyperref, % Uncomment to not load the hyperref package % doublespacing, uncomment this to set spacing in lists to single
headsepline, % Uncomment to get a line under the header %liststotoc, % Uncomment to add the list of figures/tables/etc to
%chapterinoneline, % Uncomment to place the chapter title next to the number on one line % the table of contents
%consistentlayout, % Uncomment to change the layout of the declaration, abstract and acknowledgements pages to match the default layout %toctotoc, % Uncomment to add the main table of contents to the
% table of contents
%parskip, % Uncomment to add space between paragraphs
%nohyperref, % Uncomment to not load the hyperref package
headsepline, % Uncomment to get a line under the header
%chapterinoneline, % Uncomment to place the chapter title next to
% the number on one line
%consistentlayout, % Uncomment to change the layout of the
% declaration, abstract and acknowledgements pages to match the default layout
]{MastersDoctoralThesis} % The class file specifying the document structure ]{MastersDoctoralThesis} % The class file specifying the document structure
\usepackage[utf8]{inputenc} % Required for inputting international characters \usepackage[utf8]{inputenc} % Required for inputting international characters
\usepackage[T1]{fontenc} % Output font encoding for international characters \usepackage[T1]{fontenc} % Output font encoding for international characters
\usepackage{mathpazo} % Use the Palatino font by default \usepackage{mathpazo} % Use the Palatino font by default
\usepackage{svg}
\usepackage[backend=bibtex,style=authoryear,natbib=true]{biblatex} % Use the bibtex backend with the authoryear citation style (which resembles APA) \usepackage[backend=bibtex,style=authoryear,natbib=true]{biblatex} %
% Use the bibtex backend with the authoryear citation style (which
% resembles APA)
\addbibresource{example.bib} % The filename of the bibliography \addbibresource{example.bib} % The filename of the bibliography
\addbibresource{master_citations.bib} \addbibresource{master_citations.bib}
\usepackage[autostyle=true]{csquotes} % Required to generate language-dependent quotes in the bibliography \usepackage[autostyle=true]{csquotes} % Required to generate
% language-dependent quotes in the bibliography
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
% MARGIN SETTINGS % MARGIN SETTINGS
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
\geometry{ \geometry{
paper=a4paper, % Change to letterpaper for US letter paper=a4paper, % Change to letterpaper for US letter
inner=2.5cm, % Inner margin inner=2.5cm, % Inner margin
outer=3.8cm, % Outer margin outer=3.8cm, % Outer margin
bindingoffset=.5cm, % Binding offset bindingoffset=.5cm, % Binding offset
top=1.5cm, % Top margin top=1.5cm, % Top margin
bottom=1.5cm, % Bottom margin bottom=1.5cm, % Bottom margin
%showframe, % Uncomment to show how the type block is set on the page %showframe, % Uncomment to show how the type block is set on the page
} }
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
% THESIS INFORMATION % THESIS INFORMATION
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
\thesistitle{Development and Verification of a Theoretical Model for the Clan Framework} % Your thesis title, this is used in the title and abstract, print it elsewhere with \ttitle \thesistitle{Development and Verification of a Theoretical Model for
\supervisor{\textsc{Ber Lorke}} % Your supervisor's name, this is used in the title page, print it elsewhere with \supname the Clan Framework} % Your thesis title, this is used in the title
\examiner{Prof. Dr. \textsc{Stefan Schmid}} % Your examiner's name, this is not currently used anywhere in the template, print it elsewhere with \examname % and abstract, print it elsewhere with \ttitle
\degree{Master of Computer Science} % Your degree name, this is used in the title page and abstract, print it elsewhere with \degreename \supervisor{\textsc{Ber Lorke}} % Your supervisor's name, this is
\author{\textsc{Luis Hebendanz}} % Your name, this is used in the title page and abstract, print it elsewhere with \authorname % used in the title page, print it elsewhere with \supname
\examiner{Prof. Dr. \textsc{Stefan Schmid}} % Your examiner's name,
% this is not currently used anywhere in the template, print it
% elsewhere with \examname
\degree{Master of Computer Science} % Your degree name, this is used
% in the title page and abstract, print it elsewhere with \degreename
\author{\textsc{Luis Hebendanz}} % Your name, this is used in the
% title page and abstract, print it elsewhere with \authorname
\subject{Computer Science} % Your subject area, this is not currently used anywhere in the template, print it elsewhere with \subjectname \subject{Computer Science} % Your subject area, this is not currently
\keywords{Decentralized Networks} % Keywords for your thesis, this is not currently used anywhere in the template, print it elsewhere with \keywordnames % used anywhere in the template, print it elsewhere with \subjectname
\university{\href{https://www.tu.berlin/}{TU Berlin}} % Your university's name and URL, this is used in the title page and abstract, print it elsewhere with \univname \keywords{Decentralized Networks} % Keywords for your thesis, this is
\department{\href{https://www.tu.berlin/eninet}{Internet Architecture and Management}} % Your department's name and URL, this is used in the title page and abstract, print it elsewhere with \deptname % not currently used anywhere in the template, print it elsewhere
\group{\href{https://www.tu.berlin/eninet}{INET}} % Your research group's name and URL, this is used in the title page, print it elsewhere with \groupname % with \keywordnames
\faculty{\href{https://www.tu.berlin/eecs}{Faculty IV}} % Your faculty's name and URL, this is used in the title page and abstract, print it elsewhere with \facname \university{\href{https://www.tu.berlin/}{TU Berlin}} % Your
% university's name and URL, this is used in the title page and
% abstract, print it elsewhere with \univname
\department{\href{https://www.tu.berlin/eninet}{Internet Architecture
and Management}} % Your department's name and URL, this is used in
% the title page and abstract, print it elsewhere with \deptname
\group{\href{https://www.tu.berlin/eninet}{INET}} % Your research
% group's name and URL, this is used in the title page, print it
% elsewhere with \groupname
\faculty{\href{https://www.tu.berlin/eecs}{Faculty IV}} % Your
% faculty's name and URL, this is used in the title page and
% abstract, print it elsewhere with \facname
\AtBeginDocument{ \AtBeginDocument{
\hypersetup{pdftitle=\ttitle} % Set the PDF's title to your title \hypersetup{pdftitle=\ttitle} % Set the PDF's title to your title
\hypersetup{pdfauthor=\authorname} % Set the PDF's author to your name \hypersetup{pdfauthor=\authorname} % Set the PDF's author to your name
\hypersetup{pdfkeywords=\keywordnames} % Set the PDF's keywords to your keywords \hypersetup{pdfkeywords=\keywordnames} % Set the PDF's keywords to
% your keywords
} }
\begin{document} \begin{document}
\frontmatter % Use roman page numbering style (i, ii, iii, iv...) for the pre-content pages \frontmatter % Use roman page numbering style (i, ii, iii, iv...) for
% the pre-content pages
\pagestyle{plain} % Default to the plain heading style until the thesis style is called for the body content \pagestyle{plain} % Default to the plain heading style until the
% thesis style is called for the body content
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
% TITLE PAGE % TITLE PAGE
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
\begin{titlepage} \begin{titlepage}
\begin{center} \begin{center}
\vspace*{.06\textheight} \vspace*{.06\textheight}
{\scshape\LARGE \univname\par}\vspace{1.5cm} % University name {\scshape\LARGE \univname\par}\vspace{1.5cm} % University name
\textsc{\Large Master Thesis}\\[0.5cm] % Thesis type \textsc{\Large Master Thesis}\\[0.5cm] % Thesis type
\HRule \\[0.4cm] % Horizontal line \HRule \\[0.4cm] % Horizontal line
{\huge \bfseries \ttitle\par}\vspace{0.4cm} % Thesis title {\huge \bfseries \ttitle\par}\vspace{0.4cm} % Thesis title
\HRule \\[1.5cm] % Horizontal line \HRule \\[1.5cm] % Horizontal line
\begin{minipage}[t]{0.4\textwidth}
\begin{flushleft} \large
\emph{Author:}\\
\href{http://www.johnsmith.com}{\authorname} % Author name - remove the \href bracket to remove the link
\end{flushleft}
\end{minipage}
\begin{minipage}[t]{0.4\textwidth}
\begin{flushright} \large
\emph{Supervisor:} \\
\href{http://www.jamessmith.com}{\supname} % Supervisor name - remove the \href bracket to remove the link
\end{flushright}
\end{minipage}\\[3cm]
\vfill
\large \textit{A thesis submitted in fulfillment of the requirements\\ for the degree of \degreename}\\[0.3cm] % University requirement text \begin{minipage}[t]{0.4\textwidth}
\textit{in the}\\[0.4cm] \begin{flushleft} \large
\groupname\\\deptname\\[2cm] % Research group name and department name \emph{Author:}\\
\href{http://www.johnsmith.com}{\authorname} % Author name -
\vfill % remove the \href bracket to remove the link
\end{flushleft}
\end{minipage}
\begin{minipage}[t]{0.4\textwidth}
\begin{flushright} \large
\emph{Supervisor:} \\
\href{http://www.jamessmith.com}{\supname} % Supervisor name
% - remove the \href bracket to remove the link
\end{flushright}
\end{minipage}\\[3cm]
{\large \today}\\[4cm] % Date \vfill
% \includegraphics{./TU-Berlin-Logo.png} % University/department logo - uncomment to place it
\large \textit{A thesis submitted in fulfillment of the
\vfill requirements\\ for the degree of \degreename}\\[0.3cm] %
\end{center} % University requirement text
\textit{in the}\\[0.4cm]
\groupname\\\deptname\\[2cm] % Research group name and department name
\vfill
{\large \today}\\[4cm] % Date
% \includegraphics{./TU-Berlin-Logo.png} % University/department
% logo - uncomment to place it
\vfill
\end{center}
\end{titlepage} \end{titlepage}
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
% DECLARATION PAGE % DECLARATION PAGE
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
\begin{declaration} \begin{declaration}
\addchaptertocentry{\authorshipname} % Add the declaration to the table of contents \addchaptertocentry{\authorshipname} % Add the declaration to the
\noindent I, \authorname, declare that this thesis titled, \enquote{\ttitle} and the work presented in it are my own. I confirm that: % table of contents
\noindent I, \authorname, declare that this thesis titled,
\enquote{\ttitle} and the work presented in it are my own. I confirm that:
\begin{itemize} \begin{itemize}
\item This work was done wholly or mainly while in candidature for a research degree at this University. \item This work was done wholly or mainly while in candidature
\item Where any part of this thesis has previously been submitted for a degree or any other qualification at this University or any other institution, this has been clearly stated. for a research degree at this University.
\item Where I have consulted the published work of others, this is always clearly attributed. \item Where any part of this thesis has previously been submitted
\item Where I have quoted from the work of others, the source is always given. With the exception of such quotations, this thesis is entirely my own work. for a degree or any other qualification at this University or
\item I have acknowledged all main sources of help. any other institution, this has been clearly stated.
\item Where the thesis is based on work done by myself jointly with others, I have made clear exactly what was done by others and what I have contributed myself.\\ \item Where I have consulted the published work of others, this
\end{itemize} is always clearly attributed.
\item Where I have quoted from the work of others, the source is
\noindent Signed:\\ always given. With the exception of such quotations, this
\rule[0.5em]{25em}{0.5pt} % This prints a line for the signature thesis is entirely my own work.
\item I have acknowledged all main sources of help.
\noindent Date:\\ \item Where the thesis is based on work done by myself jointly
\rule[0.5em]{25em}{0.5pt} % This prints a line to write the date with others, I have made clear exactly what was done by others
and what I have contributed myself.\\
\end{itemize}
\noindent Signed:\\
\rule[0.5em]{25em}{0.5pt} % This prints a line for the signature
\noindent Date:\\
\rule[0.5em]{25em}{0.5pt} % This prints a line to write the date
\end{declaration} \end{declaration}
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
% ABSTRACT PAGE % ABSTRACT PAGE
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
\begin{abstract} \begin{abstract}
\addchaptertocentry{\abstractname} % Add the abstract to the table of contents \addchaptertocentry{\abstractname} % Add the abstract to the table of contents
This thesis explores Clan, an open-source machine configuration management framework designed to provide a single source of truth in decentralized environments. This thesis explores Clan, an open-source machine configuration
Key to this investigation are the underlying Mesh VPN technologies, ZeroTier and Mycelium, which enable decentralized network connections, and "data-mesher," a decentralized domain name system. management framework
The study begins with an in-depth analysis of fault tolerance mechanisms in these technologies, evaluating how robust they are against node failures and network disruptions. designed to provide a single source of truth in peer-to-peer networks.
Next, scalability is examined through both theoretical models and practical implementations. Key to this investigation are the underlying peer-to-peer technologies,
Finally, the security of these technologies is evaluated through various attack scenarios. ZeroTier and Mycelium, which enable decentralized network connections,
By examining fault tolerance, scalability, and security, this thesis aims to evaluate how Clan and these supporting technologies contribute to the management of distributed systems. and the "Data-Mesher" a decentralized conflict free replicated database.
The study begins with an in-depth analysis of fault tolerance
mechanisms in these technologies,
evaluating how robust they are against node failures and network disruptions.
Next, scalability is examined through both theoretical models and
practical implementations.
Finally, the security of these technologies is evaluated through
various attack scenarios.
By examining fault tolerance, scalability, and security, this
thesis aims to evaluate how Clan
and these supporting technologies contribute to the management of
peer-to-peer systems.
\end{abstract} \end{abstract}
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
% ACKNOWLEDGEMENTS % ACKNOWLEDGEMENTS
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
\begin{acknowledgements} \begin{acknowledgements}
\addchaptertocentry{\acknowledgementname} % Add the acknowledgements to the table of contents \addchaptertocentry{\acknowledgementname} % Add the
% acknowledgements to the table of contents
I am very grateful to my supervisor, Ber Lorke, for his guidance and support during my research. His advice and feedback have been invaluable.
I also want to thank my examiner, Prof. Stefan Schmidt, for his helpful comments and insights, which improved my thesis.
A big thank you to my family and friends for their support and encouragement throughout this journey.
Lastly, thanks to everyone at INET for creating a great learning environment.
I am very grateful to my team members Mic92, Lassulus, Hsjobeki,
DavHau, Kenji and Timo with whom
I worked together with to create the Clan framework.
As well as my supervisor, Ber Lorke, for his guidance and support
during my research.
His advice and feedback have been invaluable.
I also want to thank my examiner, Prof. Stefan Schmidt, for his
helpful comments and insights.
A big thank you to my family and friends for their support and
encouragement throughout this journey.
Lastly, thanks to everyone at INET for creating a great learning environment.
\end{acknowledgements} \end{acknowledgements}
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
% LIST OF CONTENTS/FIGURES/TABLES PAGES % LIST OF CONTENTS/FIGURES/TABLES PAGES
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
\tableofcontents % Prints the main table of contents \tableofcontents % Prints the main table of contents
@@ -203,23 +266,26 @@ Lastly, thanks to everyone at INET for creating a great learning environment.
\listoftables % Prints the list of tables \listoftables % Prints the list of tables
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
% ABBREVIATIONS % ABBREVIATIONS
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
\begin{abbreviations}{ll} % Include a list of abbreviations (a table of two columns) \begin{abbreviations}{ll} % Include a list of abbreviations (a table
% of two columns)
\textbf{LAH} & \textbf{L}ist \textbf{A}bbreviations \textbf{H}ere\\ \textbf{LAH} & \textbf{L}ist \textbf{A}bbreviations \textbf{H}ere\\
\textbf{WSF} & \textbf{W}hat (it) \textbf{S}tands \textbf{F}or\\ \textbf{WSF} & \textbf{W}hat (it) \textbf{S}tands \textbf{F}or\\
\end{abbreviations} \end{abbreviations}
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
% PHYSICAL CONSTANTS/OTHER DEFINITIONS % PHYSICAL CONSTANTS/OTHER DEFINITIONS
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
% \begin{constants}{lr@{${}={}$}l} % The list of physical constants is a three column table % \begin{constants}{lr@{${}={}$}l} % The list of physical constants
% is a three column table
% % The \SI{}{} command is provided by the siunitx package, see its documentation for instructions on how to use it % % The \SI{}{} command is provided by the siunitx package, see its
% documentation for instructions on how to use it
% % Speed of Light & $c_{0}$ & \SI{2.99792458e8}{\meter\per\second} (exact)\\ % % Speed of Light & $c_{0}$ & \SI{2.99792458e8}{\meter\per\second} (exact)\\
% %Constant Name & $Symbol$ & $Constant Value$ with units\\ % %Constant Name & $Symbol$ & $Constant Value$ with units\\
@@ -227,29 +293,29 @@ Lastly, thanks to everyone at INET for creating a great learning environment.
% \end{constants} % \end{constants}
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
% SYMBOLS % SYMBOLS
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
\begin{symbols}{lll} % Include a list of Symbols (a three column table) \begin{symbols}{lll} % Include a list of Symbols (a three column table)
% $a$ & distance & \si{\meter} \\ % $a$ & distance & \si{\meter} \\
% $P$ & power & \si{\watt} (\si{\joule\per\second}) \\ % $P$ & power & \si{\watt} (\si{\joule\per\second}) \\
%Symbol & Name & Unit \\ %Symbol & Name & Unit \\
\addlinespace % Gap to separate the Roman symbols from the Greek \addlinespace % Gap to separate the Roman symbols from the Greek
% $\omega$ & angular frequency & \si{\radian} \\ % $\omega$ & angular frequency & \si{\radian} \\
\end{symbols} \end{symbols}
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
% DEDICATION % DEDICATION
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
% \dedicatory{Dedicated to/To my\ldots} % \dedicatory{Dedicated to/To my\ldots}
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
% THESIS CONTENT - CHAPTERS % THESIS CONTENT - CHAPTERS
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
\mainmatter % Begin numeric (1,2,3...) page numbering \mainmatter % Begin numeric (1,2,3...) page numbering
@@ -261,18 +327,19 @@ Lastly, thanks to everyone at INET for creating a great learning environment.
\include{Chapters/Introduction} \include{Chapters/Introduction}
%\include{Chapters/Chapter1} %\include{Chapters/Chapter1}
%\include{Chapters/Chapter2} %\include{Chapters/Chapter2}
%\include{Chapters/Chapter3} %\include{Chapters/Chapter3}
%\include{Chapters/Chapter4} %\include{Chapters/Chapter4}
%\include{Chapters/Chapter5} %\include{Chapters/Chapter5}
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
% THESIS CONTENT - APPENDICES % THESIS CONTENT - APPENDICES
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
\appendix % Cue to tell LaTeX that the following "chapters" are Appendices \appendix % Cue to tell LaTeX that the following "chapters" are Appendices
% Include the appendices of the thesis as separate files from the Appendices folder % Include the appendices of the thesis as separate files from the
% Appendices folder
% Uncomment the lines as you write the Appendices % Uncomment the lines as you write the Appendices
%\include{Appendices/AppendixA} %\include{Appendices/AppendixA}
@@ -280,11 +347,11 @@ Lastly, thanks to everyone at INET for creating a great learning environment.
%\include{Appendices/AppendixC} %\include{Appendices/AppendixC}
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
% BIBLIOGRAPHY % BIBLIOGRAPHY
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
\printbibliography[heading=bibintoc] \printbibliography[heading=bibintoc]
%---------------------------------------------------------------------------------------- %----------------------------------------------------------------------------------------
\end{document} \end{document}

291
master_citations.bib
View File

@@ -1,18 +1,289 @@
@article{kjorveziroski_full-mesh_2024, @article{kjorveziroski_full-mesh_2024,
title = {Full-mesh {VPN} performance evaluation for a secure edge-cloud continuum}, title = {Full-mesh {VPN} performance evaluation for a secure edge-cloud continuum},
volume = {54}, volume = {54},
copyright = {© 2024 The Authors. Software: Practice and Experience published by John Wiley \& Sons Ltd.}, rights = {© 2024 The Authors. Software: Practice and Experience published by John Wiley \& Sons Ltd.},
issn = {1097-024X}, issn = {1097-024X},
url = {https://onlinelibrary.wiley.com/doi/abs/10.1002/spe.3329}, url = {https://onlinelibrary.wiley.com/doi/abs/10.1002/spe.3329},
doi = {10.1002/spe.3329}, doi = {10.1002/spe.3329},
abstract = {The recent introduction of full-mesh virtual private network (VPN) solutions which offer near native performance, coupled with modern encryption algorithms and easy scalability as a result of a central control plane have a strong potential to enable the implementation of a seamless edge-cloud continuum. To test the performance of existing solutions in this domain, we present a framework consisted of both essential and optional features that full-mesh VPN solutions need to support before they can be used for interconnecting geographically dispersed compute nodes. We then apply this framework on existing offerings and select three VPN solutions for further tests: Headscale, Netbird, and ZeroTier. We evaluate their features in the context of establishing an underlay network on top of which a Kubernetes overlay network can be created. We test pod-to-pod TCP and UDP throughput as well as Kubernetes application programming interface (API) response times, in multiple scenarios, accounting for adverse network conditions such as packet loss or packet delay. Based on the obtained measurement results and through analysis of the underlying strengths and weaknesses of the individual implementations, we draw conclusions on the preferred VPN solution depending on the use-case at hand, striking a balance between usability and performance.}, abstract = {The recent introduction of full-mesh virtual private network ({VPN}) solutions which offer near native performance, coupled with modern encryption algorithms and easy scalability as a result of a central control plane have a strong potential to enable the implementation of a seamless edge-cloud continuum. To test the performance of existing solutions in this domain, we present a framework consisted of both essential and optional features that full-mesh {VPN} solutions need to support before they can be used for interconnecting geographically dispersed compute nodes. We then apply this framework on existing offerings and select three {VPN} solutions for further tests: Headscale, Netbird, and {ZeroTier}. We evaluate their features in the context of establishing an underlay network on top of which a Kubernetes overlay network can be created. We test pod-to-pod {TCP} and {UDP} throughput as well as Kubernetes application programming interface ({API}) response times, in multiple scenarios, accounting for adverse network conditions such as packet loss or packet delay. Based on the obtained measurement results and through analysis of the underlying strengths and weaknesses of the individual implementations, we draw conclusions on the preferred {VPN} solution depending on the use-case at hand, striking a balance between usability and performance.},
language = {en},
number = {8},
urldate = {2024-09-19},
journal = {Software: Practice and Experience},
author = {Kjorveziroski, Vojdan and Bernad, Cristina and Gilly, Katja and Filiposka, Sonja},
year = {2024},
keywords = {edge-cloud continuum, Kubernetes, orchestration, virtual private networks, Wireguard, ZeroTier},
pages = {1543--1564}, pages = {1543--1564},
file = {Full Text PDF:https\://onlinelibrary.wiley.com/doi/pdfdirect/10.1002/spe.3329:application/pdf}, number = {8},
journaltitle = {Software: Practice and Experience},
author = {Kjorveziroski, Vojdan and Bernad, Cristina and Gilly, Katja and Filiposka, Sonja},
urldate = {2024-09-19},
date = {2024},
langid = {english},
keywords = {edge-cloud continuum, Kubernetes, orchestration, virtual private networks, Wireguard, {ZeroTier}},
file = {Attachment:/home/lhebendanz/Zotero/storage/BGJCY48V/Softw Pract Exp - 2024 - Kjorveziroski - Full‐mesh VPN performance evaluation for a secure edge‐cloud continuum.pdf:application/pdf},
}
@inproceedings{hugerich_no-hop_2022,
location = {New York, {NY}, {USA}},
title = {No-hop: In-network Distributed Hash Tables},
isbn = {978-1-4503-9168-9},
url = {https://doi.org/10.1145/3493425.3502757},
doi = {10.1145/3493425.3502757},
series = {{ANCS} '21},
shorttitle = {No-hop},
abstract = {We make a case for a distributed hash table lookup in the network data plane. We argue that the lookup time performance of distributed hash tables can be further improved via an in-network data plane implementation. To this end, we introduce No-hop, an in-network distributed hash table implementation, which leverages the data plane programmability at line rate gained from P4. Our initial results of transporting distributed hash table logic from hosts' user space to the fast path of switches in the network data plane are promising. We show that No-hop improves the performance of locating the responsible host and maintains the properties of distributed hash tables while outperforming two baselines.},
pages = {80--87},
booktitle = {Proceedings of the Symposium on Architectures for Networking and Communications Systems},
publisher = {Association for Computing Machinery},
author = {Hügerich, Lily and Shukla, Apoorv and Smaragdakis, Georgios},
urldate = {2024-09-23},
date = {2022-01},
file = {Attachment:/home/lhebendanz/Zotero/storage/WCI9PCTE/inet_nohop_decen_hashtable.pdf:application/pdf},
}
@inproceedings{tiesel_multi-homed_2016,
location = {New York, {NY}, {USA}},
title = {Multi-Homed on a Single Link: Using Multiple {IPv}6 Access Networks},
isbn = {978-1-4503-4443-2},
url = {https://doi.org/10.1145/2959424.2959434},
doi = {10.1145/2959424.2959434},
series = {{ANRW} '16},
shorttitle = {Multi-Homed on a Single Link},
abstract = {Small companies and branch offices often have bandwidth demands and redundancy needs that go beyond the commercially available Internet access products in their price range. One way to overcome this problem is to bundle existing Internet access products. In effect, they become multi-homed often without running {BGP} or even getting an {AS} number.Currently, these users rely on proprietary L4 load balancing routers, proprietary multi-channel {VPN} routers, or sometimes {LISP}, to bundle their "cheaper" Internet access network links, e.g., via (v){DSL}, {DOCSIS}, {HSDPA}, or {LTE}. While most products claim transport-layer transparency they add complexity via middleboxes, map each {TCP} connection to a single interface, and have limited application support. Thus, in this paper we propose an alternative: Auto-configuration of multiple {IPv}6 prefixes on a single L2 link. We discuss how this enables applications to take advantage of combining multiple access networks at with minimal system changes.},
pages = {16--18},
booktitle = {Proceedings of the 2016 Applied Networking Research Workshop},
publisher = {Association for Computing Machinery},
author = {Tiesel, Philipp S. and May, Bernd and Feldmann, Anja},
urldate = {2024-09-23},
date = {2016-07},
file = {Attachment:/home/lhebendanz/Zotero/storage/W44Z4XEE/inet_ipv6_vpn.pdf:application/pdf},
}
@report{shukla_navigating_2019,
location = {Rochester, {NY}},
title = {Navigating the Landscape of Programmable Networks: Looking beyond the Regulatory Status Quo},
url = {https://papers.ssrn.com/abstract=3427455},
shorttitle = {Navigating the Landscape of Programmable Networks},
abstract = {Digital transition processes are transforming industries, economies, and societies. On the one hand, the number of connected end-users and things is growing rapidly. On the other hand, the range of applications is continuously evolving. As applications like {IoT}, the Tactile Internet, or Immersive User Interfaces gain traction, the demands for traffic delivery, privacy, and security become highly diverse and complex. In meeting these demands, programmable networks gain importance. They introduce a new networking paradigm and provide an evolved spectrum of capabilities. In this paper, we conduct a cross-disciplinary study to navigate the landscape of programmable networks. We provide a comprehensive overview of the evolution and the growing complexity of this new networking paradigm and, based on a horizon scanning approach, elucidate associated opportunities and challenges. From this, we derive and discuss insights for the design of appropriate policies and regulatory frameworks for the future Internet.},
number = {3427455},
author = {Shukla, Apoorv and Stocker, Volker},
urldate = {2024-09-23},
date = {2019-07},
langid = {english},
doi = {10.2139/ssrn.3427455},
keywords = {P4, Policy, Programmable Networks, Regulations, {SDN}},
file = {PDF:/home/lhebendanz/Zotero/storage/B768VLZ3/Shukla and Stocker - 2019 - Navigating the Landscape of Programmable Networks Looking beyond the Regulatory Status Quo.pdf:application/pdf},
}
@report{moghaddam_decentralized_2015,
title = {A Decentralized Approach to Software-Defined Networks ({SDNs})},
url = {http://arxiv.org/abs/1504.07933},
abstract = {Redistribution of the intelligence and management in the software defined networks ({SDNs}) is a potential approach to address the bottlenecks of scalability and integrity of these networks. We propose to revisit the routing concept based on the notion of regions. Using basic and consistent definition of regions, a region-based packet routing called {SmartRegion} Routing is presented. The flexibility of regions in terms of naming and addressing is then leveraged in the form of a region stack among other features placed in the associated packet header. In this way, most of complexity and dynamicity of a network is absorbed, and therefore highly fast and simplified routing at the inter-region level along with semi-autonomous intra-region routing will be feasible. In addition, multipath planning can be naturally realized at both inter and intra levels. A basic form of {SmartRegion} routing mechanism is provided. Simplicity, scalability, and manageability of the proposed approach would also bring future potentials to reduce energy consumption and environmental footprint associated to the {SDNs}. Finally, various applications, such as enabling seamless broadband access, providing beyond {IP} addressing mechanisms, and also address-equivalent naming mechanisms, are considered and discussed.},
institution = {{arXiv}},
author = {Moghaddam, Reza Farrahi and Lemieux, Yves and Cheriet, Mohamed},
urldate = {2024-09-23},
date = {2015-04},
doi = {10.48550/arXiv.1504.07933},
keywords = {Computer Science - Networking and Internet Architecture},
file = {Attachment:/home/lhebendanz/Zotero/storage/SENCWZZ4/decent_sdn.pdf:application/pdf},
}
@article{bakhshi_state_2017,
title = {State of the Art and Recent Research Advances in Software Defined Networking},
volume = {2017},
rights = {Copyright © 2017 Taimur Bakhshi.},
issn = {1530-8677},
url = {https://onlinelibrary.wiley.com/doi/abs/10.1155/2017/7191647},
doi = {10.1155/2017/7191647},
abstract = {Emerging network services and subsequent growth in the networking infrastructure have gained tremendous momentum in recent years. Application performance requiring rapid real-time network provisioning, optimized traffic management, and virtualization of shared resources has induced the conceptualization and adoption of new networking models. Software defined networking ({SDN}), one of the predominant and relatively new networking paradigms, seeks to simplify network management by decoupling network control logic from the underlying hardware and introduces real-time network programmability enabling innovation. The present work reviews the state of the art in software defined networking providing a historical perspective on complementary technologies in network programmability and the inherent shortcomings which paved the way for {SDN}. The {SDN} architecture is discussed along with popular protocols, platforms, and existing simulation and debugging solutions. Furthermore, a detailed analysis is presented around recent {SDN} development and deployment avenues ranging from mobile communications and data centers to campus networks and residential environments. The review concludes by highlighting implementation challenges and subsequent research directions being pursued in academia and industry to address issues related to application performance, control plane scalability and design, security, and interdomain connectivity in the context of {SDN}.},
pages = {7191647},
number = {1},
journaltitle = {Wireless Communications and Mobile Computing},
author = {Bakhshi, Taimur},
urldate = {2024-09-23},
date = {2017},
langid = {english},
file = {Attachment:/home/lhebendanz/Zotero/storage/TXFJ8DJB/Wireless Communications and Mobile Computing - 2017 - Bakhshi - State of the Art and Recent Research Advances in Software.pdf:application/pdf},
}
@article{han_distributed_2015,
title = {Distributed hybrid P2P networking systems},
volume = {8},
issn = {1936-6450},
url = {https://doi.org/10.1007/s12083-014-0298-7},
doi = {10.1007/s12083-014-0298-7},
pages = {555--556},
number = {4},
journaltitle = {Peer-to-Peer Netw. Appl.},
author = {Han, Jungsoo},
urldate = {2024-11-19},
date = {2015-07-01},
langid = {english},
file = {Full Text PDF:/home/lhebendanz/Zotero/storage/XVFPW4CM/Han - 2015 - Distributed hybrid P2P networking systems.pdf:application/pdf},
}
@online{noauthor_sci-hub_nodate,
title = {Sci-Hub},
url = {https://sci-hub.usualwant.com/},
abstract = {Sci-Hub,{SciHub}: Contains the latest literature in 2024,world where everyone has free access to knowledge.},
urldate = {2024-11-19},
file = {Snapshot:/home/lhebendanz/Zotero/storage/Q6ZNWGBH/sci-hub.usualwant.com.html:text/html},
}
@article{shukla_towards_2021,
title = {Towards software defined low maintenance structured peer-to-peer overlays},
volume = {14},
issn = {1936-6442, 1936-6450},
url = {https://link.springer.com/10.1007/s12083-021-01112-7},
doi = {10.1007/s12083-021-01112-7},
pages = {1242--1260},
number = {3},
journaltitle = {Peer-to-Peer Netw. Appl.},
author = {Shukla, Nitin and Datta, Dipmalya and Pandey, Mayank and Srivastava, Shashank},
urldate = {2024-11-19},
date = {2021-05},
langid = {english},
file = {PDF:/home/lhebendanz/Zotero/storage/GTPSJRYK/Shukla et al. - 2021 - Towards software defined low maintenance structured peer-to-peer overlays.pdf:application/pdf},
}
@article{naik_next_2020,
title = {Next level peer-to-peer overlay networks under high churns: a survey},
volume = {13},
issn = {1936-6442, 1936-6450},
url = {http://link.springer.com/10.1007/s12083-019-00839-8},
doi = {10.1007/s12083-019-00839-8},
shorttitle = {Next level peer-to-peer overlay networks under high churns},
pages = {905--931},
number = {3},
journaltitle = {Peer-to-Peer Netw. Appl.},
author = {Naik, Ashika R. and Keshavamurthy, Bettahally N.},
urldate = {2024-11-19},
date = {2020-05},
langid = {english},
file = {PDF:/home/lhebendanz/Zotero/storage/PWMXVDES/Naik and Keshavamurthy - 2020 - Next level peer-to-peer overlay networks under high churns a survey.pdf:application/pdf},
}
@article{dolstra_nix_nodate,
title = {Nix: A Safe and Policy-Free System for Software Deployment},
abstract = {Existing systems for software deployment are neither safe nor sufficiently flexible. Primary safety issues are the inability to enforce reliable specification of component dependencies, and the lack of support for multiple versions or variants of a component. This renders deployment operations such as upgrading or deleting components dangerous and unpredictable. A deployment system must also be flexible (i.e., policy-free) enough to support both centralised and local package management, and to allow a variety of mechanisms for transferring components. In this paper we present Nix, a deployment system that addresses these issues through a simple technique of using cryptographic hashes to compute unique paths for component instances.},
author = {Dolstra, Eelco and de Jonge, Merijn and Visser, Eelco},
langid = {english},
file = {PDF:/home/lhebendanz/Zotero/storage/FVVZ628U/Dolstra et al. - Nix A Safe and Policy-Free System for Software Deployment.pdf:application/pdf},
}
@inproceedings{guilloteau_painless_2022,
location = {Heidelberg, Germany},
title = {Painless Transposition of Reproducible Distributed Environments with {NixOS} Compose},
rights = {https://doi.org/10.15223/policy-029},
isbn = {978-1-66549-856-2},
url = {https://ieeexplore.ieee.org/document/9912715/},
doi = {10.1109/CLUSTER51413.2022.00051},
abstract = {Development of environments for distributed systems is a tedious and time-consuming iterative process. The reproducibility of such environments is a crucial factor for rigorous scientific contributions. We think that being able to smoothly test environments both locally and on a target distributed platform makes development cycles faster and reduces the friction to adopt better experimental practices. To address this issue, this paper introduces the notion of environment transposition and implements it in {NixOS} Compose, a tool that generates reproducible distributed environments. It enables users to deploy their environments on virtualized (docker, {QEMU}) or physical (Grid’5000) platforms with the same unique description of the environment. We show that {NixOS} Compose enables to build reproducible environments without overhead by comparing it to state-of-the-art solutions for the generation of distributed environments ({EnOSlib} and Kameleon). {NixOS} Compose actually enables substantial performance improvements on image building time over Kameleon (up to 11x faster for initial builds and up to 19x faster when building a variation of an existing environment).},
eventtitle = {2022 {IEEE} International Conference on Cluster Computing ({CLUSTER})},
pages = {1--12},
booktitle = {2022 {IEEE} International Conference on Cluster Computing ({CLUSTER})},
publisher = {{IEEE}},
author = {Guilloteau, Quentin and Bleuzen, Jonathan and Poquet, Millian and Richard, Olivier},
urldate = {2024-11-24},
date = {2022-09},
langid = {english},
file = {PDF:/home/lhebendanz/Zotero/storage/SEEITEJA/Guilloteau et al. - 2022 - Painless Transposition of Reproducible Distributed Environments with NixOS Compose.pdf:application/pdf},
}
@inproceedings{dolstra_nixos_2008,
location = {New York, {NY}, {USA}},
title = {{NixOS}: a purely functional Linux distribution},
isbn = {978-1-59593-919-7},
url = {https://doi.org/10.1145/1411204.1411255},
doi = {10.1145/1411204.1411255},
series = {{ICFP} '08},
shorttitle = {{NixOS}},
abstract = {Existing package and system configuration management tools suffer from an imperative model, where system administration actions such as upgrading packages or changes to system configuration files are stateful: they destructively update the state of the system. This leads to many problems, such as the inability to roll back changes easily, to run multiple versions of a package side-by-side, to reproduce a configuration deterministically on another machine, or to reliably upgrade a system. In this paper we show that we can overcome these problems by moving to a purely functional system configuration model. This means that all static parts of a system (such as software packages, configuration files and system startup scripts) are built by pure functions and are immutable, stored in a way analogously to a heap in a purely function language. We have implemented this model in {NixOS}, a non-trivial Linux distribution that uses the Nix package manager to build the entire system configuration from a purely functional specification.},
pages = {367--378},
booktitle = {Proceedings of the 13th {ACM} {SIGPLAN} international conference on Functional programming},
publisher = {Association for Computing Machinery},
author = {Dolstra, Eelco and Löh, Andres},
urldate = {2024-11-24},
date = {2008-09-20},
file = {PDF:/home/lhebendanz/Zotero/storage/GGG3BB3V/Dolstra et al. - 2010 - NixOS A Purely Functional Linux Distribution.pdf:application/pdf},
}
@inproceedings{zhang_towards_2023,
location = {Birmingham United Kingdom},
title = {Towards Dynamic and Reliable Private Key Management for Hierarchical Access Structure in Decentralized Storage},
isbn = {9798400701245},
url = {https://dl.acm.org/doi/10.1145/3583780.3615090},
doi = {10.1145/3583780.3615090},
abstract = {With the widespread development of decentralized storage, it is increasingly popular for users to store their data to the decentralized database systems for the well-understood benefits of outsourced storage. To ensure the data privacy, systems commonly require users to securely keep their private keys. Thus, the secure storage of private keys is an important issue in these systems. However, existing key-management schemes commonly rely on a Trusted Third Party ({TTP}), which raises critical security concerns such as the single point of failure and Distributed Denial of Service ({DDoS}) attacks. In this paper, we propose {HasDPSS}, a secure and efficient blockchain-based key-management scheme for decentralized storage systems. It uses secret sharing, a lightweight cryptographic technique, to build the decentralized key-management scheme. Considering that the reliability of managing participants has inherent heterogeneity, we introduce the hierarchical access structure to achieve fine-grained key management. Meanwhile, to adapt the node churn of decentralized key management, {HasDPSS} enables a dynamic management committee to provide reliable services with a proactive refresh mechanism while protecting the integrity and security of private keys. In our design, we use the dimension switch method of polynomials in the evolving process to achieve the committee change of the hierarchical access structure. The reliability of participants is guaranteed by the customized commitment protocol and the immutable property of the blockchain. We thoroughly analyze security strengths and conduct extensive experiments to demonstrate the practicality of our design.},
eventtitle = {{CIKM} '23: The 32nd {ACM} International Conference on Information and Knowledge Management},
pages = {3371--3380},
booktitle = {Proceedings of the 32nd {ACM} International Conference on Information and Knowledge Management},
publisher = {{ACM}},
author = {Zhang, Yifang and Wang, Mingyue and Guo, Yu and Guo, Fangda},
urldate = {2024-11-24},
date = {2023-10-21},
langid = {english},
file = {PDF:/home/lhebendanz/Zotero/storage/WI8JTTIS/Zhang et al. - 2023 - Towards Dynamic and Reliable Private Key Management for Hierarchical Access Structure in Decentraliz.pdf:application/pdf},
}
@inproceedings{ryeng_robust_2008,
location = {New York, {NY}, {USA}},
title = {Robust aggregation in peer-to-peer database systems},
isbn = {978-1-60558-188-0},
url = {https://doi.org/10.1145/1451940.1451946},
doi = {10.1145/1451940.1451946},
series = {{IDEAS} '08},
abstract = {Peer-to-peer database systems (P2PDBs) aim at providing database services with node autonomy, high availability and loose coupling between participating nodes by building the {DBMS} on top of a peer-to-peer network. A key feature of current peer-to-peer systems is resilience to churn in the overlay network layer. A major challenge in P2PDBs is to provide similar robustness in the data and query processing layer. In this paper we in particular describe how aggregation queries in P2PDBs can be handled in order to reduce the impact of churn on accuracy of results. We perform a formal study of data loss and accuracy of such queries, and describe new approaches that increase the accuracy of aggregation queries in P2PDBs under churn.},
pages = {29--37},
booktitle = {Proceedings of the 2008 international symposium on Database engineering \& applications},
publisher = {Association for Computing Machinery},
author = {Ryeng, Norvald H. and Nørvåg, Kjetil},
urldate = {2024-11-24},
date = {2008-09-10},
file = {PDF:/home/lhebendanz/Zotero/storage/3KC9AG7C/Ryeng and Nørvåg - 2008 - Robust aggregation in peer-to-peer database systems.pdf:application/pdf},
}
@article{tatarinov_piazza_2003,
title = {The Piazza peer data management project},
volume = {32},
issn = {0163-5808},
url = {https://doi.org/10.1145/945721.945732},
doi = {10.1145/945721.945732},
abstract = {A major problem in today's information-driven world is that sharing heterogeneous, semantically rich data is incredibly difficult. Piazza is a peer data management system that enables sharing heterogeneous data in a distributed and scalable way. Piazza assumes the participants to be interested in sharing data, and willing to define pairwise mappings between their schemas. Then, users formulate queries over their preferred schema, and a query answering system expands recursively any mappings relevant to the query, retrieving data from other peers. In this paper, we provide a brief overview of the Piazza project including our work on developing mapping languages and query reformulation algorithms, assisting the users in defining mappings, indexing, and enforcing access control over shared data.},
pages = {47--52},
number = {3},
journaltitle = {{SIGMOD} Rec.},
author = {Tatarinov, Igor and Ives, Zachary and Madhavan, Jayant and Halevy, Alon and Suciu, Dan and Dalvi, Nilesh and Dong, Xin (Luna) and Kadiyska, Yana and Miklau, Gerome and Mork, Peter},
urldate = {2024-11-24},
date = {2003-09-01},
file = {PDF:/home/lhebendanz/Zotero/storage/MRK3XWJG/Tatarinov et al. - 2003 - The Piazza peer data management project.pdf:application/pdf},
}
@article{van_der_burg_disnix_2014,
title = {Disnix: A toolset for distributed deployment},
volume = {79},
issn = {0167-6423},
url = {https://www.sciencedirect.com/science/article/pii/S0167642312000639},
doi = {10.1016/j.scico.2012.03.006},
series = {Experimental Software and Toolkits ({EST} 4): A special issue of the Workshop on Academic Software Development Tools and Techniques ({WASDeTT}-3 2010)},
shorttitle = {Disnix},
abstract = {The process of deploying a distributed system in a network of machines is often very complex, laborious and time-consuming, while it is hard to guarantee that the system will work as expected and that specific non-functional deployment requirements from the domain are supported. In this paper we describe the Disnix toolset, which provides system administrators or developers with automatic deployment of a distributed system in a network of machines from declarative specifications and offers properties such as complete dependencies, atomic upgrades and rollbacks to make this process efficient and reliable. Disnix has an extensible architecture, allowing the integration of custom modules to make the deployment more convenient and suitable for the domain in which the system is to be used. Disnix has been under development for almost four years and has been applied to several types of distributed systems, including an industrial case study.},
pages = {52--69},
journaltitle = {Science of Computer Programming},
author = {van der Burg, Sander and Dolstra, Eelco},
urldate = {2024-11-24},
date = {2014-01-01},
keywords = {Distributed systems, Service-oriented systems, Software deployment},
file = {PDF:/home/lhebendanz/Zotero/storage/XSWZ95UU/van der Burg and Dolstra - 2014 - Disnix A toolset for distributed deployment.pdf:application/pdf;ScienceDirect Snapshot:/home/lhebendanz/Zotero/storage/VHPTLVMW/S0167642312000639.html:text/html},
}
@inproceedings{dolstra_charon_2013,
title = {Charon: Declarative provisioning and deployment},
url = {https://ieeexplore.ieee.org/abstract/document/6607691},
doi = {10.1109/RELENG.2013.6607691},
shorttitle = {Charon},
abstract = {We introduce Charon, a tool for automated provisioning and deployment of networks of machines from declarative specifications. Building upon {NixOS}, a Linux distribution with a purely functional configuration management model, Charon specifications completely describe the desired configuration of sets of “logical” machines, including all software packages and services that need to be present on those machines, as well as their desired “physical” characteristics. Given such specifications, Charon will provision cloud resources (such as Amazon {EC}2 instances) as required, build and deploy packages, and activate services. We argue why declarativity and integrated provisioning and configuration management are important properties, and describe our experience with Charon.},
eventtitle = {2013 1st International Workshop on Release Engineering ({RELENG})},
pages = {17--20},
booktitle = {2013 1st International Workshop on Release Engineering ({RELENG})},
author = {Dolstra, Eelco and Vermaas, Rob and Levy, Shea},
urldate = {2024-11-24},
date = {2013-05},
keywords = {Databases, {IP} networks, Linux, Production, Servers, Software, Testing},
file = {IEEE Xplore Abstract Record:/home/lhebendanz/Zotero/storage/LDFB982I/6607691.html:text/html},
} }

17
treefmt.nix Normal file
View File

@@ -0,0 +1,17 @@
{ inputs, ... }:
{
imports = [ inputs.treefmt-nix.flakeModule ];
perSystem =
{ ... }:
{
treefmt = {
# Used to find the project root
projectRootFile = "flake.nix";
programs.deadnix.enable = true;
programs.nixfmt.enable = true;
programs.shellcheck.enable = true;
programs.yamlfmt.enable = true;
};
};
}