added source code review to methodology section

This commit is contained in:
2026-02-24 19:03:31 +01:00
parent 2e2d8880c0
commit c08df6b885
3 changed files with 110 additions and 14 deletions

View File

@@ -2,9 +2,23 @@
\label{Introduction}
TODO
Peer-to-peer overlay VPNs promise to restore genuine decentralization
by enabling direct connectivity between nodes regardless of NAT or
firewall restrictions. Yet practitioners choosing among the growing
number of mesh VPN implementations must rely largely on anecdotal
evidence: systematic, reproducible comparisons under realistic
conditions are scarce.
\subsection{Motivation}
This thesis addresses that gap. We benchmark ten peer-to-peer VPN
implementations across seven workloads and four network impairment
profiles, yielding over 300 unique measurements. We complement these
performance benchmarks with a source code analysis of each
implementation, verified through direct engagement with the respective
maintainers. The entire experimental framework is built on Nix, NixOS,
and the Clan deployment system, making every result independently
reproducible.
\section{Motivation}
Peer-to-peer architectures promise censorship-resistant, fault-tolerant
infrastructure by eliminating single points of failure
@@ -176,13 +190,39 @@ in pursuit of top rankings.
\label{fig:vision-stages}
\end{figure}
\subsection{Research Contribution}
\section{Research Contribution}
TODO
This thesis makes the following contributions:
\subsection{Related Work}
\begin{enumerate}
\item A comprehensive benchmark of ten peer-to-peer VPN
implementations across seven workloads. Including real-world
video streaming and package downloads; and four network
impairment profiles, producing over 300 unique measurements.
\item A source code analysis of all ten VPN implementations,
combining manual code review with LLM-assisted analysis,
followed by verification through direct engagement with the
respective maintainers on GitHub.
\item A fully reproducible experimental framework built on
Nix, NixOS, and the Clan deployment system, with pinned
dependencies, declarative system configuration, and
deterministic cryptographic material generation, enabling
independent replication of all results.
\item A performance analysis demonstrating that Tailscale
outperforms the Linux kernel's default networking stack under
degraded conditions, and that kernel parameter tuning; Reno
congestion control in place of CUBIC, with RACK
disabled; yields measurable throughput improvements.
\item The discovery of several security vulnerabilities across
the evaluated VPN implementations.
\item An automated benchmarking framework designed for public
leaderboard generation, intended to encourage ongoing
optimization by VPN developers.
\end{enumerate}
\subsubsection{A Comparative Study on Virtual Private Networks}
\section{Related Work}
\subsection{A Comparative Study on Virtual Private Networks}
Lackorzynski et al.\ \cite{lackorzynski_comparative_2019} evaluate
VPN protocols in the context of industrial communication systems (Industry 4.0),
@@ -192,7 +232,7 @@ latency, and CPU overhead; rather than overlay network behavior.
In contrast, this thesis evaluates VPNs that provide a full data plane
with peer-to-peer connectivity, NAT traversal, and dynamic peer discovery.
\subsubsection{Full-Mesh VPN Performance Evaluation}
\subsection{Full-Mesh VPN Performance Evaluation}
Kjorveziroski et al.\ \cite{kjorveziroski_full-mesh_2024} provide a
comprehensive evaluation of full-mesh VPN solutions for distributed
@@ -210,5 +250,5 @@ This thesis extends their work in several ways:
\item Fully reproducible experimental framework via Nix/NixOS/Clan
\end{itemize}
\subsubsection{Performance Evaluation of TCP over QUIC Tunnels}
\subsection{Performance Evaluation of TCP over QUIC Tunnels}
TODO \cite{guo_implementation_2025}

View File

@@ -4,11 +4,14 @@
\label{Methodology}
This chapter describes the methodology used to benchmark peer-to-peer
mesh VPN implementations. The experimental design prioritizes
reproducibility at every layer; from dependency management to network
conditions; enabling independent verification of results and
facilitating future comparative studies.
This chapter describes the methodology used to benchmark and analyze
peer-to-peer mesh VPN implementations. The evaluation combines
performance benchmarking under controlled network conditions with a
structured source code analysis of each implementation. The
benchmarking framework prioritizes reproducibility at every layer;
from pinned dependencies and declarative system configuration to
automated test orchestration; enabling independent verification of
results and facilitating future comparative studies.
\section{Experimental Setup}
@@ -277,6 +280,60 @@ confidence intervals, which is appropriate for benchmark data that
may not follow a normal distribution. The nix-cache test (via hyperfine)
additionally reports standard deviation.
\section{Source Code Analysis}
To complement the performance benchmarks with architectural
understanding, a structured source code analysis was conducted for
all ten VPN implementations. The analysis followed three phases.
\subsection{Repository Collection and LLM-Assisted Overview}
The latest main branch of each VPN's git repository was cloned,
together with key dependencies that implement core functionality
outside the main repository. For example, Yggdrasil delegates its
routing and cryptographic operations to the Ironwood library, which
was analyzed alongside the main codebase.
Ten LLM agents (Claude Code) were then spawned in parallel, one per
VPN. Each agent was instructed to read the full source tree and
produce an \texttt{overview.md} file documenting the following
aspects:
\begin{itemize}
\item Wire protocol and message framing
\item Encryption scheme and key exchange
\item Packet handling and performance
\item NAT traversal mechanism
\item Local routing and peer discovery
\item Security features and access control
\item Resilience / Central Point of Failure
\end{itemize}
Every claim in the generated overview was required to reference the
specific file and line range in the repository that supports it,
enabling direct verification.
\subsection{Manual Verification}
The LLM-generated overviews served as a navigational aid rather than
a trusted source. The most important code paths identified in each
overview were manually read and verified against the actual source
code, correcting inaccuracies and deepening the analysis where the
automated summaries remained superficial.
\subsection{Feature Matrix and Maintainer Review}
The findings from both the automated and manual analysis were
consolidated into a comprehensive feature matrix cataloguing 131
features across all ten VPN implementations. The matrix covers
protocol characteristics, cryptographic primitives, NAT traversal
strategies, routing behavior, and security properties.
The completed feature matrix was published and sent to the respective
VPN maintainers for review. Maintainer feedback was incorporated as
corrections and clarifications, improving the accuracy of the final
classification.
\section{Reproducibility}
Reproducibility is ensured at every layer of the experimental stack.

View File

@@ -19,7 +19,6 @@ isolated from stateful interactions typical in imperative package
management. This property is essential for ensuring identical test
environments across benchmark runs.
\subsubsection{UDP NAT and Firewall Puncturing in the Wild}
Halkes and Pouwelse~\cite{halkes_udp_2011} measure UDP hole punching