added source code review to methodology section
This commit is contained in:
@@ -2,9 +2,23 @@
|
||||
|
||||
\label{Introduction}
|
||||
|
||||
TODO
|
||||
Peer-to-peer overlay VPNs promise to restore genuine decentralization
|
||||
by enabling direct connectivity between nodes regardless of NAT or
|
||||
firewall restrictions. Yet practitioners choosing among the growing
|
||||
number of mesh VPN implementations must rely largely on anecdotal
|
||||
evidence: systematic, reproducible comparisons under realistic
|
||||
conditions are scarce.
|
||||
|
||||
\subsection{Motivation}
|
||||
This thesis addresses that gap. We benchmark ten peer-to-peer VPN
|
||||
implementations across seven workloads and four network impairment
|
||||
profiles, yielding over 300 unique measurements. We complement these
|
||||
performance benchmarks with a source code analysis of each
|
||||
implementation, verified through direct engagement with the respective
|
||||
maintainers. The entire experimental framework is built on Nix, NixOS,
|
||||
and the Clan deployment system, making every result independently
|
||||
reproducible.
|
||||
|
||||
\section{Motivation}
|
||||
|
||||
Peer-to-peer architectures promise censorship-resistant, fault-tolerant
|
||||
infrastructure by eliminating single points of failure
|
||||
@@ -176,13 +190,39 @@ in pursuit of top rankings.
|
||||
\label{fig:vision-stages}
|
||||
\end{figure}
|
||||
|
||||
\subsection{Research Contribution}
|
||||
\section{Research Contribution}
|
||||
|
||||
TODO
|
||||
This thesis makes the following contributions:
|
||||
|
||||
\subsection{Related Work}
|
||||
\begin{enumerate}
|
||||
\item A comprehensive benchmark of ten peer-to-peer VPN
|
||||
implementations across seven workloads. Including real-world
|
||||
video streaming and package downloads; and four network
|
||||
impairment profiles, producing over 300 unique measurements.
|
||||
\item A source code analysis of all ten VPN implementations,
|
||||
combining manual code review with LLM-assisted analysis,
|
||||
followed by verification through direct engagement with the
|
||||
respective maintainers on GitHub.
|
||||
\item A fully reproducible experimental framework built on
|
||||
Nix, NixOS, and the Clan deployment system, with pinned
|
||||
dependencies, declarative system configuration, and
|
||||
deterministic cryptographic material generation, enabling
|
||||
independent replication of all results.
|
||||
\item A performance analysis demonstrating that Tailscale
|
||||
outperforms the Linux kernel's default networking stack under
|
||||
degraded conditions, and that kernel parameter tuning; Reno
|
||||
congestion control in place of CUBIC, with RACK
|
||||
disabled; yields measurable throughput improvements.
|
||||
\item The discovery of several security vulnerabilities across
|
||||
the evaluated VPN implementations.
|
||||
\item An automated benchmarking framework designed for public
|
||||
leaderboard generation, intended to encourage ongoing
|
||||
optimization by VPN developers.
|
||||
\end{enumerate}
|
||||
|
||||
\subsubsection{A Comparative Study on Virtual Private Networks}
|
||||
\section{Related Work}
|
||||
|
||||
\subsection{A Comparative Study on Virtual Private Networks}
|
||||
|
||||
Lackorzynski et al.\ \cite{lackorzynski_comparative_2019} evaluate
|
||||
VPN protocols in the context of industrial communication systems (Industry 4.0),
|
||||
@@ -192,7 +232,7 @@ latency, and CPU overhead; rather than overlay network behavior.
|
||||
In contrast, this thesis evaluates VPNs that provide a full data plane
|
||||
with peer-to-peer connectivity, NAT traversal, and dynamic peer discovery.
|
||||
|
||||
\subsubsection{Full-Mesh VPN Performance Evaluation}
|
||||
\subsection{Full-Mesh VPN Performance Evaluation}
|
||||
|
||||
Kjorveziroski et al.\ \cite{kjorveziroski_full-mesh_2024} provide a
|
||||
comprehensive evaluation of full-mesh VPN solutions for distributed
|
||||
@@ -210,5 +250,5 @@ This thesis extends their work in several ways:
|
||||
\item Fully reproducible experimental framework via Nix/NixOS/Clan
|
||||
\end{itemize}
|
||||
|
||||
\subsubsection{Performance Evaluation of TCP over QUIC Tunnels}
|
||||
\subsection{Performance Evaluation of TCP over QUIC Tunnels}
|
||||
TODO \cite{guo_implementation_2025}
|
||||
|
||||
@@ -4,11 +4,14 @@
|
||||
|
||||
\label{Methodology}
|
||||
|
||||
This chapter describes the methodology used to benchmark peer-to-peer
|
||||
mesh VPN implementations. The experimental design prioritizes
|
||||
reproducibility at every layer; from dependency management to network
|
||||
conditions; enabling independent verification of results and
|
||||
facilitating future comparative studies.
|
||||
This chapter describes the methodology used to benchmark and analyze
|
||||
peer-to-peer mesh VPN implementations. The evaluation combines
|
||||
performance benchmarking under controlled network conditions with a
|
||||
structured source code analysis of each implementation. The
|
||||
benchmarking framework prioritizes reproducibility at every layer;
|
||||
from pinned dependencies and declarative system configuration to
|
||||
automated test orchestration; enabling independent verification of
|
||||
results and facilitating future comparative studies.
|
||||
|
||||
\section{Experimental Setup}
|
||||
|
||||
@@ -277,6 +280,60 @@ confidence intervals, which is appropriate for benchmark data that
|
||||
may not follow a normal distribution. The nix-cache test (via hyperfine)
|
||||
additionally reports standard deviation.
|
||||
|
||||
\section{Source Code Analysis}
|
||||
|
||||
To complement the performance benchmarks with architectural
|
||||
understanding, a structured source code analysis was conducted for
|
||||
all ten VPN implementations. The analysis followed three phases.
|
||||
|
||||
\subsection{Repository Collection and LLM-Assisted Overview}
|
||||
|
||||
The latest main branch of each VPN's git repository was cloned,
|
||||
together with key dependencies that implement core functionality
|
||||
outside the main repository. For example, Yggdrasil delegates its
|
||||
routing and cryptographic operations to the Ironwood library, which
|
||||
was analyzed alongside the main codebase.
|
||||
|
||||
Ten LLM agents (Claude Code) were then spawned in parallel, one per
|
||||
VPN. Each agent was instructed to read the full source tree and
|
||||
produce an \texttt{overview.md} file documenting the following
|
||||
aspects:
|
||||
|
||||
\begin{itemize}
|
||||
\item Wire protocol and message framing
|
||||
\item Encryption scheme and key exchange
|
||||
\item Packet handling and performance
|
||||
\item NAT traversal mechanism
|
||||
\item Local routing and peer discovery
|
||||
\item Security features and access control
|
||||
\item Resilience / Central Point of Failure
|
||||
\end{itemize}
|
||||
|
||||
Every claim in the generated overview was required to reference the
|
||||
specific file and line range in the repository that supports it,
|
||||
enabling direct verification.
|
||||
|
||||
\subsection{Manual Verification}
|
||||
|
||||
The LLM-generated overviews served as a navigational aid rather than
|
||||
a trusted source. The most important code paths identified in each
|
||||
overview were manually read and verified against the actual source
|
||||
code, correcting inaccuracies and deepening the analysis where the
|
||||
automated summaries remained superficial.
|
||||
|
||||
\subsection{Feature Matrix and Maintainer Review}
|
||||
|
||||
The findings from both the automated and manual analysis were
|
||||
consolidated into a comprehensive feature matrix cataloguing 131
|
||||
features across all ten VPN implementations. The matrix covers
|
||||
protocol characteristics, cryptographic primitives, NAT traversal
|
||||
strategies, routing behavior, and security properties.
|
||||
|
||||
The completed feature matrix was published and sent to the respective
|
||||
VPN maintainers for review. Maintainer feedback was incorporated as
|
||||
corrections and clarifications, improving the accuracy of the final
|
||||
classification.
|
||||
|
||||
\section{Reproducibility}
|
||||
|
||||
Reproducibility is ensured at every layer of the experimental stack.
|
||||
|
||||
@@ -19,7 +19,6 @@ isolated from stateful interactions typical in imperative package
|
||||
management. This property is essential for ensuring identical test
|
||||
environments across benchmark runs.
|
||||
|
||||
|
||||
\subsubsection{UDP NAT and Firewall Puncturing in the Wild}
|
||||
|
||||
Halkes and Pouwelse~\cite{halkes_udp_2011} measure UDP hole punching
|
||||
|
||||
Reference in New Issue
Block a user