added source code review to methodology section
This commit is contained in:
@@ -2,9 +2,23 @@
|
|||||||
|
|
||||||
\label{Introduction}
|
\label{Introduction}
|
||||||
|
|
||||||
TODO
|
Peer-to-peer overlay VPNs promise to restore genuine decentralization
|
||||||
|
by enabling direct connectivity between nodes regardless of NAT or
|
||||||
|
firewall restrictions. Yet practitioners choosing among the growing
|
||||||
|
number of mesh VPN implementations must rely largely on anecdotal
|
||||||
|
evidence: systematic, reproducible comparisons under realistic
|
||||||
|
conditions are scarce.
|
||||||
|
|
||||||
\subsection{Motivation}
|
This thesis addresses that gap. We benchmark ten peer-to-peer VPN
|
||||||
|
implementations across seven workloads and four network impairment
|
||||||
|
profiles, yielding over 300 unique measurements. We complement these
|
||||||
|
performance benchmarks with a source code analysis of each
|
||||||
|
implementation, verified through direct engagement with the respective
|
||||||
|
maintainers. The entire experimental framework is built on Nix, NixOS,
|
||||||
|
and the Clan deployment system, making every result independently
|
||||||
|
reproducible.
|
||||||
|
|
||||||
|
\section{Motivation}
|
||||||
|
|
||||||
Peer-to-peer architectures promise censorship-resistant, fault-tolerant
|
Peer-to-peer architectures promise censorship-resistant, fault-tolerant
|
||||||
infrastructure by eliminating single points of failure
|
infrastructure by eliminating single points of failure
|
||||||
@@ -176,13 +190,39 @@ in pursuit of top rankings.
|
|||||||
\label{fig:vision-stages}
|
\label{fig:vision-stages}
|
||||||
\end{figure}
|
\end{figure}
|
||||||
|
|
||||||
\subsection{Research Contribution}
|
\section{Research Contribution}
|
||||||
|
|
||||||
TODO
|
This thesis makes the following contributions:
|
||||||
|
|
||||||
\subsection{Related Work}
|
\begin{enumerate}
|
||||||
|
\item A comprehensive benchmark of ten peer-to-peer VPN
|
||||||
|
implementations across seven workloads. Including real-world
|
||||||
|
video streaming and package downloads; and four network
|
||||||
|
impairment profiles, producing over 300 unique measurements.
|
||||||
|
\item A source code analysis of all ten VPN implementations,
|
||||||
|
combining manual code review with LLM-assisted analysis,
|
||||||
|
followed by verification through direct engagement with the
|
||||||
|
respective maintainers on GitHub.
|
||||||
|
\item A fully reproducible experimental framework built on
|
||||||
|
Nix, NixOS, and the Clan deployment system, with pinned
|
||||||
|
dependencies, declarative system configuration, and
|
||||||
|
deterministic cryptographic material generation, enabling
|
||||||
|
independent replication of all results.
|
||||||
|
\item A performance analysis demonstrating that Tailscale
|
||||||
|
outperforms the Linux kernel's default networking stack under
|
||||||
|
degraded conditions, and that kernel parameter tuning; Reno
|
||||||
|
congestion control in place of CUBIC, with RACK
|
||||||
|
disabled; yields measurable throughput improvements.
|
||||||
|
\item The discovery of several security vulnerabilities across
|
||||||
|
the evaluated VPN implementations.
|
||||||
|
\item An automated benchmarking framework designed for public
|
||||||
|
leaderboard generation, intended to encourage ongoing
|
||||||
|
optimization by VPN developers.
|
||||||
|
\end{enumerate}
|
||||||
|
|
||||||
\subsubsection{A Comparative Study on Virtual Private Networks}
|
\section{Related Work}
|
||||||
|
|
||||||
|
\subsection{A Comparative Study on Virtual Private Networks}
|
||||||
|
|
||||||
Lackorzynski et al.\ \cite{lackorzynski_comparative_2019} evaluate
|
Lackorzynski et al.\ \cite{lackorzynski_comparative_2019} evaluate
|
||||||
VPN protocols in the context of industrial communication systems (Industry 4.0),
|
VPN protocols in the context of industrial communication systems (Industry 4.0),
|
||||||
@@ -192,7 +232,7 @@ latency, and CPU overhead; rather than overlay network behavior.
|
|||||||
In contrast, this thesis evaluates VPNs that provide a full data plane
|
In contrast, this thesis evaluates VPNs that provide a full data plane
|
||||||
with peer-to-peer connectivity, NAT traversal, and dynamic peer discovery.
|
with peer-to-peer connectivity, NAT traversal, and dynamic peer discovery.
|
||||||
|
|
||||||
\subsubsection{Full-Mesh VPN Performance Evaluation}
|
\subsection{Full-Mesh VPN Performance Evaluation}
|
||||||
|
|
||||||
Kjorveziroski et al.\ \cite{kjorveziroski_full-mesh_2024} provide a
|
Kjorveziroski et al.\ \cite{kjorveziroski_full-mesh_2024} provide a
|
||||||
comprehensive evaluation of full-mesh VPN solutions for distributed
|
comprehensive evaluation of full-mesh VPN solutions for distributed
|
||||||
@@ -210,5 +250,5 @@ This thesis extends their work in several ways:
|
|||||||
\item Fully reproducible experimental framework via Nix/NixOS/Clan
|
\item Fully reproducible experimental framework via Nix/NixOS/Clan
|
||||||
\end{itemize}
|
\end{itemize}
|
||||||
|
|
||||||
\subsubsection{Performance Evaluation of TCP over QUIC Tunnels}
|
\subsection{Performance Evaluation of TCP over QUIC Tunnels}
|
||||||
TODO \cite{guo_implementation_2025}
|
TODO \cite{guo_implementation_2025}
|
||||||
|
|||||||
@@ -4,11 +4,14 @@
|
|||||||
|
|
||||||
\label{Methodology}
|
\label{Methodology}
|
||||||
|
|
||||||
This chapter describes the methodology used to benchmark peer-to-peer
|
This chapter describes the methodology used to benchmark and analyze
|
||||||
mesh VPN implementations. The experimental design prioritizes
|
peer-to-peer mesh VPN implementations. The evaluation combines
|
||||||
reproducibility at every layer; from dependency management to network
|
performance benchmarking under controlled network conditions with a
|
||||||
conditions; enabling independent verification of results and
|
structured source code analysis of each implementation. The
|
||||||
facilitating future comparative studies.
|
benchmarking framework prioritizes reproducibility at every layer;
|
||||||
|
from pinned dependencies and declarative system configuration to
|
||||||
|
automated test orchestration; enabling independent verification of
|
||||||
|
results and facilitating future comparative studies.
|
||||||
|
|
||||||
\section{Experimental Setup}
|
\section{Experimental Setup}
|
||||||
|
|
||||||
@@ -277,6 +280,60 @@ confidence intervals, which is appropriate for benchmark data that
|
|||||||
may not follow a normal distribution. The nix-cache test (via hyperfine)
|
may not follow a normal distribution. The nix-cache test (via hyperfine)
|
||||||
additionally reports standard deviation.
|
additionally reports standard deviation.
|
||||||
|
|
||||||
|
\section{Source Code Analysis}
|
||||||
|
|
||||||
|
To complement the performance benchmarks with architectural
|
||||||
|
understanding, a structured source code analysis was conducted for
|
||||||
|
all ten VPN implementations. The analysis followed three phases.
|
||||||
|
|
||||||
|
\subsection{Repository Collection and LLM-Assisted Overview}
|
||||||
|
|
||||||
|
The latest main branch of each VPN's git repository was cloned,
|
||||||
|
together with key dependencies that implement core functionality
|
||||||
|
outside the main repository. For example, Yggdrasil delegates its
|
||||||
|
routing and cryptographic operations to the Ironwood library, which
|
||||||
|
was analyzed alongside the main codebase.
|
||||||
|
|
||||||
|
Ten LLM agents (Claude Code) were then spawned in parallel, one per
|
||||||
|
VPN. Each agent was instructed to read the full source tree and
|
||||||
|
produce an \texttt{overview.md} file documenting the following
|
||||||
|
aspects:
|
||||||
|
|
||||||
|
\begin{itemize}
|
||||||
|
\item Wire protocol and message framing
|
||||||
|
\item Encryption scheme and key exchange
|
||||||
|
\item Packet handling and performance
|
||||||
|
\item NAT traversal mechanism
|
||||||
|
\item Local routing and peer discovery
|
||||||
|
\item Security features and access control
|
||||||
|
\item Resilience / Central Point of Failure
|
||||||
|
\end{itemize}
|
||||||
|
|
||||||
|
Every claim in the generated overview was required to reference the
|
||||||
|
specific file and line range in the repository that supports it,
|
||||||
|
enabling direct verification.
|
||||||
|
|
||||||
|
\subsection{Manual Verification}
|
||||||
|
|
||||||
|
The LLM-generated overviews served as a navigational aid rather than
|
||||||
|
a trusted source. The most important code paths identified in each
|
||||||
|
overview were manually read and verified against the actual source
|
||||||
|
code, correcting inaccuracies and deepening the analysis where the
|
||||||
|
automated summaries remained superficial.
|
||||||
|
|
||||||
|
\subsection{Feature Matrix and Maintainer Review}
|
||||||
|
|
||||||
|
The findings from both the automated and manual analysis were
|
||||||
|
consolidated into a comprehensive feature matrix cataloguing 131
|
||||||
|
features across all ten VPN implementations. The matrix covers
|
||||||
|
protocol characteristics, cryptographic primitives, NAT traversal
|
||||||
|
strategies, routing behavior, and security properties.
|
||||||
|
|
||||||
|
The completed feature matrix was published and sent to the respective
|
||||||
|
VPN maintainers for review. Maintainer feedback was incorporated as
|
||||||
|
corrections and clarifications, improving the accuracy of the final
|
||||||
|
classification.
|
||||||
|
|
||||||
\section{Reproducibility}
|
\section{Reproducibility}
|
||||||
|
|
||||||
Reproducibility is ensured at every layer of the experimental stack.
|
Reproducibility is ensured at every layer of the experimental stack.
|
||||||
|
|||||||
@@ -19,7 +19,6 @@ isolated from stateful interactions typical in imperative package
|
|||||||
management. This property is essential for ensuring identical test
|
management. This property is essential for ensuring identical test
|
||||||
environments across benchmark runs.
|
environments across benchmark runs.
|
||||||
|
|
||||||
|
|
||||||
\subsubsection{UDP NAT and Firewall Puncturing in the Wild}
|
\subsubsection{UDP NAT and Firewall Puncturing in the Wild}
|
||||||
|
|
||||||
Halkes and Pouwelse~\cite{halkes_udp_2011} measure UDP hole punching
|
Halkes and Pouwelse~\cite{halkes_udp_2011} measure UDP hole punching
|
||||||
|
|||||||
Reference in New Issue
Block a user