Evaluating using the first eight DAITA servers

About a month ago Mullvad released Defense against AI-guided Traffic Analysis (DAITA) for their Windows client. In summary, DAITA consists of three components: the traffic analysis defense framework Maybenot, tailored defenses implemented in Maybenot, and (significant) integration efforts of Maybenot into Mullvad VPN’s stack.

DAITA = Maybenot + custom defenses + Mullvad VPN integration

I have been working with Mullvad on DAITA for a while, focusing on Maybenot and defense tooling. Maybenot is already open source, and so is Mullvad’s client integration. We have so much more stuff in the pipeline to share as open source and open access, but for now, I want to share some evaluation results from using the first eight DAITA servers from the perspective of a Windows client.

The servers

The first eight DAITA servers were deployed in the following locations:

It is a healthy mix of locations, providers, and types of servers.

All servers run the same customized Maybenot defenses. Details on the actual defenses are another topic, but for now, it is enough to know that they are heavily randomized, and each WireGuard connection will likely behave quite differently.

Evaluation goal

We want to compare the eight servers regarding defense level and performance overheads. Being geographically distributed, we want to understand how the defenses perform in vastly different network conditions.

Attack setup

Our attack scenario is an attacker that, somehow, knows that a client visited the website for Karlstad University, Sweden (e.g., due to a DNS leak) and wants to determine the webpage (of the university) the client visited. This type of traffic analysis attack is called a webpage fingerprinting attack in the academic literature, a particular case of website fingerprinting.

Karlstad University’s website is hosted in Sweden, which was chosen to reduce the variability caused by websites being mirrored across the world (e.g., using CDNs).

Experiment setup

We set up 20 Windows 11 VMs in Sweden connected to Sunet, the Swedish university network. Each runs the Mullvad client and Mullvad Browser, which is controlled by a Python script using Selenium. Network traffic is captured at each client using tshark.

The primary source of latency between the client and the Karlstad University website (both hosted in Sweden by Sunet) will be the location of the DAITA server (and potential congestion caused by defenses, as we will soon see). Below are the results of pinging the DAITA servers from the client VMs.

ping -c 100 nl-ams-wg-002.relays.mullvad.net
rtt min/avg/max/mdev = 20.346/20.454/20.700/0.056 ms

ping -c 100 nl-ams-wg-103.relays.mullvad.net
rtt min/avg/max/mdev = 23.442/23.558/24.461/0.101 ms

ping -c 100 gb-lon-wg-008.relays.mullvad.net
rtt min/avg/max/mdev = 25.011/25.106/25.228/0.042 ms

ping -c 100 gb-lon-wg-304.relays.mullvad.net
rtt min/avg/max/mdev = 30.718/30.799/30.961/0.041 ms

ping -c 100 us-lax-wg-203.relays.mullvad.net
rtt min/avg/max/mdev = 185.640/186.107/202.877/1.802 ms

ping -c 100 us-lax-wg-406.relays.mullvad.net
rtt min/avg/max/mdev = 176.737/177.205/178.312/0.214 ms

ping -c 100 us-nyc-wg-303.relays.mullvad.net
rtt min/avg/max/mdev = 107.457/107.563/108.538/0.109 ms

ping -c 100 us-nyc-wg-601.relays.mullvad.net
rtt min/avg/max/mdev = 100.712/100.852/101.999/0.182 ms

Data collection

Between 2024-05-29 and 2024-06-02, we collected 16 datasets in the form of PCAPs from our 20 experiment Windows clients: one for each of the 8 servers with and without DAITA (“undefended” below). Each dataset comprises 100 samples of 50 Karlstad University webpages visited using the Mullvad Browser. In total, we collected 80,000 network traces.

Results

We processed the data, computed performance metrics, and evaluated using two state-of-the-art deep-learning-based website fingerprinting attacks: Deep Fingerprinting (DF) and Robust Fingerprinting (RF).

Some key takeaways

DAITA reduces the success rate of both DF and RF attacks, with more prolonged traces (US servers) offering slightly less protection. The protection costs around 2-4 MiB of bandwidth overhead and 1-3 seconds of delay per website visit, depending on server (and website) location.

The defenses are padding-only defenses that create so much padding traffic that congestion occurs, implicitly introducing notable delays. In the future, defenses should explicitly create short, useful (for protection) delays and aim to account for congestion to minimize delays.

The user experience of enabling DAITA when browsing with Mullvad Browser is less than changing the server location from Europe to the US. Recall that the Karlstad University website is hosted in Sweden, so the traffic crossed the Atlantic twice in our experiment (unlike most regular website visits most users do). Not great, but not terrible. Much can and will be improved…stay tuned!