OpenVPN how to fix: AEAD Decrypt error: bad packet ID

When using OpenVPN to connect multiple tunnels together, you may get some errors along the way. These are normal and easy to fix. For some in particular you may get the following error.

AEAD Decrypt error: bad packet ID (may be a replay)

This error only occurs when you are running your tunnel in UDP. Of course, an easy fix would be to append the following into both your client and server configuration files to mute the messages:

--mute-replay-warning

You do not want to do this when your VPN is running in an environment where security matters greatly(enterprise and production). This error in particular may indicate an attempted attack on your VPN tunnel. Encrypted packets may be intercepted by a “middle man” or what we’d like to call “Man In The Middle Attack”(or MITM for short).

This style of malicious behavior is very unlikely given that your OpenVPN tunnel is protected and encrypted utilizing “remote-cert-tls” and “tls-auth” as well as your connection being identified by the certificate’s “EKU” value. If you’re running your tunnel in TCP and are experiencing this error, then you may want to investigate it.

MTU

Another cause(and most likely the issue at hand) is your MTU. MTU(Maximum Transmissions Unit) determines the maximum a single data packet can traverse the network, in other words the internet. There is something called “overhead” though, which takes away from the current MTU value. An overhead contains information that defines a data packet and make its way to a destination. In OpenVPN, a packet requires 41 bytes for security, 28 for the tunnel with a total for 69.

For example if you’re using DSL as your internet connection, then your MTU is most likely set to 1492 by default in your router/modem. In most cases, it is not and can be much lower. In order to find the correct value, we are going to open up Command Prompt and type in the following

ping -l 1492 -f 1.1.1.1

You are then told that “Your packets needs to be fragmented but DF set.” which means that you will need to continue to lower your tested MTU value from “1492” until you get a successful reply. In my case, the true MTU value is 1414.

MSS in OpenVPN

MSS stands for Max Segment Size. This one is specific to the TCP protocol, and restricts how big a packet can be sent through and received. This is usually calculated by your MTU size minus 40. For example we have figured out our true MTU value, which for me is 1414 – 40 is 1374. My MSS for TCP-based software and connections such as FTP, HTTP/S is 1374. In OpenVPN we are now going to subtract it by 69 which gives us 1305 for our tunnel.

We will now append both our client and server configurations with the following:

mssfix 1305

Save both configurations and restart your client and server connections. Try moving large files with a TCP-based solution such as FTP. Be wary that something such as FTPS requires additional overhead for encryption on top of the VPN tunnel, and will require you to lower your mssfix value even further. Otherwise, you will receive the “AED Decrypt error” again.

You should no longer be getting this error. If you are, then I suggest you change your protocol to “proto tcp” under the server and client configurations.