Analyzing VPN Protocols with Wireshark

In the previous post, I demonstrated how to implement a simple VPN service in Python. In this post I’ll show how to write a plugin for the packet sniffer Wireshark in order to analyze the VPN. The post is again intended as a general template, this time for the creation of Wireshark dissectors, with a focus on tunneling protocols.

Recap: The Example VPN Link to heading

As a recap, the VPN described in the previous post uses a TUN device to encapsulate IP packets in UDP packets, with each UDP packet having the following structure:

packet-beta
0-31: "Magic (4 bytes)"
32-47: "Length (2 bytes)"
48-63: "Reserved (2 bytes)"
64-95: "Ciphertext (variable length)"

The VPN uses XOR for encryption, which means that every byte of plaintext is XORed with a key byte. The key repeats to accommodate plaintexts that are longer than the key itself:

def xor(data: bytes, key: bytes):
    """XOR two byte arrays, repeating the key"""
    retval = bytearray(data)
    for i, _ in enumerate(data):
        retval[i] = data[i] ^ key[i % len(key)]
    return bytes(retval)

Wireshark Dissectors Link to heading

Most Wireshark dissectors are written in C. Their source code can be found in the Wireshark source directory /epan/dissectors. However, for research and prototyping, Wireshark also supports dissectors written in Lua. By default, Wireshark loads all Lua plugins in the plugin folders. The personal plugin folder is located at ~/.local/lib/wireshark/plugins on Linux and at %APPDATA%\Wireshark\plugins on Windows¹, and if you place a <name>.lua file there, it will be picked up at the next start or when plugins are reloaded.

The minimum viable Wireshark dissector consists of the following four components:

1. A protocol definition containing the short name and the description of the protocol (see the documentation of the Proto class):

   local my_proto = Proto("CrapVPN_UDP", "CrapVPN Protocol (UDP)")
   

2. A list of fields containing all the “keys” that will be displayed in the dissection tree. These so-called ProtoFields must be registered centrally to allow filtering on the presence or values of fields:

   local fields = {
       magic_bytes = ProtoField.string("crapvpn.magic", "Magic Bytes"),
       ciphertext_length = ProtoField.uint16("crapvpn.ciphertext_length", "Ciphertext Length", base.DEC),
       ciphertext = ProtoField.bytes("crapvpn.ciphertext", "Ciphertext")
   }
   my_proto.fields = fields
   

3. A dissector function implementing the parsing logic. The function is responsible for parsing the buffer and adding information to the packet info pinfo and the dissector tree tree. The buffer variable is usually a so-called Tvb² and subsets can be selected by calling the buffer object with an offset and a length:

   function my_proto.dissector(buffer, pinfo, tree)
       pinfo.cols.protocol = my_proto.name
       local subtree = tree:add(crapvpn_proto, buffer(), crapvpn_proto.description)
   
       subtree:add(fields.magic_bytes, buffer(0, 4))
       subtree:add(fields.ciphertext_length, buffer(4, 2))
   end
   

4. A registration in a dissector table. This registration tells Wireshark when to call your dissector (here for all packets on UDP port 1337):

   local udp_port = DissectorTable.get("udp.port")
   udp_port:add(1337, my_proto)
   

At this point, the dissector will perform like this:

In the screenshot you can see that in the packet list, the “protocol” column is set to “CRAPVPN_UDP” for packets sent to or originating from UDP port 1337. Additionally, in the lower-left-hand corner you can see that the two fields “Magic Bytes” and “Ciphertext length” have been added to the tree, with corresponding values. If you hover over the fields, the associated bytes will be highlighted in the hexdump on the lower-right-hand pane.

Working With Values Link to heading

These four components are the minimum and may even suffice for some simple applications. However, if the dissection depends on the result of a field (for example the length field), things become a bit more complicated. Contrary to (my?) intuition, it is not possible to query the tree for parsed values. Instead, during dissection one has to call the documented methods on the associated TvbRange, as shown in the following example:

function my_proto.dissector(buffer, pinfo, tree)
    local magic_bytes_buffer = buffer(0, 4)
    if magic_bytes_buffer:string() ~= "crap" then
        -- packet is not for us
        return 0
    end

    pinfo.cols.protocol = my_proto.name
    local subtree = tree:add(crapvpn_proto, buffer(), crapvpn_proto.description)    

    subtree:add(fields.magic_bytes, magic_bytes_buffer)

    local ciphertext_length_buffer = buffer(4, 2)
    local ciphertext_length = ciphertext_length_buffer:uint()
    local ciphertext_buffer = buffer(8, ciphertext_length)
    
    subtree:add(fields.ciphertext_length, ciphertext_length_buffer)
    subtree:add(fields.ciphertext, ciphertext_buffer)
end

Working With Derived Byte Arrays Link to heading

If your protocol derives byte arrays that are not directly part of the original packet, for example through reassembly or decryption, you can create a ByteArray instance, fill it with derived bytes and then call tvb("<name>") on it, for example:

function my_proto.dissector(buffer, pinfo, tree)
    local plaintext_bytes = ByteArray.new()
    plaintext:set_size(128)
    -- ... decrypt into plaintext_bytes ... --
    local plaintext_tvb = plaintext_bytes:tvb("Plaintext")
end

This will cause Wireshark to display your bytes in a second tab in the lower right corner:

Calling Sub-Dissectors Link to heading

In order to take full advantage of Wireshark’s dissection capabilities when working with tunneling/encapsulation, after unpacking the encapsulated payload, your dissector should hand over dissection to the built-in dissectors. This can be done by calling the dissectors call method, for example:

local ip_dissector = Dissector.get("ip")

function my_proto.dissector(buffer, pinfo, tree)
    ip_dissector:call(buffer(8), pinfo, tree)
end

This also works on Tvbs created from ByteArray instances:

local ip_dissector = Dissector.get("ip")

function my_proto.dissector(buffer, pinfo, tree)
    -- ...

    local plaintext_tvb = plaintext_bytes:tvb("Plaintext")
    ip_dissector:call(plaintext_tvb, pinfo, tree)
end

Calling sub-dissectors has the following advantages:

• Dissection of inner layer(s), TCP reassembly (if needed)
• Filtering for payloads
• Showing the correct protocols in the listing

The following screenshot shows the results of calling the IP dissectors:

If the sub-dissector is determined dynamically, for example by an identifier in the encapsulation header, the dissector can be fetched from a DissectorTable, which is a mapping of integers and strings to dissectors. If the encapsulation for example contained a field for the EtherType, you can fetch the sub-dissector as follows:

local ethertype_dissector_table = DissectorTable.get("ethertype")

function my_proto.dissector(buffer, pinfo, tree)
    -- ... read ethertype from the encapsulation header

    local dissector = ethertype_dissector_table:get_dissector(ethertype)
    dissector:call(plaintext_tvb, pinfo, tree)
end

Instead of getting the dissector with get_dissector and then invoking it via call, you can call the DissectorTable’s try method directly, e.g.:

ethertype_dissector_table:try(ethertype, plaintext_tvb, pinfo, tree)

for _, v in pairs(DissectorTable.list()) do print(v) end

Adding Preferences Link to heading

Instead of hard coding parameters in your dissector (like the decryption key), Wireshark makes it very easy to define preferences for your dissector. Define the preference once in the header:

my_proto.prefs.key = Pref.string("Key", "", "Key used for decryption (provided as hex string)")

Then, during dissection, the preference’s value will magically be accessible at the same symbol:

my_proto.prefs.key

From a user’s perspective, the preferences can be accessed through the context menu on a matching packet or via the dialogue Edit > Preferences…:

Full Source Code Link to heading

The full source code of the dissector created in this article is contained in the following list, and it is also available for download here:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72

do
    local crapvpn_proto = Proto("CrapVPN_UDP", "CrapVPN Protocol (UDP)")

    local ethertype_dissector_table = DissectorTable.get("ethertype")

    local fields = {
        magic_bytes = ProtoField.string("crapvpn.magic", "Magic Bytes"),
        ciphertext_length = ProtoField.uint16("crapvpn.ciphertext_length", "Ciphertext Length", base.DEC),
        ciphertext = ProtoField.bytes("crapvpn.ciphertext", "Ciphertext"),
        ethertype = ProtoField.uint16("crapvpn.ethertype", "Ether Type", base.HEX)
    }
    crapvpn_proto.fields = fields

    crapvpn_proto.prefs.key = Pref.string("Key", "", "Key used for decryption (provided as hex string)")

    local function xor_decrypt(ciphertext)
        local hex_key = crapvpn_proto.prefs.key
        if not hex_key or hex_key == "" then
            return
        end

        local key = ByteArray.new(hex_key)

        local plaintext = ByteArray.new()
        plaintext:set_size(ciphertext:len())

        for idx = 0, ciphertext:len() - 1 do
            local k = key:get_index(idx % key:len())
            local c = ciphertext:get_index(idx)
            local p = bit.bxor(c, k)
            plaintext:set_index(idx, p)
        end

        return plaintext
    end

    function crapvpn_proto.dissector(buffer, pinfo, tree)
        if buffer:len() == 0 then return end

        pinfo.cols.protocol = crapvpn_proto.name
        local subtree = tree:add(crapvpn_proto, buffer(), crapvpn_proto.description)

        local magic_bytes_buffer = buffer(0, 4)
        subtree:add(fields.magic_bytes, magic_bytes_buffer)

        if magic_bytes_buffer:string() ~= "crap" then
            -- packet is not for us
            return 0
        end

        local ciphertext_length_buffer = buffer(4, 2)
        subtree:add(fields.ciphertext_length, ciphertext_length_buffer)

        local ethertype_buffer = buffer(6, 2)
        subtree:add(fields.ethertype, ethertype_buffer)

        local ciphertext_length = ciphertext_length_buffer:uint()
        local ciphertext_buffer = buffer(8, ciphertext_length)
        subtree:add(fields.ciphertext, ciphertext_buffer)

        local ciphertext_bytes = ciphertext_buffer:bytes()
        local plaintext_bytes = xor_decrypt(ciphertext_bytes)
        if plaintext_bytes then
            local plaintext_tvb = plaintext_bytes:tvb("Plaintext")
            local ethertype = ethertype_buffer:uint()
            ethertype_dissector_table:try(ethertype, plaintext_tvb, pinfo, tree)
        end
    end

    local udp_port = DissectorTable.get("udp.port")
    udp_port:add(1337, crapvpn_proto)
end

Conclusion Link to heading

Wireshark is a powerful tool when working with unknown protocols. Writing dissectors is very easy in theory and can be done alongside the reverse-engineering process, but the documentation is a bit lacking. This post aims to serve as a first starting point for dissectors and as a reference for the most commonly used features.