This document describes how to initiate connections back into overlapping IP address ranges of multiple clients when using mast mode with `IP_IPSEC_BINDREF`.

SAref patches

Libreswan comes with two SAref patches. One adds support for setting SAref on a packet as it’s sent, and retrieving it from a packet when it’s received. This works fine for UDP when using `sendmsg()` and `recvmsg()`.

For TCP, or a connected UDP socket, the SAref needs to be assigned, or bound, to a socket. Any data written to the socket using `write()` will be sent via the assigned SAref. This feature works the same with simple tunnels as well as nested tunnels. The SAref used needs to be the inner most SA.

The `IP_IPSEC_BINDREF` patch is needed for connected sockets.

These patches can be found in the patches/kernel directory

The setup

For example, we will use 3 systems: 2 clients with private networks, connecting to a central aggregation point.

                          ,-- client1 ---- (private1)
    server ---- (internet)
                          `-- client2 ---- (private2)

The clients will initiate the connection to the server, and the `internet` could include one ore more NAT points. Furthermore, the private networks behind the clients terminating the connection could overlap. Under normal circumstances it would be impossible for the server to initiate a connection to, or in some cases reply to packets originating from, the private networks.

Here is an example ipsec.conf conn entry:

    conn server-client1
        left=server.foo.com
        leftid=@server.foo.com
        leftrsasigkey=0s...
        #
        right=%any
        rightsubnet=192.168.0.0/24
        rightid=@client1.foo.com
        rightrsasigkey=0s...
        #
        type=tunnel
        overlapip=yes
        auto=add

Normal connections

Incoming packets are relatively easy to match up with SAs, because the ESP header carries this information. Outgoing packets requires a table that maps the original packet header fields to the SA to use for IPsec. In non-mast mode, klips uses an eroute table, while in mast-mode libreswan uses iptables to manage this list.

When a packet is sent out, it will pass through iptables. Libreswan maintains an IPSEC chain in the mangle table. This allows it to tag packets using xmark (previously nfmark). When klips, in mast mode, sees a packet that’s marked with an nfmark, it will use that information to lookup the SA that will be used to tunnel/transport this packet.

Unconnected sockets

Servers, such as xl2tpd, which accept UDP connections from many peers use a single connectionless datagram socket, and address packets using `sendmsg()`. As described in the libreswan `MastRework` document, mast mode allows for each packet sent via `sendmsg()` to be tagged with a SAref by using the `IP_IPSEC_REFINFO` option of `msghdr`.

    struct msghdr msgh;
    struct cmsghdr *cmsg;
    unsigned int saref = 1234;  // some SAref

    ...

    cmsg = CMSG_FIRSTHDR(&msgh);
    cmsg->cmsg_level = IPPROTO_IP;
    cmsg->cmsg_type  = IP_IPSEC_REFINFO;
    cmsg->cmsg_len   = CMSG_LEN(sizeof(unsigned int));
    *((unsigned int *)CMSG_DATA(cmsg)) = saref;

    ...

    rc = sendmsg(socket, &msg, 0);

(refer to `sendmsg` manpage for further info)

When klips, operating in mast mode, sees a packet with this SAref, it will use it to lookup the SA to send the packet on. It is thus possible to send packets to the same IP behind two different tunnels, since the tunneled destination IP is not used for routing or SA selection.

On the incoming path, `recvmsg()` is used to read data. The `recvmsg()` system call also uses the `struct msghdr` to pass the SAref of a packet, but int he opposite direction. But before calling `recvmsg()`, a socket needs to be told that the caller is interested in SAref info.

    unsigned int arg = 1;
    rc = setsockopt(socket, IPPROTO_IP, IP_IPSEC_REFINFO, &arg, sizeof(arg));

After that, each call to `recvmsg()` will return a `cmsg` with the SAref used.

    struct msghdr msgh;
    struct cmsghdr *cmsg;
    unsigned int saref = 0;

    ...

    rc = recvmsg(socket, &msg, 0);

    ...

    for (cmsg = CMSG_FIRSTHDR(&msgh); cmsg; cmsg = CMSG_NXTHDR(&msgh,cmsg)) {
        if (cmsg->cmsg_level == IPPROTO_IP
                && cmsg->cmsg_type == IP_IPSEC_REFINFO) {
            saref = *((unsigned int *)CMSG_DATA(cmsg));
        }
    }

See xl2tpd sources for further examples of using `IP_IPSEC_REFINFO` to set and get SAref for UDP datagram packets.

Connected sockets

For TCP and connected UDP sockets, the socket only has one end point, and there is no room for selecting the SA on each packet. Instead the socket is “bound” to the SAref using the `IP_IPSEC_BINDREF` socket option:

    rc = setsockopt(socket, IPPROTO_IP, IP_IPSEC_BINDREF, &saref, sizeof(saref));

This has a similar effect to setting the SAref for each UDP datagram, but needs only be done once. Next, the aggregation-server software would call `connect()` to connect to a listening port in the private network through the tunnel selected by the SAref. Finally, `read()` and `write()` would be used to receive and send data.

Libreswan comes with two examples in its contrib/ directory that use this feature.

• sarefnc - netcat modified to use a “-S " option
• ldsaref - an LD_PRELOAD library that can be used to wrap unmodified applications