"Wire-to-wire" emulation with Proxy ARP

In my previous post, I stated a goal of mimicing the connectivity provided by the microwave T-1 connecting the studio and TV transmitter at my work.  I'm pleased to say that I've accomplished that... mostly.  There are still a few peculiar kinks to work out.

It all works through the magic of Proxy ARP.  Basically, what Proxy ARP does is allow one machine to answer ARP queries on behalf of another machine, saying, in effect "I'll take that packet; I know how to get to that machine".  The answering machine then forwards the packet on to the appropriate machine.  This differs from packet routing in that it happens in Layer 2 of IP, rather than Layer 3 (routing).

In practice, what you have to do is set up static routes on the host (which is acting in this case as a bridge) doing Proxy ARP for all hosts the bridge is providing the connection for, then turn on IP forwarding in the kernel, and turn on Proxy ARP for the appropriate interfaces.  Since I have an IPsec / L2TP VPN set up, the interfaces are going to be ppp0 for the VPN (since it's L2TP and IPsec, PPP is involved, where with straight IPsec it wouldn't be) and eth0 for the local network. I'm also going to have to do the same thing on the firewall out at the transmitter.

So, first of all, the studio.  This is based on Using Linux as an L2TP/IPsec VPN client by Jacco de Leeuw.

The xl2tpd configuration file at the studio:

; File: /etc/xl2tpd/xl2tpd.conf at STUDIO
[lac Transmitter]
; transmitter public IP obfuscated for security reasons
  lns = X.X.X.10 
  require chap = yes
  refuse pap = yes
  require authentication = yes
  ; Name should be the same as the username in the PPP authentication!
  name = bridge
  ppp debug = yes
  pppoptfile = /etc/ppp/options.l2tpd.client
  length bit = yes

Note that I'm not using the l2tp-secrets file as it really doesn't provide any additional security, as far as I can tell.

The PPP options file at the studio:

# /etc/ppp/options.l2tpd.client at STUDIO

ipcp-accept-local
ipcp-accept-remote
refuse-eap
noccp
noauth
crtscts
mtu 1410
mru 1410
nodefaultroute
debug
lock
connect-delay 5000

PPP authentication is done in /etc/ppp/chap-secrets:

# File: /etc/ppp/chap-secrets
# Secrets for authentication using CHAP
# client        server          secret                  IP addresses
bridge          *               "supersecret"
*               bridge          "supersecret"

To set up the static routes, I have set up two scripts in /etc/ppp/ip-up.d. Scripts in this directory are executed by pppd as ip-up scripts - scripts that are executed when the PPP interface is brought up. These set up static routing and turn on Proxy ARP.

#!/bin/bash

# File: /etc/ppp/ip-up.d/0001routes

PPP_INTERFACE=$1
LOCAL_ADDR=$4
REMOTE_ADDR=$5

# These are all the hosts that should be accessible both via the VPN
HOSTS=(
10.1.1.10
10.1.1.11
10.1.1.12
10.1.1.13
)

for HOST in ${HOSTS[*]}
do
        route add -host $HOST $PPP_INTERFACE
done

#!/bin/bash

# File: /etc/ppp/ip-up.d/0002proxyarp

for INTERFACE in ppp0 eth0 ; do
        /sbin/sysctl -w net.ipv4.conf.${INTERFACE}.proxy_arp=1
done

The ipsec (openswan) configuration file at the studio:

# File: /etc/ipsec.conf
# 
config setup
        protostack=netkey
conn Transmitter
        #
        # ----------------------------------------------------------
        # Use a Preshared Key. Disable Perfect Forward Secrecy.
        # Initiate rekeying.
        # Connection type _must_ be Transport Mode.
        #
        authby=secret
        pfs=no
        rekey=yes
        keyingtries=3
        type=transport
        #
        # ----------------------------------------------------------
        # The local Linux machine that connects as a client.
        #
        # The external network interface is used to connect to the server.
        # If you want to use a different interface or if there is no
        # defaultroute, you can use:   left=your.ip.addr.ess
        left=%defaultroute
        #
        leftprotoport=17/1701
        #
        # ----------------------------------------------------------
        # The remote server.
        #
        # Connect to the server at this IP address. (obfuscated for security)
        right=X.X.X.10
        #
        rightprotoport=17/1701
        # ----------------------------------------------------------
        #
        # Change 'ignore' to 'add' to enable this configuration.
        #
        auto=add
        DPDACTion=restart_by_peer
        dpdtimeout=30
        dpddelay=3

Next, the transmitter side.  Again, starting with xl2tpd:

; File: /etc/xl2tpd/xl2tpd.conf

[global]
        ipsec saref = no
        listen-addr = X.X.X.10

[lns default]
        ip range = 10.1.1.100 - 10.1.1.255
        local ip = 10.1.1.1
        assign ip = yes
        require chap = yes
        refuse pap = yes
        require authentication = yes
        name = Transmitter
        ppp debug = no
        pppoptfile = /etc/ppp/options.xl2tpd
        length bit = yes

And now PPP:

# File: /etc/ppp/options.xl2tpd

refuse-mschap-v2
refuse-mschap
ms-dns 8.8.8.8
asyncmap 0
auth
lock
hide-password
local
#debug
name l2tpd
#proxyarp
lcp-echo-interval 30
lcp-echo-failure 4

# File: /etc/ppp/chap-secrets

# Secrets for authentication using CHAP
# client                server                  secret                  IP addresses
user1                   *                       "secret1"               10.1.1.0/24
*                       user1                   "secret1"               10.1.1.0/24
user2                   *                       "secret2"               10.1.1.0/24
*                       user2                   "secret2"               10.1.1.0/24
bridge                  *                       "supersecret"           10.1.1.2
*                       bridge                  "supersecret"           10.1.1.2

Here I want to pause and note something. I have my regular "road warrior" users set up to get any address in 10.1.1.0/24, and in my xl2tpd config, I have that further restricted to addresses in the range of .100 - .255. The "bridge" user is restricted to 10.1.1.2, which is how I allow my "road warriors" and the bridge at the studio to coexist. The appropriate scripts in /etc/ppp/ip-up.d at the transmitter:

#!/bin/bash

# File: /etc/ppp/ip-up.d/0001routes
# The hosts here are made accessible to the transmitter network via the firewall, which is acting as a bridge similar to the one at the studio

PPP_INTERFACE=$1
LOCAL_ADDR=$4
REMOTE_ADDR=$5


# only set up the routes for the VPN bridge
case $REMOTE_ADDR in
10.1.1.2)
        for HOST in     10.1.1.20 \
                        10.1.1.21 \
                        10.1.1.22 \
                        10.1.1.23 ; do
                route add -host $HOST $1
        done
esac

#!/bin/bash
PPP_INTERFACE=$1
LOCAL_ADDR=$4
REMOTE_ADDR=$5

# File: /etc/ppp/ip-up.d/0002proxyarp
# only set up Proxy ARP for the VPN bridge
case $REMOTE_ADDR in
10.1.1.2)
        for INTERFACE in $PPP_INTERFACE eth0 ; do
                /sbin/sysctl -w net.ipv4.conf.${INTERFACE}.proxy_arp=1
        done
esac

And finally the ipsec (openswan) configuration, based on Configure L2TP/IPSec VPN on Ubuntu by Riobard Zhan:

#
# File: /etc/ipsec.conf
#

# Transmitter Firewall Side

config setup
    oe=off
    protostack=netkey
    nat_traversal=yes

conn L2TP-PSK-NAT
    rightsubnet=vhost:%no
    also=L2TP-PSK-noNAT

conn L2TP-PSK-noNAT
    authby=secret
    pfs=no
    auto=add
    keyingtries=3
    rekey=no
    ikelifetime=8h
    keylife=1h
    type=transport
        left=X.X.X.10
    leftprotoport=17/1701
    right=%any
    rightprotoport=17/%any
    dpdaction=restart_by_peer
    dpdtimeout=30
    dpddelay=3

So the way to accomplish this "wire-to-wire" emulation is with two bridges, one for each network to be bridged. If you want to try something like this, I wish you the best of luck, and I hope my experiences help.

Setting up an IPsec VPN

Posted here mostly for my own reference, but mayhap this will prove useful for someone else.

The setup: a remote network at the TV transmitter, with Internet access both through a commercial provider and a dedicated, but increasingly unreliable, microwave T-1 link to the studio.

The goal: mimic the behavior of the microwave T-1 (though not the unreliability, of course) using the commercial provider's service.  Set up a "virtual wire" between the network at the transmitter and the transmitter at the studio.

I've chosen an IPsec VPN to do this, and I've set things up largely as in Ch. 35 of Linux Home Networking. At the present time I've managed to achieve bidirectional communication between hosts (but not the routers/firewalls) on each network, but to do that I had to set up a private test network as shown below.

So as you can see here, I have a couple of virtual machines set up on the 172.16.1.0/24 network, which is accessible only to those VM's.  The IPsec tunnel is established between Router VM and Transmitter Firewall.  Note that I have replaced Transmitter Firewall's public IP address with "X.X.X.10" for security.

With this setup, I can achieve bidrectional communication between both networks.  Host A can reach Client VM, and Client VM can reach Host A.

Here is the relevant ipsec.conf on Router VM:

#
# File: /etc/ipsec.conf
#
# VPN Test Router VM side
config setup
        protostack=netkey
        oe=off
        nat_traversal=yes

# LEFT: Studio
# RIGHT: Transmitter
conn nettonet
  left=192.168.152.188            # Public Internet IP address of the
                                  # LEFT VPN device
  leftsubnet=172.16.1.0/24        # Subnet protected by the LEFT VPN device
  leftrsasigkey=*removed*

  right=X.X.X.10                  # Public Internet IP address of
                                  # the RIGHT VPN device
  rightsubnet=10.1.1.0/24         # Subnet protected by the RIGHT VPN device
  rightrsasigkey=*removed*
  rightnexthop=%defaultroute
  auto=start

Note here that I'm using RSA keys, rather than PSK's.  The key signatures have been removed for security.

The ipsec.conf on Transmitter Firewall is slightly different:

#
# File: /etc/ipsec.conf
#

# Transmitter Firewall Side

config setup
    oe=off
    protostack=netkey
    nat_traversal=yes

# LEFT: Studio
# RIGHT: Transmitter
conn nettonet
  left=10.1.0.2                   # Public Internet IP address of the
                                  # LEFT VPN device
  leftsubnet=172.16.1.0/24        # Subnet protected by the LEFT VPN device
  leftid=192.168.152.188
  leftrsasigkey=*removed*
  right=X.X.X.10                  # Public Internet IP address of
                                  # the RIGHT VPN device
  rightsubnet=10.1.1.0/24         # Subnet protected by the RIGHT VPN device
  rightrsasigkey=*removed*
  auto=start                      # authorizes and starts this connection
                                  # on booting

Here you'll notice that the argument to left= is 10.1.0.2, which is the public (again, these IP's have been obfuscated for security's sake) address of Studio Firewall. This is because Router VM's ultimate path to the Internet is through Studio Firewall via NAT.

Speaking of NAT, I should point out that there is no NAT of any kind being performed (currently) on Router VM or Transmitter Firewall.  Instead, the entire 172.16.1.0/24 network is visible to the 10.1.1.0/24 network at the transmitter, just as any other public network would be.

I'll note that this does not accomplish the stated goal at the beginning of this post: set up a "virtual wire" between the 10.1.1.0/24 network at the studio and the 10.1.1.0/24 network at the transmitter.  It may turn out to be the case that this "virtual wire" is extremely complicated with regard to routing and suchlike (since we would be using the same IP block on both ends) .  What this does give us, however, is a place to work from.