While using just a few nodes for DN42 has been I thought for fun I would deploy a few more POP’s around the world (The HE IPv6 video on sparse allocation might have helped!).

As such I introduce a number of new changes. I’ve extended my dn42.cycloptivity.net sub domain to include a country code for each node.

  • au01.dn42.cycloptivity.net - Aspley, Queensland

    My residential crappy ADSL2+ tail with routing via Ubiquiti EdgeRouter Lite. Currently blazes at 6/0.8Mbps. Eventually will move to a FTTH 100/40 service and host AU services for DN42. In the meantime au01.dn42.cycloptivity.net will no longer provide transit. Future nodes may include a Cisco 877 and Juniper SRX110 once bandwidth improves.

au02.dn42.cycloptivity.net - Brisbane, Queensland

aut-num:   AS4242420161
as-name:   CYCLOPTIVITY-AU02
remarks:   VYOS peering from Binary Lane Brisbane, AU.
remarks:   Peering via GRE+IPSec or OpenVPN
remarks:   Public Traceroute: au02.dn42.cycloptivity.net

jp01.dn42.cycloptivity.net - Tokyo, Japan

aut-num:   AS4242420163
as-name:   CYCLOPTIVITY-JP01
remarks:   VYOS peering from VULTR Tokyo, JP.
remarks:   Peering via GRE+IPSec or OpenVPN
remarks:   Public Traceroute: jp01.dn42.cycloptivity.net

us01.dn42.cycloptivity.net - San Jose, USA

aut-num:   AS4242420162
as-name:   CYCLOPTIVITY-US01
descr:     VYOS peering from VULTR Sillicon Valley, US. 
remarks:   Peering via GRE+IPSec or OpenVPN
remarks:   Public Traceroute: us01.dn42.cycloptivity.net

de01.dn42.cycloptivity.net - Frankfurt, Germany

aut-num:   AS4242420164
as-name:   CYCLOPTIVITY-DE01
descr:     VYOS peering from VULTR Frankfurt, DE. 
remarks:   Peering via GRE+IPSec or OpenVPN
remarks:   Public Traceroute: de01.dn42.cycloptivity.net

Currently the last remaining POP to go live will be jp01. All other nodes are already alive and able to provide transit (albeit sub-optimal in some cases). At this time all nodes are providing IPv4 only. Native IPv6 is planned next.

I’ve created a brief diagram showing average RTT between each node.


I am starting to make use of the phpipam tool to track address usage.

CYCLOPTIVITY-DN42 ( has 4 directly nested subnets:

Subnet description	    Subnet	            Used	% Free	
au02.cycloptivity.dn42	    11/30   63.33		
jp01.cycloptivity.dn42	0/30	100		
us01.cycloptivity.dn42	2/30	93.33		
de01.cycloptivity.dn42	1/30	96.67	

Hopefully, I can make use of the phpipam API to create a automatic rdns tool. Tracking all the various interfaces is getting fairly tedious!

Thats all for now! If your interested in this sort of shenanigans please visit dn42.net!

Its not quite the next blog post that I promised but what the heck. Today we are talking about my recent experiences in peering VyOS into DN42 via OpenVPN.

This should help you to form site-to-site tunnels using OpenVPN!

Generating the keys

First we need to generate a PSK to exchange with our peer. Also note that on VyOS you need to store keys in /config/auth/ or suffer key loss during image updates.

[email protected]:~$ generate openvpn key /config/auth/peer.key

After sharing the pre-shared key with your peer we move onto config.

Tunnel configuration

Your peer will likely be running OpenVPN and actually using its config file syntax. Take this file for example.

mode p2p
rport 1194
lport 1194
proto udp
dev-type tun
dev peer.example
encryption bf128
hash sha1
secret /etc/openvpn/peer.key
user nobody
group nobody
script-security 2
status /var/log/openvpn_status_peer.log
log-append /var/log/openvpn_peer.log
keepalive 10 60

Now we can convert to VyOS configuration statements.

set interfaces openvpn vtun1 encryption 'bf128'
set interfaces openvpn vtun1 hash 'sha1'
set interfaces openvpn vtun1 local-address ''
set interfaces openvpn vtun1 local-host ''
set interfaces openvpn vtun1 local-port '1194'
set interfaces openvpn vtun1 mode 'site-to-site'
set interfaces openvpn vtun1 openvpn-option 'comp-lzo'
set interfaces openvpn vtun1 remote-address ''
set interfaces openvpn vtun1 remote-host ''
set interfaces openvpn vtun1 remote-port '1194'
set interfaces openvpn vtun1 shared-secret-key-file '/config/auth/peer.key'

VyOS seems to expose most of the more arcane OpenVPN configuration statements via the set interfaces openvpn vtun1 openvpn-option command, so you should be able to match your peers configuration. By default VyOS will run the OpenVPN processes with a --ping 10 --ping-restart 60 so you can omit that.

Once configured you can run show openvpn site-to-site status to view your tunnels!

I have now provisioned a dedicated VPS with Binarylane in the NEXTDC B1 facility. From here we have a good place for DN42 transit until I am eventually connected to the NBN. Those that are interested, please feel free to traceroute dn42.cycloptivity.net.

Good luck and happy peering!

I have recently stumbled across DN42 and have been fairly excited to transition my networking know how from layer 3 meshing via OLSR (back in the NTFreenet days) to now understanding more of complex enterprise networks.

What better way to learn about such things than to:

  1. Install a complex enterprise network in your own home
  2. Make sure you have outage sensitive customers on the network. Any Wife, kids or friends will usually do

And so with this in place I introduce my humble home lab.


The network is so far comprised of:

  • 2x Ubiquiti ERL
  • 2x Ubiquiti Rocket M2
  • 1x Cisco SG200-26
  • 1x Linksys SLM2008
  • 1x Fritzbox 7270 + Fritzfon DECT handset
  • 1x Asus WL500GPV2
  • 1x Netcomm NB6PLUS4
  • 1x Dovado GO + Seirra Wireless 320U

There are also some servers

  • 1x SmartOS host - 2x gbit nics plus 1x IPMI
  • 1x QNAP TS-410 - 2x gbit nics

And finally some 11 clients between wired clients, mobile clients and media devices.

Layer 1 and 2

Things a pretty boring here. You can see most of the detail already in the picture at the start of this post. None the less here is the documentation for my benifit.

I have not really found a cleaner way for this short of tables so you will have to suffer through or suggest a better alternative!



Interface Endpoint
ADSL Filter
Port 1 RocketM2-eth0



Interface Endpoint VLANS
eth0 SG200-26-ge21 lag1
eth1 SG200-26-ge22 lag1
eth2 ERL-02-eth2 NA


Interface Endpoint VLANS
eth0 SG200-26-ge23 lag1
eth1 SG200-26-ge24 lag1
eth2 ERL-01-eth2 NA


Interface Endpoint VLANS
ge1 Desktop-eth0 110UP
ge2 Desktop-eth0 110UP
ge3 NA 110UP
ge4 NA 1UP
ge5 NA 1UP
ge6 NA 1UP
ge7 NA 1UP
ge8 NA 1UP
ge9 NA 1UP
ge10 SLM2008-ge1 1UP,130T,140T
ge11 Fritz-lan1 140UP
ge12 WL500GPv2-eth0 150UP
ge13 NA 1UP
ge14 NA 1UP
ge15 NA 1UP
ge16 NA 1UP
ge17 NA 1UP
ge18 Server-ipmi0 130UP
ge19 Server-eth0 120UP
ge20 Server-eth1 130UP
ge21 ERL-01-eth0 lag1
ge22 ERL-01-eth1 lag1
ge23 ERL-02-eth0 lag2
ge24 ERL-02-eth1 lag2
ge25 RocketM2-eth0 191UP
ge26 Dovado-lan0 192UP
lag1 ERL-01-bond0 1UP,120T,150T,161T,191T,192T
lag2 ERL-02-bond0 1UP,110T,130T,140T,162T,191T,192T



Interface Endpoint VLANS
ge1 SG200-26-ge10 1UP,130T,140T
ge2 Wired Client 140UP
ge3 Wired Client 140UP
ge4 Wired Client 140UP
ge5 Wired Client 140UP
ge6 NA 140UP
ge7 TS410-eth0 lag1
ge8 TS410-eth1 lag1
lag1 TS410-bond0 140UP

In the next installment we will describe the layer 3 networks!

Heres a short post for today. A while back I remember being very excited to see that asciiflow.com had updated their platform and the tool had become far richer.

What I did not realise at the time was that the source code which once use to be available had vanished from the internet (aka github). I am not exactly sure what the motivation for this would be since its either a thinly veiled attempt to siphon up technical details from those less security concious or just a plain change of heart. Either way it left me feeling fairly disappointed in the project.

Learning to love perl - asciio

After being faced with the need to document things quickly and in an ever changing way I went on the hunt for a replacement to asciiflow and I found asciio lurking in the Fedora repositories.

I’ve been using asciio now for a few weeks and can say that this application is simply awesome! A GTK interface to interact with layer based ascii diagrams is exactly what I was after. Most particularly I can be confident that these documents stay with me and my /dev/sda.

Some assembly required

There is mostly undocumented feature in the save and save as function. That being the all documents saved with .txt extension are NOT able to be loaded as these are the ascii exports.

To save a document to reuse give it another extension. Personally I have used .asciio to keep things obvious and not had any future trouble. Details are in BZ1097538.

Happy documenting!

I was recently asked to help one of our elderly neighbours get access to Skype via iPad to keep in touch with a very spread out family.

Unfortunately, for the time being our ADSL2+ bandwidth is very constrained with about 5Mbit down and 1Mbit up. Naturally you can see how an existing network of consumers might make sharing this link difficult.

Enter my recent WRT54GL replacement, the Ubiquiti Edge Router Lite. I am the first to admit that the CLI at first was very off putting coming from the familiar environment of OpenWRT. However, the lower power consumption, price point and possibility of getting high speed inter-vlan routing made it an attractive choice.

The network

+-----------------+         XXXX  Wireless                             
| Guest Client    |                                                    
| 10.0.150.xxx/24 |         +--+  Wired                                
|                 |                                                    
        X                                EdgeRouter Lite               
        X                 +-------------------------------------------+
        X                 |                                           |
+-------------+---+       | +---------------+       +---------------+ |
| Asus WL500GP-V2 |       | | Guest Network |       | PPPOE Bridge  | |
| L2 Bridge       +---------+ |       | | |
| OpenWRT         |       | | bond0.150     |       | eth1          | |
+-----------------+       | +---------------+       +---------------+ |
                          |                                           |

The network diagram does not show the other networks the ERL is managing but you get the picture. Since I am ultimately greedy, I have NOT configured any shaping on the other networks. The goal with this exercise is to ensure that the neighbors cant give me any surprises by hammering the available bandwidth.

We set out with the goal of:

  • Downstream speeds shaped to 1Mbit
  • Upstream speeds shaped to 256Kbit
  • Guests must use a different external IP for NAT
  • No transfer limits or other restrictions (Only a WPA2 PSK)

The configuration

First we need to establish our traffic policies.

# show traffic-policy
 rate-control guestnet-ratecontrol {
     bandwidth 1mbit
     burst 15k
     latency 50ms
 rate-control guestnet-out-ratecontrol {
     bandwidth 256kbit
     burst 15k
     latency 50ms

Next, we need to implement a workaround to apply traffic policy to an outbound interface. We achieve this using the IFB interfaces to redirect our traffic a second time (At these speeds I have not seen a particular overhead for this double handling of traffic).

# show interfaces input ifb2
 traffic-policy {
     out guestnet-out-ratecontrol

Finally, we assign the inbound policy to the guest interface and apply our traffic redirection.

# show interfaces bonding bond0 vif 150
 description OPENWRT
 firewall {
     in {
         name OPENWRT_IN
     local {
         name OPENWRT_LOCAL
 redirect ifb2
 traffic-policy {
     out guestnet-ratecontrol

There is vastly more detail available on the Ubiquiti wiki if your heading down the QOS or rate limiting paths. For me, the above works like a charm!


The outcome as measured by Speedtest.net is right on the spot.