The core protocol of WoopChain

blockchain consensus woopchain bls distributed-ledger-technology pbft proof-of-stake sharding

11 KiB

Raw Blame History

NAT Hole Punching Spike Report

(That is, can we run Harmony on a home network?)

Purpose

Find out if we can easily punch a TCP port forwarding hole through a home gateway, so that we can run Harmony node at home.

Goal

We should identify a way to easily tell the home gateway to punch a TCP hole so that it is forwarded to a local IP address and port number of our choice, that is, we configure our Harmony client to run on that address/port.

Background

VoIP and games have needed this for a long time, and there are a few protocols that can do this:

NAT Port Mapping Protocol (NAT-PMP)
- Apple’s version
- Used by Back To My Mac
- IETF Informational (recognized but not standardized)
Port Control Protocol (PCP)
- Successor of NAT-PMP
- Works with large-scale NAT
- IETF Proposed Standard
Internet Gateway Device (IGD) Control Protocol
- Microsoft’s version, originally part of Universal Plug and Play (UPnP)
  - UPnP is now ISO/IEC 29341
- Older, dates back to first Xbox (gaming was the first use case)

Most home gateways support one or both of these. IGD/UPnP is more common because it’s older and the use case is more compelling – PC/console gaming. We first focus on IGD therefore. Our office is behind a Google Wi-Fi router, which supports both IGD and PCP.

Plan & “tl;dr” Result

(See Transcript for details.)

Find a CLI-based IGD/UPnP implementation we can use in our scripts.
⇒ MiniUPnP seemed promising
Examine docs to see if it can setup and teardown the port mapping.
⇒ list (upnpc -l), setup (upnp -r), teardown (upnp -d)
Set up a port forwarding to SSH daemon my Mac
⇒ upnpc -r 22 TCP
Figure out the port mapping established.
⇒ found in the upnpc -r command output, also shown in upnpc -l output
SSH to a server out of the home network.
SSH back to my Mac, using the public IP/port obtained from step 4.
⇒ WORKS!
Tear down the port mapping.
⇒ upnpc -d 22 TCP

Next Step

Use this for the startup script to use in Docker.
See if we can also find a suitable CLI tool for PCP or NAT-PMP. a. I believe Apple AirPort access point implements only NAT-PMP, and not UPnP.
Figure out how to deal with timeout issue. a. Can we refresh existing mapping so the public port number does not change?
Figure out how to deal with changes in public IP address. a. When this happens, public port may also change, or mapping may be gone.
Figure out how to deal with router reboots. a. Depends on implementation, but chances are, the mapping may be gone.

Also, everyone:

Try this at home. a. Let me know how it goes.

Appendix A. Transcript

Docs Examination

Command-line help; bold ones seem to fit our purposes.

quelthalas 09:07:25 ~ $ 11 upnpc -h
upnpc : miniupnpc library test client, version 2.1.
 (c) 2005-2018 Thomas Bernard.
Go to http://miniupnp.free.fr/ or https://miniupnp.tuxfamily.org/
for more information.
Usage :	upnpc [options] -a ip port external_port protocol [duration]
		Add port redirection
       	upnpc [options] -d external_port protocol <remote host>
		Delete port redirection
       	upnpc [options] -s
		Get Connection status
       	upnpc [options] -l
		List redirections
       	upnpc [options] -L
		List redirections (using GetListOfPortMappings (for IGD:2 only)
       	upnpc [options] -n ip port external_port protocol [duration]
		Add (any) port redirection allowing IGD to use alternative external_port (for IGD:2 only)
       	upnpc [options] -N external_port_start external_port_end protocol [manage]
		Delete range of port redirections (for IGD:2 only)
       	upnpc [options] -r port1 [external_port1] protocol1 [port2 [external_port2] protocol2] [...]
		Add all redirections to the current host
       	upnpc [options] -A remote_ip remote_port internal_ip internal_port protocol lease_time
		Add Pinhole (for IGD:2 only)
       	upnpc [options] -U uniqueID new_lease_time
		Update Pinhole (for IGD:2 only)
       	upnpc [options] -C uniqueID
		Check if Pinhole is Working (for IGD:2 only)
       	upnpc [options] -K uniqueID
		Get Number of packets going through the rule (for IGD:2 only)
       	upnpc [options] -D uniqueID
		Delete Pinhole (for IGD:2 only)
       	upnpc [options] -S
		Get Firewall status (for IGD:2 only)
       	upnpc [options] -G remote_ip remote_port internal_ip internal_port protocol
		Get Outbound Pinhole Timeout (for IGD:2 only)
       	upnpc [options] -P
		Get Presentation url

protocol is UDP or TCP
Options:
  -e description : set description for port mapping.
  -6 : use ip v6 instead of ip v4.
  -u url : bypass discovery process by providing the XML root description url.
  -m address/interface : provide ip address (ip v4) or interface name (ip v4 or v6) to use for sending SSDP multicast packets.
  -z localport : SSDP packets local (source) port (1024-65535).
  -p path : use this path for MiniSSDPd socket.
  -t ttl : set multicast TTL. Default value is 2.

Forwarding Setup

Precondition

quelthalas 09:07:27 ~ $ 12 upnpc -l
upnpc : miniupnpc library test client, version 2.1.
 (c) 2005-2018 Thomas Bernard.
Go to http://miniupnp.free.fr/ or https://miniupnp.tuxfamily.org/
for more information.
List of UPNP devices found on the network :
 desc: http://192.168.86.1:5000/rootDesc.xml
 st: urn:schemas-upnp-org:device:InternetGatewayDevice:1

Found valid IGD : http://192.168.86.1:5000/ctl/IPConn
Local LAN ip address : 192.168.86.250
Connection Type : IP_Routed
Status : Connected, uptime=4460733s, LastConnectionError : ERROR_NONE
  Time started : Wed Jan 16 18:02:03 2019
MaxBitRateDown : 1000000000 bps (1000.0 Mbps)   MaxBitRateUp 1000000000 bps (1000.0 Mbps)
ExternalIPAddress = 73.71.156.214
 i protocol exPort->inAddr:inPort description remoteHost leaseTime
 0 UDP 58956->192.168.86.230:58956 'WhatsApp (1552097793) ()' '' 551337
 1 UDP 57027->192.168.86.230:57027 'WhatsApp (1552098628) ()' '' 552173
 2 UDP 58671->192.168.86.230:58671 'WhatsApp (1552149239) ()' '' 602783
 3 UDP 55355->192.168.86.230:55355 'WhatsApp (1552150382) ()' '' 603926
GetGenericPortMappingEntry() returned 713 (SpecifiedArrayIndexInvalid)

Trigger

quelthalas 09:07:36 ~ $ 13 upnpc -r 22 TCP
upnpc : miniupnpc library test client, version 2.1.
 (c) 2005-2018 Thomas Bernard.
Go to http://miniupnp.free.fr/ or https://miniupnp.tuxfamily.org/
for more information.
List of UPNP devices found on the network :
 desc: http://192.168.86.1:5000/rootDesc.xml
 st: urn:schemas-upnp-org:device:InternetGatewayDevice:1

Found valid IGD : http://192.168.86.1:5000/ctl/IPConn
Local LAN ip address : 192.168.86.250
ExternalIPAddress = 73.71.156.214
InternalIP:Port = 192.168.86.250:22
external 73.71.156.214:22 TCP is redirected to internal 192.168.86.250:22 (duration=604800)

It assigned the external port number (exPort) 22. The external IP address is 73.71.156.214. It also has a one-week (604800 seconds) lifetime.

Postcondition

quelthalas 09:07:41 ~ $ 14 upnpc -l
upnpc : miniupnpc library test client, version 2.1.
 (c) 2005-2018 Thomas Bernard.
Go to http://miniupnp.free.fr/ or https://miniupnp.tuxfamily.org/
for more information.
List of UPNP devices found on the network :
 desc: http://192.168.86.1:5000/rootDesc.xml
 st: urn:schemas-upnp-org:device:InternetGatewayDevice:1

Found valid IGD : http://192.168.86.1:5000/ctl/IPConn
Local LAN ip address : 192.168.86.250
Connection Type : IP_Routed
Status : Connected, uptime=4460745s, LastConnectionError : ERROR_NONE
  Time started : Wed Jan 16 18:02:03 2019
MaxBitRateDown : 1000000000 bps (1000.0 Mbps)   MaxBitRateUp 1000000000 bps (1000.0 Mbps)
ExternalIPAddress = 73.71.156.214
 i protocol exPort->inAddr:inPort description remoteHost leaseTime
 0 UDP 58956->192.168.86.230:58956 'WhatsApp (1552097793) ()' '' 551325
 1 UDP 57027->192.168.86.230:57027 'WhatsApp (1552098628) ()' '' 552161
 2 UDP 58671->192.168.86.230:58671 'WhatsApp (1552149239) ()' '' 602771
 3 UDP 55355->192.168.86.230:55355 'WhatsApp (1552150382) ()' '' 603914
 4 TCP    22->192.168.86.250:22    'libminiupnpc' '' 604793
GetGenericPortMappingEntry() returned 713 (SpecifiedArrayIndexInvalid)

Index 4 is the new entry we just set up.

SSH Roundtrip Test

SSH to an out-of-home server

We use our Jenkins server on AWS.

quelthalas 09:07:48 ~ $ 15 ssh jenkins
Last login: Sat Mar  9 16:45:50 2019 from c-73-71-156-214.hsd1.ca.comcast.net

       __|  __|_  )
       _|  (     /   Amazon Linux 2 AMI
      ___|\___|___|

https://aws.amazon.com/amazon-linux-2/
8 package(s) needed for security, out of 11 available
Run "sudo yum update" to apply all updates.
[ec2-user@ip-172-31-14-35 ~]$

SSH back to my Mac

Use the external address and port number we gathered.

[ec2-user@ip-172-31-14-35 ~]$ ssh -p 22 ek@73.71.156.214
Password:
Last login: Sat Mar  9 08:46:14 2019 from 54.183.5.66
Agent running at /var/folders/bj/fgmkk8fj715_jcbdzrxjlrh80000gn/T//ssh-73WRpgJuix8J/agent.9152 (PID 9153)
quelthalas 09:08:13 ~ $ 1 logout
Connection to 73.71.156.214 closed.
[ec2-user@ip-172-31-14-35 ~]$ logout
Shared connection to jenkins.harmony.one closed.
quelthalas 09:08:30 ~ $ 16

Forwarding Teardown

Trigger

quelthalas 09:29:12 ~ $ 18 upnpc -d 22 TCP
upnpc : miniupnpc library test client, version 2.1.
 (c) 2005-2018 Thomas Bernard.
Go to http://miniupnp.free.fr/ or https://miniupnp.tuxfamily.org/
for more information.
List of UPNP devices found on the network :
 desc: http://192.168.86.1:5000/rootDesc.xml
 st: urn:schemas-upnp-org:device:InternetGatewayDevice:1

Found valid IGD : http://192.168.86.1:5000/ctl/IPConn
Local LAN ip address : 192.168.86.250
UPNP_DeletePortMapping() returned : 0

Postcondition

quelthalas 09:29:19 ~ $ 19 upnpc -l
upnpc : miniupnpc library test client, version 2.1.
 (c) 2005-2018 Thomas Bernard.
Go to http://miniupnp.free.fr/ or https://miniupnp.tuxfamily.org/
for more information.
List of UPNP devices found on the network :
 desc: http://192.168.86.1:5000/rootDesc.xml
 st: urn:schemas-upnp-org:device:InternetGatewayDevice:1

Found valid IGD : http://192.168.86.1:5000/ctl/IPConn
Local LAN ip address : 192.168.86.250
Connection Type : IP_Routed
Status : Connected, uptime=4462039s, LastConnectionError : ERROR_NONE
  Time started : Wed Jan 16 18:02:03 2019
MaxBitRateDown : 1000000000 bps (1000.0 Mbps)   MaxBitRateUp 1000000000 bps (1000.0 Mbps)
ExternalIPAddress = 73.71.156.214
 i protocol exPort->inAddr:inPort description remoteHost leaseTime
 0 UDP 58956->192.168.86.230:58956 'WhatsApp (1552097793) ()' '' 550031
 1 UDP 57027->192.168.86.230:57027 'WhatsApp (1552098628) ()' '' 550867
 2 UDP 58671->192.168.86.230:58671 'WhatsApp (1552149239) ()' '' 601477
GetGenericPortMappingEntry() returned 713 (SpecifiedArrayIndexInvalid)

Note that the TCP port 22 entry is gone. Teardown worked.

11 KiB Raw Blame History

NAT Hole Punching Spike Report

Purpose

Goal

Background

Plan & “tl;dr” Result

Next Step

Appendix A. Transcript

Docs Examination

Forwarding Setup

Precondition

Trigger

Postcondition

SSH Roundtrip Test

SSH to an out-of-home server

SSH back to my Mac

Forwarding Teardown

Trigger

Postcondition

11 KiB

Raw Blame History