chore(phantun) release v0.3.2

chore(fake-tcp) release v0.3.1
perf(fake-tcp) use flume to avoid locking in receiver, improved single
2025-09-16 04:04:29 +08:00 · 2022-04-10 06:10:57 -07:00 · 2022-04-10 06:09:48 -07:00 · 2022-04-10 21:07:12 +08:00 · 2022-04-10 18:47:22 +08:00 · 2022-04-10 18:37:06 +08:00
20 changed files with 1061 additions and 293 deletions
--- a/.github/dependabot.yml
+++ b/.github/dependabot.yml
@@ -0,0 +1,12 @@
+version: 2
+updates:
+
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    schedule:
+      interval: "daily"
+
+  - package-ecosystem: "cargo"
+    directory: "/"
+    schedule:
+      interval: "daily"
--- a/.github/workflows/release.yml
+++ b/.github/workflows/release.yml
@@ -16,10 +16,22 @@ jobs:
      matrix:
        target:
          - x86_64-unknown-linux-gnu
+          - x86_64-unknown-linux-musl
+          - i686-unknown-linux-gnu
+          - i686-unknown-linux-musl
          - armv7-unknown-linux-gnueabihf
+          - armv7-unknown-linux-musleabihf
+          - arm-unknown-linux-gnueabihf
+          - arm-unknown-linux-musleabihf
+          - aarch64-unknown-linux-gnu
+          - aarch64-unknown-linux-musl
+          - mips-unknown-linux-gnu
+          - mips-unknown-linux-musl
+          - mipsel-unknown-linux-gnu
+          - mipsel-unknown-linux-musl

    steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v3
      - uses: actions-rs/toolchain@v1
        with:
          toolchain: stable
--- a/.github/workflows/rust.yml
+++ b/.github/workflows/rust.yml
@@ -11,7 +11,10 @@ jobs:
    runs-on: ubuntu-latest

    steps:
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v3
+    - uses: actions-rs/toolchain@v1
+      with:
+        toolchain: stable
    - name: Run lint
      run: cargo clippy --verbose
    - name: Build
--- a/201
+++ b/201
@@ -0,0 +1,201 @@
+                              Apache License
+                        Version 2.0, January 2004
+                     http://www.apache.org/licenses/
+
+TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+1. Definitions.
+
+   "License" shall mean the terms and conditions for use, reproduction,
+   and distribution as defined by Sections 1 through 9 of this document.
+
+   "Licensor" shall mean the copyright owner or entity authorized by
+   the copyright owner that is granting the License.
+
+   "Legal Entity" shall mean the union of the acting entity and all
+   other entities that control, are controlled by, or are under common
+   control with that entity. For the purposes of this definition,
+   "control" means (i) the power, direct or indirect, to cause the
+   direction or management of such entity, whether by contract or
+   otherwise, or (ii) ownership of fifty percent (50%) or more of the
+   outstanding shares, or (iii) beneficial ownership of such entity.
+
+   "You" (or "Your") shall mean an individual or Legal Entity
+   exercising permissions granted by this License.
+
+   "Source" form shall mean the preferred form for making modifications,
+   including but not limited to software source code, documentation
+   source, and configuration files.
+
+   "Object" form shall mean any form resulting from mechanical
+   transformation or translation of a Source form, including but
+   not limited to compiled object code, generated documentation,
+   and conversions to other media types.
+
+   "Work" shall mean the work of authorship, whether in Source or
+   Object form, made available under the License, as indicated by a
+   copyright notice that is included in or attached to the work
+   (an example is provided in the Appendix below).
+
+   "Derivative Works" shall mean any work, whether in Source or Object
+   form, that is based on (or derived from) the Work and for which the
+   editorial revisions, annotations, elaborations, or other modifications
+   represent, as a whole, an original work of authorship. For the purposes
+   of this License, Derivative Works shall not include works that remain
+   separable from, or merely link (or bind by name) to the interfaces of,
+   the Work and Derivative Works thereof.
+
+   "Contribution" shall mean any work of authorship, including
+   the original version of the Work and any modifications or additions
+   to that Work or Derivative Works thereof, that is intentionally
+   submitted to Licensor for inclusion in the Work by the copyright owner
+   or by an individual or Legal Entity authorized to submit on behalf of
+   the copyright owner. For the purposes of this definition, "submitted"
+   means any form of electronic, verbal, or written communication sent
+   to the Licensor or its representatives, including but not limited to
+   communication on electronic mailing lists, source code control systems,
+   and issue tracking systems that are managed by, or on behalf of, the
+   Licensor for the purpose of discussing and improving the Work, but
+   excluding communication that is conspicuously marked or otherwise
+   designated in writing by the copyright owner as "Not a Contribution."
+
+   "Contributor" shall mean Licensor and any individual or Legal Entity
+   on behalf of whom a Contribution has been received by Licensor and
+   subsequently incorporated within the Work.
+
+2. Grant of Copyright License. Subject to the terms and conditions of
+   this License, each Contributor hereby grants to You a perpetual,
+   worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+   copyright license to reproduce, prepare Derivative Works of,
+   publicly display, publicly perform, sublicense, and distribute the
+   Work and such Derivative Works in Source or Object form.
+
+3. Grant of Patent License. Subject to the terms and conditions of
+   this License, each Contributor hereby grants to You a perpetual,
+   worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+   (except as stated in this section) patent license to make, have made,
+   use, offer to sell, sell, import, and otherwise transfer the Work,
+   where such license applies only to those patent claims licensable
+   by such Contributor that are necessarily infringed by their
+   Contribution(s) alone or by combination of their Contribution(s)
+   with the Work to which such Contribution(s) was submitted. If You
+   institute patent litigation against any entity (including a
+   cross-claim or counterclaim in a lawsuit) alleging that the Work
+   or a Contribution incorporated within the Work constitutes direct
+   or contributory patent infringement, then any patent licenses
+   granted to You under this License for that Work shall terminate
+   as of the date such litigation is filed.
+
+4. Redistribution. You may reproduce and distribute copies of the
+   Work or Derivative Works thereof in any medium, with or without
+   modifications, and in Source or Object form, provided that You
+   meet the following conditions:
+
+   (a) You must give any other recipients of the Work or
+       Derivative Works a copy of this License; and
+
+   (b) You must cause any modified files to carry prominent notices
+       stating that You changed the files; and
+
+   (c) You must retain, in the Source form of any Derivative Works
+       that You distribute, all copyright, patent, trademark, and
+       attribution notices from the Source form of the Work,
+       excluding those notices that do not pertain to any part of
+       the Derivative Works; and
+
+   (d) If the Work includes a "NOTICE" text file as part of its
+       distribution, then any Derivative Works that You distribute must
+       include a readable copy of the attribution notices contained
+       within such NOTICE file, excluding those notices that do not
+       pertain to any part of the Derivative Works, in at least one
+       of the following places: within a NOTICE text file distributed
+       as part of the Derivative Works; within the Source form or
+       documentation, if provided along with the Derivative Works; or,
+       within a display generated by the Derivative Works, if and
+       wherever such third-party notices normally appear. The contents
+       of the NOTICE file are for informational purposes only and
+       do not modify the License. You may add Your own attribution
+       notices within Derivative Works that You distribute, alongside
+       or as an addendum to the NOTICE text from the Work, provided
+       that such additional attribution notices cannot be construed
+       as modifying the License.
+
+   You may add Your own copyright statement to Your modifications and
+   may provide additional or different license terms and conditions
+   for use, reproduction, or distribution of Your modifications, or
+   for any such Derivative Works as a whole, provided Your use,
+   reproduction, and distribution of the Work otherwise complies with
+   the conditions stated in this License.
+
+5. Submission of Contributions. Unless You explicitly state otherwise,
+   any Contribution intentionally submitted for inclusion in the Work
+   by You to the Licensor shall be under the terms and conditions of
+   this License, without any additional terms or conditions.
+   Notwithstanding the above, nothing herein shall supersede or modify
+   the terms of any separate license agreement you may have executed
+   with Licensor regarding such Contributions.
+
+6. Trademarks. This License does not grant permission to use the trade
+   names, trademarks, service marks, or product names of the Licensor,
+   except as required for reasonable and customary use in describing the
+   origin of the Work and reproducing the content of the NOTICE file.
+
+7. Disclaimer of Warranty. Unless required by applicable law or
+   agreed to in writing, Licensor provides the Work (and each
+   Contributor provides its Contributions) on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+   implied, including, without limitation, any warranties or conditions
+   of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+   PARTICULAR PURPOSE. You are solely responsible for determining the
+   appropriateness of using or redistributing the Work and assume any
+   risks associated with Your exercise of permissions under this License.
+
+8. Limitation of Liability. In no event and under no legal theory,
+   whether in tort (including negligence), contract, or otherwise,
+   unless required by applicable law (such as deliberate and grossly
+   negligent acts) or agreed to in writing, shall any Contributor be
+   liable to You for damages, including any direct, indirect, special,
+   incidental, or consequential damages of any character arising as a
+   result of this License or out of the use or inability to use the
+   Work (including but not limited to damages for loss of goodwill,
+   work stoppage, computer failure or malfunction, or any and all
+   other commercial damages or losses), even if such Contributor
+   has been advised of the possibility of such damages.
+
+9. Accepting Warranty or Additional Liability. While redistributing
+   the Work or Derivative Works thereof, You may choose to offer,
+   and charge a fee for, acceptance of support, warranty, indemnity,
+   or other liability obligations and/or rights consistent with this
+   License. However, in accepting such obligations, You may act only
+   on Your own behalf and on Your sole responsibility, not on behalf
+   of any other Contributor, and only if You agree to indemnify,
+   defend, and hold each Contributor harmless for any liability
+   incurred by, or claims asserted against, such Contributor by reason
+   of your accepting any such warranty or additional liability.
+
+END OF TERMS AND CONDITIONS
+
+APPENDIX: How to apply the Apache License to your work.
+
+   To apply the Apache License to your work, attach the following
+   boilerplate notice, with the fields enclosed by brackets "[]"
+   replaced with your own identifying information. (Don't include
+   the brackets!)  The text should be enclosed in the appropriate
+   comment syntax for the file format. We also recommend that a
+   file or class name and description of purpose be included on the
+   same "printed page" as the copyright notice for easier
+   identification within third-party archives.
+
+Copyright 2021-2022 Datong Sun (dndx@idndx.com)
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+	http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
--- a/27
+++ b/27
@@ -0,0 +1,27 @@
+MIT License
+
+Copyright (c) 2021-2022 Datong Sun (dndx@idndx.com)
+
+Permission is hereby granted, free of charge, to any
+person obtaining a copy of this software and associated
+documentation files (the "Software"), to deal in the
+Software without restriction, including without
+limitation the rights to use, copy, modify, merge,
+publish, distribute, sublicense, and/or sell copies of
+the Software, and to permit persons to whom the Software
+is furnished to do so, subject to the following
+conditions:
+
+The above copyright notice and this permission notice
+shall be included in all copies or substantial portions
+of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
+ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
+TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
+PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
+SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
+IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+DEALINGS IN THE SOFTWARE.
--- a/README.md
+++ b/README.md
@@ -2,35 +2,49 @@

 A lightweight and fast UDP to TCP obfuscator.

+![GitHub Workflow Status](https://img.shields.io/github/workflow/status/dndx/phantun/Rust?style=flat-square)
+![docs.rs](https://img.shields.io/docsrs/fake-tcp)
+
 Table of Contents
 =================

 * [Phantun](#phantun)
+* [Latest release](#latest-release)
 * [Overview](#overview)
 * [Usage](#usage)
-    * [Enable Kernel IP forwarding](#enable-kernel-ip-forwarding)
-    * [Add required firewall rules (using nftables as an example)](#add-required-firewall-rules-using-nftables-as-an-example)
+    * [1. Enable Kernel IP forwarding](#1-enable-kernel-ip-forwarding)
+    * [2. Add required firewall rules](#2-add-required-firewall-rules)
        * [Client](#client)
+            * [Using nftables](#using-nftables)
+            * [Using iptables](#using-iptables)
        * [Server](#server)
-    * [Give Phantun binaries required capability to it can be run as non-root (Optional)](#give-phantun-binaries-required-capability-to-it-can-be-run-as-non-root-optional)
-    * [Start](#start)
+            * [Using nftables](#using-nftables)
+            * [Using iptables](#using-iptables)
+    * [3. Run Phantun binaries as non-root (Optional)](#3-run-phantun-binaries-as-non-root-optional)
+    * [4. Start Phantun daemon](#4-start-phantun-daemon)
        * [Server](#server)
        * [Client](#client)
 * [MTU overhead](#mtu-overhead)
+    * [MTU calculation for WireGuard](#mtu-calculation-for-wireguard)
 * [Version compatibility](#version-compatibility)
+* [Documentations](#documentations)
 * [Performance](#performance)
 * [Future plans](#future-plans)
 * [Compariation to udp2raw](#compariation-to-udp2raw)
 * [License](#license)

+# Latest release
+
+[v0.3.1](https://github.com/dndx/phantun/releases/tag/v0.3.1)
+
 # Overview

-Phanton is a project that obfuscated UDP packets into TCP connections. It aims to
+Phantun is a project that obfuscated UDP packets into TCP connections. It aims to
 achieve maximum performance with minimum processing and encapsulation overhead.

 It is commonly used in environments where UDP is blocked/throttled but TCP is allowed through.

-Phanton simply converts a stream of UDP packets into obfuscated TCP stream packets. The TCP stack
+Phantun simply converts a stream of UDP packets into obfuscated TCP stream packets. The TCP stack
 used by Phantun is designed to pass through most L3/L4 stateful/stateless firewalls/NAT
 devices. It will **not** be able to pass through L7 proxies.
 However, the advantage of this approach is that none of the common UDP over TCP performance killer
@@ -41,22 +55,57 @@ connection from the perspective of firewalls/NAT devices.
 Phantun means Phantom TUN, as it is an obfuscator for UDP traffic that does just enough work
 to make it pass through stateful firewall/NATs as TCP packets.

+Phantun is written in 100% safe Rust. It has been optimized extensively to scale well on multi-core
+systems and has no issue saturating all available CPU resources on a fast connection.
+See the [Performance](#performance) section for benchmarking results.
+
 ![Traffic flow diagram](images/traffic-flow.png)

 # Usage

+For the example below, it is assumed that **Phantun Server** listens for incoming Phantun Client connections at
+port `4567` (the `--local` option for server), and it forwards UDP packets to UDP server at `127.0.0.1:1234`
+(the `--remote` option for server).
+
+It is also assumed that **Phantun Client** listens for incoming UDP packets at
+`127.0.0.1:1234` (the `--local` option for client) and connects to Phantun Server at `10.0.0.1:4567`
+(the `--remote` option for client).
+
 Phantun creates TUN interface for both the Client and Server. For Client, Phantun assigns itself the IP address
-`192.168.200.2` and for Server, it assigns `192.168.201.2`. Therefore, your Kernel must have
+`192.168.200.2` by default and for Server, it assigns `192.168.201.2` by default. Therefore, your Kernel must have
 `net.ipv4.ip_forward` enabled and setup appropriate iptables rules for NAT between your physical
 NIC address and Phantun's TUN interface address.

-## Enable Kernel IP forwarding
+You may customize the name of Tun interface created by Phantun and the assigned addresses. Please
+run the executable with `-h` options to see how to change them.
+
+Another way to help understand this network topology (please see the diagram above for an illustration of this topology):
+
+Phantun Client is like a machine with private IP address (`192.168.200.2`) behind a router.
+In order for it to reach the Internet, you will need to SNAT the private IP address before it's traffic
+leaves the NIC.
+
+Phantun Server is like a server with private IP address (`192.168.201.2`) behind a router.
+In order to access it from the Internet, you need to `DNAT` it's listening port on the router
+and change the destination IP address to where the server is listening for incoming connections.
+
+In those cases, the machine/iptables running Phantun acts as the "router" that allows Phantun
+to communicate with outside using it's private IP addresses.
+
+As of Phantun v0.2.2, IPv6 support for UDP endpoints has been added, however Fake TCP IPv6 support
+has not been finished yet. To specify an IPv6 address, use the following format: `[::1]:1234` with
+the command line options.
+
+[Back to TOC](#table-of-contents)
+
+## 1. Enable Kernel IP forwarding

 Edit `/etc/sysctl.conf`, add `net.ipv4.ip_forward=1` and run `sudo sysctl -p /etc/sysctl.conf`.

 [Back to TOC](#table-of-contents)

-## Add required firewall rules (using nftables as an example)
+## 2. Add required firewall rules
+

 ### Client

@@ -65,6 +114,10 @@ one that can be used on the physical network. This can be done simply with masqu

 Note: change `eth0` to whatever actual physical interface name is

+[Back to TOC](#table-of-contents)
+
+#### Using nftables
+
 ```
 table inet nat {
    chain postrouting {
@@ -76,12 +129,24 @@ table inet nat {

 [Back to TOC](#table-of-contents)

+#### Using iptables
+
+```
+iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
+```
+
+[Back to TOC](#table-of-contents)
+
 ### Server

 Server needs to DNAT the TCP listening port to Phantun's TUN interface address.

 Note: change `eth0` to whatever actual physical interface name is and `4567` to
-actual TCP port number used by Phanton server
+actual TCP port number used by Phantun server
+
+[Back to TOC](#table-of-contents)
+
+#### Using nftables

 ```
 table ip nat {
@@ -94,7 +159,15 @@ table ip nat {

 [Back to TOC](#table-of-contents)

-## Give Phantun binaries required capability to it can be run as non-root (Optional)
+#### Using iptables
+
+```
+iptables -t nat -A PREROUTING -p tcp -i eth0 --dport 4567 -j DNAT --to-destination 192.168.201.2
+```
+
+[Back to TOC](#table-of-contents)
+
+## 3. Run Phantun binaries as non-root (Optional)

 It is ill-advised to run network facing applications as root user. Phantun can be run fully
 as non-root user with the `cap_net_admin` capability.
@@ -107,7 +180,11 @@ sudo setcap cap_net_admin=+pe phantun_client

 [Back to TOC](#table-of-contents)

-## Start
+## 4. Start Phantun daemon
+
+**Note:** Run Phantun executable with `-h` option to see full detailed options.
+
+[Back to TOC](#table-of-contents)

 ### Server

@@ -118,6 +195,12 @@ rule specified above. `127.0.0.1:1234` is the UDP Server to connect to for new c
 RUST_LOG=info /usr/local/bin/phantun_server --local 4567 --remote 127.0.0.1:1234
 ```

+Or use host name with `--remote`:
+
+```
+RUST_LOG=info /usr/local/bin/phantun_server --local 4567 --remote example.com:1234
+```
+
 [Back to TOC](#table-of-contents)

 ### Client
@@ -129,6 +212,12 @@ the Phantun Server to connect.
 RUST_LOG=info /usr/local/bin/phantun_client --local 127.0.0.1:1234 --remote 10.0.0.1:4567
 ```

+Or use host name with `--remote`:
+
+```
+RUST_LOG=info /usr/local/bin/phantun_client --local 127.0.0.1:1234 --remote example.com:4567
+```
+
 [Back to TOC](#table-of-contents)

 # MTU overhead
@@ -136,13 +225,42 @@ RUST_LOG=info /usr/local/bin/phantun_client --local 127.0.0.1:1234 --remote 10.0
 Phantun aims to keep tunneling overhead to the minimum. The overhead compared to a plain UDP packet
 is the following:

-Plain UDP packet: 20 byte IP header + 8 byte UDP header = 28 bytes
-Phantun obfuscated UDP packet: 20 byte IP header + 20 byte TCP header = 40 bytes
+**Standard UDP packet:** `20 byte IP header + 8 byte UDP header = 28 bytes`

-Phantun's additional overhead: 12 bytes. I other words, when using Phantun, the usable payload for
+**Obfuscated packet:** `20 byte IP header + 20 byte TCP header = 40 bytes`
+
+
+Note that Phantun does not add any additional header other than IP and TCP headers in order to pass through
+stateful packet inspection!
+
+Phantun's additional overhead: `12 bytes`. I other words, when using Phantun, the usable payload for
 UDP packet is reduced by 12 bytes. This is the minimum overhead possible when doing such kind
 of obfuscation.

+![Packet header diagram](images/packet-headers.png)
+
+[Back to TOC](#table-of-contents)
+
+## MTU calculation for WireGuard
+
+For people who use Phantun to tunnel [WireGuard®](https://www.wireguard.com) UDP packets, here are some guidelines on figuring
+out the correct MTU to use for your WireGuard interface.
+
+```
+WireGuard MTU = Interface MTU - IP header (20 bytes) - TCP header (20 bytes) - WireGuard overhead (32 bytes)
+```
+
+For example, for a Ethernet interface with 1500 bytes MTU, the WireGuard interface MTU should be set as:
+
+```
+1500 - 20 - 20 - 32 = 1428 bytes
+```
+
+The resulted Phantun TCP data packet will be 1500 bytes which does not exceed the
+interface MTU of 1500. Please note it is strongly recommended to use the same interface
+MTU for both ends of a WireGuard tunnel, or unexected packet loss may occur and these issues are
+generally very hard to troubleshoot.
+
 [Back to TOC](#table-of-contents)

 # Version compatibility
@@ -152,30 +270,41 @@ of Server/Client of Phantun on both ends to ensure maximum compatibility.

 [Back to TOC](#table-of-contents)

+# Documentations
+
+For users who wish to use `fake-tcp` library inside their own project, refer to the documentations for the library at:
+[https://docs.rs/fake-tcp](https://docs.rs/fake-tcp).
+
+[Back to TOC](#table-of-contents)
+
 # Performance

-Performance was tested on AWS t3.xlarge instance with 4 vCPUs and 5 Gb/s NIC. WireGuard was used
-for tunneling TCP/UDP traffic between two test instances and MTU has been tuned to avoid fragmentation.
+Performance was tested on 2 AWS `t4g.xlarge` instances with 4 vCPUs and 5 Gb/s NIC over LAN. `nftables` was used to redirect
+UDP stream of `iperf3` to go through the Phantun/udp2raw tunnel between two test instances and MTU has been tuned to avoid fragmentation.

-|                 | WireGuard   | WireGuard + Phantun | WireGuard + udp2raw (cipher-mode=none auth-mode=none disable-anti-replay) |
-|-----------------|-------------|---------------------|---------------------------------------------------------------------------|
-| iperf3 -c IP -R | 1.56 Gbit/s | 540 Mbit/s          | 369 Mbit/s                                                                |
-| iperf3 -c IP    | 1.71 Gbit/s | 519 Mbit/s          | 312 Mbit/s                                                                |
+Test command: `iperf3 -c <IP> -p <PORT> -R -u -l 1400 -b 1000m -t 30 -P 5`
+
+| Mode                                                          | Speed          | Overall CPU Usage        |
+|---------------------------------------------------------------|----------------|--------------------------|
+| Direct connection                                             | 3.35 Gbits/sec | 25% (1 core at 100%)     |
+| Phantun                                                       | 2.03 Gbits/sec | 95% (all cores utilized) |
+| udp2raw (cipher-mode=none auth-mode=none disable-anti-replay) | 876 Mbits/sec  | 50% (2 cores at 100%)    |

 [Back to TOC](#table-of-contents)

 # Future plans

-* IPv6 support
+* IPv6 support for fake-tcp
 * Load balancing a single UDP stream into multiple TCP streams
-* Iteration tests
+* Integration tests
+* Auto insertion/removal of required firewall rules

 [Back to TOC](#table-of-contents)

 # Compariation to udp2raw
 [udp2raw](https://github.com/wangyu-/udp2raw-tunnel) is another popular project by [@wangyu-](https://github.com/wangyu-)
 that is very similar to what Phantun can do. In fact I took inspirations of Phantun from udp2raw. The biggest reason for
-developing Phanton is because of lack of performance when running udp2raw (especially on multi-core systems such as Raspberry Pi).
+developing Phantun is because of lack of performance when running udp2raw (especially on multi-core systems such as Raspberry Pi).
 However, the goal is never to be as feature complete as udp2raw and only support the most common use cases. Most notably, UDP over ICMP
 and UDP over UDP mode are not supported and there is no anti-replay nor encryption support. The benefit of this is much better
 performance overall and less MTU overhead because lack of additional headers inside the TCP payload.
@@ -189,17 +318,17 @@ Here is a quick overview of comparison between those two to help you choose:
 | UDP over UDP obfuscation                         |       ❌       |         ✅         |
 | Multi-threaded                                   |       ✅       |         ❌         |
 | Throughput                                       |     Better    |        Good       |
-| Raw IP mode                                      | TUN interface | Raw sockets + BPF |
+| L4 IP mode                                       | TUN interface | Raw sockets + BPF |
 | Tunneling MTU overhead                           |    12 bytes   |      44 bytes     |
 | Seprate TCP connections for each UDP connection  | Client/Server |    Server only    |
 | Anti-replay, encryption                          |       ❌       |         ✅         |
-| IPv6                                             |    Planned    |         ✅         |
+| IPv6                                             |    UDP only    |         ✅         |

 [Back to TOC](#table-of-contents)

 # License

-Copyright 2021 Datong Sun <dndx@idndx.com>
+Copyright 2021-2022 Datong Sun (dndx@idndx.com)

 Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
 [https://www.apache.org/licenses/LICENSE-2.0](https://www.apache.org/licenses/LICENSE-2.0)> or the MIT license
--- a/fake-tcp/Cargo.toml
+++ b/fake-tcp/Cargo.toml
@@ -1,7 +1,7 @@
 [package]
 name = "fake-tcp"
-version = "0.1.0"
-edition = "2018"
+version = "0.3.1"
+edition = "2021"
 authors = ["Datong Sun <dndx@idndx.com>"]
 license = "MIT OR Apache-2.0"
 repository = "https://github.com/dndx/phantun"
@@ -10,10 +10,16 @@ description = """
 A TUN interface based, user space, asynchronous and high performance TCP stack that allows
 packet oriented tunneling with minimum overhead.
 """
+
+[features]
+benchmark = []
+
 [dependencies]
 bytes = "1"
-pnet = "0.28.0"
-tokio-tun = "0.3.15"
-tokio = { version = "1.11.0", features = ["full"] }
-rand = { version = "0.8.4", features = ["small_rng"] }
+pnet = "0.29"
+tokio = { version = "1.14", features = ["full"] }
+rand = { version = "0.8", features = ["small_rng"] }
 log = "0.4"
+internet-checksum = "0.2"
+tokio-tun = "0.5"
+flume = "0.10"
--- a/fake-tcp/LICENSE-APACHE
+++ b/fake-tcp/LICENSE-APACHE
@@ -186,7 +186,7 @@ APPENDIX: How to apply the Apache License to your work.
   same "printed page" as the copyright notice for easier
   identification within third-party archives.

-Copyright 2014-2021 The Rust Project Developers
+Copyright 2021-2022 Datong Sun (dndx@idndx.com)

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
--- a/fake-tcp/LICENSE-MIT
+++ b/fake-tcp/LICENSE-MIT
@@ -1,6 +1,6 @@
 MIT License

-Copyright (c) 2014-2021 The Rust Project Developers
+Copyright (c) 2021-2022 Datong Sun (dndx@idndx.com)

 Permission is hereby granted, free of charge, to any
 person obtaining a copy of this software and associated
--- a/fake-tcp/src/lib.rs
+++ b/fake-tcp/src/lib.rs
@@ -1,7 +1,49 @@
+//! A minimum, userspace TCP based datagram stack
+//!
+//! # Overview
+//!
+//! `fake-tcp` is a reusable library that implements a minimum TCP stack in
+//! user space using the Tun interface. It allows programs to send datagrams
+//! as if they are part of a TCP connection. `fake-tcp` has been tested to
+//! be able to pass through a variety of NAT and stateful firewalls while
+//! fully preserves certain desirable behavior such as out of order delivery
+//! and no congestion/flow controls.
+//!
+//! # Core Concepts
+//!
+//! The core of the `fake-tcp` crate compose of two structures. [`Stack`] and
+//! [`Socket`].
+//!
+//! ## [`Stack`]
+//!
+//! [`Stack`] represents a virtual TCP stack that operates at
+//! Layer 3. It is responsible for:
+//!
+//! * TCP active and passive open and handshake
+//! * `RST` handling
+//! * Interact with the Tun interface at Layer 3
+//! * Distribute incoming datagrams to corresponding [`Socket`]
+//!
+//! ## [`Socket`]
+//!
+//! [`Socket`] represents a TCP connection. It registers the identifying
+//! tuple `(src_ip, src_port, dest_ip, dest_port)` inside the [`Stack`] so
+//! so that incoming packets can be distributed to the right [`Socket`] with
+//! using a channel. It is also what the client should use for
+//! sending/receiving datagrams.
+//!
+//! # Examples
+//!
+//! Please see [`client.rs`](https://github.com/dndx/phantun/blob/main/phantun/src/bin/client.rs)
+//! and [`server.rs`](https://github.com/dndx/phantun/blob/main/phantun/src/bin/server.rs) files
+//! from the `phantun` crate for how to use this library in client/server mode, respectively.
+
+#![cfg_attr(feature = "benchmark", feature(test))]
+
 pub mod packet;

 use bytes::{Bytes, BytesMut};
-use log::{info, trace};
+use log::{error, info, trace, warn};
 use packet::*;
 use pnet::packet::{tcp, Packet};
 use rand::prelude::*;
@@ -10,19 +52,19 @@ use std::fmt;
 use std::net::{Ipv4Addr, SocketAddrV4};
 use std::sync::atomic::{AtomicU32, Ordering};
 use std::sync::{Arc, RwLock};
-use tokio::io::{AsyncReadExt, AsyncWriteExt};
-use tokio::sync::mpsc::{self, Receiver, Sender};
-use tokio::sync::watch;
-use tokio::sync::Mutex as AsyncMutex;
-use tokio::{io, time};
+use tokio::sync::broadcast;
+use tokio::sync::mpsc;
+use tokio::time;
 use tokio_tun::Tun;

 const TIMEOUT: time::Duration = time::Duration::from_secs(1);
 const RETRIES: usize = 6;
-const MPSC_BUFFER_LEN: usize = 512;
+const MPMC_BUFFER_LEN: usize = 512;
+const MPSC_BUFFER_LEN: usize = 128;
+const MAX_UNACKED_LEN: u32 = 128 * 1024 * 1024; // 128MB

-#[derive(Debug, Hash, Eq, PartialEq)]
-pub struct AddrTuple {
+#[derive(Hash, Eq, PartialEq, Clone, Debug)]
+struct AddrTuple {
    local_addr: SocketAddrV4,
    remote_addr: SocketAddrV4,
 }
@@ -36,21 +78,20 @@ impl AddrTuple {
    }
 }

-#[derive(Debug)]
 struct Shared {
-    tuples: RwLock<HashMap<AddrTuple, Arc<Sender<Bytes>>>>,
+    tuples: RwLock<HashMap<AddrTuple, flume::Sender<Bytes>>>,
    listening: RwLock<HashSet<u16>>,
-    outgoing: Sender<Bytes>,
-    ready: Sender<Socket>,
+    tun: Vec<Arc<Tun>>,
+    ready: mpsc::Sender<Socket>,
+    tuples_purge: broadcast::Sender<AddrTuple>,
 }

 pub struct Stack {
    shared: Arc<Shared>,
    local_ip: Ipv4Addr,
-    ready: Receiver<Socket>,
+    ready: mpsc::Receiver<Socket>,
 }

-#[derive(Debug)]
 pub enum State {
    Idle,
    SynSent,
@@ -58,138 +99,138 @@ pub enum State {
    Established,
 }

-#[derive(Debug)]
-pub enum Mode {
-    Client,
-    Server,
-}
-
-#[derive(Debug)]
 pub struct Socket {
-    mode: Mode,
    shared: Arc<Shared>,
-    incoming: AsyncMutex<Receiver<Bytes>>,
+    tun: Arc<Tun>,
+    incoming: flume::Receiver<Bytes>,
    local_addr: SocketAddrV4,
    remote_addr: SocketAddrV4,
    seq: AtomicU32,
    ack: AtomicU32,
+    last_ack: AtomicU32,
    state: State,
-    closing_tx: watch::Sender<()>,
-    closing_rx: watch::Receiver<()>,
 }

+/// A socket that represents a unique TCP connection between a server and client.
+///
+/// The `Socket` object itself satisfies `Sync` and `Send`, which means it can
+/// be safely called within an async future.
+///
+/// To close a TCP connection that is no longer needed, simply drop this object
+/// out of scope.
 impl Socket {
    fn new(
-        mode: Mode,
        shared: Arc<Shared>,
+        tun: Arc<Tun>,
        local_addr: SocketAddrV4,
        remote_addr: SocketAddrV4,
        ack: Option<u32>,
        state: State,
-    ) -> (Socket, Sender<Bytes>) {
-        let (incoming_tx, incoming_rx) = mpsc::channel(MPSC_BUFFER_LEN);
-        let (closing_tx, closing_rx) = watch::channel(());
+    ) -> (Socket, flume::Sender<Bytes>) {
+        let (incoming_tx, incoming_rx) = flume::bounded(MPMC_BUFFER_LEN);

        (
            Socket {
-                mode,
                shared,
-                incoming: AsyncMutex::new(incoming_rx),
+                tun,
+                incoming: incoming_rx,
                local_addr,
                remote_addr,
                seq: AtomicU32::new(0),
                ack: AtomicU32::new(ack.unwrap_or(0)),
+                last_ack: AtomicU32::new(ack.unwrap_or(0)),
                state,
-                closing_tx,
-                closing_rx,
            },
            incoming_tx,
        )
    }

    fn build_tcp_packet(&self, flags: u16, payload: Option<&[u8]>) -> Bytes {
+        let ack = self.ack.load(Ordering::Relaxed);
+        self.last_ack.store(ack, Ordering::Relaxed);
+
        build_tcp_packet(
            self.local_addr,
            self.remote_addr,
            self.seq.load(Ordering::Relaxed),
-            self.ack.load(Ordering::Relaxed),
+            ack,
            flags,
            payload,
        )
    }

+    /// Sends a datagram to the other end.
+    ///
+    /// This method takes `&self`, and it can be called safely by multiple threads
+    /// at the same time.
+    ///
+    /// A return of `None` means the Tun socket returned an error
+    /// and this socket must be closed.
    pub async fn send(&self, payload: &[u8]) -> Option<()> {
-        let mut closing = self.closing_rx.clone();
-
        match self.state {
            State::Established => {
                let buf = self.build_tcp_packet(tcp::TcpFlags::ACK, Some(payload));
-                self.seq.fetch_add(buf.len() as u32, Ordering::Relaxed);
-
-                tokio::select! {
-                    res = self.shared.outgoing.send(buf) => {
-                        res.unwrap();
-                        Some(())
-                    },
-                    _ = closing.changed() => {
-                        None
-                    }
-                }
+                self.seq.fetch_add(payload.len() as u32, Ordering::Relaxed);
+                self.tun.send(&buf).await.ok().and(Some(()))
            }
            _ => unreachable!(),
        }
    }

+    /// Attempt to receive a datagram from the other end.
+    ///
+    /// This method takes `&self`, and it can be called safely by multiple threads
+    /// at the same time.
+    ///
+    /// A return of `None` means the TCP connection is broken
+    /// and this socket must be closed.
    pub async fn recv(&self, buf: &mut [u8]) -> Option<usize> {
-        let mut closing = self.closing_rx.clone();
-
        match self.state {
            State::Established => {
-                let mut incoming = self.incoming.lock().await;
-                tokio::select! {
-                    Some(raw_buf) = incoming.recv() => {
-                        let (_v4_packet, tcp_packet) = parse_ipv4_packet(&raw_buf);
+                self.incoming.recv_async().await.ok().and_then(|raw_buf| {
+                    let (_v4_packet, tcp_packet) = parse_ipv4_packet(&raw_buf);

-                        if (tcp_packet.get_flags() & tcp::TcpFlags::RST) != 0 {
-                            info!("Connection {} reset by peer", self);
-                            self.close();
-                            return None;
-                        }
-
-                        let payload = tcp_packet.payload();
-
-                        self.ack
-                            .store(tcp_packet.get_sequence().wrapping_add(1), Ordering::Relaxed);
-
-                        buf[..payload.len()].copy_from_slice(payload);
-
-                        Some(payload.len())
-                    },
-                    _ = closing.changed() => {
-                        None
+                    if (tcp_packet.get_flags() & tcp::TcpFlags::RST) != 0 {
+                        info!("Connection {} reset by peer", self);
+                        return None;
                    }
-                }
+
+                    let payload = tcp_packet.payload();
+
+                    let new_ack = tcp_packet.get_sequence().wrapping_add(payload.len() as u32);
+                    let last_ask = self.last_ack.load(Ordering::Relaxed);
+                    self.ack.store(new_ack, Ordering::Relaxed);
+
+                    if new_ack.overflowing_sub(last_ask).0 > MAX_UNACKED_LEN {
+                        let buf = self.build_tcp_packet(tcp::TcpFlags::ACK, None);
+                        if let Err(e) = self.tun.try_send(&buf) {
+                            // This should not really happen as we have not sent anything for
+                            // quite some time...
+                            info!("Connection {} unable to send idling ACK back: {}", self, e)
+                        }
+                    }
+
+                    buf[..payload.len()].copy_from_slice(payload);
+
+                    Some(payload.len())
+                })
            }
            _ => unreachable!(),
        }
    }

-    pub fn close(&self) {
-        self.closing_tx.send(()).unwrap();
-    }
-
    async fn accept(mut self) {
        for _ in 0..RETRIES {
            match self.state {
                State::Idle => {
                    let buf = self.build_tcp_packet(tcp::TcpFlags::SYN | tcp::TcpFlags::ACK, None);
                    // ACK set by constructor
-                    self.shared.outgoing.send(buf).await.unwrap();
+                    self.tun.send(&buf).await.unwrap();
                    self.state = State::SynReceived;
                    info!("Sent SYN + ACK to client");
                }
                State::SynReceived => {
-                    let res = time::timeout(TIMEOUT, self.incoming.lock().await.recv()).await;
+                    let res = time::timeout(TIMEOUT, self.incoming.recv_async()).await;
                    if let Ok(buf) = res {
                        let buf = buf.unwrap();
                        let (_v4_packet, tcp_packet) = parse_ipv4_packet(&buf);
@@ -208,7 +249,9 @@ impl Socket {

                            info!("Connection from {:?} established", self.remote_addr);
                            let ready = self.shared.ready.clone();
-                            ready.send(self).await.unwrap();
+                            if let Err(e) = ready.send(self).await {
+                                error!("Unable to send accepted socket to ready queue: {}", e);
+                            }
                            return;
                        }
                    } else {
@@ -226,12 +269,12 @@ impl Socket {
            match self.state {
                State::Idle => {
                    let buf = self.build_tcp_packet(tcp::TcpFlags::SYN, None);
-                    self.shared.outgoing.send(buf).await.unwrap();
+                    self.tun.send(&buf).await.unwrap();
                    self.state = State::SynSent;
                    info!("Sent SYN to server");
                }
                State::SynSent => {
-                    match time::timeout(TIMEOUT, self.incoming.lock().await.recv()).await {
+                    match time::timeout(TIMEOUT, self.incoming.recv_async()).await {
                        Ok(buf) => {
                            let buf = buf.unwrap();
                            let (_v4_packet, tcp_packet) = parse_ipv4_packet(&buf);
@@ -251,7 +294,7 @@ impl Socket {

                                // send ACK to finish handshake
                                let buf = self.build_tcp_packet(tcp::TcpFlags::ACK, None);
-                                self.shared.outgoing.send(buf).await.unwrap();
+                                self.tun.send(&buf).await.unwrap();

                                self.state = State::Established;

@@ -274,24 +317,32 @@ impl Socket {
 }

 impl Drop for Socket {
+    /// Drop the socket and close the TCP connection
    fn drop(&mut self) {
+        let tuple = AddrTuple::new(self.local_addr, self.remote_addr);
        // dissociates ourself from the dispatch map
-        assert!(self
-            .shared
-            .tuples
-            .write()
-            .unwrap()
-            .remove(&AddrTuple::new(self.local_addr, self.remote_addr))
-            .is_some());
+        assert!(self.shared.tuples.write().unwrap().remove(&tuple).is_some());
+        // purge cache
+        self.shared.tuples_purge.send(tuple).unwrap();
+
+        let buf = build_tcp_packet(
+            self.local_addr,
+            self.remote_addr,
+            self.seq.load(Ordering::Relaxed),
+            0,
+            tcp::TcpFlags::RST,
+            None,
+        );
+        if let Err(e) = self.tun.try_send(&buf) {
+            warn!("Unable to send RST to remote end: {}", e);
+        }

-        let buf = self.build_tcp_packet(tcp::TcpFlags::RST, None);
-        self.shared.outgoing.try_send(buf).unwrap();
-        self.close();
        info!("Fake TCP connection to {} closed", self);
    }
 }

 impl fmt::Display for Socket {
+    /// User-friendly string representation of the socket
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        write!(
            f,
@@ -301,19 +352,33 @@ impl fmt::Display for Socket {
    }
 }

+/// A userspace TCP state machine
 impl Stack {
-    pub fn new(tun: Tun) -> Stack {
-        let (outgoing_tx, outgoing_rx) = mpsc::channel(MPSC_BUFFER_LEN);
+    /// Create a new stack, `tun` is an array of [`Tun`](tokio_tun::Tun).
+    /// When more than one [`Tun`](tokio_tun::Tun) object is passed in, same amount
+    /// of reader will be spawned later. This allows user to utilize the performance
+    /// benefit of Multiqueue Tun support on machines with SMP.
+    pub fn new(tun: Vec<Tun>) -> Stack {
+        let tun: Vec<Arc<Tun>> = tun.into_iter().map(Arc::new).collect();
        let (ready_tx, ready_rx) = mpsc::channel(MPSC_BUFFER_LEN);
+        let (tuples_purge_tx, _tuples_purge_rx) = broadcast::channel(16);
        let shared = Arc::new(Shared {
            tuples: RwLock::new(HashMap::new()),
-            outgoing: outgoing_tx,
+            tun: tun.clone(),
            listening: RwLock::new(HashSet::new()),
            ready: ready_tx,
+            tuples_purge: tuples_purge_tx.clone(),
        });
-        let local_ip = tun.destination().unwrap();
+        let local_ip = tun[0].destination().unwrap();
+
+        for t in tun {
+            tokio::spawn(Stack::reader_task(
+                t,
+                shared.clone(),
+                tuples_purge_tx.subscribe(),
+            ));
+        }

-        tokio::spawn(Stack::dispatch(tun, outgoing_rx, shared.clone()));
        Stack {
            shared,
            local_ip,
@@ -321,22 +386,26 @@ impl Stack {
        }
    }

+    /// Listens for incoming connections on the given `port`.
    pub fn listen(&mut self, port: u16) {
        assert!(self.shared.listening.write().unwrap().insert(port));
    }

+    /// Accepts an incoming connection.
    pub async fn accept(&mut self) -> Socket {
        self.ready.recv().await.unwrap()
    }

+    /// Connects to the remote end. `None` returned means
+    /// the connection attempt failed.
    pub async fn connect(&mut self, addr: SocketAddrV4) -> Option<Socket> {
        let mut rng = SmallRng::from_entropy();
        let local_port: u16 = rng.gen_range(1024..65535);
        let local_addr = SocketAddrV4::new(self.local_ip, local_port);
        let tuple = AddrTuple::new(local_addr, addr);
        let (mut sock, incoming) = Socket::new(
-            Mode::Client,
            self.shared.clone(),
+            self.shared.tun.choose(&mut rng).unwrap().clone(),
            local_addr,
            addr,
            None,
@@ -345,53 +414,88 @@ impl Stack {

        {
            let mut tuples = self.shared.tuples.write().unwrap();
-            assert!(tuples.insert(tuple, Arc::new(incoming.clone())).is_none());
+            assert!(tuples.insert(tuple, incoming.clone()).is_none());
        }

        sock.connect().await.map(|_| sock)
    }

-    async fn dispatch(tun: Tun, mut outgoing: Receiver<Bytes>, shared: Arc<Shared>) {
-        let (mut tun_r, mut tun_w) = io::split(tun);
+    async fn reader_task(
+        tun: Arc<Tun>,
+        shared: Arc<Shared>,
+        mut tuples_purge: broadcast::Receiver<AddrTuple>,
+    ) {
+        let mut tuples: HashMap<AddrTuple, flume::Sender<Bytes>> = HashMap::new();

        loop {
            let mut buf = BytesMut::with_capacity(MAX_PACKET_LEN);
+            buf.resize(MAX_PACKET_LEN, 0);

            tokio::select! {
-                buf = outgoing.recv() => {
-                    let buf = buf.unwrap();
-                    tun_w.write_all(&buf).await.unwrap();
-                },
-                s = tun_r.read_buf(&mut buf) => {
-                    s.unwrap();
+                size = tun.recv(&mut buf) => {
+                    let size = size.unwrap();
+                    buf.truncate(size);
                    let buf = buf.freeze();
+
                    if buf[0] >> 4 != 4 {
                        // not an IPv4 packet
                        continue;
                    }

                    let (ip_packet, tcp_packet) = parse_ipv4_packet(&buf);
-                    let local_addr = SocketAddrV4::new(ip_packet.get_destination(), tcp_packet.get_destination());
+                    let local_addr =
+                        SocketAddrV4::new(ip_packet.get_destination(), tcp_packet.get_destination());
                    let remote_addr = SocketAddrV4::new(ip_packet.get_source(), tcp_packet.get_source());

                    let tuple = AddrTuple::new(local_addr, remote_addr);
+                    if let Some(c) = tuples.get(&tuple) {
+                        if c.send_async(buf).await.is_err() {
+                            trace!("Cache hit, but receiver already closed, dropping packet");
+                        }

-                    let sender;
-                    {
-                        let tuples = shared.tuples.read().unwrap();
-                        sender = tuples.get(&tuple).cloned();
-                    }
-
-                    if let Some(c) = sender {
-                        c.send(buf).await.unwrap();
                        continue;
+
+                        // If not Ok, receiver has been closed and just fall through to the slow
+                        // path below
+
+                    } else {
+                        trace!("Cache miss, checking the shared tuples table for connection");
+                        let sender = {
+                            let tuples = shared.tuples.read().unwrap();
+                            tuples.get(&tuple).cloned()
+                        };
+
+                        if let Some(c) = sender {
+                            trace!("Storing connection information into local tuples");
+                            tuples.insert(tuple, c.clone());
+                            c.send_async(buf).await.unwrap();
+                            continue;
+                        }
                    }

-                    if tcp_packet.get_flags() == tcp::TcpFlags::SYN && shared.listening.read().unwrap().contains(&tcp_packet.get_destination()) {
+                    if tcp_packet.get_flags() == tcp::TcpFlags::SYN
+                        && shared
+                            .listening
+                            .read()
+                            .unwrap()
+                            .contains(&tcp_packet.get_destination())
+                    {
                        // SYN seen on listening socket
                        if tcp_packet.get_sequence() == 0 {
-                            let (sock, incoming) = Socket::new(Mode::Server, shared.clone(), local_addr, remote_addr, Some(tcp_packet.get_sequence() + 1), State::Idle);
-                            assert!(shared.tuples.write().unwrap().insert(tuple, Arc::new(incoming)).is_none());
+                            let (sock, incoming) = Socket::new(
+                                shared.clone(),
+                                tun.clone(),
+                                local_addr,
+                                remote_addr,
+                                Some(tcp_packet.get_sequence() + 1),
+                                State::Idle,
+                            );
+                            assert!(shared
+                                .tuples
+                                .write()
+                                .unwrap()
+                                .insert(tuple, incoming)
+                                .is_none());
                            tokio::spawn(sock.accept());
                        } else {
                            trace!("Bad TCP SYN packet from {}, sending RST", remote_addr);
@@ -399,11 +503,11 @@ impl Stack {
                                local_addr,
                                remote_addr,
                                0,
-                                tcp_packet.get_sequence() + 1,
-                                tcp::TcpFlags::RST,
+                                tcp_packet.get_sequence() + tcp_packet.payload().len() as u32 + 1, // +1 because of SYN flag set
+                                tcp::TcpFlags::RST | tcp::TcpFlags::ACK,
                                None,
                            );
-                            shared.outgoing.try_send(buf).unwrap();
+                            shared.tun[0].try_send(&buf).unwrap();
                        }
                    } else if (tcp_packet.get_flags() & tcp::TcpFlags::RST) == 0 {
                        info!("Unknown TCP packet from {}, sending RST", remote_addr);
@@ -411,12 +515,17 @@ impl Stack {
                            local_addr,
                            remote_addr,
                            tcp_packet.get_acknowledgement(),
-                            0,
-                            tcp::TcpFlags::RST,
+                            tcp_packet.get_sequence() + tcp_packet.payload().len() as u32,
+                            tcp::TcpFlags::RST | tcp::TcpFlags::ACK,
                            None,
                        );
-                        shared.outgoing.try_send(buf).unwrap();
+                        shared.tun[0].try_send(&buf).unwrap();
                    }
+                },
+                tuple = tuples_purge.recv() => {
+                    let tuple = tuple.unwrap();
+                    tuples.remove(&tuple);
+                    trace!("Removed cached tuple: {:?}", tuple);
                }
            }
        }
--- a/fake-tcp/src/packet.rs
+++ b/fake-tcp/src/packet.rs
@@ -1,4 +1,6 @@
 use bytes::{Bytes, BytesMut};
+use internet_checksum::Checksum;
+use pnet::packet::Packet;
 use pnet::packet::{ip, ipv4, tcp};
 use std::convert::TryInto;
 use std::net::SocketAddrV4;
@@ -16,8 +18,8 @@ pub fn build_tcp_packet(
    payload: Option<&[u8]>,
 ) -> Bytes {
    let wscale = (flags & tcp::TcpFlags::SYN) != 0;
-    let tcp_total_len = TCP_HEADER_LEN + if wscale {4} else {0} // nop + wscale
-                        + payload.map_or(0, |payload| payload.len());
+    let tcp_header_len = TCP_HEADER_LEN + if wscale { 4 } else { 0 }; // nop + wscale
+    let tcp_total_len = tcp_header_len + payload.map_or(0, |payload| payload.len());
    let total_len = IPV4_HEADER_LEN + tcp_total_len;
    let mut buf = BytesMut::with_capacity(total_len);
    buf.resize(total_len, 0);
@@ -30,12 +32,14 @@ pub fn build_tcp_packet(
    v4.set_version(4);
    v4.set_header_length(IPV4_HEADER_LEN as u8 / 4);
    v4.set_next_level_protocol(ip::IpNextHeaderProtocols::Tcp);
-    v4.set_ttl(32);
+    v4.set_ttl(64);
    v4.set_source(*local_addr.ip());
    v4.set_destination(*remote_addr.ip());
    v4.set_total_length(total_len.try_into().unwrap());
    v4.set_flags(ipv4::Ipv4Flags::DontFragment);
-    v4.set_checksum(ipv4::checksum(&v4.to_immutable()));
+    let mut cksm = Checksum::new();
+    cksm.add_bytes(v4.packet());
+    v4.set_checksum(u16::from_be_bytes(cksm.checksum()));

    let mut tcp = tcp::MutableTcpPacket::new(&mut tcp_buf).unwrap();
    tcp.set_window(0xffff);
@@ -54,11 +58,17 @@ pub fn build_tcp_packet(
        tcp.set_payload(payload);
    }

-    let checksum = tcp::ipv4_checksum(&tcp.to_immutable(), local_addr.ip(), remote_addr.ip());
-    tcp.set_checksum(checksum);
+    let mut cksm = Checksum::new();
+    cksm.add_bytes(&local_addr.ip().octets());
+    cksm.add_bytes(&remote_addr.ip().octets());
+    let ip::IpNextHeaderProtocol(tcp_protocol) = ip::IpNextHeaderProtocols::Tcp;
+    let mut pseudo = [0u8, tcp_protocol, 0, 0];
+    pseudo[2..].copy_from_slice(&(tcp_total_len as u16).to_be_bytes());
+    cksm.add_bytes(&pseudo);
+    cksm.add_bytes(tcp.packet());
+    tcp.set_checksum(u16::from_be_bytes(cksm.checksum()));

    v4_buf.unsplit(tcp_buf);
-
    v4_buf.freeze()
 }

@@ -68,3 +78,61 @@ pub fn parse_ipv4_packet(buf: &Bytes) -> (ipv4::Ipv4Packet, tcp::TcpPacket) {

    (v4, tcp)
 }
+
+#[cfg(all(test, feature = "benchmark"))]
+mod benchmarks {
+    extern crate test;
+    use super::*;
+    use test::{black_box, Bencher};
+
+    #[bench]
+    fn bench_build_tcp_packet_1460(b: &mut Bencher) {
+        let local_addr = "127.0.0.1:1234".parse().unwrap();
+        let remote_addr = "127.0.0.2:1234".parse().unwrap();
+        let payload = black_box([123u8; 1460]);
+        b.iter(|| {
+            build_tcp_packet(
+                local_addr,
+                remote_addr,
+                123,
+                456,
+                tcp::TcpFlags::ACK,
+                Some(&payload),
+            )
+        });
+    }
+
+    #[bench]
+    fn bench_build_tcp_packet_512(b: &mut Bencher) {
+        let local_addr = "127.0.0.1:1234".parse().unwrap();
+        let remote_addr = "127.0.0.2:1234".parse().unwrap();
+        let payload = black_box([123u8; 512]);
+        b.iter(|| {
+            build_tcp_packet(
+                local_addr,
+                remote_addr,
+                123,
+                456,
+                tcp::TcpFlags::ACK,
+                Some(&payload),
+            )
+        });
+    }
+
+    #[bench]
+    fn bench_build_tcp_packet_128(b: &mut Bencher) {
+        let local_addr = "127.0.0.1:1234".parse().unwrap();
+        let remote_addr = "127.0.0.2:1234".parse().unwrap();
+        let payload = black_box([123u8; 128]);
+        b.iter(|| {
+            build_tcp_packet(
+                local_addr,
+                remote_addr,
+                123,
+                456,
+                tcp::TcpFlags::ACK,
+                Some(&payload),
+            )
+        });
+    }
+}
--- a/images/packet-headers.png
+++ b/images/packet-headers.png
--- a/images/traffic-flow.png
+++ b/images/traffic-flow.png
--- a/phantun/Cargo.toml
+++ b/phantun/Cargo.toml
@@ -1,20 +1,22 @@
 [package]
 name = "phantun"
-version = "0.1.0"
-edition = "2018"
+version = "0.3.2"
+edition = "2021"
 authors = ["Datong Sun <dndx@idndx.com>"]
 license = "MIT OR Apache-2.0"
 repository = "https://github.com/dndx/phantun"
 readme = "README.md"
 description = """
-Turns transforms UDP stream into (fake) TCP streams that can go through
-Layer 4 firewalls.
+Transforms UDP stream into (fake) TCP streams that can go through
+Layer 3 & Layer 4 (NAPT) firewalls/NATs.
 """
 [dependencies]
-clap = "2.33.3"
-socket2 = { version = "0.4.2", features = ["all"] }
-fake-tcp = "0.1.0"
-tokio-tun = "0.3.15"
-tokio = { version = "1.11.0", features = ["full"] }
+clap = { version = "3.0", features = ["cargo"] }
+socket2 = { version = "0.4", features = ["all"] }
+fake-tcp = { path = "../fake-tcp", version = "0.3" }
+tokio = { version = "1.14", features = ["full"] }
+tokio-util = "0.7"
 log = "0.4"
-pretty_env_logger = "0.4.0"
+pretty_env_logger = "0.4"
+tokio-tun = "0.5"
+num_cpus = "1.13"
--- a/phantun/LICENSE-APACHE
+++ b/phantun/LICENSE-APACHE
@@ -186,7 +186,7 @@ APPENDIX: How to apply the Apache License to your work.
   same "printed page" as the copyright notice for easier
   identification within third-party archives.

-Copyright 2014-2021 The Rust Project Developers
+Copyright 2021-2022 Datong Sun (dndx@idndx.com)

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
--- a/phantun/LICENSE-MIT
+++ b/phantun/LICENSE-MIT
@@ -1,6 +1,6 @@
 MIT License

-Copyright (c) 2014-2021 The Rust Project Developers
+Copyright (c) 2021-2022 Datong Sun (dndx@idndx.com)

 Permission is hereby granted, free of charge, to any
 person obtaining a copy of this software and associated
--- a/phantun/src/bin/client.rs
+++ b/phantun/src/bin/client.rs
@@ -1,82 +1,120 @@
-use clap::{App, Arg};
+use clap::{crate_version, Arg, Command};
 use fake_tcp::packet::MAX_PACKET_LEN;
 use fake_tcp::{Socket, Stack};
 use log::{debug, error, info};
+use phantun::utils::new_udp_reuseport;
 use std::collections::HashMap;
-use std::convert::TryInto;
-use std::net::{SocketAddr, SocketAddrV4};
+use std::net::{Ipv4Addr, SocketAddr};
 use std::sync::Arc;
-use std::time::Duration;
-use tokio::net::UdpSocket;
-use tokio::sync::RwLock;
+use tokio::sync::{Notify, RwLock};
 use tokio::time;
 use tokio_tun::TunBuilder;
+use tokio_util::sync::CancellationToken;

-const UDP_TTL: Duration = Duration::from_secs(180);
-
-fn new_udp_reuseport(addr: SocketAddrV4) -> UdpSocket {
-    let udp_sock = socket2::Socket::new(socket2::Domain::IPV4, socket2::Type::DGRAM, None).unwrap();
-    udp_sock.set_reuse_port(true).unwrap();
-    // from tokio-rs/mio/blob/master/src/sys/unix/net.rs
-    udp_sock.set_cloexec(true).unwrap();
-    udp_sock.set_nonblocking(true).unwrap();
-    udp_sock.bind(&socket2::SockAddr::from(addr)).unwrap();
-    let udp_sock: std::net::UdpSocket = udp_sock.into();
-    udp_sock.try_into().unwrap()
-}
+use phantun::UDP_TTL;

 #[tokio::main]
 async fn main() {
    pretty_env_logger::init();

-    let matches = App::new("Phantun Client")
-        .version("1.0")
-        .author("dndx@GitHub")
+    let matches = Command::new("Phantun Client")
+        .version(crate_version!())
+        .author("Datong Sun (github.com/dndx)")
        .arg(
-            Arg::with_name("local")
-                .short("l")
+            Arg::new("local")
+                .short('l')
                .long("local")
                .required(true)
                .value_name("IP:PORT")
-                .help("Sets the listening socket address")
+                .help("Sets the IP and port where Phantun Client listens for incoming UDP datagrams, IPv6 address need to be specified as: \"[IPv6]:PORT\"")
                .takes_value(true),
        )
        .arg(
-            Arg::with_name("remote")
-                .short("r")
+            Arg::new("remote")
+                .short('r')
                .long("remote")
                .required(true)
-                .value_name("IP:PORT")
-                .help("Sets the connecting socket address")
+                .value_name("IP or HOST NAME:PORT")
+                .help("Sets the address or host name and port where Phantun Client connects to Phantun Server")
+                .takes_value(true),
+        )
+        .arg(
+            Arg::new("tun")
+                .long("tun")
+                .required(false)
+                .value_name("tunX")
+                .help("Sets the Tun interface name, if absent, pick the next available name")
+                .default_value("")
+                .takes_value(true),
+        )
+        .arg(
+            Arg::new("tun_local")
+                .long("tun-local")
+                .required(false)
+                .value_name("IP")
+                .help("Sets the Tun interface local address (O/S's end)")
+                .default_value("192.168.200.1")
+                .takes_value(true),
+        )
+        .arg(
+            Arg::new("tun_peer")
+                .long("tun-peer")
+                .required(false)
+                .value_name("IP")
+                .help("Sets the Tun interface destination (peer) address (Phantun Client's end). \
+                       You will need to setup SNAT/MASQUERADE rules on your Internet facing interface \
+                       in order for Phantun Client to connect to Phantun Server")
+                .default_value("192.168.200.2")
                .takes_value(true),
        )
        .get_matches();

-    let local_addr: SocketAddrV4 = matches
+    let local_addr: SocketAddr = matches
        .value_of("local")
        .unwrap()
        .parse()
        .expect("bad local address");
-    let remote_addr: SocketAddrV4 = matches
-        .value_of("remote")
+
+    let remote_addr = tokio::net::lookup_host(matches.value_of("remote").unwrap())
+        .await
+        .expect("bad remote address or host")
+        .find(|addr| addr.is_ipv4())
+        .expect("unable to resolve remote host name or no valid A record was returned");
+    let remote_addr = if let SocketAddr::V4(addr) = remote_addr {
+        addr
+    } else {
+        unreachable!();
+    };
+    info!("Remote address is: {}", remote_addr);
+
+    let tun_local: Ipv4Addr = matches
+        .value_of("tun_local")
        .unwrap()
        .parse()
-        .expect("bad remote address");
+        .expect("bad local address for Tun interface");
+    let tun_peer: Ipv4Addr = matches
+        .value_of("tun_peer")
+        .unwrap()
+        .parse()
+        .expect("bad peer address for Tun interface");
+
+    let num_cpus = num_cpus::get();
+    info!("{} cores available", num_cpus);

    let tun = TunBuilder::new()
-        .name("") // if name is empty, then it is set by kernel.
+        .name(matches.value_of("tun").unwrap()) // if name is empty, then it is set by kernel.
        .tap(false) // false (default): TUN, true: TAP.
        .packet_info(false) // false: IFF_NO_PI, default is true.
        .up() // or set it up manually using `sudo ip link set <tun-name> up`.
-        .address("192.168.200.1".parse().unwrap())
-        .destination("192.168.200.2".parse().unwrap())
-        .try_build()
+        .address(tun_local)
+        .destination(tun_peer)
+        .try_build_mq(num_cpus)
        .unwrap();

-    info!("Created TUN device {}", tun.name());
+    info!("Created TUN device {}", tun[0].name());

    let udp_sock = Arc::new(new_udp_reuseport(local_addr));
-    let connections = Arc::new(RwLock::new(HashMap::<SocketAddrV4, Arc<Socket>>::new()));
+    let connections = Arc::new(RwLock::new(HashMap::<SocketAddr, Arc<Socket>>::new()));

    let mut stack = Stack::new(tun);

@@ -85,7 +123,7 @@ async fn main() {

        loop {
            tokio::select! {
-                Ok((size, SocketAddr::V4(addr))) = udp_sock.recv_from(&mut buf_r) => {
+                Ok((size, addr)) = udp_sock.recv_from(&mut buf_r) => {
                    // seen UDP packet to listening socket, this means:
                    // 1. It is a new UDP connection, or
                    // 2. It is some extra packets not filtered by more specific
@@ -112,48 +150,85 @@ async fn main() {
                    assert!(connections.write().await.insert(addr, sock.clone()).is_none());
                    debug!("inserted fake TCP socket into connection table");

-                    let connections = connections.clone();
-
                    // spawn "fastpath" UDP socket and task, this will offload main task
                    // from forwarding UDP packets
-                    tokio::spawn(async move {
-                        let mut buf_udp = [0u8; MAX_PACKET_LEN];
-                        let mut buf_tcp = [0u8; MAX_PACKET_LEN];
-                        let udp_sock = new_udp_reuseport(local_addr);
-                        udp_sock.connect(addr).await.unwrap();

+                    let packet_received = Arc::new(Notify::new());
+                    let quit = CancellationToken::new();
+
+                    for i in 0..num_cpus {
+                        let sock = sock.clone();
+                        let quit = quit.clone();
+                        let packet_received = packet_received.clone();
+
+                        tokio::spawn(async move {
+                            let mut buf_udp = [0u8; MAX_PACKET_LEN];
+                            let mut buf_tcp = [0u8; MAX_PACKET_LEN];
+                            let udp_sock = new_udp_reuseport(local_addr);
+                            udp_sock.connect(addr).await.unwrap();
+
+                            loop {
+                                tokio::select! {
+                                    Ok(size) = udp_sock.recv(&mut buf_udp) => {
+                                        if sock.send(&buf_udp[..size]).await.is_none() {
+                                            debug!("removed fake TCP socket from connections table");
+                                            quit.cancel();
+                                            return;
+                                        }
+
+                                        packet_received.notify_one();
+                                    },
+                                    res = sock.recv(&mut buf_tcp) => {
+                                        match res {
+                                            Some(size) => {
+                                                if size > 0 {
+                                                    if let Err(e) = udp_sock.send(&buf_tcp[..size]).await {
+                                                        error!("Unable to send UDP packet to {}: {}, closing connection", e, addr);
+                                                        quit.cancel();
+                                                        return;
+                                                    }
+                                                }
+                                            },
+                                            None => {
+                                                debug!("removed fake TCP socket from connections table");
+                                                quit.cancel();
+                                                return;
+                                            },
+                                        }
+
+                                        packet_received.notify_one();
+                                    },
+                                    _ = quit.cancelled() => {
+                                        debug!("worker {} terminated", i);
+                                        return;
+                                    },
+                                };
+                            }
+                        });
+                    }
+
+                    let connections = connections.clone();
+                    tokio::spawn(async move {
                        loop {
                            let read_timeout = time::sleep(UDP_TTL);
+                            let packet_received_fut = packet_received.notified();

                            tokio::select! {
-                                Ok(size) = udp_sock.recv(&mut buf_udp) => {
-                                    if sock.send(&buf_udp[..size]).await.is_none() {
-                                        connections.write().await.remove(&addr);
-                                        debug!("removed fake TCP socket from connections table");
-                                        return;
-                                    }
-                                },
-                                res = sock.recv(&mut buf_tcp) => {
-                                    match res {
-                                        Some(size) => {
-                                            if size > 0 {
-                                                udp_sock.send(&buf_tcp[..size]).await.unwrap();
-                                            }
-                                        },
-                                        None => {
-                                            connections.write().await.remove(&addr);
-                                            debug!("removed fake TCP socket from connections table");
-                                            return;
-                                        },
-                                    }
-                                },
                                _ = read_timeout => {
                                    info!("No traffic seen in the last {:?}, closing connection", UDP_TTL);
                                    connections.write().await.remove(&addr);
                                    debug!("removed fake TCP socket from connections table");
+
+                                    quit.cancel();
                                    return;
-                                }
-                            };
+                                },
+                                _ = quit.cancelled() => {
+                                    connections.write().await.remove(&addr);
+                                    debug!("removed fake TCP socket from connections table");
+                                    return;
+                                },
+                                _ = packet_received_fut => {},
+                            }
                        }
                    });
                },
--- a/phantun/src/bin/server.rs
+++ b/phantun/src/bin/server.rs
@@ -1,36 +1,70 @@
-use clap::{App, Arg};
+use clap::{crate_version, Arg, Command};
 use fake_tcp::packet::MAX_PACKET_LEN;
 use fake_tcp::Stack;
-use log::info;
-use std::net::SocketAddrV4;
+use log::{debug, error, info};
+use phantun::utils::new_udp_reuseport;
+use std::net::Ipv4Addr;
+use std::sync::Arc;
 use tokio::net::UdpSocket;
-use tokio::time::{self, Duration};
+use tokio::sync::Notify;
+use tokio::time;
 use tokio_tun::TunBuilder;
-const UDP_TTL: Duration = Duration::from_secs(180);
+use tokio_util::sync::CancellationToken;
+
+use phantun::UDP_TTL;

 #[tokio::main]
 async fn main() {
    pretty_env_logger::init();

-    let matches = App::new("Phantun Server")
-        .version("1.0")
-        .author("dndx@GitHub")
+    let matches = Command::new("Phantun Server")
+        .version(crate_version!())
+        .author("Datong Sun (github.com/dndx)")
        .arg(
-            Arg::with_name("local")
-                .short("l")
+            Arg::new("local")
+                .short('l')
                .long("local")
                .required(true)
                .value_name("PORT")
-                .help("Sets the listening port")
+                .help("Sets the port where Phantun Server listens for incoming Phantun Client TCP connections")
                .takes_value(true),
        )
        .arg(
-            Arg::with_name("remote")
-                .short("r")
+            Arg::new("remote")
+                .short('r')
                .long("remote")
                .required(true)
-                .value_name("IP:PORT")
-                .help("Sets the connecting socket address")
+                .value_name("IP or HOST NAME:PORT")
+                .help("Sets the address or host name and port where Phantun Server forwards UDP packets to, IPv6 address need to be specified as: \"[IPv6]:PORT\"")
+                .takes_value(true),
+        )
+        .arg(
+            Arg::new("tun")
+                .long("tun")
+                .required(false)
+                .value_name("tunX")
+                .help("Sets the Tun interface name, if absent, pick the next available name")
+                .default_value("")
+                .takes_value(true),
+        )
+        .arg(
+            Arg::new("tun_local")
+                .long("tun-local")
+                .required(false)
+                .value_name("IP")
+                .help("Sets the Tun interface local address (O/S's end)")
+                .default_value("192.168.201.1")
+                .takes_value(true),
+        )
+        .arg(
+            Arg::new("tun_peer")
+                .long("tun-peer")
+                .required(false)
+                .value_name("IP")
+                .help("Sets the Tun interface destination (peer) address (Phantun Server's end). \
+                       You will need to setup DNAT rules to this address in order for Phantun Server \
+                       to accept TCP traffic from Phantun Client")
+                .default_value("192.168.201.2")
                .takes_value(true),
        )
        .get_matches();
@@ -40,22 +74,40 @@ async fn main() {
        .unwrap()
        .parse()
        .expect("bad local port");
-    let remote_addr: SocketAddrV4 = matches
-        .value_of("remote")
+
+    let remote_addr = tokio::net::lookup_host(matches.value_of("remote").unwrap())
+        .await
+        .expect("bad remote address or host")
+        .next()
+        .expect("unable to resolve remote host name");
+    info!("Remote address is: {}", remote_addr);
+
+    let tun_local: Ipv4Addr = matches
+        .value_of("tun_local")
        .unwrap()
        .parse()
-        .expect("bad remote address");
+        .expect("bad local address for Tun interface");
+    let tun_peer: Ipv4Addr = matches
+        .value_of("tun_peer")
+        .unwrap()
+        .parse()
+        .expect("bad peer address for Tun interface");
+
+    let num_cpus = num_cpus::get();
+    info!("{} cores available", num_cpus);

    let tun = TunBuilder::new()
-        .name("") // if name is empty, then it is set by kernel.
+        .name(matches.value_of("tun").unwrap()) // if name is empty, then it is set by kernel.
        .tap(false) // false (default): TUN, true: TAP.
        .packet_info(false) // false: IFF_NO_PI, default is true.
        .up() // or set it up manually using `sudo ip link set <tun-name> up`.
-        .address("192.168.201.1".parse().unwrap())
-        .destination("192.168.201.2".parse().unwrap())
-        .try_build()
+        .address(tun_local)
+        .destination(tun_peer)
+        .try_build_mq(num_cpus)
        .unwrap();

+    info!("Created TUN device {}", tun[0].name());
+
    //thread::sleep(time::Duration::from_secs(5));
    let mut stack = Stack::new(tun);
    stack.listen(local_port);
@@ -66,37 +118,82 @@ async fn main() {
        let mut buf_tcp = [0u8; MAX_PACKET_LEN];

        loop {
-            let sock = stack.accept().await;
+            let sock = Arc::new(stack.accept().await);
            info!("New connection: {}", sock);

-            tokio::spawn(async move {
-                let udp_sock = UdpSocket::bind("0.0.0.0:0").await.unwrap();
-                udp_sock.connect(remote_addr).await.unwrap();
+            let packet_received = Arc::new(Notify::new());
+            let quit = CancellationToken::new();
+            let udp_sock = UdpSocket::bind(if remote_addr.is_ipv4() {
+                "0.0.0.0:0"
+            } else {
+                "[::]:0"
+            })
+            .await
+            .unwrap();
+            let local_addr = udp_sock.local_addr().unwrap();
+            drop(udp_sock);

+            for i in 0..num_cpus {
+                let sock = sock.clone();
+                let quit = quit.clone();
+                let packet_received = packet_received.clone();
+                let udp_sock = new_udp_reuseport(local_addr);
+
+                tokio::spawn(async move {
+                    udp_sock.connect(remote_addr).await.unwrap();
+
+                    loop {
+                        tokio::select! {
+                            Ok(size) = udp_sock.recv(&mut buf_udp) => {
+                                if sock.send(&buf_udp[..size]).await.is_none() {
+                                    quit.cancel();
+                                    return;
+                                }
+
+                                packet_received.notify_one();
+                            },
+                            res = sock.recv(&mut buf_tcp) => {
+                                match res {
+                                    Some(size) => {
+                                        if size > 0 {
+                                            if let Err(e) = udp_sock.send(&buf_tcp[..size]).await {
+                                                error!("Unable to send UDP packet to {}: {}, closing connection", e, remote_addr);
+                                                quit.cancel();
+                                                return;
+                                            }
+                                        }
+                                    },
+                                    None => {
+                                        quit.cancel();
+                                        return;
+                                    },
+                                }
+
+                                packet_received.notify_one();
+                            },
+                            _ = quit.cancelled() => {
+                                debug!("worker {} terminated", i);
+                                return;
+                            },
+                        };
+                    }
+                });
+            }
+
+            tokio::spawn(async move {
                loop {
                    let read_timeout = time::sleep(UDP_TTL);
+                    let packet_received_fut = packet_received.notified();

                    tokio::select! {
-                        Ok(size) = udp_sock.recv(&mut buf_udp) => {
-                            if sock.send(&buf_udp[..size]).await.is_none() {
-                                return;
-                            }
-                        },
-                        res = sock.recv(&mut buf_tcp) => {
-                            match res {
-                                Some(size) => {
-                                    if size > 0 {
-                                        udp_sock.send(&buf_tcp[..size]).await.unwrap();
-                                    }
-                                },
-                                None => { return; },
-                            }
-                        },
                        _ = read_timeout => {
                            info!("No traffic seen in the last {:?}, closing connection", UDP_TTL);
+
+                            quit.cancel();
                            return;
-                        }
-                    };
+                        },
+                        _ = packet_received_fut => {},
+                    }
                }
            });
        }
--- a/phantun/src/lib.rs
+++ b/phantun/src/lib.rs
@@ -0,0 +1,5 @@
+use std::time::Duration;
+
+pub mod utils;
+
+pub const UDP_TTL: Duration = Duration::from_secs(180);
--- a/phantun/src/utils.rs
+++ b/phantun/src/utils.rs
@@ -0,0 +1,22 @@
+use std::net::SocketAddr;
+use tokio::net::UdpSocket;
+
+pub fn new_udp_reuseport(local_addr: SocketAddr) -> UdpSocket {
+    let udp_sock = socket2::Socket::new(
+        if local_addr.is_ipv4() {
+            socket2::Domain::IPV4
+        } else {
+            socket2::Domain::IPV6
+        },
+        socket2::Type::DGRAM,
+        None,
+    )
+    .unwrap();
+    udp_sock.set_reuse_port(true).unwrap();
+    // from tokio-rs/mio/blob/master/src/sys/unix/net.rs
+    udp_sock.set_cloexec(true).unwrap();
+    udp_sock.set_nonblocking(true).unwrap();
+    udp_sock.bind(&socket2::SockAddr::from(local_addr)).unwrap();
+    let udp_sock: std::net::UdpSocket = udp_sock.into();
+    udp_sock.try_into().unwrap()
+}
Author	SHA1	Message	Date
Datong Sun	99bff568f6	chore(phantun) release `v0.3.2`	2022-04-10 06:10:57 -07:00
Datong Sun	91ad2c03a1	chore(fake-tcp) release `v0.3.1`	2022-04-10 06:09:48 -07:00
Datong Sun	581d80d08c	perf(fake-tcp) use flume to avoid locking in receiver, improved single connection performance by 300%	2022-04-10 21:07:12 +08:00
Datong Sun	55da4d6a62	docs(README) style improvements	2022-04-10 18:47:22 +08:00
Datong Sun	bb859be6b6	docs(README) add build status and docs.rs badge	2022-04-10 18:37:06 +08:00
Datong Sun	8d315ea4e7	docs(README) add packet header diagram	2022-04-10 18:26:28 +08:00
Datong Sun	21eabe8b82	docs(README) add description for safe Rust and bump latest release to `v0.3.1`	2022-04-10 01:44:33 -07:00
Datong Sun	8a74b31c6e	chore(phantun) bump `fake-tcp` dependency to `v0.3.0` and release `v0.3.1`	2022-04-10 01:37:59 -07:00
Datong Sun	ca14ba457f	chore(fake-tcp) bump to `v0.3.0`	2022-04-10 01:36:45 -07:00
Datong Sun	33a0cfe567	docs(README) updated benchmarking results	2022-04-10 01:35:07 -07:00
Datong Sun	95dfd8ab54	fix(fake-tcp) fix an issue where `RST` generated is not following the proper RFC requirement. Send ACK every 128MB in lieu of data packets.	2022-04-10 16:33:53 +08:00
Datong Sun	1c35635091	docs(README) bump latest release version to `v0.3.0`	2022-04-09 08:49:29 -07:00
Datong Sun	b8a6c8853b	chore(phantun) bump to `v0.3.0`	2022-04-09 08:39:44 -07:00
Datong Sun	d97a27778b	style(phantun) refactor out common functions and constants	2022-04-09 21:32:07 +08:00
Datong Sun	35f7b35ff5	perf(phantun) spawn multiple threads for UDP send/receive	2022-04-09 21:32:07 +08:00
Datong Sun	dff0c4ca28	docs(readme) add link for `fake-tcp` docs	2022-04-09 12:17:11 +08:00
Datong Sun	9bf78adc92	chore(fake-tcp) bump to `v0.2.4` with new documentations	2022-04-08 21:10:36 -07:00
Datong Sun	5d4e3bf8c0	docs(fake-tcp) added documentations for `fake-tcp`	2022-04-09 12:10:13 +08:00
Datong Sun	9c85b43e94	style(phantun) use the `clap::Command` struct, removed the deprecated `clap::App` usage	2022-04-09 11:00:20 +08:00
Datong Sun	66b0bc11b0	chore(phantun) use path dependency for `fake-tcp` crate	2022-04-09 11:00:20 +08:00
Datong Sun	02b00dfc3a	docs(images) updated the flow diagram	2022-03-22 05:16:31 -07:00
dependabot[bot]	0ee7774d03	chore(deps): bump actions/checkout from 2 to 3 Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](https://github.com/actions/checkout/compare/v2...v3) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>	2022-03-03 10:40:48 +08:00
dependabot[bot]	11fdac78f1	chore(deps): update pnet requirement from 0.28 to 0.29 Updates the requirements on [pnet](https://github.com/libpnet/libpnet) to permit the latest version. - [Release notes](https://github.com/libpnet/libpnet/releases) - [Commits](https://github.com/libpnet/libpnet/compare/v0.28.0...v0.29.0) --- updated-dependencies: - dependency-name: pnet dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>	2022-01-14 23:44:23 +08:00
Datong Sun	ed686ce9fa	docs(licenses) updated to year 2022	2022-01-03 07:57:09 -08:00
Datong Sun	d9001b08aa	docs(readme) bumped latest release to `v0.2.5`	2022-01-03 07:54:41 -08:00
Datong Sun	726ecac9cf	chore(phantun) bump `phantun` to `v0.2.5`	2022-01-03 07:47:48 -08:00
dependabot[bot]	2ef0a056be	chore(deps): update clap requirement from 2.34 to 3.0 Updates the requirements on [clap](https://github.com/clap-rs/clap) to permit the latest version. - [Release notes](https://github.com/clap-rs/clap/releases) - [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md) - [Commits](https://github.com/clap-rs/clap/compare/clap_generate-v3.0.0-rc.0...clap_complete-v3.0.0) --- updated-dependencies: - dependency-name: clap dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>	2022-01-03 23:45:13 +08:00
Datong Sun	cb9dd3e931	fix(client) disable AAAA resolve, since tokio-tun does not yet have IPv6 support. See: https://github.com/yaa110/tokio-tun/pull/8	2022-01-03 23:37:31 +08:00
Datong Sun	7db7164193	chore(*) use `tokio-tun` v0.5 instead of forked version. Bumped `fake-tcp` to `v0.2.3`	2021-12-07 17:07:54 +08:00
Datong Sun	def134d73b	docs(readme) bump latest version to `v0.2.4`	2021-12-05 07:13:01 -08:00
Datong Sun	b3c781cdc5	chore(phantun) bump `phantun` to v0.2.4	2021-12-05 07:10:42 -08:00
Datong Sun	d5e30c113f	chore(phantun) bump `clap` and `tokio` dependencies to latest	2021-12-05 07:01:58 -08:00
Datong Sun	e2a9194f6f	chore(fake-tcp) bump to v0.2.2	2021-12-05 06:57:23 -08:00
Datong Sun	d0eaefe5d0	chore(phantun) specify exact version for `fake-tcp` dependency	2021-12-05 06:49:35 -08:00
Datong Sun	299646a54f	chore(fake-tcp) bump to v0.2.1	2021-12-05 06:43:18 -08:00
Datong Sun	8b28cdc6c2	chore(crates) do not include bugfix version in dependency as it prevents auto update from Dependabot	2021-12-05 22:41:05 +08:00
Datong Sun	a8ad203754	docs(readme) fixed some typos	2021-12-01 15:10:52 +08:00
Datong Sun	33e510e7ba	fix(fake-tcp) remove unneeded `State::Closed` as it is not in a reachable code path	2021-11-19 10:45:13 -08:00
Datong Sun	521a3f1a01	docs(readme) bump release version and add host name support info for `--remote`	2021-11-18 20:48:53 -08:00
Datong Sun	c5a5116808	chore(phantun) bump to v0.2.3, bump `fake-tcp` dependency to v0.2.0	2021-11-18 20:37:28 -08:00
Datong Sun	e8f2457cb5	chore(fake-tcp) bump to v0.2.0	2021-11-18 20:36:05 -08:00
Datong Sun	583cdbe300	perf(fake-tcp) reduce the number of `clone()` calls in hot path	2021-11-19 12:35:21 +08:00
Datong Sun	91988520e5	feat(*) add DNS name support for `--remote` argument in both Client and Server	2021-11-19 12:30:47 +08:00
Datong Sun	49cc6a6865	chore(phantun) update fake-tcp dependency version to v0.1.3	2021-11-02 18:29:53 +08:00
Datong Sun	7390d4bf27	chore(fake-tcp) release version v0.1.3	2021-11-02 18:29:53 +08:00
dependabot[bot]	95e762f5fd	chore(deps): update dndx-fork-tokio-tun requirement from 0.3.16 to 0.4.0 Updates the requirements on [dndx-fork-tokio-tun](https://github.com/yaa110/tokio-tun) to permit the latest version. - [Release notes](https://github.com/yaa110/tokio-tun/releases) - [Commits](https://github.com/yaa110/tokio-tun/compare/0.3.16...0.4.0) --- updated-dependencies: - dependency-name: dndx-fork-tokio-tun dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>	2021-11-02 18:29:53 +08:00
Datong Sun	c9043015f2	docs(readme) update latest release to v0.2.2	2021-10-30 09:32:14 -07:00
Datong Sun	494abf37c5	docs(readme) add IPv6 support info	2021-10-30 09:21:23 -07:00
Datong Sun	cab87bd75b	chore(cargo) bump Phantun to v0.2.2	2021-10-30 09:15:23 -07:00
Datong Sun	042f5af49f	feat(phantun) add support for connecting to/from IPv6 based UDP endpoint	2021-10-31 00:09:02 +08:00
Datong Sun	f667f56747	chore(actions) always use the latest stable toolchain	2021-10-21 03:47:57 -07:00
Datong Sun	49665b906f	chore(cargo) use Rust edition 2021	2021-10-21 03:41:42 -07:00
Datong Sun	e9cde27923	ci(dependabot) add Dependabot checks	2021-10-20 02:07:05 -07:00
Datong Sun	cf588db042	ci(release) add musl libc targets	2021-10-19 01:13:56 -07:00
Datong Sun	0e27822995	docs(readme) update WireGuard MTU calculation to take into account for padding truncation	2021-10-16 10:55:01 -07:00
Datong Sun	eeca1d0108	docs(readme) bump latest release version to v0.2.1	2021-10-12 07:26:49 -07:00
Datong Sun	35541df7e2	chore(phantun) bump version to v0.2.1, bump fake-tcp dependency to v0.1.2	2021-10-09 09:33:16 -07:00
Datong Sun	3219cb9f38	chore(fake-tcp) bump version to 0.1.2	2021-10-09 09:31:55 -07:00
Datong Sun	41e86521b7	fix(fake-tcp) more robust checking for receiving end closing, avoids panicking tasks from causing server to stop completely	2021-10-10 00:29:28 +08:00
Datong Sun	427fb7c19a	fix(fake-tcp) use 64 as TTL value, fixes #11	2021-10-09 23:57:47 +08:00
Datong Sun	ef96a5161d	chore(phantun) release v0.2.0	2021-09-28 23:35:37 -07:00
Datong Sun	2b7588adfe	docs(phantun) add `-h` option information	2021-09-28 23:31:07 -07:00
Datong Sun	6285efd0d7	feat(phantun) make Tun interface name and address configurable. Improved documentations of Clap CLI help Closes #8	2021-09-29 14:27:00 +08:00
Datong Sun	5a6ebf52ea	fix(phantun) better UDP error handling When UDP send fails, print a proper error message and close the connection, instead of `panic`ing.	2021-09-28 00:53:52 +08:00
Datong Sun	e3e50f8a9e	docs(readme) add more network topology explanation	2021-09-24 08:44:42 -07:00
Datong Sun	e97a2d1cad	docs(readme) update latest release version	2021-09-22 22:23:00 -07:00
Datong Sun	4dc9f867ed	chore(release) add more cross compilation targets	2021-09-22 22:17:05 -07:00
Datong Sun	a54d1b3759	chore(phantun): update fake-tcp used to 0.1.1	2021-09-22 22:04:15 -07:00
Datong Sun	8371256f0b	perf(fake-tcp) use cached tuples per dispatcher task to avoid RwLock contentions. Use multi queue Tun. Upgraded tokio to 1.12.0. This makes the entire Phantun forward process completely lock contention free	2021-09-23 12:59:49 +08:00
Datong Sun	04b0e97c1d	perf(fake-tcp) faster checksum calculation Before: ``` test packet::benchmarks::bench_build_tcp_packet_128 ... bench: 207 ns/iter (+/- 25) test packet::benchmarks::bench_build_tcp_packet_1460 ... bench: 670 ns/iter (+/- 14) test packet::benchmarks::bench_build_tcp_packet_512 ... bench: 334 ns/iter (+/- 21) ``` After: ``` test packet::benchmarks::bench_build_tcp_packet_128 ... bench: 206 ns/iter (+/- 9) test packet::benchmarks::bench_build_tcp_packet_1460 ... bench: 259 ns/iter (+/- 16) test packet::benchmarks::bench_build_tcp_packet_512 ... bench: 212 ns/iter (+/- 11) ```	2021-09-20 23:12:53 +08:00
Datong Sun	212f9b6f9d	docs(readme) add iptables config guide	2021-09-20 04:15:55 -07:00
Datong Sun	9b644166a5	docs(readme) fixed typo	2021-09-20 04:09:12 -07:00
Datong Sun	e0eb0f03cd	docs(readme) changed section header to be shorter	2021-09-19 03:59:27 -07:00
Datong Sun	9418096fe3	docs(readme) fix styling	2021-09-19 03:58:35 -07:00
Datong Sun	f70c466399	docs(readme) fix WireGuard section header format	2021-09-19 03:53:47 -07:00
Datong Sun	fb66286db3	docs(images) update diagram to include more descriptions	2021-09-19 03:46:19 -07:00
Datong Sun	d4d941971f	ci(release) add aarch64 target	2021-09-19 03:32:05 -07:00
Datong Sun	62df34983f	docs(readme) add future plans	2021-09-19 03:09:16 -07:00
Datong Sun	d2b8bbeaa8	docs(readme) add better styling for `MAX_OF_16`	2021-09-19 03:02:37 -07:00
Datong Sun	10468b48ce	docs(readme) add WireGuard MTU calculation steps	2021-09-19 02:59:50 -07:00
Datong Sun	3405dd38ed	docs(license) add license files to repository root	2021-09-19 02:40:43 -07:00