Update README.zh-cn.md

Signed-off-by: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>

README.md

@@ -1,40 +1,42 @@
 # Udp2raw-tunnel
 
 
-A Tunnel which turns UDP Traffic into Encrypted FakeTCP/UDP/ICMP Traffic by using Raw Socket, helps you Bypass UDP FireWalls(or Unstable UDP Environment). It can defend Replay-Attack and supports Multiplexing. It also acts as a Connection Stabilizer.
+A Tunnel which turns UDP Traffic into Encrypted FakeTCP/UDP/ICMP Traffic by using Raw Socket, helps you Bypass UDP FireWalls(or Unstable UDP Environment).
-
-![image0](images/image0.PNG)
 
 When used alone,udp2raw tunnels only UDP traffic. Nevertheless,if you used udp2raw + any UDP-based VPN together,you can tunnel any traffic(include TCP/UDP/ICMP),currently OpenVPN/L2TP/ShadowVPN and [tinyfecVPN](https://github.com/wangyu-/tinyfecVPN) are confirmed to be supported.
 
-![image_vpn](images/udp2rawopenvpn.PNG)
 
-[简体中文](/doc/README.zh-cn.md)(内容更丰富)
+![image0](images/image0.PNG)
+
+or
+
+![image_vpn](images/udp2rawopenvpn.PNG)
 
 [udp2raw wiki](https://github.com/wangyu-/udp2raw-tunnel/wiki)
 
+[简体中文](/doc/README.zh-cn.md)(内容更丰富)
+
+
 # Support Platforms
-Linux host (including desktop Linux,Android phone/tablet,OpenWRT router,or Raspberry PI) with root access.
+Linux host (including desktop Linux,Android phone/tablet,OpenWRT router,or Raspberry PI) with root account or cap_net_raw capability.
 
 For Windows and MacOS users, use the udp2raw in [this repo](https://github.com/wangyu-/udp2raw-multiplatform).
 
-<del>For Windows and MacOS You can run udp2raw inside [this](https://github.com/wangyu-/udp2raw-tunnel/releases/download/20171108.0/lede-17.01.2-x86_virtual_machine_image.zip) 7.5mb virtual machine image(make sure network adapter runs at bridged mode).</del>
-
-
-
 # Features
 ### Send/Receive UDP Packets with ICMP/FakeTCP/UDP headers
 ICMP/FakeTCP headers help you bypass UDP blocking, UDP QOS or improper UDP NAT behavior on some ISPs. In ICMP header mode,udp2raw works like an ICMP tunnel.
 
-UDP headers are also supported. In UDP header mode, it behaves just like a normal UDP tunnel, and you can just make use of the other features (such as encrytion, anti-replay, or connection stalization).
+UDP headers are also supported. In UDP header mode, it behaves just like a normal UDP tunnel, and you can just make use of the other features (such as encryption, anti-replay, or connection stalization).
 
 ### Simulated TCP with Real-time/Out-of-Order Delivery
-In FakeTCP header mode,udp2raw simulates 3-way handshake while establishing a connection,simulates seq and ack_seq while data transferring. It also simulates following TCP options: `MSS`, `sackOk`, `TS`, `TS_ack`, `wscale`.Firewalls will regard FakeTCP as a TCP connection, but its essentially UDP: it supports real-time/out-of-order delivery(just as normal UDP does), no congrestion control or re-transmission. So there wont be any TCP over TCP problem when using OpenVPN.
+In FakeTCP header mode,udp2raw simulates 3-way handshake while establishing a connection,simulates seq and ack_seq while data transferring. It also simulates a few TCP options such as: `MSS`, `sackOk`, `TS`, `TS_ack`, `wscale`. Firewalls will regard FakeTCP as a TCP connection, but its essentially UDP: it supports real-time/out-of-order delivery(just as normal UDP does), no congestion control or re-transmission. So there wont be any TCP over TCP problem when using OpenVPN.
 
-### Encrpytion, Anti-Replay
+### Encryption, Anti-Replay
 * Encrypt your traffic with AES-128-CBC.
 * Protect data integrity by HMAC-SHA1 (or weaker MD5/CRC32).
-* Defense replay attack with an anti-replay window, smiliar to IPSec and OpenVPN. 
+* Defense replay attack with anti-replay window.
+
+[Notes on encryption](https://github.com/wangyu-/udp2raw-tunnel/wiki/Notes-on-encryption)
 
 ### Failure Dectection & Stablization (Connection Recovery)
 Conection failures are detected by heartbeats. If timed-out, client will automatically change port number and reconnect. If reconnection is successful, the previous connection will be recovered, and all existing UDP conversations will stay vaild.
@@ -218,63 +220,6 @@ raw_mode: faketcp  cipher_mode: aes128cbc  auth_mode: md5
 
 (reverse speed was simliar and not uploaded)
 
-# Application
-## Tunneling any traffic via raw traffic by using udp2raw +openvpn
-![image_vpn](images/udp2rawopenvpn.PNG)
-1. Bypasses UDP block/UDP QOS
-
-2. No TCP over TCP problem (TCP over TCP problem http://sites.inka.de/bigred/devel/tcp-tcp.html ,https://community.openvpn.net/openvpn/ticket/2 )
-
-3. OpenVpn over ICMP also becomes a choice
-
-4. Supports almost any UDP-based VPN
-
-More details at [openvpn+udp2raw_guide](https://github.com/wangyu-/udp2raw-tunnel/wiki/udp2raw-openvpn-config-guide)
-## Speed-up tcp connection via raw traffic by using udp2raw+kcptun
-kcptun is a tcp connection speed-up program,it speeds-up tcp connection by using kcp protocol on-top of udp.by using udp2raw,you can use kcptun while udp is QoSed or blocked.
-(kcptun, https://github.com/xtaci/kcptun)
-
-## Speed-up tcp connection via raw traffic by using udp2raw+finalspeed
-finalspeed is a tcp connection speed-up program similiar to kcptun,it speeds-up tcp connection by using kcp protocol on-top of udp or tcp.but its tcp mode doesnt support openvz,you can bypass this problem if you use udp2raw+finalspeed together,and icmp mode also becomes avaliable.
-
-# How to build
-read [build_guide](/doc/build_guide.md)
-
-# Other
-### Easier installation on ArchLinux
-```
-yaourt -S udp2raw-tunnel # or
-pacaur -S udp2raw-tunnel
-```
-
-# Related work
-### kcptun-raw
-udp2raw was inspired by kcptun-raw,which modified kcptun to support tcp mode.
-
-https://github.com/Chion82/kcptun-raw
-### relayRawSocket
-kcptun-raw was inspired by relayRawSocket. A simple  udp to raw tunnel,wrote in python
-
-https://github.com/linhua55/some_kcptun_tools/tree/master/relayRawSocket
-### kcpraw
-another project of kcptun with tcp mode
-
-https://github.com/ccsexyz/kcpraw
-
-### icmptunnel
-Transparently tunnel your IP traffic through ICMP echo and reply packets.
-
-https://github.com/DhavalKapil/icmptunnel
-
-### Tcp Minion
-Tcp Minion is a project which modifid the code of tcp stack in kernel,and implemented real-time out-order udp packet delivery through this modified tcp stack.I failed to find the implementation,but there are some papers avaliable:
-
-https://arxiv.org/abs/1103.0463
-
-http://korz.cs.yale.edu/2009/tng/papers/pfldnet10.pdf
-
-https://pdfs.semanticscholar.org/9e6f/e2306f4385b4eb5416d1fcab16e9361d6ba3.pdf
-
 # wiki
 
 Check wiki for more info:

client.cpp

@@ -7,6 +7,18 @@
 #include "encrypt.h"
 #include "fd_manager.h"
 
+#ifdef UDP2RAW_MP
+u32_t detect_interval=1500;
+u64_t laste_detect_time=0;
+
+int use_udp_for_detection=0;
+int use_tcp_for_detection=1;
+
+
+extern pcap_t *pcap_handle;
+
+extern int pcap_captured_full_len;
+#endif
 
 int client_on_timer(conn_info_t &conn_info) //for client. called when a timer is ready in epoll
 {
@@ -20,6 +32,75 @@ int client_on_timer(conn_info_t &conn_info) //for client. called when a timer is
 
 	mylog(log_trace,"<client_on_timer,send_info.ts_ack= %u>\n",send_info.ts_ack);
 
+#ifdef UDP2RAW_MP
+	//mylog(log_debug,"pcap cnt :%d\n",pcap_cnt);
+	if(send_with_pcap&&!pcap_header_captured)
+	{
+
+		if(get_current_time()-laste_detect_time>detect_interval)
+		{
+			laste_detect_time=get_current_time();
+		}
+		else
+		{
+			return 0;
+		}
+/*
+		struct sockaddr_in remote_addr_in={0};
+
+		socklen_t slen = sizeof(sockaddr_in);
+		int port=get_true_random_number()%65534+1;
+		remote_addr_in.sin_family = AF_INET;
+		remote_addr_in.sin_port = htons(port);
+		remote_addr_in.sin_addr.s_addr = remote_ip_uint32;*/
+		int port=get_true_random_number()%65534+1;
+		address_t tmp_addr=remote_addr;
+		tmp_addr.set_port(port);
+
+		if(use_udp_for_detection)
+		{
+			int new_udp_fd=socket(tmp_addr.get_type(), SOCK_DGRAM, IPPROTO_UDP);
+			if(new_udp_fd<0)
+			{
+				mylog(log_warn,"create new_udp_fd error\n");
+				return -1;
+			}
+			setnonblocking(new_udp_fd);
+			u64_t tmp=get_true_random_number();
+
+			int ret=sendto(new_udp_fd,(char*)(&tmp),sizeof(tmp),0,(struct sockaddr *)&tmp_addr.inner,tmp_addr.get_len());
+			if(ret==-1)
+			{
+				mylog(log_warn,"sendto() failed\n");
+			}
+			sock_close(new_udp_fd);
+		}
+
+		if(use_tcp_for_detection)
+		{
+			static int last_tcp_fd=-1;
+
+			int new_tcp_fd=socket(tmp_addr.get_type(), SOCK_STREAM, IPPROTO_TCP);
+			if(new_tcp_fd<0)
+			{
+				mylog(log_warn,"create new_tcp_fd error\n");
+				return -1;
+			}
+			setnonblocking(new_tcp_fd);
+			connect(new_tcp_fd,(struct sockaddr *)&tmp_addr.inner,tmp_addr.get_len());
+			if(last_tcp_fd!=-1)
+				sock_close(last_tcp_fd);
+			last_tcp_fd=new_tcp_fd;
+			//close(new_tcp_fd);
+		}
+
+
+
+		mylog(log_info,"waiting for a use-able packet to be captured\n");
+
+		return 0;
+	}
+#endif
 	if(raw_info.disabled)
 	{
 		conn_info.state.client_current_state=client_idle;
@@ -105,7 +186,8 @@ int client_on_timer(conn_info_t &conn_info) //for client. called when a timer is
 			{
 				setnonblocking(bind_fd);
 				int ret=connect(bind_fd,(struct sockaddr *)&remote_addr.inner,remote_addr.get_len());
-				mylog(log_info,"ret=%d,errno=%s,%d %s\n",ret,get_sock_error(),bind_fd,remote_addr.get_str());
+				mylog(log_debug,"ret=%d,errno=%s, %d %s\n",ret,get_sock_error(),bind_fd,remote_addr.get_str());
+				//mylog(log_info,"ret=%d,errno=,%d %s\n",ret,bind_fd,remote_addr.get_str());
 				conn_info.state.client_current_state=client_tcp_handshake_dummy;
 				mylog(log_info,"state changed from client_idle to client_tcp_handshake_dummy\n");
 			}
@@ -306,6 +388,77 @@ int client_on_timer(conn_info_t &conn_info) //for client. called when a timer is
 	}
 	return 0;
 }
+int client_on_raw_recv_hs2_or_ready(conn_info_t &conn_info,char type,char *data,int data_len)
+{
+    packet_info_t &send_info=conn_info.raw_info.send_info;
+    packet_info_t &recv_info=conn_info.raw_info.recv_info;
+
+    if(!recv_info.new_src_ip.equal(send_info.new_dst_ip)||recv_info.src_port!=send_info.dst_port)
+    {
+        mylog(log_warn,"unexpected adress %s %s %d %d,this shouldnt happen.\n",recv_info.new_src_ip.get_str1(),send_info.new_dst_ip.get_str2(),recv_info.src_port,send_info.dst_port);
+        return -1;
+    }
+
+    if(conn_info.state.client_current_state==client_handshake2)
+    {
+        mylog(log_info,"changed state from to client_handshake2 to client_ready\n");
+        conn_info.state.client_current_state=client_ready;
+        conn_info.last_hb_sent_time=0;
+        conn_info.last_hb_recv_time=get_current_time();
+        conn_info.last_oppsite_roller_time=conn_info.last_hb_recv_time;
+        client_on_timer(conn_info);
+    }
+    if(data_len>=0&&type=='h')
+    {
+        mylog(log_debug,"[hb]heart beat received,oppsite_roller=%d\n",int(conn_info.oppsite_roller));
+        conn_info.last_hb_recv_time=get_current_time();
+        return 0;
+    }
+    else if(data_len>= int( sizeof(u32_t))&&type=='d')
+    {
+        mylog(log_trace,"received a data from fake tcp,len:%d\n",data_len);
+
+        if(hb_mode==0)
+            conn_info.last_hb_recv_time=get_current_time();
+
+        u32_t tmp_conv_id;
+        memcpy(&tmp_conv_id,&data[0],sizeof(tmp_conv_id));
+        tmp_conv_id=ntohl(tmp_conv_id);
+
+        if(!conn_info.blob->conv_manager.c.is_conv_used(tmp_conv_id))
+        {
+            mylog(log_info,"unknow conv %d,ignore\n",tmp_conv_id);
+            return 0;
+        }
+
+        conn_info.blob->conv_manager.c.update_active_time(tmp_conv_id);
+
+        //u64_t u64=conn_info.blob->conv_manager.c.find_data_by_conv(tmp_conv_id);
+        address_t tmp_addr=conn_info.blob->conv_manager.c.find_data_by_conv(tmp_conv_id);
+
+        //sockaddr_in tmp_sockaddr={0};
+
+        //tmp_sockaddr.sin_family = AF_INET;
+        //tmp_sockaddr.sin_addr.s_addr=(u64>>32u);
+
+        //tmp_sockaddr.sin_port= htons(uint16_t((u64<<32u)>>32u));
+
+
+        int ret=sendto(udp_fd,data+sizeof(u32_t),data_len -(sizeof(u32_t)),0,(struct sockaddr *)&tmp_addr.inner,tmp_addr.get_len());
+
+        if(ret<0)
+        {
+            mylog(log_warn,"sento returned %d,%s,%02x,%s\n",ret,get_sock_error(),int(tmp_addr.get_type()),tmp_addr.get_str());
+            //perror("ret<0");
+        }
+    }
+    else
+    {
+        mylog(log_warn,"unknown packet,this shouldnt happen.\n");
+        return -1;
+    }
+    return 0;
+}
 int client_on_raw_recv(conn_info_t &conn_info) //called when raw fd received a packet.
 {
 	char* data;int data_len;
@@ -315,7 +468,10 @@ int client_on_raw_recv(conn_info_t &conn_info) //called when raw fd received a p
 	raw_info_t &raw_info=conn_info.raw_info;
 
 	mylog(log_trace,"<client_on_raw_recv,send_info.ts_ack= %u>\n",send_info.ts_ack);
+
+#ifdef UDP2RAW_LINUX
 	if(pre_recv_raw_packet()<0) return -1;
+#endif
 
 	if(conn_info.state.client_current_state==client_idle )
 	{
@@ -329,6 +485,11 @@ int client_on_raw_recv(conn_info_t &conn_info) //called when raw fd received a p
 		{
 			return -1;
 		}
+		if(data_len>=max_data_len+1)
+		{
+			mylog(log_debug,"data_len=%d >= max_data_len+1,ignored",data_len);
+			return -1;
+		}
 		if(!recv_info.new_src_ip.equal(send_info.new_dst_ip)||recv_info.src_port!=send_info.dst_port)
 		{
 			mylog(log_debug,"unexpected adress %s %s %d %d\n",recv_info.new_src_ip.get_str1(),send_info.new_dst_ip.get_str2(),recv_info.src_port,send_info.dst_port);
@@ -426,74 +587,21 @@ int client_on_raw_recv(conn_info_t &conn_info) //called when raw fd received a p
 	}
 	else if(conn_info.state.client_current_state==client_handshake2||conn_info.state.client_current_state==client_ready)//received heartbeat or data
 	{
-		char type;
+		vector<char> type_vec;
-		if(recv_safer(conn_info,type,data,data_len)!=0)
+		vector<string> data_vec;
+        recv_safer_multi(conn_info,type_vec,data_vec);
+		if(data_vec.empty())
 		{
 			mylog(log_debug,"recv_safer failed!\n");
 			return -1;
 		}
-		if(!recv_info.new_src_ip.equal(send_info.new_dst_ip)||recv_info.src_port!=send_info.dst_port)
+
+		for(int i=0;i<(int)type_vec.size();i++)
         {
-			mylog(log_warn,"unexpected adress %s %s %d %d,this shouldnt happen.\n",recv_info.new_src_ip.get_str1(),send_info.new_dst_ip.get_str2(),recv_info.src_port,send_info.dst_port);
+		    char type=type_vec[i];
-			return -1;
+		    char *data=(char *)data_vec[i].c_str(); //be careful, do not append data to it
-		}
+		    int data_len=data_vec[i].length();
-		if(conn_info.state.client_current_state==client_handshake2)
+            client_on_raw_recv_hs2_or_ready(conn_info, type, data,data_len);
-		{
-			mylog(log_info,"changed state from to client_handshake2 to client_ready\n");
-			conn_info.state.client_current_state=client_ready;
-			conn_info.last_hb_sent_time=0;
-			conn_info.last_hb_recv_time=get_current_time();
-			conn_info.last_oppsite_roller_time=conn_info.last_hb_recv_time;
-			client_on_timer(conn_info);
-		}
-		if(data_len>=0&&type=='h')
-		{
-			mylog(log_debug,"[hb]heart beat received,oppsite_roller=%d\n",int(conn_info.oppsite_roller));
-			conn_info.last_hb_recv_time=get_current_time();
-			return 0;
-		}
-		else if(data_len>= int( sizeof(u32_t))&&type=='d')
-		{
-			mylog(log_trace,"received a data from fake tcp,len:%d\n",data_len);
-
-			if(hb_mode==0)
-				conn_info.last_hb_recv_time=get_current_time();
-
-			u32_t tmp_conv_id;
-			memcpy(&tmp_conv_id,&data[0],sizeof(tmp_conv_id));
-			tmp_conv_id=ntohl(tmp_conv_id);
-
-			if(!conn_info.blob->conv_manager.c.is_conv_used(tmp_conv_id))
-			{
-				mylog(log_info,"unknow conv %d,ignore\n",tmp_conv_id);
-				return 0;
-			}
-
-			conn_info.blob->conv_manager.c.update_active_time(tmp_conv_id);
-
-			//u64_t u64=conn_info.blob->conv_manager.c.find_data_by_conv(tmp_conv_id);
-			address_t tmp_addr=conn_info.blob->conv_manager.c.find_data_by_conv(tmp_conv_id);
-
-			//sockaddr_in tmp_sockaddr={0};
-
-			//tmp_sockaddr.sin_family = AF_INET;
-			//tmp_sockaddr.sin_addr.s_addr=(u64>>32u);
-
-			//tmp_sockaddr.sin_port= htons(uint16_t((u64<<32u)>>32u));
-
-
-			int ret=sendto(udp_fd,data+sizeof(u32_t),data_len -(sizeof(u32_t)),0,(struct sockaddr *)&tmp_addr.inner,tmp_addr.get_len());
-
-			if(ret<0)
-			{
-		    	mylog(log_warn,"sento returned %d,%s,%02x,%s\n",ret,get_sock_error(),int(tmp_addr.get_type()),tmp_addr.get_str());
-				//perror("ret<0");
-			}
-		}
-		else
-		{
-			mylog(log_warn,"unknown packet,this shouldnt happen.\n");
-						return -1;
         }
 
 		return 0;
@@ -509,11 +617,12 @@ int client_on_udp_recv(conn_info_t &conn_info)
 {
 	int recv_len;
 	char buf[buf_len];
-	address_t::storage_t udp_new_addr_in={0};
+	address_t::storage_t udp_new_addr_in={{0}};
 	socklen_t udp_new_addr_len = sizeof(address_t::storage_t);
 	if ((recv_len = recvfrom(udp_fd, buf, max_data_len+1, 0,
 			(struct sockaddr *) &udp_new_addr_in, &udp_new_addr_len)) == -1) {
-		mylog(log_warn,"recv_from error,%s\n",get_sock_error());
+		mylog(log_debug,"recv_from error,%s\n",get_sock_error());
+		return -1;
 		//myexit(1);
 	};
 
@@ -563,10 +672,71 @@ void udp_accept_cb(struct ev_loop *loop, struct ev_io *watcher, int revents)
 }
 void raw_recv_cb(struct ev_loop *loop, struct ev_io *watcher, int revents)
 {
-	//assert(0==1);
+	if(is_udp2raw_mp)assert(0==1);
 	conn_info_t & conn_info= *((conn_info_t*)watcher->data);
 	client_on_raw_recv(conn_info);
 }
+#ifdef UDP2RAW_MP
+void async_cb(struct ev_loop *loop, struct ev_async *watcher, int revents)
+{
+	conn_info_t & conn_info= *((conn_info_t*)watcher->data);
+
+	if(send_with_pcap&&!pcap_header_captured)
+	{
+		int empty=0;char *p;int len;
+		pthread_mutex_lock(&queue_mutex);
+		empty=my_queue.empty();
+		if(!empty)
+		{
+			my_queue.peek_front(p,len);
+			my_queue.pop_front();
+		}
+		pthread_mutex_unlock(&queue_mutex);
+		if(empty) return;
+
+		pcap_header_captured=1;
+		assert(pcap_link_header_len!=-1);
+		memcpy(pcap_header_buf,p,max_data_len);
+
+		log_bare(log_info,"link level header captured:\n");
+		unsigned char *tmp=(unsigned char*)pcap_header_buf;
+		pcap_captured_full_len=len;
+		for(int i=0;i<pcap_link_header_len;i++)
+		log_bare(log_info,"<%x>",(u32_t)tmp[i]);
+
+		log_bare(log_info,"\n");
+		return ;
+	}
+
+	//mylog(log_info,"async_cb called\n");
+	while(1)
+	{
+		int empty=0;char *p;int len;
+		pthread_mutex_lock(&queue_mutex);
+		empty=my_queue.empty();
+		if(!empty)
+		{
+			my_queue.peek_front(p,len);
+			my_queue.pop_front();
+		}
+		pthread_mutex_unlock(&queue_mutex);
+
+		if(empty) break;
+		if(g_fix_gro==0&&len>max_data_len)
+		{
+		    mylog(log_warn,"huge packet %d > %d, dropped\n",len,max_data_len);
+		    break;
+		}
+
+		int new_len=len-pcap_link_header_len;
+		memcpy(g_packet_buf,p+pcap_link_header_len,new_len);
+		g_packet_buf_len=new_len;
+		assert(g_packet_buf_cnt==0);
+		g_packet_buf_cnt++;
+		client_on_raw_recv(conn_info);
+	}
+}
+#endif
 void clear_timer_cb(struct ev_loop *loop, struct ev_timer *watcher, int revents)
 {
 	conn_info_t & conn_info= *((conn_info_t*)watcher->data);
@@ -612,6 +782,7 @@ int client_event_loop()
 	packet_info_t &send_info=conn_info.raw_info.send_info;
 	packet_info_t &recv_info=conn_info.raw_info.recv_info;
 
+#ifdef UDP2RAW_LINUX
 	if(lower_level)
 	{
 		if(lower_level_manual)
@@ -687,6 +858,116 @@ int client_event_loop()
 		}
 
 	}
+#endif
+	
+#ifdef UDP2RAW_MP
+
+	address_t tmp_addr;
+	if(get_src_adress2(tmp_addr,remote_addr)!=0)
+	{
+		mylog(log_error,"get_src_adress() failed\n");
+		myexit(-1);
+	}
+	if(strcmp(dev,"")==0)
+	{
+		mylog(log_info,"--dev have not been set, trying to detect automatically, available devices:\n");
+
+		mylog(log_info,"available device(device name: ip address ; description):\n");
+
+		char errbuf[PCAP_ERRBUF_SIZE];
+
+		int found=0;
+
+		pcap_if_t *interfaces,*d;
+		if(pcap_findalldevs(&interfaces,errbuf)==-1)
+		{
+			mylog(log_fatal,"error in pcap_findalldevs(),%s\n",errbuf);
+			myexit(-1);
+		}
+
+		for(pcap_if_t *d=interfaces; d!=NULL; d=d->next) {
+			log_bare(log_warn,"%s:", d->name);
+			int cnt=0;
+			for(pcap_addr_t *a=d->addresses; a!=NULL; a=a->next) {
+				if(a->addr==NULL)
+				{
+					log_bare(log_debug," [a->addr==NULL]");
+					continue;
+				}
+				if(a->addr->sa_family == AF_INET||a->addr->sa_family == AF_INET6)
+				{
+					cnt++;
+
+					if(a->addr->sa_family ==AF_INET)
+					{
+						char s[max_addr_len];
+						inet_ntop(AF_INET, &((struct sockaddr_in*)a->addr)->sin_addr, s,max_addr_len);
+						log_bare(log_warn," [%s]", s);
+
+						if(a->addr->sa_family==raw_ip_version)
+						{
+							if(((struct sockaddr_in*)a->addr)->sin_addr.s_addr ==tmp_addr.inner.ipv4.sin_addr.s_addr)
+							{
+								found++;
+								strcpy(dev,d->name);
+							}
+						}
+					}
+					else
+					{
+						assert(a->addr->sa_family ==AF_INET6);
+
+						char s[max_addr_len];
+						inet_ntop(AF_INET6, &((struct sockaddr_in6*)a->addr)->sin6_addr, s,max_addr_len);
+						log_bare(log_warn," [%s]", s);
+
+						if(a->addr->sa_family==raw_ip_version)
+						{
+							if(  memcmp( &((struct sockaddr_in6*)a->addr)->sin6_addr,&tmp_addr.inner.ipv6.sin6_addr,sizeof(struct in6_addr))==0 )
+							{
+								found++;
+								strcpy(dev,d->name);
+							}
+						}
+					}
+				}
+				else
+				{
+					log_bare(log_debug," [unknow:%d]",int(a->addr->sa_family));
+				}
+			}
+			if(cnt==0) log_bare(log_warn," [no ip found]");
+			if(d->description==0)
+			{
+				log_bare(log_warn,"; (no description available)");
+			}
+			else
+			{
+				log_bare(log_warn,"; %s", d->description);
+			}
+			log_bare(log_warn,"\n");
+		}
+
+		if(found==0)
+		{
+			mylog(log_fatal,"no matched device found for ip: [%s]\n",tmp_addr.get_ip());
+			myexit(-1);
+		}
+		else if(found==1)
+		{
+			mylog(log_info,"using device:[%s], ip: [%s]\n",dev,tmp_addr.get_ip());
+		}
+		else
+		{
+			mylog(log_fatal,"more than one devices found for ip: [%s] , you need to use --dev manually\n",tmp_addr.get_ip());
+			myexit(-1);
+		}
+	}
+	else
+	{
+		mylog(log_info,"--dev has been manually set, using device:[%s]\n",dev);
+	}
+#endif
 
 	send_info.src_port=0;
 	memset(&send_info.new_src_ip,0,sizeof(send_info.new_src_ip));
@@ -746,11 +1027,22 @@ int client_event_loop()
 	//	myexit(-1);
 	//}
 
+#ifdef UDP2RAW_LINUX
 	struct ev_io raw_recv_watcher;
 
 	raw_recv_watcher.data=&conn_info;
     ev_io_init(&raw_recv_watcher, raw_recv_cb, raw_recv_fd, EV_READ);
     ev_io_start(loop, &raw_recv_watcher);
+#endif
+
+#ifdef UDP2RAW_MP
+	g_default_loop=loop;
+	async_watcher.data=&conn_info;
+	ev_async_init(&async_watcher,async_cb);
+	ev_async_start(loop,&async_watcher);
+
+	init_raw_socket();//must be put after dev detection
+#endif
 
 	//set_timer(epollfd,timer_fd);
 	struct ev_timer clear_timer;

common.cpp

@@ -345,6 +345,57 @@ int my_ip_t::from_str(char * str)
 	}
 	return 0;
 }*/
+#ifdef UDP2RAW_MP
+
+int init_ws()
+{
+#if defined(__MINGW32__)
+	WORD wVersionRequested;
+	WSADATA wsaData;
+	int err;
+
+	/* Use the MAKEWORD(lowbyte, highbyte) macro declared in Windef.h */
+	wVersionRequested = MAKEWORD(2, 2);
+
+	err = WSAStartup(wVersionRequested, &wsaData);
+	if (err != 0) {
+		/* Tell the user that we could not find a usable */
+		/* Winsock DLL.                                  */
+		printf("WSAStartup failed with error: %d\n", err);
+		exit(-1);
+	}
+
+	/* Confirm that the WinSock DLL supports 2.2.*/
+	/* Note that if the DLL supports versions greater    */
+	/* than 2.2 in addition to 2.2, it will still return */
+	/* 2.2 in wVersion since that is the version we      */
+	/* requested.                                        */
+
+	if (LOBYTE(wsaData.wVersion) != 2 || HIBYTE(wsaData.wVersion) != 2) {
+		/* Tell the user that we could not find a usable */
+		/* WinSock DLL.                                  */
+		printf("Could not find a usable version of Winsock.dll\n");
+		WSACleanup();
+		exit(-1);
+	}
+	else
+	{
+		printf("The Winsock 2.2 dll was found okay");
+	}
+
+	int tmp[]={0,100,200,300,500,800,1000,2000,3000,4000,-1};
+	int succ=0;
+	for(int i=1;tmp[i]!=-1;i++)
+	{
+		if(_setmaxstdio(100)==-1) break;
+		else succ=i;
+	}
+	printf(", _setmaxstdio() was set to %d\n",tmp[succ]);
+#endif
+return 0;
+}
+
+#endif
 
 #if defined(__MINGW32__)
 int inet_pton(int af, const char *src, void *dst)
@@ -404,7 +455,11 @@ char *get_sock_error()
 			(LPWSTR)&s, 0, NULL);
 	sprintf(buf, "%d:%S", e,s);
 	int len=strlen(buf);
-	if(len>0&&buf[len-1]=='\n') buf[len-1]=0;
+	while(len>0 && (buf[len-1]=='\r'||buf[len-1]=='\n' ))
+	{
+		len--;
+		buf[len]=0;
+	}
 	LocalFree(s);
 	return buf;
 }
@@ -426,13 +481,31 @@ int get_sock_errno()
 #endif
 
 
+u64_t get_current_time_us()
+{
+        static u64_t value_fix=0;
+        static u64_t largest_value=0;
+
+        u64_t raw_value=(u64_t)(ev_time()*1000*1000);
+
+        u64_t fixed_value=raw_value+value_fix;
+
+        if(fixed_value< largest_value)
+        {
+                value_fix+= largest_value- fixed_value;
+        }
+        else
+        {
+                largest_value=fixed_value;
+        }
+
+	//printf("<%lld,%lld,%lld>\n",raw_value,value_fix,raw_value + value_fix);
+        return raw_value + value_fix; //new fixed value
+}
+
 u64_t get_current_time()
 {
-	timespec tmp_time;
+	return get_current_time_us()/1000;
-	clock_gettime(CLOCK_MONOTONIC, &tmp_time);
-	return ((u64_t)tmp_time.tv_sec)*1000llu+((u64_t)tmp_time.tv_nsec)/(1000*1000llu);
-
-	//return (u64_t)(ev_time()*1000); //todo change to this later
 }
 
 u64_t pack_u64(u32_t a,u32_t b)
@@ -470,6 +543,8 @@ void init_random_number_fd()
 	}
 	setnonblocking(random_number_fd);
 }*/
+
+#if !defined(__MINGW32__)
 struct random_fd_t
 {
 	int random_number_fd;
@@ -489,8 +564,60 @@ struct random_fd_t
 		return random_number_fd;
 	}
 }random_fd;
+#else
+struct my_random_t
+{
+    std::random_device rd;
+    std::mt19937 gen;
+    std::uniform_int_distribution<u64_t> dis64;
+    std::uniform_int_distribution<u32_t> dis32;
+
+    std::uniform_int_distribution<unsigned char> dis8;
+
+    my_random_t()
+	{
+    	//std::mt19937 gen_tmp(rd());  //random device is broken on mingw
+	timespec tmp_time;
+	clock_gettime(CLOCK_MONOTONIC, &tmp_time);
+	long  long  a=((u64_t)tmp_time.tv_sec)*1000000000llu+((u64_t)tmp_time.tv_nsec);
+    	std::mt19937 gen_tmp(a);
+    	gen=gen_tmp;
+    	gen.discard(700000);  //magic
+	}
+    u64_t gen64()
+    {
+    	return dis64(gen);
+    }
+    u32_t gen32()
+    {
+    	return dis32(gen);
+    }
+
+    unsigned char gen8()
+    {
+    	return dis8(gen);
+    }
+	/*int random_number_fd;
+	random_fd_t()
+	{
+			random_number_fd=open("/dev/urandom",O_RDONLY);
+			if(random_number_fd==-1)
+			{
+				mylog(log_fatal,"error open /dev/urandom\n");
+				myexit(-1);
+			}
+			setnonblocking(random_number_fd);
+	}
+	int get_fd()
+	{
+		return random_number_fd;
+	}*/
+}my_random;
+#endif
+
 u64_t get_true_random_number_64()
 {
+#if !defined(__MINGW32__)
 	u64_t ret;
 	int size=read(random_fd.get_fd(),&ret,sizeof(ret));
 	if(size!=sizeof(ret))
@@ -499,9 +626,13 @@ u64_t get_true_random_number_64()
 		myexit(-1);
 	}
 	return ret;
+#else
+	return my_random.gen64();  //fake random number
+#endif
 }
 u32_t get_true_random_number()
 {
+#if !defined(__MINGW32__)
 	u32_t ret;
 	int size=read(random_fd.get_fd(),&ret,sizeof(ret));
 	if(size!=sizeof(ret))
@@ -510,6 +641,9 @@ u32_t get_true_random_number()
 		myexit(-1);
 	}
 	return ret;
+#else
+	return my_random.gen32();  //fake random number
+#endif
 }
 u32_t get_true_random_number_nz() //nz for non-zero
 {
@@ -672,6 +806,13 @@ int set_buf_size(int fd,int socket_buf_size)
 {
 	if(force_socket_buf)
 	{
+		if(is_udp2raw_mp)
+		{
+		mylog(log_fatal,"force_socket_buf not supported in this verion\n");
+		myexit(-1);
+		}
+		//assert(0==1);
+#ifdef UDP2RAW_LINUX
 		if(setsockopt(fd, SOL_SOCKET, SO_SNDBUFFORCE, &socket_buf_size, sizeof(socket_buf_size))<0)
 		{
 			mylog(log_fatal,"SO_SNDBUFFORCE fail  socket_buf_size=%d  errno=%s\n",socket_buf_size,strerror(errno));
@@ -682,6 +823,8 @@ int set_buf_size(int fd,int socket_buf_size)
 			mylog(log_fatal,"SO_RCVBUFFORCE fail  socket_buf_size=%d  errno=%s\n",socket_buf_size,strerror(errno));
 			myexit(1);
 		}
+#endif
+
 	}
 	else
 	{
@@ -824,6 +967,7 @@ void myexit(int a)
 {
     if(enable_log_color)
    	printf("%s\n",RESET);
+#ifdef UDP2RAW_LINUX
     if(keep_thread_running)
     {
 		if(pthread_cancel(keep_thread))
@@ -836,6 +980,7 @@ void myexit(int a)
 		}
     }
 	clear_iptables_rule();
+#endif
 	exit(a);
 }
 
@@ -905,6 +1050,12 @@ int read_file(const char * file,string &output)
 	return 0;
 }
 int run_command(string command0,char * &output,int flag) {
+if(is_udp2raw_mp)
+{
+    mylog(log_fatal,"run_command not supported in this version\n");
+    myexit(-1);
+}
+#ifdef UDP2RAW_LINUX
     FILE *in;
 
 
@@ -958,6 +1109,7 @@ int run_command(string command0,char * &output,int flag) {
     	return -4;
     }
 
+#endif
     return 0;
 
 }
@@ -1078,6 +1230,7 @@ vector<string> parse_conf_line(const string& s0)
 
 int create_fifo(char * file)
 {
+#if !defined(__MINGW32__)
 	if(mkfifo (file, 0666)!=0)
 	{
 		if(errno==EEXIST)
@@ -1111,6 +1264,11 @@ int create_fifo(char * file)
 
 	setnonblocking(fifo_fd);
 	return fifo_fd;
+#else
+        mylog(log_fatal,"--fifo not supported in this version\n");
+        myexit(-1);
+	return 0;
+#endif
 }
 
 /*

common.h

@@ -27,8 +27,12 @@
 #include <assert.h>
 #include <pthread.h>
 
-#if defined(UDP2RAW_MP)
+#ifndef USE_LIBNET
+#define NO_LIBNET
+#endif
 
+#if defined(UDP2RAW_MP)
+const int is_udp2raw_mp=1;
 #if !defined(__CYGWIN__) && !defined(__MINGW32__)
 #include <pcap.h>
 #else
@@ -40,8 +44,10 @@
 #include <libnet.h>
 #endif
 
-#else
 
+#else
+#define UDP2RAW_LINUX
+const int is_udp2raw_mp=0;
 //#include <linux/if_ether.h>
 #include <linux/filter.h>
 #include <linux/if_packet.h>
@@ -53,7 +59,11 @@
 
 #endif
 
+#if !defined(NO_LIBEV_EMBED)
 #include <my_ev.h>
+#else
+#include "ev.h"
+#endif
 
 #if defined(__MINGW32__)
 #include <winsock2.h>
@@ -159,6 +169,8 @@ const int max_addr_len=100;
 
 extern int force_socket_buf;
 
+extern int g_fix_gro;
+
 /*
 struct ip_port_t
 {
@@ -345,12 +357,60 @@ struct not_copy_able_t
 	}
 };
 
+const int huge_data_len=65535+100; //a packet with link level header might be larger than 65535
+const int huge_buf_len=huge_data_len+100;
 
 const int max_data_len=1800;
 const int buf_len=max_data_len+400;
 
 //const int max_address_len=512;
 
+#ifdef UDP2RAW_MP
+const int queue_len=200;
+
+struct queue_t
+{
+	char data[queue_len][huge_buf_len];
+	int data_len[queue_len];
+
+	int head=0;
+	int tail=0;
+	void clear()
+	{
+		head=tail=0;
+	}
+	int empty()
+	{
+		if(head==tail) return 1;
+		else return 0;
+	}
+	int full()
+	{
+		if( (tail+1)%queue_len==head  ) return 1;
+		else return 0;
+	}
+	void peek_front(char * & p,int &len)
+	{
+		assert(!empty());
+		p=data[head];
+		len=data_len[head];
+	}
+	void pop_front()
+	{
+		assert(!empty());
+		head++;head%=queue_len;
+	}
+	void push_back(char * p,int len)
+	{
+		assert(!full());
+		memcpy(data[tail],p,len);
+		data_len[tail]=len;
+		tail++;tail%=queue_len;
+	}
+};
+
+int init_ws();
+#endif
 u64_t get_current_time();
 u64_t pack_u64(u32_t a,u32_t b);
 

connection.cpp

@@ -11,8 +11,6 @@
 
 int disable_anti_replay=0;//if anti_replay windows is diabled
 
-
-
 const int disable_conn_clear=0;//a raw connection is called conn.
 
 conn_manager_t conn_manager;
@@ -418,6 +416,13 @@ int recv_bare(raw_info_t &raw_info,char* & data,int & len)//recv function with e
 		//printf("recv_raw_fail in recv bare\n");
 		return -1;
 	}
+
+	if(len>=max_data_len+1)
+	{
+		mylog(log_debug,"data_len=%d >= max_data_len+1,ignored",len);
+		return -1;
+	}
+
 	mylog(log_trace,"data len=%d\n",len);
 	if ((raw_mode == mode_faketcp && (recv_info.syn == 1 || recv_info.ack != 1)))
 	{
@@ -462,12 +467,9 @@ int send_safer(conn_info_t &conn_info,char type,const char* data,int len)  //saf
 	}
 
 
-
 	char send_data_buf[buf_len];  //buf for send data and send hb
 	char send_data_buf2[buf_len];
 
-
-
 	my_id_t n_tmp_id=htonl(conn_info.my_id);
 
 	memcpy(send_data_buf,&n_tmp_id,sizeof(n_tmp_id));
@@ -488,10 +490,32 @@ int send_safer(conn_info_t &conn_info,char type,const char* data,int len)  //saf
 
 	int new_len=len+sizeof(n_seq)+sizeof(n_tmp_id)*2+2;
 
+	if(g_fix_gro==0)
+    {
         if (my_encrypt(send_data_buf, send_data_buf2, new_len) != 0)
         {
             return -1;
         }
+    }
+	else
+    {
+        if (my_encrypt(send_data_buf, send_data_buf2+2, new_len) != 0)
+        {
+            return -1;
+        }
+        write_u16(send_data_buf2,new_len);
+        new_len+=2;
+	if(cipher_mode==cipher_xor)
+	{
+	    send_data_buf2[0]^=gro_xor[0];
+	    send_data_buf2[1]^=gro_xor[1];
+	}
+	else if(cipher_mode==cipher_aes128cbc||cipher_mode==cipher_aes128cbc)
+	{
+	    aes_ecb_encrypt1(send_data_buf2);
+	}
+    }
+
 
 	if(send_raw0(conn_info.raw_info,send_data_buf2,new_len)!=0) return -1;
 
@@ -598,23 +622,105 @@ int reserved_parse_safer(conn_info_t &conn_info,const char * input,int input_len
 	}
 
 
-	if(after_recv_raw0(conn_info.raw_info)!=0) return -1;
+	if(after_recv_raw0(conn_info.raw_info)!=0) return -1;  //TODO might need to move this function to somewhere else after --fix-gro is introduced
 
 	return 0;
 }
-int recv_safer(conn_info_t &conn_info,char &type,char* &data,int &len)///safer transfer function with anti-replay,when mutually verification is done.
+int recv_safer_notused(conn_info_t &conn_info,char &type,char* &data,int &len)///safer transfer function with anti-replay,when mutually verification is done.
 {
 	packet_info_t &send_info=conn_info.raw_info.send_info;
 	packet_info_t &recv_info=conn_info.raw_info.recv_info;
 
 	char * recv_data;int recv_len;
-	static char recv_data_buf[buf_len];
+	//static char recv_data_buf[buf_len];
 
 	if(recv_raw0(conn_info.raw_info,recv_data,recv_len)!=0) return -1;
 
 	return reserved_parse_safer(conn_info,recv_data,recv_len,type,data,len);
 }
 
+int recv_safer_multi(conn_info_t &conn_info,vector<char> &type_arr,vector<string> &data_arr)///safer transfer function with anti-replay,when mutually verification is done.
+{
+    packet_info_t &send_info=conn_info.raw_info.send_info;
+    packet_info_t &recv_info=conn_info.raw_info.recv_info;
+
+    char * recv_data;int recv_len;
+    assert(type_arr.empty());
+    assert(data_arr.empty());
+
+    if(recv_raw0(conn_info.raw_info,recv_data,recv_len)!=0) return -1;
+
+    char type;
+    char *data;
+    int len;
+
+    if(g_fix_gro==0)
+    {
+        int ret = reserved_parse_safer(conn_info, recv_data, recv_len, type, data, len);
+        if(ret==0)
+        {
+            type_arr.push_back(type);
+            data_arr.emplace_back(data,data+len);
+            //std::copy(data,data+len,data_arr[0]);
+        }
+        return 0;
+    } else
+    {
+        char *ori_recv_data=recv_data;
+        int ori_recv_len=recv_len;
+        //mylog(log_debug,"recv_len:%d\n",recv_len);
+        int cnt=0;
+        while(recv_len>=16)
+        {
+            cnt++;
+            int single_len_no_xor;
+            single_len_no_xor=read_u16(recv_data);
+            int single_len;
+	    if(cipher_mode==cipher_xor)
+	    {
+		recv_data[0]^=gro_xor[0];
+		recv_data[1]^=gro_xor[1];
+	    }
+	    else if(cipher_mode==cipher_aes128cbc||cipher_mode==cipher_aes128cbc)
+	    {
+		aes_ecb_decrypt1(recv_data);
+	    }
+            single_len=read_u16(recv_data);
+            recv_len-=2;
+            recv_data+=2;
+            if(single_len > recv_len)
+            {
+                mylog(log_debug,"illegal single_len %d(%d), recv_len %d left,dropped\n",single_len,single_len_no_xor,recv_len);
+                break;
+            }
+            if(single_len> max_data_len )
+            {
+                mylog(log_warn,"single_len %d(%d) > %d, maybe you need to turn down mtu at upper level\n",single_len,single_len_no_xor,max_data_len);
+		break;
+            }
+
+            int ret = reserved_parse_safer(conn_info, recv_data, single_len, type, data, len);
+
+            if(ret!=0)
+            {
+                mylog(log_debug,"parse failed, offset= %d,single_len=%d(%d)\n",(int)(recv_data-ori_recv_data),single_len,single_len_no_xor);
+            } else{
+                type_arr.push_back(type);
+                data_arr.emplace_back(data,data+len);
+                //std::copy(data,data+len,data_arr[data_arr.size()-1]);
+            }
+            recv_data+=single_len;
+            recv_len-=single_len;
+        }
+        if(cnt>1)
+        {
+            mylog(log_debug,"got a suspected gro packet, %d packets recovered, recv_len=%d, loop_cnt=%d\n",(int)data_arr.size(),ori_recv_len,cnt);
+        }
+        return 0;
+    }
+}
+
+
 void server_clear_function(u64_t u64)//used in conv_manager in server mode.for server we have to use one udp fd for one conv(udp connection),
 //so we have to close the fd when conv expires
 {

connection.h

@@ -18,8 +18,6 @@ extern int disable_anti_replay;
 
 const int disable_conv_clear=0;//a udp connection in the multiplexer is called conversation in this program,conv for short.
 
-
-
 struct anti_replay_t  //its for anti replay attack,similar to openvpn/ipsec 's anti replay window
 {
 	u64_t max_packet_received;
@@ -346,5 +344,8 @@ int send_handshake(raw_info_t &raw_info,my_id_t id1,my_id_t id2,my_id_t id3);//
 int send_safer(conn_info_t &conn_info,char type,const char* data,int len);  //safer transfer function with anti-replay,when mutually verification is done.
 int send_data_safer(conn_info_t &conn_info,const char* data,int len,u32_t conv_num);//a wrap for  send_safer for transfer data.
 //int reserved_parse_safer(conn_info_t &conn_info,const char * input,int input_len,char &type,char* &data,int &len);//subfunction for recv_safer,allow overlap
-int recv_safer(conn_info_t &conn_info,char &type,char* &data,int &len);///safer transfer function with anti-replay,when mutually verification is done.
+
+//int recv_safer(conn_info_t &conn_info,char &type,char* &data,int &len);///safer transfer function with anti-replay,when mutually verification is done.
+
+int recv_safer_multi(conn_info_t &conn_info,vector<char> &type_arr,vector<string> &data_arr);//new api for handle gro
 #endif /* CONNECTION_H_ */

doc/README.zh-cn.md

@@ -14,21 +14,19 @@ udp2raw tunnel，通过raw socket给UDP包加上TCP或ICMP header，进而绕过
 
 **提示：**
 
-udp2raw不是加速器，只是一个帮助你绕过UDP限制的工具。如果你需要UDP加速器，请看UDPspeeder。
+udp2raw不是加速器，只是一个帮助你绕过UDP限制的工具。如果你需要UDP“加速器” (改善UDP丢包)，请看UDPspeeder。
 
 UDPspeeder的repo:
 
 https://github.com/wangyu-/UDPspeeder
 # 支持的平台
-Linux主机，有root权限。可以是PC、android手机/平板、openwrt路由器、树莓派。主机上最好安装了iptables命令(apt/yum很容易安装)。
+Linux主机，有root权限或cap_net_raw capability.。可以是PC、android手机/平板、openwrt路由器、树莓派。主机上最好安装了iptables命令(apt/yum很容易安装)。
 
 Release中提供了`amd64`、`x86`、`arm`、`mips_be`、`mips_le`的预编译binary.
 
 ##### 对于windows和mac用户：
 
-可以用[这个repo](https://github.com/wangyu-/udp2raw-multiplatform)里的udp2raw，原生运行。
+可以用[这个repo](https://github.com/wangyu-/udp2raw-multiplatform)里的udp2raw。
-
-<del>可以把udp2raw运行在虚拟机上(网络必须是桥接模式)。可以参考： https://github.com/wangyu-/udp2raw-tunnel/wiki/在windows-mac上运行udp2raw客户端，带图形界面 </del>
 
 ##### 对于ios和游戏主机用户：
 
@@ -44,10 +42,10 @@ Release中提供了`amd64`、`x86`、`arm`、`mips_be`、`mips_le`的预编译bi
 ### 心跳保活、自动重连，连接恢复
 心跳保活、自动重连，udp2raw重连可以恢复上次的连接，重连后上层连接继续有效，底层掉线上层不掉线。有效解决上层连接断开的问题。 （功能借鉴自[kcptun-raw](https://github.com/Chion82/kcptun-raw)）（**就算你拔掉网线重插，或者重新拨号获得新ip，上层应用也不会断线**）
 
-### 加密 防重放攻击
+### 加密、防重放攻击
 用aes128cbc加密(或更弱的xor)，hmac-sha1(或更弱的md5/crc32/simple)做数据完整校验。用类似ipsec/openvpn的replay window机制来防止重放攻击。
 
-设计目标是，即使攻击者可以监听到tunnel的所有包，可以选择性丢弃tunnel的任意包，可以重放任意包；攻击者也没办法获得tunnel承载的任何数据，也没办法向tunnel的数据流中通过包构造/包重放插入任何数据。
+[Notes on encryption](https://github.com/wangyu-/udp2raw-tunnel/wiki/Notes-on-encryption)
 
 ### 其他特性
 信道复用，client的udp端支持多个连接。
@@ -56,7 +54,7 @@ server支持多个client，也能正确处理多个连接的重连和连接恢
 
 NAT 穿透 ，tcp icmp udp模式都支持nat穿透。
 
-支持Openvz，配合finalspeed使用，可以在openvz上用tcp模式的finalspeed
+支持Openvz，配合finalspeed使用，可以在openvz上用tcp模式的finalspeed.
 
 支持Openwrt，没有编译依赖，容易编译到任何平台上。
 
@@ -113,10 +111,10 @@ usage:
     run as server : ./this_program -s -l server_listen_ip:server_port -r remote_address:remote_port  [options]
 
 common options,these options must be same on both side:
-    --raw-mode            <string>        avaliable values:faketcp(default),udp,icmp
+    --raw-mode            <string>        available values:faketcp(default),udp,icmp
     -k,--key              <string>        password to gen symetric key,default:"secret key"
-    --cipher-mode         <string>        avaliable values:aes128cbc(default),xor,none
+    --cipher-mode         <string>        available values:aes128cbc(default),xor,none
-    --auth-mode           <string>        avaliable values:hmac_sha1,md5(default),crc32,simple,none
+    --auth-mode           <string>        available values:hmac_sha1,md5(default),crc32,simple,none
     -a,--auto-rule                        auto add (and delete) iptables rule
     -g,--gen-rule                         generate iptables rule then exit,so that you can copy and
                                           add it manually.overrides -a
@@ -264,25 +262,6 @@ raw_mode: faketcp  cipher_mode: aes128cbc  auth_mode: md5
 [udp2raw+kcptun step_by_step教程](kcptun_step_by_step.md)
 ### 中转 finalspeed
 [udp2raw+finalspeed step_by_step教程](finalspeed_step_by_step.md)
-# 如何自己编译
-[编译教程](build_guide.zh-cn.md)
-# 相关repo
-### kcptun-raw
-udp2raw was inspired by kcptun-raw,which modified kcptun to support tcp mode.
-
-https://github.com/Chion82/kcptun-raw
-### relayRawSocket
-kcptun-raw was inspired by relayRawSocket. A simple  udp to raw tunnel,wrote in python
-
-https://github.com/linhua55/some_kcptun_tools/tree/master/relayRawSocket
-### kcpraw
-another project of kcptun with tcp mode
-
-https://github.com/ccsexyz/kcpraw
-### icmptunnel
-Transparently tunnel your IP traffic through ICMP echo and reply packets.
-
-https://github.com/DhavalKapil/icmptunnel
 
 # wiki
 

encrypt.cpp

@@ -26,6 +26,8 @@ unsigned char hmac_key_decrypt[hmac_key_len + 100];  //key for hmac
 unsigned char cipher_key_encrypt[cipher_key_len + 100];  //key for aes etc.
 unsigned char cipher_key_decrypt[cipher_key_len + 100];  //key for aes etc.
 
+char gro_xor[256+100];//dirty fix for gro
+
 unordered_map<int, const char *> auth_mode_tostring = {{auth_none, "none"}, {auth_md5, "md5"}, {auth_crc32, "crc32"},{auth_simple,"simple"},{auth_hmac_sha1,"hmac_sha1"},};
 
 unordered_map<int, const char *> cipher_mode_tostring={{cipher_none,"none"},{cipher_aes128cfb,"aes128cfb"},{cipher_aes128cbc,"aes128cbc"},{cipher_xor,"xor"},};
@@ -35,6 +37,8 @@ auth_mode_t auth_mode=auth_md5;
 cipher_mode_t cipher_mode=cipher_aes128cbc;
 int is_hmac_used=0;
 
+int aes128cfb_old=0;
+
 //TODO key negotiation and forward secrecy
 
 int my_init_keys(const char * user_passwd,int is_client)
@@ -48,9 +52,10 @@ int my_init_keys(const char * user_passwd,int is_client)
 
 	md5((uint8_t*)tmp,strlen(tmp),(uint8_t*)normal_key);
 
+
 	if(auth_mode==auth_hmac_sha1)
 		is_hmac_used=1;
-	if(is_hmac_used)
+	if(is_hmac_used||g_fix_gro||1)
 	{
 		unsigned char salt[400]="";
 		char salt_text[400]="udp2raw_salt1";
@@ -82,6 +87,9 @@ int my_init_keys(const char * user_passwd,int is_client)
 		assert( hkdf_sha256_expand( pbkdf2_output1,32, (unsigned char *)info_cipher_decrypt,strlen(info_cipher_decrypt), cipher_key_decrypt, cipher_key_len )  ==0);
 		assert( hkdf_sha256_expand( pbkdf2_output1,32, (unsigned char *)info_hmac_encrypt,strlen(info_hmac_encrypt), hmac_key_encrypt, hmac_key_len )  ==0);
 		assert( hkdf_sha256_expand( pbkdf2_output1,32, (unsigned char *)info_hmac_decrypt,strlen(info_hmac_decrypt), hmac_key_decrypt, hmac_key_len )  ==0);
+
+        const char *gro_info="gro";
+        assert( hkdf_sha256_expand( pbkdf2_output1,32, (unsigned char *)gro_info,strlen(gro_info), (unsigned char *)gro_xor, 256 )  ==0);
 	}
 	
 	print_binary_chars(normal_key,16);
@@ -283,6 +291,7 @@ int padding(char *data ,int &data_len,int padding_num)
 
 int de_padding(const char *data ,int &data_len,int padding_num)
 {
+	if(data_len==0) return -1;
 	if((uint8_t)data[data_len-1]  >padding_num) return -1;
 	data_len-=(uint8_t)data[data_len-1];
 	if(data_len<0)
@@ -291,6 +300,40 @@ int de_padding(const char *data ,int &data_len,int padding_num)
 	}
 	return 0;
 }
+void aes_ecb_encrypt(const char *data,char *output)
+{
+	static int first_time=1;
+	char *key=(char*)cipher_key_encrypt;
+	if(aes_key_optimize)
+	{
+		if(first_time==0) key=0;
+		else first_time=0;
+	}	
+	AES_ECB_encrypt_buffer((uint8_t*)data,(uint8_t*)key,(uint8_t*)output);
+}
+void aes_ecb_encrypt1(char *data)
+{
+    char buf[16];
+    memcpy(buf,data,16);
+    aes_ecb_encrypt(buf,data);
+}
+void aes_ecb_decrypt(const char *data,char *output)
+{
+	static int first_time=1;
+	char *key=(char*)cipher_key_decrypt;
+	if(aes_key_optimize)
+	{
+		if(first_time==0) key=0;
+		else first_time=0;
+	}	
+	AES_ECB_decrypt_buffer((uint8_t*)data,(uint8_t*)key,(uint8_t*)output);
+}
+void aes_ecb_decrypt1(char *data)
+{
+    char buf[16];
+    memcpy(buf,data,16);
+    aes_ecb_decrypt(buf,data);
+}
 int cipher_aes128cbc_encrypt(const char *data,char *output,int &len,char * key)
 {
 	static int first_time=1;
@@ -312,6 +355,7 @@ int cipher_aes128cbc_encrypt(const char *data,char *output,int &len,char * key)
 int cipher_aes128cfb_encrypt(const char *data,char *output,int &len,char * key)
 {
 	static int first_time=1;
+	assert(len>=16);
 
 	char buf[buf_len];
 	memcpy(buf,data,len);//TODO inefficient code
@@ -320,6 +364,10 @@ int cipher_aes128cfb_encrypt(const char *data,char *output,int &len,char * key)
 		if(first_time==0) key=0;
 		else first_time=0;
 	}
+	if(!aes128cfb_old)
+	{
+	    aes_ecb_encrypt(data,buf); //encrypt the first block
+	}
 
 	AES_CFB_encrypt_buffer((unsigned char *)output,(unsigned char *)buf,len,(unsigned char *)key,(unsigned char *)zero_iv);
 	return 0;
@@ -363,12 +411,19 @@ int cipher_aes128cbc_decrypt(const char *data,char *output,int &len,char * key)
 int cipher_aes128cfb_decrypt(const char *data,char *output,int &len,char * key)
 {
 	static int first_time=1;
+	if(len<16) return -1;
+    
 	if(aes_key_optimize)
 	{
 		if(first_time==0) key=0;
 		else first_time=0;
 	}
+	
+	
 	AES_CFB_decrypt_buffer((unsigned char *)output,(unsigned char *)data,len,(unsigned char *)key,(unsigned char *)zero_iv);
+	
+	if(!aes128cfb_old)
+	    aes_ecb_decrypt1(output); //decrypt the first block
 	//if(de_padding(output,len,16)<0) return -1;
 	return 0;
 }

encrypt.h

@@ -12,6 +12,7 @@
 //extern char key[16];
 
 const int aes_key_optimize=1; //if enabled,once you used a key for aes,you cant change it anymore
+extern int aes128cfb_old;
 
 int my_init_keys(const char *,int);
 
@@ -34,10 +35,15 @@ extern cipher_mode_t cipher_mode;
 extern unordered_map<int, const char *> auth_mode_tostring;
 extern unordered_map<int, const char *> cipher_mode_tostring;
 
-
+extern char gro_xor[256+100];
-
 
 int cipher_decrypt(const char *data,char *output,int &len,char * key);//internal interface ,exposed for test only
 int cipher_encrypt(const char *data,char *output,int &len,char * key);//internal interface ,exposed for test only
 
+void aes_ecb_encrypt(const char *data,char *output);
+void aes_ecb_decrypt(const char *data,char *output);
+
+void aes_ecb_encrypt1(char *data);
+void aes_ecb_decrypt1(char *data);
+
 #endif

lib/aes-common.h

@@ -7,9 +7,8 @@
 #include <stdint.h>
 
 
-//not used
+void AES_ECB_encrypt_buffer(const uint8_t* input, const uint8_t* key, uint8_t *output);
-//void AES_ECB_encrypt(const uint8_t* input, const uint8_t* key, uint8_t *output, const uint32_t length);
+void AES_ECB_decrypt_buffer(const uint8_t* input, const uint8_t* key, uint8_t *output);
-//void AES_ECB_decrypt(const uint8_t* input, const uint8_t* key, uint8_t *output, const uint32_t length);
 
 void AES_CBC_encrypt_buffer(uint8_t* output, uint8_t* input, uint32_t length, const uint8_t* key, const uint8_t* iv);
 void AES_CBC_decrypt_buffer(uint8_t* output, uint8_t* input, uint32_t length, const uint8_t* key, const uint8_t* iv);

lib/aes_acc/aesacc.c

@@ -366,32 +366,25 @@ void AES_CBC_decrypt_buffer(uint8_t* output, uint8_t* input, uint32_t length, co
   decrypt_cbc(rk, length, iv_tmp, input, output);
 }
 
-/*
+void AES_ECB_encrypt_buffer(const uint8_t* input, const uint8_t* key, uint8_t* output)
-void AES_ECB_encrypt(const uint8_t* input, const uint8_t* key, uint8_t* output, const uint32_t length)
 {
-  uint8_t rk[AES_RKSIZE];
+  static uint8_t rk[AES_RKSIZE];
 
-  if (key == NULL)
-  {
-    return;
-  }
   aeshw_init();
+  if(key!=NULL)
     setkey_enc(rk, key);
   encrypt_ecb(AES_NR, rk, input, output);
 }
 
-void AES_ECB_decrypt(const uint8_t* input, const uint8_t* key, uint8_t *output, const uint32_t length)
+void AES_ECB_decrypt_buffer(const uint8_t* input, const uint8_t* key, uint8_t *output)
 {
-  uint8_t rk[AES_RKSIZE];
+  static uint8_t rk[AES_RKSIZE];
 
-  if (key == NULL)
-  {
-    return;
-  }
   aeshw_init();
+  if(key!=NULL)
     setkey_dec(rk, key);
   decrypt_ecb(AES_NR, rk, input, output);
-}*/
+}
 
 static void encrypt_cfb( uint8_t* rk,
                          uint32_t length,size_t *iv_off,

lib/aes_faster_c/wrapper.cpp

@@ -12,15 +12,29 @@
 #endif
 
 
-void AES_ECB_encrypt(const uint8_t* input, const uint8_t* key, uint8_t *output, const uint32_t length)
+void AES_ECB_encrypt_buffer(const uint8_t* input, const uint8_t* key, uint8_t *output)
 {
-	printf("AES_ECB_encrypt not implemented\n");
+	static aes_context ctx;
-	exit(-1);
+	if(key!=0)
+	{
+		aes_init( &ctx);
+		aes_setkey_enc(&ctx,key,AES_KEYSIZE);
 	}
-void AES_ECB_decrypt(const uint8_t* input, const uint8_t* key, uint8_t *output, const uint32_t length)
+	int ret=aes_crypt_ecb( &ctx, AES_ENCRYPT, (const unsigned char*)input,(unsigned char*) output );
+	assert(ret==0);
+	return ;
+}
+void AES_ECB_decrypt_buffer(const uint8_t* input, const uint8_t* key, uint8_t *output)
 {
-	printf("AES_ECB_encrypt not implemented\n");
+	static aes_context ctx;
-	exit(-1);
+	if(key!=0)
+	{
+		aes_init( &ctx);
+		aes_setkey_dec(&ctx,key,AES_KEYSIZE);
+	}
+	int ret=aes_crypt_ecb( &ctx, AES_DECRYPT, (const unsigned char*)input,(unsigned char*) output );
+	assert(ret==0);
+    return ;
 }
 
 void AES_CBC_encrypt_buffer(uint8_t* output, uint8_t* input, uint32_t length, const uint8_t* key, const uint8_t* iv)

main.cpp

@@ -33,6 +33,10 @@ int main(int argc, char *argv[])
 	assert(sizeof(unsigned int)==4);
 	assert(sizeof(unsigned long long)==8);
 
+#ifdef UDP2RAW_MP
+	init_ws();
+#endif
+
 	dup2(1, 2);//redirect stderr to stdout
 #if defined(__MINGW32__)
     enable_log_color=0;
@@ -59,11 +63,17 @@ int main(int argc, char *argv[])
 	}
 	else
 	{
+#ifdef UDP2RAW_LINUX
 		signal(SIGINT, signal_handler);
 		signal(SIGHUP, signal_handler);
 		signal(SIGKILL, signal_handler);
 		signal(SIGTERM, signal_handler);
 		signal(SIGQUIT, signal_handler);
+#else
+		mylog(log_fatal,"server mode not supported in multi-platform version\n");
+		myexit(-1);	
+#endif
+	      
 	}
 #if !defined(__MINGW32__)
 	if(geteuid() != 0)
@@ -87,7 +97,10 @@ int main(int argc, char *argv[])
 	my_init_keys(key_string,program_mode==client_mode?1:0);
 
 	iptables_rule();
+	
+#ifdef UDP2RAW_LINUX
 	init_raw_socket();
+#endif
 
 	if(program_mode==client_mode)
 	{
@@ -95,7 +108,12 @@ int main(int argc, char *argv[])
 	}
 	else
 	{
+#ifdef UDP2RAW_LINUX
 		server_event_loop();
+#else
+		mylog(log_fatal,"server mode not supported in multi-platform version\n");
+		myexit(-1);
+#endif
 	}
 
 	return 0;

makefile

@@ -1,27 +1,55 @@
 cc_cross=/home/wangyu/Desktop/arm-2014.05/bin/arm-none-linux-gnueabi-g++
 cc_local=g++
-#cc_local=/opt/cross/x86_64-linux-musl/bin/x86_64-linux-musl-g++
-#cc_mips34kc=/toolchains/OpenWrt-SDK-ar71xx-for-linux-x86_64-gcc-4.8-linaro_uClibc-0.9.33.2/staging_dir/toolchain-mips_34kc_gcc-4.8-linaro_uClibc-0.9.33.2/bin/mips-openwrt-linux-g++
 cc_mips24kc_be=/toolchains/lede-sdk-17.01.2-ar71xx-generic_gcc-5.4.0_musl-1.1.16.Linux-x86_64/staging_dir/toolchain-mips_24kc_gcc-5.4.0_musl-1.1.16/bin/mips-openwrt-linux-musl-g++
 cc_mips24kc_le=/toolchains/lede-sdk-17.01.2-ramips-mt7621_gcc-5.4.0_musl-1.1.16.Linux-x86_64/staging_dir/toolchain-mipsel_24kc_gcc-5.4.0_musl-1.1.16/bin/mipsel-openwrt-linux-musl-g++
-#cc_arm= /toolchains/gcc-linaro-4.9.4-2017.01-x86_64_arm-linux-gnueabi/bin/arm-linux-gnueabi-g++ -march=armv6 -marm 
 cc_arm= /toolchains/arm-2014.05/bin/arm-none-linux-gnueabi-g++
-#cc_arm=/toolchains/lede-sdk-17.01.2-brcm2708-bcm2708_gcc-5.4.0_musl-1.1.16_eabi.Linux-x86_64/staging_dir/toolchain-arm_arm1176jzf-s+vfp_gcc-5.4.0_musl-1.1.16_eabi/bin/arm-openwrt-linux-muslgnueabi-g++
+cc_mingw_cross=i686-w64-mingw32-g++-posix
+cc_mac_cross=o64-clang++ -stdlib=libc++
 #cc_bcm2708=/home/wangyu/raspberry/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian/bin/arm-linux-gnueabihf-g++ 
+
+
 FLAGS= -std=c++11   -Wall -Wextra -Wno-unused-variable -Wno-unused-parameter -Wno-missing-field-initializers ${OPT}
 
-COMMON=main.cpp lib/md5.cpp lib/pbkdf2-sha1.cpp lib/pbkdf2-sha256.cpp encrypt.cpp log.cpp network.cpp common.cpp  connection.cpp misc.cpp fd_manager.cpp client.cpp server.cpp -lpthread my_ev.cpp -isystem libev
+COMMON=main.cpp lib/md5.cpp lib/pbkdf2-sha1.cpp lib/pbkdf2-sha256.cpp encrypt.cpp log.cpp network.cpp common.cpp  connection.cpp misc.cpp fd_manager.cpp client.cpp server.cpp -lpthread
-SOURCES= $(COMMON) lib/aes_faster_c/aes.cpp lib/aes_faster_c/wrapper.cpp
+
-SOURCES_TINY_AES= $(COMMON) lib/aes.c
+SOURCES0= $(COMMON) lib/aes_faster_c/aes.cpp lib/aes_faster_c/wrapper.cpp
-SOURCES_AES_ACC=$(COMMON) $(wildcard lib/aes_acc/aes*.c)
+SOURCES= ${SOURCES0} my_ev.cpp -isystem libev
+SOURCES_AES_ACC= $(COMMON) $(wildcard lib/aes_acc/aes*.c) my_ev.cpp -isystem libev
+PCAP="-lpcap"
+MP="-DUDP2RAW_MP"
+
 
 NAME=udp2raw
-TARGETS=amd64 arm amd64_hw_aes arm_asm_aes mips24kc_be mips24kc_be_asm_aes x86 x86_asm_aes mips24kc_le mips24kc_le_asm_aes
-TAR=${NAME}_binaries.tar.gz `echo ${TARGETS}|sed -r 's/([^ ]+)/udp2raw_\1/g'` version.txt
 
+TARGETS=amd64 arm amd64_hw_aes arm_asm_aes mips24kc_be mips24kc_be_asm_aes x86 x86_asm_aes mips24kc_le mips24kc_le_asm_aes
+
+TAR=${NAME}_binaries.tar.gz `echo ${TARGETS}|sed -r 's/([^ ]+)/${NAME}_\1/g'` version.txt
+
+TARGETS_MP= mingw_cross mingw_cross_wepoll mac_cross
+
+export STAGING_DIR=/tmp/    #just for supress warning of staging_dir not define
+
+# targets for nativei (non-cross) compile 
 all:git_version
 	rm -f ${NAME}
-	${cc_local}   -o ${NAME}          -I. ${SOURCES} ${FLAGS} -lrt -ggdb -static -O3
+	${cc_local}   -o ${NAME}          -I. ${SOURCES} ${FLAGS} -lrt -ggdb -static -O2
+
+#dynamic link
+dynamic: git_version
+	${cc_local}   -o ${NAME}_$@          -I. ${SOURCES} ${FLAGS} -lrt -O3
+
+#targes for general cross compile
+
+cross:git_version
+	${cc_cross}   -o ${NAME}_cross    -I. ${SOURCES} ${FLAGS} -lrt -O2
+
+cross2:git_version
+	${cc_cross}   -o ${NAME}_cross    -I. ${SOURCES} ${FLAGS} -lrt -static -lgcc_eh -O2
+
+cross3:git_version
+	${cc_cross}   -o ${NAME}_cross    -I. ${SOURCES} ${FLAGS} -lrt -static -O2
+
+#targets only for debug purpose
 fast: git_version
 	rm -f ${NAME}
 	${cc_local}   -o ${NAME}          -I. ${SOURCES} ${FLAGS} -lrt -ggdb
@@ -30,29 +58,20 @@ debug: git_version
 	${cc_local}   -o ${NAME}          -I. ${SOURCES} ${FLAGS} -lrt -Wformat-nonliteral -D MY_DEBUG 
 debug2: git_version
 	rm -f ${NAME}
-	${cc_local}   -o ${NAME}          -I. ${SOURCES} ${FLAGS} -lrt -Wformat-nonliteral -ggdb
+	${cc_local}   -o ${NAME}          -I. ${SOURCES} ${FLAGS} -lrt -Wformat-nonliteral -ggdb -fsanitize=address
 
-dynamic: git_version
+#targets only for 'make release'
-	${cc_local}   -o ${NAME}_$@          -I. ${SOURCES} ${FLAGS} -lrt -O3
 
 mips24kc_be: git_version
 	${cc_mips24kc_be}  -o ${NAME}_$@   -I. ${SOURCES} ${FLAGS} -lrt -lgcc_eh -static -O3
 mips24kc_be_asm_aes: git_version
 	${cc_mips24kc_be}  -o ${NAME}_$@   -I. ${SOURCES_AES_ACC} ${FLAGS} -lrt -lgcc_eh -static -O3 lib/aes_acc/asm/mips_be.S
-
 mips24kc_le: git_version
 	${cc_mips24kc_le}  -o ${NAME}_$@   -I. ${SOURCES} ${FLAGS} -lrt -lgcc_eh -static -O3
 mips24kc_le_asm_aes: git_version
 	${cc_mips24kc_le}  -o ${NAME}_$@   -I. ${SOURCES_AES_ACC} ${FLAGS} -lrt -lgcc_eh -static -O3 lib/aes_acc/asm/mips.S
-
-#bcm2708:
-#	${cc_bcm2708} -o ${NAME}_bcm2708  -I. ${SOURCES} ${FLAGS} -lrt -static -O3
 amd64:git_version
 	${cc_local}   -o ${NAME}_$@    -I. ${SOURCES} ${FLAGS} -lrt -static -O3
-
-amd64_perf:git_version
-	${cc_local}   -o ${NAME}_$@    -I. ${SOURCES} ${FLAGS} -lrt -static -O0 -fno-omit-frame-pointer -g
-
 amd64_hw_aes:git_version
 	${cc_local}   -o ${NAME}_$@   -I. ${SOURCES_AES_ACC} ${FLAGS} -lrt -static -O3 lib/aes_acc/asm/x64.S
 x86:git_version
@@ -61,31 +80,53 @@ x86_asm_aes:git_version
 	${cc_local}   -o ${NAME}_$@    -I. ${SOURCES_AES_ACC} ${FLAGS} -lrt -static -O3 -m32 lib/aes_acc/asm/x86.S
 arm:git_version
 	${cc_arm}   -o ${NAME}_$@      -I. ${SOURCES} ${FLAGS} -lrt -static -O3
-
-arm_perf:git_version
-	${cc_arm}   -o ${NAME}_$@      -I. ${SOURCES} ${FLAGS} -lrt -static -mapcs-frame -fno-omit-frame-pointer -g -O0 -lgcc_eh
-
 arm_asm_aes:git_version
 	${cc_arm}   -o ${NAME}_$@    -I. ${SOURCES_AES_ACC} ${FLAGS} -lrt -static -O3 lib/aes_acc/asm/arm.S
 
-cross:git_version
-	${cc_cross}   -o ${NAME}_cross    -I. ${SOURCES} ${FLAGS} -lrt -O3
-
-cross2:git_version
-	${cc_cross}   -o ${NAME}_cross    -I. ${SOURCES} ${FLAGS} -lrt -static -lgcc_eh -O3   
-
-cross3:git_version
-	${cc_cross}   -o ${NAME}_cross    -I. ${SOURCES} ${FLAGS} -lrt -static -O3
-
 release: ${TARGETS}
 	cp git_version.h version.txt
 	tar -zcvf ${TAR}
 
+#targets for multi-platform version (native compile)
+cygwin:git_version
+	${cc_local}   -o ${NAME}_$@          -I. ${SOURCES} pcap_wrapper.cpp ${FLAGS} -lrt -ggdb -static -O2 -D_GNU_SOURCE
+
+mingw:git_version
+	${cc_local}   -o ${NAME}_$@         -I. ${SOURCES} pcap_wrapper.cpp ${FLAGS} -ggdb -static -O2 -lws2_32
+
+mingw_wepoll:git_version
+	${cc_local}   -o ${NAME}_$@        -I. ${SOURCES0} pcap_wrapper.cpp ${FLAGS} -ggdb -static -O2 -DNO_LIBEV_EMBED -D_WIN32 -lev -lws2_32
+
+linux:git_version
+	${cc_local}   -o ${NAME}_$@          -I. ${SOURCES} ${PCAP} ${FLAGS} -lrt -ggdb -static -O2
+
+freebsd:git_version
+	${cc_local}   -o ${NAME}_$@        -I. ${SOURCES} ${PCAP} ${FLAGS} -lrt -ggdb -static -O2
+
+mac:git_version
+	${cc_local}   -o ${NAME}_$@        -I. ${SOURCES} ${PCAP} ${FLAGS} -ggdb -O2
+
+#targets for multi-platform version (cross compile)
+
+mingw_cross:git_version
+	${cc_mingw_cross}   -o ${NAME}_mp.exe          -I. ${SOURCES} pcap_wrapper.cpp ${FLAGS} -ggdb -static -O2 -lws2_32 ${MP}
+
+mingw_cross_wepoll:git_version
+	${cc_mingw_cross}   -o ${NAME}_mp_wepoll.exe   -I. ${SOURCES0} pcap_wrapper.cpp ${FLAGS} -ggdb -static -O2 -DNO_LIBEV_EMBED -D_WIN32 -lev -lws2_32 ${MP}
+
+mac_cross:git_version
+	${cc_mac_cross}   -o ${NAME}_mp_mac            -I. ${SOURCES} ${PCAP} ${FLAGS} -ggdb -O2 ${MP}
+
+release_mp:${TARGETS_MP}
+	cp git_version.h version.txt
+	tar -zcvf ${NAME}_mp_binaries.tar.gz ${NAME}_mp.exe ${NAME}_mp_wepoll.exe ${NAME}_mp_mac version.txt
+
+
 clean:	
 	rm -f ${TAR}
-	rm -f udp2raw udp2raw_cross udp2raw_cmake udp2raw_dynamic
+	rm -f ${NAME} ${NAME}_cross ${NAME}.exe ${NAME}_wepoll.exe ${NAME}_mac
+	rm -f ${NAME}_mp_binaries.tar.gz ${NAME}_mp.exe ${NAME}_mp_wepoll.exe ${NAME}_mp_mac
 	rm -f git_version.h
 
 git_version:
 	    echo "const char *gitversion = \"$(shell git rev-parse HEAD)\";" > git_version.h
-	

misc.cpp

@@ -47,8 +47,10 @@ my_id_t const_id=0;//an id used for connection recovery,its generated randomly,i
 
 int udp_fd=-1;  //for client only. client use this fd to listen and handle udp connection
 int bind_fd=-1; //bind only,never send or recv.  its just a dummy fd for bind,so that other program wont occupy the same port
+#ifdef UDP2RAW_LINUX
 int epollfd=-1; //fd for epoll
 int timer_fd=-1;   //the general timer fd for client and server.for server this is not the only timer find,every connection has a timer fd.
+#endif
 int fail_time_counter=0;//determine if the max_fail_time is reached
 int epoll_trigger_counter=0;//for debug only
 int debug_flag=0;//for debug only
@@ -70,12 +72,14 @@ char fifo_file[1000]="";
 
 int clear_iptables=0;
 int wait_xtables_lock=0;
+#ifdef UDP2RAW_LINUX
 string iptables_command0="iptables/ip6tables ";
 string iptables_command="";
 string iptables_pattern="";
 int iptables_rule_added=0;
 int iptables_rule_keeped=0;
 int iptables_rule_keep_index=0;
+#endif
 
 program_mode_t program_mode=unset_mode;//0 unset; 1client 2server
 raw_mode_t raw_mode=mode_faketcp;
@@ -94,6 +98,7 @@ int socket_buf_size=1024*1024;
 
 
 //char lower_level_arg[1000];
+#ifdef UDP2RAW_LINUX
 int process_lower_level_arg()//handle --lower-level option
 {
 	lower_level=1;
@@ -122,6 +127,7 @@ int process_lower_level_arg()//handle --lower-level option
 	}
 	return 0;
 }
+#endif
 void print_help()
 {
 	char git_version_buf[100]={0};
@@ -131,19 +137,27 @@ void print_help()
 	printf("build date:%s %s\n",__DATE__,__TIME__);
 	printf("repository: https://github.com/wangyu-/udp2raw-tunnel\n");
 	printf("\n");
+#ifdef UDP2RAW_MP
+#ifdef NO_LIBNET
+	printf("libnet is disabled at compile time\n");
+	printf("\n");
+#endif
+#endif
 	printf("usage:\n");
 	printf("    run as client : ./this_program -c -l local_listen_ip:local_port -r server_address:server_port  [options]\n");
 	printf("    run as server : ./this_program -s -l server_listen_ip:server_port -r remote_address:remote_port  [options]\n");
 	printf("\n");
 	printf("common options,these options must be same on both side:\n");
-	printf("    --raw-mode            <string>        avaliable values:faketcp(default),udp,icmp\n");
+	printf("    --raw-mode            <string>        available values:faketcp(default),udp,icmp and easy-faketcp\n");
 	printf("    -k,--key              <string>        password to gen symetric key,default:\"secret key\"\n");
-	printf("    --cipher-mode         <string>        avaliable values:aes128cfb,aes128cbc(default),xor,none\n");
+	printf("    --cipher-mode         <string>        available values:aes128cfb,aes128cbc(default),xor,none\n");
-	printf("    --auth-mode           <string>        avaliable values:hmac_sha1,md5(default),crc32,simple,none\n");
+	printf("    --auth-mode           <string>        available values:hmac_sha1,md5(default),crc32,simple,none\n");
 	printf("    -a,--auto-rule                        auto add (and delete) iptables rule\n");
 	printf("    -g,--gen-rule                         generate iptables rule then exit,so that you can copy and\n");
 	printf("                                          add it manually.overrides -a\n");
 	printf("    --disable-anti-replay                 disable anti-replay,not suggested\n");
+	printf("    --fix-gro                             try to fix huge packet caused by GRO. this option is at an early stage.\n");
+	printf("                                          make sure client and server are at same version.\n");
 
 	//printf("\n");
 	printf("client options:\n");
@@ -164,7 +178,9 @@ void print_help()
 	printf("    --disable-bpf                         disable the kernel space filter,most time its not necessary\n");
 	printf("                                          unless you suspect there is a bug\n");
 //	printf("\n");
+#ifdef UDP2RAW_LINUX
 	printf("    --dev                 <string>        bind raw socket to a device, not necessary but improves performance\n");
+#endif
 	printf("    --sock-buf            <number>        buf size for socket,>=10 and <=10240,unit:kbyte,default:1024\n");
 	printf("    --force-sock-buf                      bypass system limitation while setting sock-buf\n");
 	printf("    --seq-mode            <number>        seq increase mode for faketcp:\n");
@@ -186,7 +202,6 @@ void print_help()
 	printf("    --clear                               clear any iptables rules added by this program.overrides everything\n");
 	printf("    --retry-on-error                      retry on error, allow to start udp2raw before network is initialized\n");
 	printf("    -h,--help                             print this help message\n");
-
 	//printf("common options,these options must be same on both side\n");
 }
 
@@ -298,6 +313,11 @@ void process_arg(int argc, char *argv[])  //process all options
 		{"dev", required_argument,    0, 1},
 		{"dns-resolve", no_argument,    0, 1},
 		{"easy-tcp", no_argument,    0, 1},
+#ifdef UDP2RAW_MP
+		{"pcap-send", no_argument,    0, 1},
+		{"no-pcap-mutex", no_argument,    0, 1},
+#endif
+        {"fix-gro", no_argument,    0, 1},
 		{NULL, 0, 0, 0}
 	  };
 
@@ -466,6 +486,11 @@ void process_arg(int argc, char *argv[])  //process all options
 		case 'h':
 			break;
 		case 'a':
+if(is_udp2raw_mp)
+{
+			mylog(log_fatal,"-a not supported in this version, check -g or --raw-mode easyfaketcp\n");
+			myexit(-1);
+}
 			auto_add_iptables_rule=1;
 			break;
 		case 'g':
@@ -479,6 +504,12 @@ void process_arg(int argc, char *argv[])  //process all options
 			mylog(log_debug,"option_index: %d\n",option_index);
 			if(strcmp(long_options[option_index].name,"clear")==0)
 			{
+if(is_udp2raw_mp)
+{
+				mylog(log_fatal,"--clear not supported in this version\n");
+				myexit(-1);
+}
+
 				clear_iptables=1;
 			}
 			else if(strcmp(long_options[option_index].name,"source-ip")==0)
@@ -561,9 +592,16 @@ void process_arg(int argc, char *argv[])  //process all options
 			}
 			else if(strcmp(long_options[option_index].name,"cipher-mode")==0)
 			{
+				string s=optarg;
+				if(s=="aes128cfb_0")
+				{
+					s="aes128cfb";
+					aes128cfb_old=1;
+					mylog(log_warn,"aes128cfb_0 is used\n");
+				}
 				for(i=0;i<cipher_end;i++)
 				{
-					if(strcmp(optarg,cipher_mode_tostring[i])==0)
+					if(strcmp(s.c_str(),cipher_mode_tostring[i])==0)
 					{
 						cipher_mode=(cipher_mode_t)i;
 						break;
@@ -581,20 +619,44 @@ void process_arg(int argc, char *argv[])  //process all options
 			}
 			else if(strcmp(long_options[option_index].name,"lower-level")==0)
 			{
+if(is_udp2raw_mp)
+{
+				mylog(log_fatal,"--lower-level not supported in this version\n");
+				myexit(-1);
+}
+
+#ifdef UDP2RAW_LINUX
 				process_lower_level_arg();
+#endif
+				//process_lower_level_arg();
 				//lower_level=1;
 				//strcpy(lower_level_arg,optarg);
 			}
 			else if(strcmp(long_options[option_index].name,"simple-rule")==0)
 			{
+if(is_udp2raw_mp)
+{
+				mylog(log_fatal,"--simple-rule not supported in this version\n");
+				myexit(-1);
+}
 				simple_rule=1;
 			}
 			else if(strcmp(long_options[option_index].name,"keep-rule")==0)
 			{
+if(is_udp2raw_mp)
+{
+				mylog(log_fatal,"--keep-rule not supported in this version\n");
+				myexit(-1);
+}
 				keep_rule=1;
 			}
 			else if(strcmp(long_options[option_index].name,"gen-add")==0)
 			{
+if(is_udp2raw_mp)
+{
+				mylog(log_fatal,"--gen-add not supported in this version\n");
+				myexit(-1);
+}
 				generate_iptables_rule_add=1;
 			}
 			else if(strcmp(long_options[option_index].name,"disable-color")==0)
@@ -627,6 +689,11 @@ void process_arg(int argc, char *argv[])  //process all options
 			}
 			else if(strcmp(long_options[option_index].name,"force-sock-buf")==0)
 			{
+if(is_udp2raw_mp)
+{
+				mylog(log_fatal,"--force-sock-buf not supported in this version\n");
+				myexit(-1);
+}
 				force_socket_buf=1;
 			}
 			else if(strcmp(long_options[option_index].name,"retry-on-error")==0)
@@ -683,6 +750,11 @@ void process_arg(int argc, char *argv[])  //process all options
 			}
 			else if(strcmp(long_options[option_index].name,"fifo")==0)
 			{
+if(is_udp2raw_mp)
+{
+				mylog(log_fatal,"--fifo not supported in this version\n");
+				myexit(-1);
+}
 				sscanf(optarg,"%s",fifo_file);
 
 				mylog(log_info,"fifo_file =%s \n",fifo_file);
@@ -733,11 +805,28 @@ void process_arg(int argc, char *argv[])  //process all options
 				enable_dns_resolve=1;
 				mylog(log_info,"dns-resolve enabled\n");
 			}
+#ifdef UDP2RAW_MP
+			else if(strcmp(long_options[option_index].name,"pcap-send")==0)
+			{
+				send_with_pcap=1;
+				mylog(log_info,"--pcap-send enabled, now pcap will be used for sending packet instead of libnet\n");
+			}
+			else if(strcmp(long_options[option_index].name,"no-pcap-mutex")==0)
+			{
+				use_pcap_mutex=0;
+				mylog(log_warn,"--no-pcap-mutex enabled, we will assume the underlying pcap calls are threadsafe\n");
+			}
+#endif
 			else if(strcmp(long_options[option_index].name,"easy-tcp")==0)
 			{
 				use_tcp_dummy_socket=1;
 				mylog(log_info,"--easy-tcp enabled, now a dummy tcp socket will be created for handshake and block rst\n");
 			}
+            else if(strcmp(long_options[option_index].name,"fix-gro")==0)
+            {
+                mylog(log_info,"--fix-gro enabled\n");
+                g_fix_gro=1;
+            }
 			else
 			{
 				mylog(log_warn,"ignored unknown long option ,option_index:%d code:<%x>\n",option_index, optopt);
@@ -890,6 +979,7 @@ void pre_process_arg(int argc, char *argv[])//mainly for load conf file
 	process_arg(new_argc,new_argv_char);
 
 }
+#ifdef UDP2RAW_LINUX
 void *run_keep(void *none)  //called in a new thread for --keep-rule option
 {
 
@@ -1078,6 +1168,7 @@ void iptables_rule()  // handles -a -g --gen-add  --keep-rule --clear --wait-loc
 		mylog(log_warn," -a has not been set, make sure you have added the needed iptables rules manually\n");
 	}
 }
+#endif
 
 int unit_test()
 {
@@ -1137,6 +1228,7 @@ int unit_test()
 	return 0;
 }
 
+#ifdef UDP2RAW_LINUX
 int set_timer(int epollfd,int &timer_fd)//put a timer_fd into epoll,general function,used both in client and server
 {
 	int ret;
@@ -1365,6 +1457,117 @@ int clear_iptables_rule()
 	}
 	return 0;
 }
+#endif
+
+#ifdef UDP2RAW_MP
+void iptables_rule()  // handles -a -g --gen-add  --keep-rule --clear --wait-lock
+{
+
+	if(generate_iptables_rule)
+	{
+		if(raw_mode==mode_faketcp && use_tcp_dummy_socket==1)
+		{
+			mylog(log_fatal, "failed,-g doesnt work with easy-faketcp mode\n");
+			myexit(-1);
+		}
+		if(raw_mode==mode_udp)
+		{
+			mylog(log_warn, "It not necessary to use iptables/firewall rule in udp mode\n");
+		}
+		log_bare(log_warn,"for linux, use:\n");
+		if(raw_ip_version==AF_INET)
+		{
+			if(raw_mode==mode_faketcp)
+				printf("iptables -I INPUT -s %s -p tcp -m tcp --sport %d -j DROP\n",remote_addr.get_ip(),remote_addr.get_port());
+			if(raw_mode==mode_udp)
+				printf("iptables -I INPUT -s %s -p udp -m udp --sport %d -j DROP\n",remote_addr.get_ip(),remote_addr.get_port());
+			if(raw_mode==mode_icmp)
+				printf("iptables -I INPUT -s %s -p icmp --icmp-type 0 -j DROP\n",remote_addr.get_ip());
+			printf("\n");
+		}
+		else
+		{
+			assert(raw_ip_version==AF_INET6);
+			if(raw_mode==mode_faketcp)
+				printf("ip6tables -I INPUT -s %s -p tcp -m tcp --sport %d -j DROP\n",remote_addr.get_ip(),remote_addr.get_port());
+			if(raw_mode==mode_udp)
+				printf("ip6tables -I INPUT -s %s -p udp -m udp --sport %d -j DROP\n",remote_addr.get_ip(),remote_addr.get_port());
+			if(raw_mode==mode_icmp)
+				printf("ip6tables -I INPUT -s %s -p -p icmpv6 --icmpv6-type 129 -j DROP\n",remote_addr.get_ip());
+			printf("\n");
+		}
+
+		log_bare(log_warn,"for mac/bsd use:\n");
+		if(raw_ip_version==AF_INET)
+		{
+			if(raw_mode==mode_faketcp)
+				printf("echo 'block drop inet proto tcp from %s port %d to any' > ./1.conf\n",remote_addr.get_ip(),remote_addr.get_port());
+			if(raw_mode==mode_udp)
+				printf("echo 'block drop inet proto udp from %s port %d to any' > ./1.conf\n",remote_addr.get_ip(),remote_addr.get_port());
+			if(raw_mode==mode_icmp)
+				printf("echo 'block drop inet proto icmp from %s to any' > ./1.conf\n",remote_addr.get_ip());
+		}
+		else
+		{
+			assert(raw_ip_version==AF_INET6);
+			if(raw_mode==mode_faketcp)
+				printf("echo 'block drop inet6 proto tcp from %s port %d to any' > ./1.conf\n",remote_addr.get_ip(),remote_addr.get_port());
+			if(raw_mode==mode_udp)
+				printf("echo 'block drop inet6 proto udp from %s port %d to any' > ./1.conf\n",remote_addr.get_ip(),remote_addr.get_port());
+			if(raw_mode==mode_icmp)
+				printf("echo 'block drop inet6 proto icmp6 from %s to any' > ./1.conf\n",remote_addr.get_ip());
+		}
+		printf("pfctl -f ./1.conf\n");
+		printf("pfctl -e\n");
+		printf("\n");
+
+		log_bare(log_warn,"for windows vista and above use:\n");
+		if(raw_ip_version==AF_INET)
+		{
+			if(raw_mode==mode_faketcp)
+			{
+				printf("netsh advfirewall firewall add rule name=udp2raw protocol=TCP dir=in remoteip=%s remoteport=%d action=block\n",remote_addr.get_ip(),remote_addr.get_port());
+				printf("netsh advfirewall firewall add rule name=udp2raw protocol=TCP dir=out remoteip=%s remoteport=%d action=block\n",remote_addr.get_ip(),remote_addr.get_port());
+			}
+			if(raw_mode==mode_udp)
+			{
+				printf("netsh advfirewall firewall add rule name=udp2raw protocol=UDP dir=in remoteip=%s remoteport=%d action=block\n",remote_addr.get_ip(),remote_addr.get_port());
+				printf("netsh advfirewall firewall add rule name=udp2raw protocol=UDP dir=out remoteip=%s remoteport=%d action=block\n",remote_addr.get_ip(),remote_addr.get_port());
+			}
+
+			if(raw_mode==mode_icmp)
+			{
+				printf("netsh advfirewall firewall add rule name=udp2raw protocol=ICMPV4 dir=in remoteip=%s action=block\n",remote_addr.get_ip());
+				printf("netsh advfirewall firewall add rule name=udp2raw protocol=ICMPV4 dir=out remoteip=%s action=block\n",remote_addr.get_ip());
+			}
+		}
+		else
+		{
+			assert(raw_ip_version==AF_INET6);
+			if(raw_mode==mode_faketcp)
+			{
+				printf("netsh advfirewall firewall add rule name=udp2raw protocol=TCP dir=in remoteip=%s remoteport=%d action=block\n",remote_addr.get_ip(),remote_addr.get_port());
+				printf("netsh advfirewall firewall add rule name=udp2raw protocol=TCP dir=out remoteip=%s remoteport=%d action=block\n",remote_addr.get_ip(),remote_addr.get_port());
+			}
+			if(raw_mode==mode_udp)
+			{
+				printf("netsh advfirewall firewall add rule name=udp2raw protocol=UDP dir=in remoteip=%s remoteport=%d action=block\n",remote_addr.get_ip(),remote_addr.get_port());
+				printf("netsh advfirewall firewall add rule name=udp2raw protocol=UDP dir=out remoteip=%s remoteport=%d action=block\n",remote_addr.get_ip(),remote_addr.get_port());
+			}
+
+			if(raw_mode==mode_icmp)
+			{
+				printf("netsh advfirewall firewall add rule name=udp2raw protocol=ICMPV6 dir=in remoteip=%s action=block\n",remote_addr.get_ip());
+				printf("netsh advfirewall firewall add rule name=udp2raw protocol=ICMPV6 dir=out remoteip=%s action=block\n",remote_addr.get_ip());
+			}
+		}
+
+		myexit(0);
+
+	}
+
+}
+#endif
 
 void  signal_handler(int sig)
 {

network.cpp

@@ -9,6 +9,8 @@
 #include "log.h"
 #include "misc.h"
 
+int g_fix_gro=0;
+
 int raw_recv_fd=-1;
 int raw_send_fd=-1;
 u32_t link_level_header_len=0;//set it to 14 if SOCK_RAW is used in socket(PF_PACKET, SOCK_RAW, htons(ETH_P_IP));
@@ -32,19 +34,22 @@ char if_name[100]="";
 char dev[100]="";
 
 unsigned short g_ip_id_counter=0;
-
+#ifdef UDP2RAW_LINUX
 unsigned char dest_hw_addr[sizeof(sockaddr_ll::sll_addr)]=
     {0xff,0xff,0xff,0xff,0xff,0xff,0,0};
+#endif
+
 //{0x00,0x23,0x45,0x67,0x89,0xb9};
 
 const u32_t receive_window_lower_bound=40960;
 const u32_t receive_window_random_range=512;
 const unsigned char wscale=0x05;
 
-char g_packet_buf[buf_len]; //looks dirty but works well
+char g_packet_buf[huge_buf_len]; //looks dirty but works well
 int g_packet_buf_len=-1;
 int g_packet_buf_cnt=0;
 
+#ifdef UDP2RAW_LINUX
 union
 {
 	sockaddr_ll ll;
@@ -52,6 +57,43 @@ union
 	sockaddr_in6 ipv6;
 }g_sockaddr;
 socklen_t g_sockaddr_len = -1;
+#endif
+
+#ifdef UDP2RAW_MP
+
+#ifndef NO_LIBNET
+libnet_t *libnet_handle;
+libnet_ptag_t g_ptag=0;
+int send_with_pcap=0;
+#else
+int send_with_pcap=1;
+#endif
+
+int pcap_header_captured=0;
+int pcap_header_buf[buf_len];
+int pcap_captured_full_len=-1;
+
+pcap_t *pcap_handle;
+int pcap_link_header_len=-1;
+//int pcap_cnt=0;
+queue_t my_queue;
+
+pthread_mutex_t queue_mutex = PTHREAD_MUTEX_INITIALIZER;
+pthread_mutex_t pcap_mutex = PTHREAD_MUTEX_INITIALIZER;
+int use_pcap_mutex=1;
+
+ev_async async_watcher;
+
+struct ev_loop* g_default_loop;
+
+pthread_t pcap_recv_thread;
+
+struct bpf_program g_filter;
+long long g_filter_compile_cnt=0;
+
+#endif
+
+#ifdef UDP2RAW_LINUX
 
 struct sock_filter code_tcp_old[] = {
 		{ 0x28, 0, 0, 0x0000000c },//0
@@ -216,6 +258,7 @@ tcpdump -i eth1  ip and icmp -dd
 (010) ret      #0
 
  */
+#endif
 
 packet_info_t::packet_info_t()
 {
@@ -253,8 +296,94 @@ packet_info_t::packet_info_t()
 	}
 
 }
+#ifdef UDP2RAW_MP
+void my_packet_handler(
+    u_char *args,
+    const struct pcap_pkthdr *packet_header,
+    const u_char *pkt_data
+)
+{
+	/*printf("<%d %d>\n",(int)packet_header->caplen,(int)packet_header->len );
+	for(int i=0;i<sizeof(pcap_pkthdr);i++)
+	{
+		char *p=(char *) packet_header;
+		printf("<%x>",int( p[i] ));
+	}
+	printf("\n");*/
+	//mylog(log_debug,"received a packet!\n");
+	assert(packet_header->caplen <= packet_header->len);
+	assert(packet_header->caplen <= max_data_len);
+	//if(packet_header->caplen > max_data_len) return ;
+	if(g_fix_gro==0&&packet_header->caplen<packet_header->len) return;
 
+	if((int)packet_header->caplen<pcap_link_header_len) return;
+	//mylog(log_debug,"and its vaild!\n");
 
+	pthread_mutex_lock(&queue_mutex);
+	if(!my_queue.full())
+		my_queue.push_back((char *)pkt_data,(int)(packet_header->caplen));
+	pthread_mutex_unlock(&queue_mutex);
+
+	//pcap_cnt++;
+
+	ev_async_send (g_default_loop,&async_watcher);
+    return;
+}
+
+void *pcap_recv_thread_entry(void *none)
+{
+	struct pcap_pkthdr *packet_header;
+	const u_char *pkt_data;
+
+	while(1)
+	{
+		if(use_pcap_mutex) pthread_mutex_lock(&pcap_mutex);
+		int ret=pcap_loop(pcap_handle, -1, my_packet_handler, NULL); //use -1 instead of 0 as cnt, since 0 is undefined in old versions
+		if(use_pcap_mutex) pthread_mutex_unlock(&pcap_mutex);
+		if(ret==-1)
+			mylog(log_warn,"pcap_loop exited with value %d\n",ret);
+		else
+		{
+			mylog(log_debug,"pcap_loop exited with value %d\n",ret);
+		}
+		ev_sleep(1.0);
+		//myexit(-1);
+	}
+	/*
+	while(1)
+	{
+		//printf("!!!\n");
+		pthread_mutex_lock(&pcap_mutex);
+		int ret=pcap_next_ex(pcap_handle,&packet_header,&pkt_data);
+		pthread_mutex_unlock(&pcap_mutex);
+
+		switch (ret)
+		{
+			case 0:
+				continue;
+			case 1:
+
+				break;
+
+			case -1:
+				mylog(log_fatal,"pcap_next_ex error [%s]\n",pcap_geterr(pcap_handle));
+				myexit(-1);
+				break;
+			case -2:
+				assert(0==1);//
+				break;
+			default:
+				assert(0==1);//
+		}
+	}
+	myexit(-1);*/
+	return 0;
+}
+
+extern void async_cb(struct ev_loop *loop, struct ev_async *watcher, int revents);
+#endif
+
+#ifdef UDP2RAW_LINUX
 int init_raw_socket()
 {
 	assert(raw_ip_version==AF_INET||raw_ip_version==AF_INET6);
@@ -379,6 +508,226 @@ int init_raw_socket()
 
 	return 0;
 }
+#endif
+#ifdef UDP2RAW_MP
+int init_raw_socket()
+{
+
+#ifndef NO_LIBNET
+	char libnet_errbuf[LIBNET_ERRBUF_SIZE];
+
+	if(raw_ip_version==AF_INET)
+	{
+		libnet_handle = libnet_init(LIBNET_RAW4, dev, libnet_errbuf);
+	}
+	else
+	{
+		assert(raw_ip_version==AF_INET6);
+		libnet_handle = libnet_init(LIBNET_RAW6, dev, libnet_errbuf);
+	}
+
+	if(libnet_handle==0)
+	{
+		mylog(log_fatal,"libnet_init failed bc of [%s]\n",libnet_errbuf);
+		myexit(-1);
+	}
+	g_ptag=0;
+    libnet_clear_packet(libnet_handle);
+#endif
+
+	char pcap_errbuf[PCAP_ERRBUF_SIZE];
+
+	//pcap_handle=pcap_open_live(dev,max_data_len,0,1000,pcap_errbuf);
+
+	pcap_handle = pcap_create( dev, pcap_errbuf );
+
+
+	if(pcap_handle==0)
+	{
+		mylog(log_fatal,"pcap_create failed bc of [%s]\n",pcap_errbuf);
+		myexit(-1);
+	}
+
+	assert( pcap_set_snaplen(pcap_handle, huge_data_len) ==0);
+	assert( pcap_set_promisc(pcap_handle, 0) ==0);
+	assert( pcap_set_timeout(pcap_handle, 1) ==0);
+	assert( pcap_set_immediate_mode(pcap_handle,1) ==0);
+
+	int ret = pcap_activate( pcap_handle );
+	if( ret < 0 )
+	{
+		 printf("pcap_activate failed  %s\n", pcap_geterr(pcap_handle));
+		 myexit(-1);
+	}
+
+	if(send_with_pcap)
+	{
+		ret=pcap_setdirection(pcap_handle,PCAP_D_INOUT);//must be used after being actived
+		if(ret!=0) mylog(log_debug,"pcap_setdirection(pcap_handle,PCAP_D_INOUT) failed with value %d, %s\n",ret,pcap_geterr(pcap_handle));
+	}
+	else
+	{
+		ret=pcap_setdirection(pcap_handle,PCAP_D_IN);
+		if(ret!=0) mylog(log_debug,"pcap_setdirection(pcap_handle,PCAP_D_IN) failed with value %d, %s\n",ret,pcap_geterr(pcap_handle));
+	}
+
+
+	ret=pcap_datalink(pcap_handle);
+
+	if(ret==DLT_EN10MB)
+	{
+		pcap_link_header_len=14;
+	}
+	else if(ret==DLT_NULL)
+	{
+		pcap_link_header_len=4;
+	}
+	else if(ret==DLT_LINUX_SLL)
+	{
+		pcap_link_header_len=16;
+	}
+	else
+	{
+		mylog(log_fatal,"unknown pcap link type : %d\n",ret);
+		myexit(-1);
+	}
+
+	char filter_exp[1000];
+
+	address_t tmp_addr;
+	if(get_src_adress2(tmp_addr,remote_addr)!=0)
+	{
+		mylog(log_error,"get_src_adress() failed, maybe you dont have internet\n");
+		myexit(-1);
+	}
+
+	string src=tmp_addr.get_ip();
+	string dst=remote_addr.get_ip();
+	if(raw_ip_version==AF_INET)
+	{
+		//sprintf(filter_exp,"ip and src %s and dst %s and (tcp or udp or icmp)",my_ntoa(source_ip_uint32),dst.c_str());
+		sprintf(filter_exp,"ip and src %s and dst %s and (tcp or udp or icmp)",src.c_str(),dst.c_str());
+	}
+	else
+	{
+		assert(raw_ip_version==AF_INET6);
+		sprintf(filter_exp,"ip6 and src %s and dst %s and (tcp or udp or icmp6)",src.c_str(),dst.c_str());
+
+	}
+
+	 if (pcap_compile(pcap_handle, &g_filter, filter_exp, 0, PCAP_NETMASK_UNKNOWN ) == -1) {
+		 printf("Bad filter - %s\n", pcap_geterr(pcap_handle));
+		 myexit(-1);
+	 }
+	 g_filter_compile_cnt++;
+
+
+	 if (pcap_setfilter(pcap_handle, &g_filter) == -1) {
+		 printf("Error setting filter - %s\n", pcap_geterr(pcap_handle));
+		 myexit(-1);
+	 }
+
+///////////////////////////////////////////////////////////////new thread created here
+		if(pthread_create(&pcap_recv_thread, NULL, pcap_recv_thread_entry, 0)) {
+			mylog(log_fatal, "Error creating thread\n");
+			myexit(-1);
+		}
+////////////////////////////////////////////////////////////////////////////////
+
+
+	g_ip_id_counter=get_true_random_number()%65535;
+
+	/*
+	if(lower_level==0)
+	{
+		raw_send_fd = socket(AF_INET , SOCK_RAW , IPPROTO_TCP);
+
+	    if(raw_send_fd == -1) {
+	    	mylog(log_fatal,"Failed to create raw_send_fd\n");
+	        //perror("Failed to create raw_send_fd");
+	        myexit(1);
+	    }
+
+	    int one = 1;
+	    const int *val = &one;
+	    if (setsockopt (raw_send_fd, IPPROTO_IP, IP_HDRINCL, val, sizeof (one)) < 0) {
+	    	mylog(log_fatal,"Error setting IP_HDRINCL %d\n",errno);
+	        //perror("Error setting IP_HDRINCL");
+	        myexit(2);
+	    }
+
+
+	}
+	else
+	{
+		raw_send_fd = socket(PF_PACKET , SOCK_DGRAM , htons(ETH_P_IP));
+
+	    if(raw_send_fd == -1) {
+	    	mylog(log_fatal,"Failed to create raw_send_fd\n");
+	        //perror("Failed to create raw_send_fd");
+	        myexit(1);
+	    }
+		//init_ifindex(if_name);
+
+	}
+
+	if(force_socket_buf)
+	{
+		if(setsockopt(raw_send_fd, SOL_SOCKET, SO_SNDBUFFORCE, &socket_buf_size, sizeof(socket_buf_size))<0)
+		{
+			mylog(log_fatal,"SO_SNDBUFFORCE fail  socket_buf_size=%d  errno=%s\n",socket_buf_size,strerror(errno));
+			myexit(1);
+		}
+	}
+	else
+	{
+		if(setsockopt(raw_send_fd, SOL_SOCKET, SO_SNDBUF, &socket_buf_size, sizeof(socket_buf_size))<0)
+		{
+			mylog(log_fatal,"SO_SNDBUF fail  socket_buf_size=%d  errno=%s\n",socket_buf_size,strerror(errno));
+			myexit(1);
+		}
+	}
+
+
+
+	//raw_fd = socket(AF_PACKET, SOCK_DGRAM, htons(ETH_P_ALL));
+
+	raw_recv_fd= socket(PF_PACKET, SOCK_DGRAM, htons(ETH_P_IP));
+
+    if(raw_recv_fd == -1) {
+    	mylog(log_fatal,"Failed to create raw_recv_fd\n");
+        //perror("");
+        myexit(1);
+    }
+
+	if(force_socket_buf)
+	{
+		if(setsockopt(raw_recv_fd, SOL_SOCKET, SO_RCVBUFFORCE, &socket_buf_size, sizeof(socket_buf_size))<0)
+		{
+			mylog(log_fatal,"SO_RCVBUFFORCE fail  socket_buf_size=%d  errno=%s\n",socket_buf_size,strerror(errno));
+			myexit(1);
+		}
+	}
+	else
+	{
+		if(setsockopt(raw_recv_fd, SOL_SOCKET, SO_RCVBUF, &socket_buf_size, sizeof(socket_buf_size))<0)
+		{
+			mylog(log_fatal,"SO_RCVBUF fail  socket_buf_size=%d  errno=%s\n",socket_buf_size,strerror(errno));
+			myexit(1);
+		}
+	}
+
+    //IP_HDRINCL to tell the kernel that headers are included in the packet
+
+
+
+    setnonblocking(raw_send_fd); //not really necessary
+    setnonblocking(raw_recv_fd);*/
+
+	return 0;
+}
+#endif
+#ifdef UDP2RAW_LINUX
 void init_filter(int port)
 {
 	sock_fprog bpf;
@@ -448,9 +797,160 @@ void init_filter(int port)
 		myexit(-1);
 	}
 }
+#endif
+
+#ifdef UDP2RAW_MP
+void init_filter(int port)
+{
+	/*
+	sock_fprog bpf;*/
+	if(raw_mode==mode_faketcp||raw_mode==mode_udp)
+	{
+		filter_port=port;
+	}
+
+
+
+	 char filter_exp[1000];
+
+	if(raw_ip_version==AF_INET)
+	{
+		if(raw_mode==mode_faketcp)
+		{
+			sprintf(filter_exp,"ip and tcp and src %s and src port %d and dst port %d",remote_addr.get_ip(),remote_addr.get_port(),port);
+		}
+		else if(raw_mode==mode_udp)
+		{
+			sprintf(filter_exp,"ip and udp and src %s and src port %d and dst port %d",remote_addr.get_ip(),remote_addr.get_port(),port);
+		}
+		else if(raw_mode==mode_icmp)
+		{
+			sprintf(filter_exp,"ip and icmp and src %s",remote_addr.get_ip());
+		}
+		else
+		{
+			mylog(log_fatal,"unknow raw mode\n");
+			myexit(-1);
+		}
+	}
+	else
+	{
+		assert(raw_ip_version==AF_INET6);
+		if(raw_mode==mode_faketcp)
+		{
+			sprintf(filter_exp,"ip6 and tcp and src %s and src port %d and dst port %d",remote_addr.get_ip(),remote_addr.get_port(),port);
+		}
+		else if(raw_mode==mode_udp)
+		{
+			sprintf(filter_exp,"ip6 and udp and src %s and src port %d and dst port %d",remote_addr.get_ip(),remote_addr.get_port(),port);
+		}
+		else if(raw_mode==mode_icmp)
+		{
+			sprintf(filter_exp,"ip6 and icmp6 and src %s",remote_addr.get_ip());
+		}
+		else
+		{
+			mylog(log_fatal,"unknow raw mode\n");
+			myexit(-1);
+		}
+	}
+
+	mylog(log_info,"filter expression is [%s]\n",filter_exp);
+
+	//pthread_mutex_lock(&pcap_mutex);//not sure if mutex is needed here
+
+	long long tmp_cnt=0;
+	if(use_pcap_mutex)
+	{
+		while(pthread_mutex_trylock(&pcap_mutex)!=0)
+		{
+			tmp_cnt++;
+			pcap_breakloop(pcap_handle);
+			if(tmp_cnt==100)
+			{
+				mylog(log_warn,"%lld attempts of pcap_breakloop()\n", tmp_cnt);
+			}
+			if(tmp_cnt%1000==0)
+			{
+				mylog(log_warn,"%lld attempts of pcap_breakloop()\n", tmp_cnt);
+				if(tmp_cnt>5000)
+				{
+					 mylog(log_fatal,"we might have already run into a deadlock\n");
+				}
+			}
+			ev_sleep(0.001);
+		}
+		mylog(log_info,"breakloop() succeed after %lld attempt(s)\n", tmp_cnt);
+	}
+
+	if(1)
+	{
+		int ret=pcap_setdirection(pcap_handle,PCAP_D_IN);
+		if(ret!=0) mylog(log_debug,"pcap_setdirection(pcap_handle,PCAP_D_IN) failed with value %d, %s\n",ret,pcap_geterr(pcap_handle));
+	}
+
+	assert(g_filter_compile_cnt!=0);
+	pcap_freecode(&g_filter);
+
+	 if (pcap_compile(pcap_handle, &g_filter, filter_exp, 0, PCAP_NETMASK_UNKNOWN ) == -1) {
+		 mylog(log_fatal,"Bad filter - %s\n", pcap_geterr(pcap_handle));
+		 myexit(-1);
+	 }
+	 g_filter_compile_cnt++;
+
+	 if (pcap_setfilter(pcap_handle, &g_filter) == -1)
+	 {
+		 mylog(log_fatal,"Error setting filter - %s\n", pcap_geterr(pcap_handle));
+		 myexit(-1);
+	 }
+
+
+	if(use_pcap_mutex) pthread_mutex_unlock(&pcap_mutex);
+	/*
+	if(disable_bpf_filter) return;
+	//if(raw_mode==mode_icmp) return ;
+	//code_tcp[8].k=code_tcp[10].k=port;
+	if(raw_mode==mode_faketcp)
+	{
+		bpf.len = sizeof(code_tcp)/sizeof(code_tcp[0]);
+		code_tcp[code_tcp_port_index].k=port;
+		bpf.filter = code_tcp;
+	}
+	else if(raw_mode==mode_udp)
+	{
+		bpf.len = sizeof(code_udp)/sizeof(code_udp[0]);
+		code_udp[code_udp_port_index].k=port;
+		bpf.filter = code_udp;
+	}
+	else if(raw_mode==mode_icmp)
+	{
+		bpf.len = sizeof(code_icmp)/sizeof(code_icmp[0]);
+		bpf.filter = code_icmp;
+	}
+
+	int dummy;
+
+	int ret=setsockopt(raw_recv_fd, SOL_SOCKET, SO_DETACH_FILTER, &dummy, sizeof(dummy)); //in case i forgot to remove
+	if (ret != 0)
+	{
+		mylog(log_debug,"error remove fiter\n");
+		//perror("filter");
+		//exit(-1);
+	}
+	ret = setsockopt(raw_recv_fd, SOL_SOCKET, SO_ATTACH_FILTER, &bpf, sizeof(bpf));
+	if (ret != 0)
+	{
+		mylog(log_fatal,"error set fiter\n");
+		//perror("filter");
+		myexit(-1);
+	}*/
+}
+#endif
+
 void remove_filter()
 {
 	filter_port=0;
+#ifdef UDP2RAW_LINUX
 	int dummy;
 	int ret=setsockopt(raw_recv_fd, SOL_SOCKET, SO_DETACH_FILTER, &dummy, sizeof(dummy));
 	if (ret != 0)
@@ -459,10 +959,12 @@ void remove_filter()
 		//perror("filter");
 		//exit(-1);
 	}
+#endif
 }
 
 int init_ifindex(const char * if_name,int fd,int &index)
 {
+#ifdef UDP2RAW_LINUX
 	struct ifreq ifr;
 	size_t if_name_len=strlen(if_name);
 	if (if_name_len<sizeof(ifr.ifr_name)) {
@@ -479,8 +981,11 @@ int init_ifindex(const char * if_name,int fd,int &index)
 	}
 	index=ifr.ifr_ifindex;
 	mylog(log_info,"ifname:%s  ifindex:%d\n",if_name,index);
+#endif
 	return 0;
 }
+
+#ifdef UDP2RAW_LINUX
 bool interface_has_arp(const char * interface) {
     struct ifreq ifr;
    // int sock = socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP);
@@ -700,7 +1205,9 @@ int find_lower_level_info(u32_t ip,u32_t &dest_ip,string &if_name,string &hw)
 	dest_ip=ntohl(dest_ip);
 	return 0;
 }
+#endif
 
+#ifdef UDP2RAW_LINUX
 int send_raw_packet(raw_info_t &raw_info,const char * packet,int len)
 {
 	const packet_info_t &send_info=raw_info.send_info;
@@ -751,6 +1258,76 @@ int send_raw_packet(raw_info_t &raw_info,const char * packet,int len)
     }
     return 0;
 }
+#endif
+
+#ifdef UDP2RAW_MP
+
+int send_raw_packet(raw_info_t &raw_info,const char * packet,int len)
+{
+	const packet_info_t &send_info=raw_info.send_info;
+	const packet_info_t &recv_info=raw_info.recv_info;
+
+    if(! send_with_pcap)
+    {
+#ifndef NO_LIBNET
+
+		//g_ptag=libnet_build_ipv4(ip_tot_len, iph->tos, ntohs(iph->id), ntohs(iph->frag_off),
+		//	iph->ttl , iph->protocol , iph->check , iph->saddr, iph->daddr,
+		//	(const unsigned char *)payload, payloadlen, libnet_handle, g_ptag);
+
+		//assert(g_ptag!=-1 &&g_ptag!=0);
+
+		//int ret;
+		//ret= libnet_write(libnet_handle);
+
+		//assert(ret!=-1);
+
+
+		//iph->tot_len=htons(ip_tot_len);
+		//iph->check =csum ((unsigned short *) send_raw_ip_buf, iph->ihl*4);
+    	if(raw_ip_version==AF_INET)
+    	{
+    		libnet_write_raw_ipv4(libnet_handle,(const unsigned char *)packet,len);  //todo, this api is marked as internal, maybe we should avoid using it.
+    	}
+    	else
+    	{
+    		assert(raw_ip_version==AF_INET6);
+    		libnet_write_raw_ipv6(libnet_handle,(const unsigned char *)packet,len);
+    	}
+#endif
+    }
+    else
+    {
+    	char buf[buf_len];
+    	assert(pcap_header_captured==1);
+    	assert(pcap_link_header_len!=-1);
+    	memcpy(buf,pcap_header_buf,pcap_link_header_len);
+    	memcpy(buf+pcap_link_header_len,packet,len);
+    	//pthread_mutex_lock(&pcap_mutex); looks like this is not necessary, and it harms performance
+    	int ret=pcap_sendpacket(pcap_handle,(const unsigned char *)buf,len+pcap_link_header_len);
+		if(ret!=0)
+		{
+			mylog(log_fatal,"pcap_sendpcaket failed with vaule %d,%s\n",ret,pcap_geterr(pcap_handle));
+			//pthread_mutex_unlock(&pcap_mutex);
+			myexit(-1);
+		}
+    	//pthread_mutex_unlock(&pcap_mutex);
+		/*
+	unsigned char *p=(unsigned char *)send_raw_ip_buf0;
+	for(int i=0;i<ip_tot_len+pcap_link_header_len;i++)
+		printf("<%02x>",int(p[i]));
+	printf("\n");
+    	assert(pcap_sendpacket(pcap_handle,(const unsigned char *)pcap_header_buf,cap_len)==0);
+	p=(unsigned char *)pcap_header_buf;
+	for(int i=0;i<cap_len;i++)
+		printf("<%02x>",int(p[i]));
+	printf("\n");
+	printf("pcap send!\n");*/
+    }
+    return 0;
+}
+#endif
+
 int send_raw_ip(raw_info_t &raw_info,const char * payload,int payloadlen)
 {
 	const packet_info_t &send_info=raw_info.send_info;
@@ -794,16 +1371,28 @@ int send_raw_ip(raw_info_t &raw_info,const char * payload,int payloadlen)
 		iph->daddr = send_info.new_dst_ip.v4;
 
 		ip_tot_len=sizeof (struct my_iphdr)+payloadlen;
+#ifdef UDP2RAW_LINUX
 		if(lower_level)iph->tot_len = htons(ip_tot_len);            //this is not necessary ,kernel will always auto fill this  //http://man7.org/linux/man-pages/man7/raw.7.html
 		else
 			iph->tot_len = 0;
+#endif
+
+#ifdef UDP2RAW_MP
+		iph->tot_len = htons(ip_tot_len);//always fill for mp version
+#endif
 
 		memcpy(send_raw_ip_buf+sizeof(my_iphdr) , payload, payloadlen);
 
+#ifdef UDP2RAW_LINUX
 		if(lower_level) iph->check =
 				csum ((unsigned short *) send_raw_ip_buf, iph->ihl*4); //this is not necessary ,kernel will always auto fill this
 		else
 			iph->check=0;
+#endif
+
+#ifdef UDP2RAW_MP
+		iph->check =csum((unsigned short *) send_raw_ip_buf, iph->ihl*4);//always cal checksum for mp version
+#endif
     }
     else
     {
@@ -829,18 +1418,44 @@ int send_raw_ip(raw_info_t &raw_info,const char * payload,int payloadlen)
 
 int pre_recv_raw_packet()
 {
+#ifdef UDP2RAW_LINUX
 	assert(g_packet_buf_cnt==0);
 
 	g_sockaddr_len=sizeof(g_sockaddr.ll);
-	g_packet_buf_len = recvfrom(raw_recv_fd, g_packet_buf, max_data_len+1, 0 ,(sockaddr*)&g_sockaddr , &g_sockaddr_len);
+	g_packet_buf_len = recvfrom(raw_recv_fd, g_packet_buf, huge_data_len+1, 0 ,(sockaddr*)&g_sockaddr , &g_sockaddr_len);
 	//assert(g_sockaddr_len==sizeof(g_sockaddr.ll)); //g_sockaddr_len=18, sizeof(g_sockaddr.ll)=20, why its not equal? maybe its bc sll_halen is 6?
 
 	//assert(g_addr_ll_size==sizeof(g_addr_ll));
 
-	if(g_packet_buf_len==max_data_len+1)
+	if(g_packet_buf_len==huge_data_len+1)
 	{
-		mylog(log_warn,"huge packet, data_len > %d,dropped\n",max_data_len);
+		if(g_fix_gro==0)
+		{
+		mylog(log_warn,"huge packet, data_len %d > %d,dropped\n",g_packet_buf_len,huge_data_len);
 		return -1;
+		}
+		else
+		{
+			mylog(log_debug,"huge packet, data_len %d > %d,not dropped\n",g_packet_buf_len,huge_data_len);  
+			g_packet_buf_len=huge_data_len;
+		}
+	}
+
+    if(g_packet_buf_len>= max_data_len+1)
+    {
+        if(g_fix_gro==0)
+        {
+            mylog(log_warn, "huge packet, data_len %d > %d(max_data_len) dropped, maybe you need to turn down mtu at upper level, or you may take a look at --fix-gro\n", g_packet_buf_len,
+                  max_data_len);
+            return -1;
+        }
+        else
+        {
+            mylog(log_debug, "huge packet, data_len %d > %d(max_data_len) not dropped\n", g_packet_buf_len,
+                  max_data_len);
+            //return -1;
+        }
+
     }
 
 	if(g_packet_buf_len<0)
@@ -849,6 +1464,7 @@ int pre_recv_raw_packet()
 		return -1;
 	}
 	g_packet_buf_cnt++;
+#endif
 	return 0;
 }
 int discard_raw_packet()
@@ -857,6 +1473,7 @@ int discard_raw_packet()
 	g_packet_buf_cnt--;
 	return 0;
 }
+#ifdef UDP2RAW_LINUX
 int recv_raw_packet(char * &packet,int &len,int peek)
 {
 	assert(g_packet_buf_cnt==1);
@@ -885,6 +1502,19 @@ int recv_raw_packet(char * &packet,int &len,int peek)
 	len=g_packet_buf_len-int(link_level_header_len);
 	return 0;
 }
+#endif
+#ifdef UDP2RAW_MP
+int recv_raw_packet(char * &packet,int &len,int peek)
+{
+	assert(g_packet_buf_cnt==1);
+	if(!peek)
+		g_packet_buf_cnt--;
+
+	packet=g_packet_buf;
+	len=g_packet_buf_len;
+	return 0;
+}
+#endif
 int recv_raw_ip(raw_info_t &raw_info,char * &payload,int &payloadlen)
 {
 	char *raw_packet_buf;
@@ -939,11 +1569,12 @@ int recv_raw_ip(raw_info_t &raw_info,char * &payload,int &payloadlen)
 			return -1;
 		}
 	}
-
+#ifdef UDP2RAW_LINUX
 	if(lower_level)
 	{
 		memcpy(&recv_info.addr_ll,&g_sockaddr.ll,sizeof(recv_info.addr_ll));
 	}
+#endif
 
 
 
@@ -1925,9 +2556,8 @@ int recv_raw_tcp(raw_info_t &raw_info,char * &payload,int &payloadlen)
 
     if(tcp_chk!=0)
     {
-    	mylog(log_debug,"tcp_chk:%x\n",tcp_chk);
+    	mylog(log_debug,"tcp_chk:%x, tcp checksum failed, ignored\n",tcp_chk);
-    	mylog(log_debug,"tcp header error\n");
+    	//return -1;
-    	return -1;
 
     }
 
@@ -2509,7 +3139,7 @@ int try_to_list_and_bind2(int &fd,address_t address)  //try to bind to a port,ma
 	temp_bind_addr.sin_port = htons(port);
 	temp_bind_addr.sin_addr.s_addr = local_ip_uint32;*/
 
-	if (bind(fd, (struct sockaddr*)&address.inner, address.get_len()) !=0)
+	if (::bind(fd, (struct sockaddr*)&address.inner, address.get_len()) !=0)
 	{
 		mylog(log_debug,"bind fail\n");
 		return -1;
@@ -2543,9 +3173,9 @@ int client_bind_to_a_new_port(int &fd,u32_t local_ip_uint32)//find a free port a
 int client_bind_to_a_new_port2(int &fd,const address_t& address)//find a free port and bind to it.
 {
 	address_t tmp=address;
-	int raw_send_port=10000+get_true_random_number()%(65535-10000);
 	for(int i=0;i<1000;i++)//try 1000 times at max,this should be enough
 	{
+		int raw_send_port=10000+get_true_random_number()%(65535-10000);
 		tmp.set_port(raw_send_port);
 		if (try_to_list_and_bind2(fd,tmp)==0)
 		{

network.h

@@ -27,10 +27,35 @@ extern int random_drop;
 
 extern int ifindex;
 
-extern char g_packet_buf[buf_len];
+extern char g_packet_buf[huge_buf_len];
 extern int g_packet_buf_len;
 extern int g_packet_buf_cnt;
+#ifdef UDP2RAW_MP
+extern queue_t my_queue;
 
+extern ev_async async_watcher;
+extern struct ev_loop* g_default_loop;
+
+extern pthread_mutex_t queue_mutex;
+extern int use_pcap_mutex;
+
+extern int pcap_cnt;
+
+extern int pcap_link_header_len;
+
+extern int send_with_pcap;
+extern int pcap_header_captured;
+extern int pcap_header_buf[buf_len];
+
+struct icmphdr
+{
+	uint8_t type;
+	uint8_t code;
+	uint16_t check_sum;
+	uint16_t id;
+	uint16_t seq;
+};
+#endif
 
 struct my_iphdr
   {
@@ -213,7 +238,9 @@ struct packet_info_t  //todo change this to union
 
 	bool has_ts;
 
+#ifdef UDP2RAW_LINUX
 	sockaddr_ll addr_ll;
+#endif
 
 	i32_t data_len;
 
@@ -242,7 +269,13 @@ void init_filter(int port);
 
 void remove_filter();
 
+#ifdef UDP2RAW_LINUX
 int init_ifindex(const char * if_name,int fd,int &index);
+#endif
+
+#ifdef UDP2RAW_MP
+int init_ifindex(const char * if_name,int &index);
+#endif
 
 int find_lower_level_info(u32_t ip,u32_t &dest_ip,string &if_name,string &hw);
 

pcap_wrapper.cpp

@@ -0,0 +1,125 @@
+#include <windows.h>
+#include <pcap_wrapper.h>
+#include <assert.h>
+#include <stdio.h>
+int	(*pcap_loop )(pcap_t *, int, pcap_handler, u_char *);
+int	(*pcap_breakloop )(pcap_t *);
+
+pcap_t* (*pcap_create)(const char *, char *);
+
+int	(*pcap_set_snaplen) (pcap_t *, int)=0;
+int	(*pcap_set_promisc) (pcap_t *, int)=0;
+int	(*pcap_can_set_rfmon) (pcap_t *)=0;
+int	(*pcap_set_rfmon )(pcap_t *, int)=0;
+int	(*pcap_set_timeout)(pcap_t *, int)=0;
+int	(*pcap_set_buffer_size)(pcap_t *, int)=0;
+int	(*pcap_activate)(pcap_t *)=0;
+
+int	(*pcap_setfilter)(pcap_t *, struct bpf_program *)=0;
+int 	(*pcap_setdirection)(pcap_t *, pcap_direction_t)=0;
+
+int	(*pcap_datalink)(pcap_t *)=0;
+
+void	(*pcap_freecode)(struct bpf_program *)=0;
+
+int	(*pcap_compile)(pcap_t *, struct bpf_program *, const char *, int,
+     bpf_u_int32)=0;
+
+char*   (*pcap_geterr)(pcap_t *)=0;
+int	(*pcap_sendpacket)(pcap_t *, const u_char *, int)=0;
+
+char* (*pcap_lookupdev)(char *)=0;
+
+int	(*pcap_findalldevs)(pcap_if_t **, char *)=0;
+
+struct init_pcap_t
+{
+	init_pcap_t()
+	{
+		init_pcap();
+	}
+	
+}do_it;
+
+static void init_npcap_dll_path()
+{
+	BOOL(WINAPI *SetDllDirectory)(LPCTSTR);
+	char sysdir_name[512];
+	int len;
+
+	SetDllDirectory = (BOOL(WINAPI *)(LPCTSTR)) GetProcAddress(GetModuleHandle("kernel32.dll"), "SetDllDirectoryA");
+	if (SetDllDirectory == NULL) {
+		printf("Error in SetDllDirectory\n");
+	}
+	else {
+		len = GetSystemDirectory(sysdir_name, 480);	//	be safe
+		if (!len)
+			printf("Error in GetSystemDirectory (%d)\n", (int)GetLastError());
+		strcat(sysdir_name, "\\Npcap");
+		if (SetDllDirectory(sysdir_name) == 0)
+			printf("Error in SetDllDirectory(\"System32\\Npcap\")\n");
+	}
+}
+
+#define EXPORT_FUN(XXX) do{ XXX= (__typeof__(XXX)) GetProcAddress(wpcap, #XXX); }while(0)
+int init_pcap()
+{
+	HMODULE wpcap=LoadLibrary("wpcap.dll");
+	if(wpcap!=0)
+	{
+		printf("using system32/wpcap.dll\n");
+	}
+	else
+	{
+		init_npcap_dll_path();
+		//SetDllDirectory("C:\\Windows\\System32\\Npcap\\");
+		wpcap=LoadLibrary("wpcap.dll");
+		if(wpcap!=0)
+			printf("using system32/npcap/wpcap.dll\n");
+	}
+	if(wpcap==0)
+	{
+		printf("cant not open wpcap.dll, make sure winpcap/npcap is installed\n");
+		exit(-1);
+	}
+	assert(wpcap!=0);
+
+	EXPORT_FUN(pcap_loop);
+	EXPORT_FUN(pcap_breakloop);
+	EXPORT_FUN(pcap_create);
+	EXPORT_FUN(pcap_set_snaplen);
+	EXPORT_FUN(pcap_set_promisc);
+	EXPORT_FUN(pcap_set_timeout);
+	EXPORT_FUN(pcap_activate);
+	EXPORT_FUN(pcap_setfilter);
+	EXPORT_FUN(pcap_setdirection);
+	EXPORT_FUN(pcap_datalink);
+	EXPORT_FUN(pcap_freecode);
+	EXPORT_FUN(pcap_compile);
+	EXPORT_FUN(pcap_geterr);
+	EXPORT_FUN(pcap_sendpacket);
+	EXPORT_FUN(pcap_lookupdev);
+	EXPORT_FUN(pcap_findalldevs);
+	/*
+	pcap_loop = (__typeof__(pcap_loop))GetProcAddress(wpcap, "pcap_loop");
+	pcap_create = (__typeof__(pcap_create))GetProcAddress(wpcap, "pcap_create");
+	pcap_set_snaplen = (__typeof__(pcap_set_snaplen))GetProcAddress(wpcap, "pcap_set_snaplen");
+	pcap_set_promisc = (__typeof__(pcap_set_promisc))GetProcAddress(wpcap, "pcap_set_promisc");
+	pcap_set_timeout = (__typeof__(pcap_set_timeout))GetProcAddress(wpcap, "pcap_set_timeout");
+	pcap_activate = (__typeof__(pcap_activate))GetProcAddress(wpcap, "pcap_activate");
+	pcap_setfilter = (__typeof__(pcap_setfilter))GetProcAddress(wpcap, "pcap_setfilter");
+	pcap_setdirection = (__typeof__(pcap_setdirection))GetProcAddress(wpcap, "pcap_setdirection");
+	pcap_datalink = (__typeof__(pcap_datalink))GetProcAddress(wpcap, "pcap_datalink");
+	pcap_freecode = (__typeof__(pcap_freecode))GetProcAddress(wpcap, "pcap_freecode");
+	pcap_compile = (__typeof__(pcap_compile))GetProcAddress(wpcap, "pcap_compile");
+	pcap_geterr = (__typeof__(pcap_geterr))GetProcAddress(wpcap, "pcap_geterr");
+	pcap_sendpacket = (__typeof__(pcap_sendpacket))GetProcAddress(wpcap, "pcap_sendpacket");
+	pcap_lookupdev = (__typeof__(pcap_lookupdev))GetProcAddress(wpcap, "pcap_lookupdev");
+	pcap_findalldevs = (__typeof__(pcap_findalldevs))GetProcAddress(wpcap, "pcap_findalldevs");
+	//pcap_loop = (__typeof__(pcap_loop))GetProcAddress(wpcap, "pcap_loop");
+	//pcap_loop = (__typeof__(pcap_loop))GetProcAddress(wpcap, "pcap_loop");
+	//pcap_loop = (__typeof__(pcap_loop))GetProcAddress(wpcap, "pcap_loop");
+	*/
+	return 0;
+
+}

pcap_wrapper.h

@@ -0,0 +1,127 @@
+#pragma once
+
+//#ifdef __cplusplus
+//extern "C" {
+//#endif
+
+//#include <sys/time.h>
+//#include <stdint.h>
+
+struct bpf_program
+{
+ char a[4096];
+};
+
+struct pcap_t
+{
+ char a[4096];
+};
+
+typedef unsigned int bpf_u_int32;
+
+typedef struct my_timeval {
+  int tv_sec;
+  int tv_usec;
+} my_timeval;
+
+struct pcap_pkthdr {
+	struct my_timeval ts;	/* time stamp */
+	bpf_u_int32 caplen;	/* length of portion present */
+	bpf_u_int32 len;	/* length this packet (off wire) */
+};
+
+typedef enum {
+       PCAP_D_INOUT = 0,
+       PCAP_D_IN,
+       PCAP_D_OUT
+} pcap_direction_t;
+
+
+struct pcap_addr {
+	struct pcap_addr *next;
+	struct sockaddr *addr;		/* address */
+	struct sockaddr *netmask;	/* netmask for that address */
+	struct sockaddr *broadaddr;	/* broadcast address for that address */
+	struct sockaddr *dstaddr;	/* P2P destination address for that address */
+};
+
+struct pcap_if {
+	struct pcap_if *next;
+	char *name;		/* name to hand to "pcap_open_live()" */
+	char *description;	/* textual description of interface, or NULL */
+	struct pcap_addr *addresses;
+	bpf_u_int32 flags;	/* PCAP_IF_ interface flags */
+};
+
+typedef struct pcap_if pcap_if_t;
+typedef struct pcap_addr pcap_addr_t;
+
+
+
+typedef unsigned char u_char;
+
+
+#define PCAP_ERRBUF_SIZE 256
+
+#define DLT_NULL	0	/* BSD loopback encapsulation */
+#define DLT_EN10MB	1	/* Ethernet (10Mb) */
+#define DLT_EN3MB	2	/* Experimental Ethernet (3Mb) */
+#define DLT_AX25	3	/* Amateur Radio AX.25 */
+#define DLT_PRONET	4	/* Proteon ProNET Token Ring */
+#define DLT_CHAOS	5	/* Chaos */
+#define DLT_IEEE802	6	/* 802.5 Token Ring */
+#define DLT_ARCNET	7	/* ARCNET, with BSD-style header */
+#define DLT_SLIP	8	/* Serial Line IP */
+#define DLT_PPP		9	/* Point-to-point Protocol */
+#define DLT_FDDI	10	/* FDDI */
+#define DLT_LINUX_SLL   113
+
+#define PCAP_NETMASK_UNKNOWN	0xffffffff
+
+typedef void (*pcap_handler)(u_char *, const struct pcap_pkthdr *,
+			     const u_char *);
+
+extern int	(*pcap_loop )(pcap_t *, int, pcap_handler, u_char *);
+
+extern int	(*pcap_breakloop )(pcap_t *);
+
+extern pcap_t*	(*pcap_create)(const char *, char *);
+
+extern int	(*pcap_set_snaplen) (pcap_t *, int);
+extern int	(*pcap_set_promisc) (pcap_t *, int);
+extern int	(*pcap_can_set_rfmon) (pcap_t *);
+extern int	(*pcap_set_rfmon )(pcap_t *, int);
+extern int	(*pcap_set_timeout)(pcap_t *, int);
+extern int	(*pcap_set_buffer_size)(pcap_t *, int);
+extern int	(*pcap_activate)(pcap_t *);
+
+extern int	(*pcap_setfilter)(pcap_t *, struct bpf_program *);
+extern int 	(*pcap_setdirection)(pcap_t *, pcap_direction_t);
+
+extern int	(*pcap_datalink)(pcap_t *);
+
+extern void	(*pcap_freecode)(struct bpf_program *);
+
+extern int	(*pcap_compile)(pcap_t *, struct bpf_program *, const char *, int,
+	    bpf_u_int32);
+
+extern char*   (*pcap_geterr)(pcap_t *);
+extern int	(*pcap_sendpacket)(pcap_t *, const u_char *, int);
+
+extern char* (*pcap_lookupdev)(char *);
+
+extern int	(*pcap_findalldevs)(pcap_if_t **, char *);
+
+inline int pcap_set_immediate_mode(pcap_t *,int)
+{
+	return 0;
+}
+
+
+
+//#ifdef __cplusplus
+//}
+//#endif
+
+int init_pcap();
+

server.cpp

@@ -5,6 +5,7 @@
  *      Author: root
  */
 
+#ifndef UDP2RAW_MP
 
 #include "common.h"
 #include "network.h"
@@ -18,7 +19,7 @@
 int server_on_timer_multi(conn_info_t &conn_info)  //for server. called when a timer is ready in epoll.for server,there will be one timer for every connection
 // there is also a global timer for server,but its not handled here
 {
-	char ip_port[40];
+	char ip_port[max_addr_len];
 	//u32_t ip=conn_info.raw_info.send_info.dst_ip;
 	//u32_t port=conn_info.raw_info.send_info.dst_port;
 
@@ -411,7 +412,10 @@ int server_on_raw_recv_handshake1(conn_info_t &conn_info,char * ip_port,char * d
 	}
 	return 0;
 }
-
+int server_on_recv_safer_multi(conn_info_t &conn_info,char type,char *data,int data_len)
+{
+    return 0;
+}
 int server_on_raw_recv_multi() //called when server received an raw packet
 {
 	char dummy_buf[buf_len];
@@ -441,7 +445,7 @@ int server_on_raw_recv_multi() //called when server received an raw packet
 	address_t addr;
 	addr.from_ip_port_new(raw_ip_version,&peek_info.new_src_ip,peek_info.src_port);
 
-	char ip_port[40];
+	char ip_port[max_addr_len];
 	addr.to_str(ip_port);
 	//sprintf(ip_port,"%s:%d",my_ntoa(ip),port);
 	mylog(log_trace,"[%s]peek_raw\n",ip_port);
@@ -456,6 +460,11 @@ int server_on_raw_recv_multi() //called when server received an raw packet
 			{
 				return 0;
 			}
+			if(data_len>=max_data_len+1)
+			{
+				mylog(log_debug,"data_len=%d >= max_data_len+1,ignored",data_len);
+				return -1;
+			}
 			if(use_tcp_dummy_socket!=0)
 				return 0;
 			raw_info_t &raw_info=tmp_raw_info;
@@ -590,13 +599,24 @@ int server_on_raw_recv_multi() //called when server received an raw packet
 	}
 	if(conn_info.state.server_current_state==server_ready)
 	{
-		char type;
+        vector<char> type_vec;
-		//mylog(log_info,"before recv_safer\n");
+        vector<string> data_vec;
-		if (recv_safer(conn_info,type, data, data_len) != 0) {
+        recv_safer_multi(conn_info,type_vec,data_vec);
+        if(data_vec.empty())
+        {
+            mylog(log_debug,"recv_safer failed!\n");
             return -1;
         }
-		//mylog(log_info,"after recv_safer\n");
+
-		return server_on_raw_recv_ready(conn_info,ip_port,type,data,data_len);
+        for(int i=0;i<(int)type_vec.size();i++)
+        {
+            char type=type_vec[i];
+            char *data=(char *)data_vec[i].c_str(); //be careful, do not append data to it
+            int data_len=data_vec[i].length();
+            server_on_raw_recv_ready(conn_info,ip_port,type,data,data_len);
+        }
+        return 0;
+
 	}
 
 	if(conn_info.state.server_current_state==server_idle)
@@ -897,3 +917,5 @@ int server_event_loop()
 	}
 	return 0;
 }
+
+#endif