HTTP Expect Continue performance - c#

In several Internet sources I've seen a general recommendation to disable sending Expect: 100-continue HTTP header in order to increase performance if the client is NOT actually going to send a large body.
However, testing the following code reveals that sending the header makes overall time decrease by ~50ms in average.
var hc = new HttpClient();
hc.DefaultRequestHeaders.ExpectContinue = ?;
hc.BaseAddress = new Uri("http://XXX/api/");
var r = new HttpRequestMessage(HttpMethod.Post, new Uri("YYY", UriKind.Relative))
{
Content = new StringContent("{}", Encoding.UTF8, #"application/json")
};
var tt = hc.SendAsync(r).Result;
tt.Content.ReadAsStringAsync().Result.Dump();
hc.Dispose();
Here is the WireShark dump for request with Expect: 100-continue
1 0.000000000 ss.ss.ss.176 dd.dd.dd.150 TCP 66 54515→80 [SYN] Seq=0 Win=8192 Len=0 MSS=1260 WS=4 SACK_PERM=1
2 0.342137000 dd.dd.dd.150 ss.ss.ss.176 TCP 66 80→54515 [SYN, ACK] Seq=0 Ack=1 Win=8192 Len=0 MSS=1380 WS=1 SACK_PERM=1
3 0.342687000 ss.ss.ss.176 dd.dd.dd.150 TCP 54 54515→80 [ACK] Seq=1 Ack=1 Win=66780 Len=0
4 *REF* ss.ss.ss.176 dd.dd.dd.150 HTTP 272 POST /XXX/api/YYY HTTP/1.1
5 0.361158000 dd.dd.dd.150 ss.ss.ss.176 HTTP 79 HTTP/1.1 100 Continue
6 0.361846000 ss.ss.ss.176 dd.dd.dd.150 TCP 56 54515→80 [PSH, ACK] Seq=219 Ack=26 Win=66752 Len=2
7 0.705497000 dd.dd.dd.150 ss.ss.ss.176 HTTP 461 HTTP/1.1 200 OK (application/json)
8 0.726029000 ss.ss.ss.176 dd.dd.dd.150 TCP 54 54515→80 [FIN, ACK] Seq=221 Ack=433 Win=66348 Len=0
9 1.067923000 dd.dd.dd.150 ss.ss.ss.176 TCP 54 80→54515 [FIN, ACK] Seq=433 Ack=222 Win=65535 Len=0
10 1.068466000 ss.ss.ss.176 dd.dd.dd.150 TCP 54 54515→80 [ACK] Seq=222 Ack=434 Win=66348 Len=0
The same request without the header:
11 9.300455000 ss.ss.ss.176 dd.dd.dd.150 TCP 66 54516→80 [SYN] Seq=0 Win=8192 Len=0 MSS=1260 WS=4 SACK_PERM=1
12 9.640626000 dd.dd.dd.150 ss.ss.ss.176 TCP 66 80→54516 [SYN, ACK] Seq=0 Ack=1 Win=8192 Len=0 MSS=1380 WS=1 SACK_PERM=1
13 9.641393000 ss.ss.ss.176 dd.dd.dd.150 TCP 54 54516→80 [ACK] Seq=1 Ack=1 Win=66780 Len=0
14 *REF* ss.ss.ss.176 dd.dd.dd.150 HTTP 250 POST /XXX/api/YYY HTTP/1.1
15 0.406794000 dd.dd.dd.150 ss.ss.ss.176 TCP 54 80→54516 [ACK] Seq=1 Ack=197 Win=65535 Len=0
16 0.406963000 ss.ss.ss.176 dd.dd.dd.150 TCP 56 54516→80 [PSH, ACK] Seq=197 Ack=1 Win=66780 Len=2
17 0.749589000 dd.dd.dd.150 ss.ss.ss.176 HTTP 461 HTTP/1.1 200 OK (application/json)
18 0.769053000 ss.ss.ss.176 dd.dd.dd.150 TCP 54 54516→80 [FIN, ACK] Seq=199 Ack=408 Win=66372 Len=0
19 1.109276000 dd.dd.dd.150 ss.ss.ss.176 TCP 54 80→54516 [FIN, ACK] Seq=408 Ack=200 Win=65535 Len=0
20 1.109742000 ss.ss.ss.176 dd.dd.dd.150 TCP 54 54516→80 [ACK] Seq=200 Ack=409 Win=66372 Len=0
Same results were received for IIS 7.5, IIS 8.0
The questions are:
What makes the request with the Expect header execute faster, when theoretically the opposite shall take place?
Is it always the case that the body of POST request goes within a separate TCP packet (I've looked through only a couple of samples, there this is true)? Here I mean why TCP packet at line 14 in dump does not contain the data (POST body) that was sent in TCP packet at line 16?


I had the same problem (~50 ms of delay). I guess it's a bug on HttpClient implementation of .NET Framework.
I made some test with .NET Core 2.1 and I was able to remove Continue message without performance decay.


Resolved same problem with TcpClient
var tcpClient = new TcpClient();
var uri = new Uri(httpAddress);
tcpClient.Connect(uri.Host, uri.Port);
string httpResponse = null;
using (NetworkStream networkStream = tcpClient.GetStream())
{
var httpRequestBuilder = new StringBuilder();
httpRequestBuilder.AppendLine("POST / HTTP/1.1");
httpRequestBuilder.Append("Host: ").AppendLine(uri.Host);
httpRequestBuilder.AppendLine("Content-Type: application/json");
httpRequestBuilder.Append("Content-Length: ").AppendLine(postJsonBody.Length.ToString());
httpRequestBuilder.AppendLine();
httpRequestBuilder.AppendLine(postJsonBody);
var httpRequest = httpRequestBuilder.ToString();
var requestBytes = Encoding.UTF8.GetBytes(httpRequest);
// sending request as one package
// (without PSH)
networkStream.Write(requestBytes, 0, requestBytes.Length);
// reading response
using (var sr = new StreamReader(networkStream, Encoding.UTF8))
{
httpResponse = sr.ReadToEnd();
}
}
tcpClient.Close();

Related

Windows Firewall Rule doesn't work - Firewall API - C#

i programmed a small chat program which is coded in C#. It uses WCF, to communicate with other users in the local network. Furthermore it uses the Firewall API dll in System32 to add 4 rules which allows all ports for incoming/outgoing TCP and UDP connections for this program. This rules are adding themselfe in the current network configuration(Public, Private). When the program Is closing, it deletes the 4 rules.
The problem is, that the firewall ignores this rules and drops the data which Is incomming. When i deactivate the firewall on both pcs(the two which are communicating) it works without problems.
The firewall has this 4 rules activated but i don't really get it why the firewall drops the incoming data...
Tested on 4 different Windows 10 machines.
After I think 1,5 hours the firewall rules worked on one machine which Is also confusing me...
This is the method with that i create this rules:
private void FWRule(NET_FW_RULE_DIRECTION_ direction,
NET_FW_ACTION_ fwaction, NET_FW_IP_PROTOCOL_ protocol, bool add)
{
try
{
INetFwRule firewallRule = (INetFwRule)Activator.CreateInstance(Type.GetTypeFromProgID("HNetCfg.FWRule"));
INetFwMgr fwMgr = (INetFwMgr)Activator.CreateInstance(Type.GetTypeFromProgID("HNetCfg.FWMgr"));
firewallRule.Action = fwaction;
firewallRule.Enabled = true;
firewallRule.InterfaceTypes = "All";
firewallRule.serviceName ="Chatty";
firewallRule.Grouping = "Chatty";
firewallRule.Profiles = (int)NET_FW_PROFILE_TYPE_.NET_FW_PROFILE_CURRENT;
firewallRule.ApplicationName = Assembly.GetExecutingAssembly().Location;
if (protocol == NET_FW_IP_PROTOCOL_.NET_FW_IP_PROTOCOL_TCP)
firewallRule.Name = Assembly.GetExecutingAssembly().GetName().Name + " Server Remote TCP";
else if (protocol == NET_FW_IP_PROTOCOL_.NET_FW_IP_PROTOCOL_UDP)
firewallRule.Name = Assembly.GetExecutingAssembly().GetName().Name + " Server Remote UDP";
firewallRule.Protocol = (int)protocol;
INetFwPolicy2 firewallPolicy = (INetFwPolicy2)Activator.CreateInstance
(Type.GetTypeFromProgID("HNetCfg.FwPolicy2"));
firewallRule.Direction = direction;
if (add == true)
{
firewallPolicy.Rules.Add(firewallRule);
}
else
{
firewallPolicy.Rules.Remove(firewallRule.Name);
}
}
catch (Exception ex) { MessageBox.Show(ex.Message, "ERROR"); }
}
The log from the firewall of the Server program shows that the firewall drops the incoming TCP data from the client(the server port for TCP is 2310, the client port for TCP is random/not known):
2016-10-19 10:37:56 DROP ICMP :: ff02::1 - - 80 - - - - 130 0 - RECEIVE
2016-10-19 10:38:54 DROP TCP fe80::91c:1ba7:abfc:5e5f fe80::1c96:dc05:3b26:1eaa 50248 2310 72 S 1175516710 0 8192 - - - RECEIVE
2016-10-19 10:38:57 DROP TCP fe80::91c:1ba7:abfc:5e5f fe80::1c96:dc05:3b26:1eaa 50248 2310 72 S 1175516710 0 8192 - - - RECEIVE
2016-10-19 10:39:03 DROP TCP fe80::91c:1ba7:abfc:5e5f fe80::1c96:dc05:3b26:1eaa 50248 2310 68 S 1175516710 0 8192 - - - RECEIVE
2016-10-19 10:40:02 DROP ICMP :: ff02::1 - - 80 - - - - 130 0 - RECEIVE
2016-10-19 10:41:02 DROP TCP fe80::91c:1ba7:abfc:5e5f fe80::1c96:dc05:3b26:1eaa 50252 2310 72 S 4091559822 0 8192 - - - RECEIVE
2016-10-19 10:41:05 DROP TCP fe80::91c:1ba7:abfc:5e5f fe80::1c96:dc05:3b26:1eaa 50252 2310 72 S 4091559822 0 8192 - - - RECEIVE
2016-10-19 10:41:11 DROP TCP fe80::91c:1ba7:abfc:5e5f fe80::1c96:dc05:3b26:1eaa 50252 2310 68 S 4091559822 0 8192 - - - RECEIVE
2016-10-19 10:41:23 DROP TCP 192.168.1.100 192.168.1.121 50253 2310 52 S 3013060805 0 8192 - - - RECEIVE
2016-10-19 10:41:26 DROP TCP 192.168.1.100 192.168.1.121 50253 2310 52 S 3013060805 0 8192 - - - RECEIVE
2016-10-19 10:41:30 DROP TCP fe80::91c:1ba7:abfc:5e5f fe80::1c96:dc05:3b26:1eaa 50261 2310 72 S 1985828196 0 8192 - - - RECEIVE
2016-10-19 10:41:33 DROP TCP fe80::91c:1ba7:abfc:5e5f fe80::1c96:dc05:3b26:1eaa 50261 2310 72 S 1985828196 0 8192 - - - RECEIVE
2016-10-19 10:41:39 DROP TCP fe80::91c:1ba7:abfc:5e5f fe80::1c96:dc05:3b26:1eaa 50261 2310 68 S 1985828196 0 8192 - - - RECEIVE
2016-10-19 10:42:07 DROP ICMP :: ff02::1 - - 80 - - - - 130 0 - RECEIVE
2016-10-19 10:44:13 DROP ICMP :: ff02::1 - - 80 - - - - 130 0 - RECEIVE
2016-10-19 10:44:28 DROP TCP fe80::91c:1ba7:abfc:5e5f fe80::1c96:dc05:3b26:1eaa 50285 2310 72 S 2762432965 0 8192 - - - RECEIVE
2016-10-19 10:44:29 DROP TCP fe80::91c:1ba7:abfc:5e5f fe80::1c96:dc05:3b26:1eaa 50285 2310 72 S 2762432965 0 8192 - - - RECEIVE
2016-10-19 10:44:35 DROP TCP fe80::91c:1ba7:abfc:5e5f fe80::1c96:dc05:3b26:1eaa 50285 2310 68 S 2762432965 0 8192 - - - RECEIVE
The rule is registered in the Windows Firewall(sorry OS is in German):

Unrecognized Ciphers from mobile application

I'm writing an Android application with Xamarin (C#) that makes calls to Exchange Web Services. The code sets the out of office message. The code I have written works from a standard Windows Forms application, but fails with a "Request Invalid" SOAP exception. I've captured the requests using Fiddler, and it appears to be a cipher issue.
The Windows Forms request is this:
A SSLv3-compatible ClientHello handshake was found. Fiddler extracted the parameters below.
Version: 3.1 (TLS/1.0)
Random: 53 B1 31 EF DD 96 01 7E 0A 0A 1F 85 60 8F 2F BD D6 77 08 2D B0 C6 52 91 BC A0 19 2F 20 08 F1
SessionID: empty
Extensions:
renegotiation_info 00
server_name
elliptic_curves secp256r1 [0x17], secp384r1 [0x18]
ec_point_formats uncompressed [0x0]
Ciphers:
[002F] TLS_RSA_AES_128_SHA
[0035] TLS_RSA_AES_256_SHA
[0005] SSL_RSA_WITH_RC4_128_SHA
[000A] SSL_RSA_WITH_3DES_EDE_SHA
[C013] TLS1_CK_ECDHE_RSA_WITH_AES_128_CBC_SHA
[C014] TLS1_CK_ECDHE_RSA_WITH_AES_256_CBC_SHA
[C009] TLS1_CK_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
[C00A] TLS1_CK_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
[0032] TLS_DHE_DSS_WITH_AES_128_SHA
[0038] TLS_DHE_DSS_WITH_AES_256_SHA
[0013] SSL_DHE_DSS_WITH_3DES_EDE_SHA
[0004] SSL_RSA_WITH_RC4_128_MD5
Compression:
[00] NO_COMPRESSION
The Android request is this:
A SSLv3-compatible ClientHello handshake was found. Fiddler extracted the parameters below.
Version: 3.1 (TLS/1.0)
Random: 53 B1 30 9C 9A A2 AE 57 A9 98 A3 C8 DF 94 E0 04 1D 3F E1 34 A8 8A DF 8B 5E 8B 60 67 96 6D 1B
SessionID: empty
Extensions:
server_name
Ciphers:
[0035] TLS_RSA_AES_256_SHA
[002F] TLS_RSA_AES_128_SHA
[000A] SSL_RSA_WITH_3DES_EDE_SHA
[0005] SSL_RSA_WITH_RC4_128_SHA
[0004] SSL_RSA_WITH_RC4_128_MD5
[0009] SSL_RSA_WITH_DES_SHA
[0003] SSL_RSA_EXPORT_WITH_RC4_40_MD5
[0006] SSL_RSA_EXPORT_WITH_RC2_40_MD5
[0008] SSL_RSA_EXPORT_WITH_DES40_SHA
[0060] Unrecognized cipher - See http://www.iana.org/assignments/tls-parameters/
[0061] Unrecognized cipher - See http://www.iana.org/assignments/tls-parameters/
[0062] TLS_RSA_EXPORT1024_WITH_DES_SHA
[0064] TLS_RSA_EXPORT1024_WITH_RC4_56_SHA
Compression:
[00] NO_COMPRESSION
The "unrecognized cipher" message leaps out here. Any help on how to resolve this issue would be great.
thanks

That two ciphers are not recognized does not mean that there are no ciphers to choose from. Unless the TLS is explicitly broken down further on, it's likely another issue.
I think you are better off looking at the SOAP messages.

Why I have different compression byte array sizes using DeflateStream on different machines

I'm compressing short string on my machine using DeflateStream
public byte[] Compress(byte[] oryginalBytes)
{
using (var msi = new MemoryStream(oryginalBytes))
using (var mso = new MemoryStream())
{
using (var gs = new DeflateStream(mso, CompressionMode.Compress))
{
msi.CopyTo(gs);
}
return mso.ToArray();
}
}
And run this code:
Compress(Encoding.Unicode.GetBytes("[TEST]"));
On my machine (windows 7 64 bit) I get 12 elements byte array:
139 102 8 97 112 101 8 6 146 177 12 0
And running this on my build server (Windows 2008 64bit) I have 120 elements byte array.
236 189 7 96 28 73 150 37 38 47 109 202 123 127 74 245 74 215 224 116 161 8 128 96 19 36 216 144 64 16 236 193 136 205 230 146 236 29 105 71 35 41 171 42 129 202 101 86 101 93 102 22 64 204 237 157 188 247 222 123 239 189 247 222 123 239 189 247 186 59 157 78 39 247 223 255 63 92 102 100 1 108 246 206 74 218 201 158 33 128 170 200 31 63 126 124 31 63 34 190 247 107 188 249 53 78 127 141 215 244 239 247 127 141 255 39 0 0 255 255
Anybody has an idea why this behave so strange?

DeflateStream manual
This class represents the Deflate algorithm, which is an industry-standard algorithm for lossless file compression and decompression. Starting with the .NET Framework 4.5, the DeflateStream class uses the zlib library. As a result, it provides a better compression algorithm and, in most cases, a smaller compressed file than it provides in earlier versions of the .NET Framework.

GNUPlot: From 3D scatter plot to Countour map

Given the following list of x,y,z points (z is actually a signal strength indicator in dBm):
63 371 -21
142 371 -9
233 374 -18
288 371 -36
310 373 -38
349 374 -39
415 348 -44
507 334 -49
689 337 -56
635 254 -57
422 284 -42
380 278 -39
281 280 -39
214 299 -34
146 285 -30
81 302 -39
76 246 -39
80 214 -44
137 200 -44
64 134 -48
73 87 -48
200 101 -46
230 202 -44
246 105 -53
285 109 -53
278 191 -50
334 87 -54
395 189 -56
513 208 -58
510 99 -61
553 101 -62
593 100 -65
634 101 -64
679 102 -68
731 196 -70
I can quite easily generate a 3D scatter plot using this command in gnuplot:
splot "C:/path/to/points.dat" with points
What I'm really after is a way to "fill in the gaps" and create a surface. The ultimate goal is to use data like the points shown above to generate an RF map for a given WiFi access point. The resulting map would be used to calculate the possible locations of a device, given its RSSI (received signal strength indicator).
(Several APs and their associated RF maps will be used, however I don't think the specifics are relevant to this question.)
What would be the best way to interpolate the z-values (RSSI in dBm, in this case) for the remaining points?
EDIT :
I seem to have made some progress. Using set dgrid3d 50,100 splines before the splot command seems to generate more or less what I'm looking for. So the next question is: how can I replicate this kind of function in code (C#, ideally)?

TCP Socket Sending Delays and Retransmission

I have a .NET 3.5 C# application that sends 2000-6000 byte packets to a linux machine running sles 10. The machines are on the same subnet.
About 90% of the time, everything works fine. The linux machine processes my request and responds in 5-15ms. But about 10% of the time, there is an approx 200ms-800ms delay.
Looking at the logs on the linux machine, it seems the delay is on my end. That is, if my call to socket.Send(...) returns at 1:15:00.000 and I get a response at 1:15:00.210, the log on the linux machine says that it received the request at 1:15:00.200 and then processed it in 10ms. (I'm using System.Diagnostics.Stopwatch for timing on my machine.)
To debug, I captured the traffic using wireshark. Here is the traffic. Between No. 8 and 9 is where a 600 ms delay occurs. (137.34.210.108 is my machine and 137.34.210.95 is the linux machine).
"1","11:56:27.380318","137.34.210.95","137.34.210.108","TCP","20700 > 17479 [PSH, ACK] Seq=1 Ack=1 Win=32767 Len=76"
"2","11:56:27.380393","HewlettP_29:37:0f","Broadcast","ARP","Who has 137.34.210.95? Tell 137.34.210.108"
"3","11:56:27.380558","HewlettP_29:39:93","HewlettP_29:37:0f","ARP","137.34.210.95 is at 00:1b:78:29:39:93"
"4","11:56:27.380564","137.34.210.108","137.34.210.95","TCP","17479 > 20700 [ACK] Seq=1 Ack=77 Win=65459 [TCP CHECKSUM INCORRECT] Len=0"
"5","12:04:48.096892","HewlettP_29:37:0f","Broadcast","ARP","Who has 137.34.210.95? Tell 137.34.210.108"
"6","12:04:48.097216","HewlettP_29:39:93","HewlettP_29:37:0f","ARP","137.34.210.95 is at 00:1b:78:29:39:93"
"7","12:04:48.097229","137.34.210.108","137.34.210.95","TCP","17480 > 20600 [PSH, ACK] Seq=1 Ack=1 Win=64198 [TCP CHECKSUM INCORRECT] Len=458"
"8","12:04:48.097457","137.34.210.95","137.34.210.108","TCP","20600 > 17480 [ACK] Seq=1 Ack=4294964377 Win=32767 Len=0 SLE=1 SRE=459"
"9","12:04:49.700966","137.34.210.108","137.34.210.95","TCP","17479 > 20700 [ACK] Seq=1 Ack=77 Win=65459 [TCP CHECKSUM INCORRECT] Len=1460"
"10","12:04:49.701190","137.34.210.108","137.34.210.95","TCP","[TCP Retransmission] 17480 > 20600 [ACK] Seq=4294964377 Ack=1 Win=64198 [TCP CHECKSUM INCORRECT] Len=1460"
"11","12:04:49.703970","137.34.210.95","137.34.210.108","TCP","20600 > 17480 [ACK] Seq=1 Ack=4294965837 Win=32767 Len=0 SLE=1 SRE=459"
"12","12:04:49.703993","137.34.210.108","137.34.210.95","TCP","[TCP Retransmission] 17480 > 20600 [ACK] Seq=4294965837 Ack=1 Win=64198 [TCP CHECKSUM INCORRECT] Len=1460"
"13","12:04:49.704002","137.34.210.108","137.34.210.95","TCP","[TCP Retransmission] 17480 > 20600 [PSH, ACK] Seq=1 Ack=1 Win=64198 [TCP CHECKSUM INCORRECT] Len=458"
"14","12:04:49.704211","137.34.210.95","137.34.210.108","TCP","20600 > 17480 [ACK] Seq=1 Ack=459 Win=32767 Len=0"
"15","12:04:49.704215","137.34.210.95","137.34.210.108","TCP","[TCP Dup ACK 14#1] 20600 > 17480 [ACK] Seq=1 Ack=459 Win=32767 Len=0 SLE=1 SRE=459"
"16","12:04:49.705425","137.34.210.95","137.34.210.108","TCP","20700 > 17479 [PSH, ACK] Seq=77 Ack=1461 Win=32767 Len=44"
Can someone help me to interpret this? I see that a re-transmit is occurring. But I'm not sure why. The switch shows no dropped packets. And even if the packets are being lost, why would it take 600ms to re-transmit?
I thought that this (http://support.microsoft.com/kb/328890) might have something to do with the 200ms delays but I've tried changing the TcpAckFrequency and it didn't help.
Thanks,
Mike

Let's start by pruning some of that Wireshark output. We can toss the ARPs in packets 2, 3, 5 and 6. Looking at the rest, you have two sets of traffic in there. Packets 8 and 9 are two different connections, so you can't compare them. 7, 8 and 10, however, are part of one connection so let's examine those.
Packet 7 is 458 bytes of data being sent to the Linux box with a TCP sequence number of 1. However, the ACK that the Linux box returns is 4294964377. This means that Wireshark is showing relative TCP values and that the Linux box is not sending an ACK for packet 7, but for an earlier packet. Your PC then waits for a follow-up ACK and, when it doesn't get one, retransmits the required data. In this case the 458 bytes from packet 7 along with a previous 1002 bytes. That's why the sequence number from packet 10 matches the ACK from packet 8.
Unfortunately this doesn't tell you why data is being dropped. Packet 8 shows the Linux box indicating it still has a full 32k of input buffer available for this connection ("Win=32767").

This only shows the TCP packets on the Linux machine, but I'd recommend to look at the ip stats with the 'netstat -s' command. One reason for the retransmissions might be socket buffer overflows, which will be shown with this command.

I don't recall if Windows has it, but on UNIX you'd enable TCP_NODELAY.
This disables TCP's Nagle Algorithm which makes the system wait for a small time in case more data is going to be added to the transmit buffer.
int nodelay = 1;
setsockopt(s, IPPROTO_TCP, TCP_NODELAY, &nodelay, sizeof(nodelay));

Categories