We Keep Coding

Why does this C# TcpClient code not always see a response?

I am using the following code to read a psuedo-HTTP request response. It works sometimes but not always and I do not understand.
Background: I have a device that takes HTTP GET requests and sends a chunked HTTP response. In one case, the response is not a proper chunked HTTP response. It leaves out the null chunk that indicates the end of data. I have fixed that problem in the device, but I am trying to figure out how to read the non-comforming HTTP response. I found code from Create http request using TcpClient that sometimes works and sometimes doesn't and I do not understand why.
If I use the code unaltered, it works fine. If I use it by replacing the "www.bing.com" with my device's IP, "192.1.168.89" in both places the string appears, for example, and change the GET command line to "GET /index.htm HTTP/1.1", it works fine. This version of the command returns a web page that is constructed by the device and sends several TCP buffers (about 1400 bytes in my device) of chunked data.
However, if I change to another command that my device understands, "GET /request.htm?T HTTP/1.1", but returns less than 500 bytes of chunked data, then I never see the response. In fact it never gets past the call to "CopyToAsync(memory)" and I do not understand why. The device sees the request, parses it and sends a proper HTTP response. (I know it is a proper response because I have code that uses HTTPClient to read the response and it sees the response fine. And I see the response data from the device side is exactly the same going out in both cases. I can see the device data because I am writing the device's firmware and can change it to printf() the data being sent out to the TCP routines.)
Anyone have an explanation for why the code below isn't always seeing a response?
private static async Task<string> HttpRequestAsync() {
string result = string.Empty;
using (var tcp = new TcpClient("www.bing.com", 80))
using (var stream = tcp.GetStream())
{
tcp.SendTimeout = 500;
tcp.ReceiveTimeout = 1000;
// Send request headers
var builder = new StringBuilder();
builder.AppendLine("GET /?scope=images&nr=1 HTTP/1.1");
builder.AppendLine("Host: www.bing.com");
//builder.AppendLine("Content-Length: " + data.Length); // only for POST request
builder.AppendLine("Connection: close");
builder.AppendLine();
var header = Encoding.ASCII.GetBytes(builder.ToString());
await stream.WriteAsync(header, 0, header.Length);
// Send payload data if you are POST request
//await stream.WriteAsync(data, 0, data.Length);
// receive data
using (var memory = new MemoryStream())
{
await stream.CopyToAsync(memory);
memory.Position = 0;
var data = memory.ToArray();
var index = BinaryMatch(data, Encoding.ASCII.GetBytes("\r\n\r\n")) + 4;
var headers = Encoding.ASCII.GetString(data, 0, index);
memory.Position = index;
if (headers.IndexOf("Content-Encoding: gzip") > 0)
{
using (GZipStream decompressionStream = new GZipStream(memory, CompressionMode.Decompress))
using (var decompressedMemory = new MemoryStream())
{
decompressionStream.CopyTo(decompressedMemory);
decompressedMemory.Position = 0;
result = Encoding.UTF8.GetString(decompressedMemory.ToArray());
}
}
else
{
result = Encoding.UTF8.GetString(data, index, data.Length - index);
//result = Encoding.GetEncoding("gbk").GetString(data, index, data.Length - index);
}
}
//Debug.WriteLine(result);
return result;
}
}
private static int BinaryMatch(byte[] input, byte[] pattern)
{
int sLen = input.Length - pattern.Length + 1;
for (int i = 0; i < sLen; ++i)
{
bool match = true;
for (int j = 0; j < pattern.Length; ++j)
{
if (input[i + j] != pattern[j])
{
match = false;
break;
}
}
if (match)
{
return i;
}
}
return -1;
}
=====================
Let me edit the function above to show what it is now and maybe clarify things.
static async Task<byte[]> getTcpClientHttpDataRequestAsync(string ipAddress, string request)
{
string result = string.Empty;
List<byte> arrayList = new List<byte>();
using (var tcp = new TcpClient("192.168.1.89", 80))
using (var stream = tcp.GetStream())
using (var memory = new MemoryStream())
{
tcp.SendTimeout = 500;
tcp.ReceiveTimeout = 10000;
tcp.NoDelay = true;
// Send request headers
var builder = new StringBuilder();
builder.AppendLine("GET /request.htm?x01011920000000000001 HTTP/1.1");
builder.AppendLine("Host: 192.168.1.89");
builder.AppendLine("Connection: Close");
builder.AppendLine();
var header = Encoding.ASCII.GetBytes(builder.ToString());
Console.WriteLine("======");
Console.WriteLine(builder.ToString());
Console.WriteLine("======");
await stream.WriteAsync(header, 0, header.Length);
do { } while (stream.DataAvailable == 0);
Console.WriteLine("Data available");
bool done = false;
do
{
int next = stream.ReadByte();
if (next < 0)
{
done = true;
}
else
{
arrayList.Add(Convert.ToByte(next));
}
} while (stream.DataAvailable && !done);
byte[] data = arrayList.ToArray();
return data;
}
}
The GET command is what my device is responding to. If the command starts with 'x' as shown then it responds with a proper HTTP response and the function above reads the data. If it starts with 'd' it is missing the 0 length chunk at the end and the function above never sees any data from the device.
With Wireshark, I am seeing the following responses for the 'x' and 'd' commands.
The 'x' command returns 2 TCP frames with the following data:
0000 1c 6f 65 d3 f0 e2 4c 60 de 41 3f 67 08 00 45 00 .oe...L`.A?g..E.
0010 00 9c 00 47 00 00 64 06 d2 49 c0 a8 01 59 c0 a8 ...G..d..I...Y..
0020 01 22 00 50 05 5d fc f5 9e 72 ad 75 e3 2c 50 18 .".P.]...r.u.,P.
0030 00 01 a9 cd 00 00 48 54 54 50 2f 31 2e 31 20 32 ......HTTP/1.1 2
0040 30 30 20 4f 4b 0d 0a 43 6f 6e 6e 65 63 74 69 6f 00 OK..Connectio
0050 6e 3a 20 63 6c 6f 73 65 0d 0a 43 6f 6e 74 65 6e n: close..Conten
0060 74 2d 54 79 70 65 3a 20 74 65 78 74 2f 68 74 6d t-Type: text/htm
0070 6c 0d 0a 43 61 63 68 65 2d 43 6f 6e 74 72 6f 6c l..Cache-Control
0080 3a 20 6e 6f 2d 63 61 63 68 65 0d 0a 54 72 61 6e : no-cache..Tran
0090 73 66 65 72 2d 45 6e 63 6f 64 69 6e 67 3a 20 63 sfer-Encoding: c
00a0 68 75 6e 6b 65 64 0d 0a 0d 0a hunked....
0000 1c 6f 65 d3 f0 e2 4c 60 de 41 3f 67 08 00 45 00 .oe...L`.A?g..E.
0010 00 45 00 48 00 00 64 06 d2 9f c0 a8 01 59 c0 a8 .E.H..d......Y..
0020 01 22 00 50 05 5d fc f5 9e e6 ad 75 e3 2c 50 18 .".P.].....u.,P.
0030 00 01 fc 20 00 00 30 30 31 0d 0a 2b 0d 0a 30 30 ... ..001..+..00
0040 37 0d 0a 01 85 86 00 00 0d 0a 0d 0a 30 30 30 0d 7...........000.
0050 0a 0d 0a ...
By comparison the 'd' command returns data in 2 TCP frames as:
0000 1c 6f 65 d3 f0 e2 4c 60 de 41 3f 67 08 00 45 00 .oe...L`.A?g..E.
0010 00 9c 00 4e 00 00 64 06 d2 42 c0 a8 01 59 c0 a8 ...N..d..B...Y..
0020 01 22 00 50 05 5e d3 c3 f9 f5 69 cc 6d a3 50 18 .".P.^....i.m.P.
0030 00 01 30 ae 00 00 48 54 54 50 2f 31 2e 31 20 32 ..0...HTTP/1.1 2
0040 30 30 20 4f 4b 0d 0a 43 6f 6e 6e 65 63 74 69 6f 00 OK..Connectio
0050 6e 3a 20 63 6c 6f 73 65 0d 0a 43 6f 6e 74 65 6e n: close..Conten
0060 74 2d 54 79 70 65 3a 20 74 65 78 74 2f 68 74 6d t-Type: text/htm
0070 6c 0d 0a 43 61 63 68 65 2d 43 6f 6e 74 72 6f 6c l..Cache-Control
0080 3a 20 6e 6f 2d 63 61 63 68 65 0d 0a 54 72 61 6e : no-cache..Tran
0090 73 66 65 72 2d 45 6e 63 6f 64 69 6e 67 3a 20 63 sfer-Encoding: c
00a0 68 75 6e 6b 65 64 0d 0a 0d 0a hunked....
0000 1c 6f 65 d3 f0 e2 4c 60 de 41 3f 67 08 00 45 00 .oe...L`.A?g..E.
0010 00 36 00 4f 00 00 64 06 d2 a7 c0 a8 01 59 c0 a8 .6.O..d......Y..
0020 01 22 00 50 05 5e d3 c3 fa 69 69 cc 6d a3 50 18 .".P.^...ii.m.P.
0030 00 01 64 c2 00 00 30 30 37 0d 0a 01 90 91 00 00 ..d...007.......
0040 0d 0a 0d 0a ....
The only discernible differences that I see is that in the second frame of the 'd' command it is missing a 1 byte chunk that is part of our protocol (and shouldn't have any effect on the TCP/HTTP function) and the last 7 bytes of data that the 'x' command provides, which is the 0 length chunk expected for HTTP.
Going back to the code in HttpRequestAsync(), if the 'd' command is sent then the code never sees stream.DataAvailable become true, even though the data has been sent. Why?

await stream.CopyToAsync()
will not complete until
stream.DataAvailable == false
You have indicated to the server, in the headers that you will close the TCP connection when done, but have not done so. The server will eventually close the connection when it thinks you're gone. The server is not obligated to obey your "Connection: close" request and that should be indicated in the headers the server returns.
Before you call stream.CopyToAsync() you should check the headers to determine if what Content-Length has been supplied and pass a buffer length to stream.CopyToAsync() and then call TcpClient.Close()

ADB Public Key Format

I'm trying to implement ADB protocol in C#. When i tried using AUTH(2), the device always says that my signature is wrong by replying AUTH(1) over and over again until i sent AUTH(3) (which will trigger the allow this pc dialog). I have saved the public key to the phone by checking the "Always allow from this PC" option on the device.
I am suspecting that my public key format i sent is wrong because seeing the size my implementation sends vs the original ADB's is different. The public keys example below does not use the same private key.
ADB:
000022: Bulk or Interrupt Transfer (DOWN), 2019-08-16 08:23:37,2615937 +0,0000492 (1. Device: SAMSUNG_Android (Galaxy series, misc. (MTP mode)))
Pipe Handle: Control PipeSend 0x2bd bytes to the device
51 41 41 41 41 45 39 37 36 35 78 52 52 44 4D 45 QAAAAE9765xRRDME
6B 43 47 59 4F 7A 43 63 72 43 68 71 62 53 73 56 kCGYOzCcrChqbSsV
6D 32 4A 35 57 46 58 71 4A 78 59 79 4B 41 63 55 m2J5WFXqJxYyKAcU
2B 51 66 57 6A 57 7A 35 54 78 68 58 32 45 46 43 +QfWjWz5TxhX2EFC
52 4E 4F 38 55 56 4A 4D 68 49 44 33 49 58 72 47 RNO8UVJMhID3IXrG
38 49 48 54 77 41 4F 76 6D 78 2F 33 67 4F 41 73 8IHTwAOvmx/3gOAs
55 4F 53 36 4A 2F 5A 74 43 6C 38 38 35 4E 72 64 UOS6J/ZtCl885Nrd
67 66 30 63 61 33 36 6E 70 51 62 43 54 42 56 45 gf0ca36npQbCTBVE
56 59 7A 54 52 49 65 59 4A 63 69 65 78 30 37 4F VYzTRIeYJciex07O
77 38 38 51 6A 52 4C 53 76 63 53 62 52 56 6D 72 w88QjRLSvcSbRVmr
73 78 47 70 32 4C 7A 77 47 69 6E 77 43 4D 48 46 sxGp2LzwGinwCMHF
45 79 69 64 7A 52 42 2B 73 74 4F 2B 66 67 6D 44 EyidzRB+stO+fgmD
47 64 50 4D 31 51 74 59 47 73 51 74 65 53 76 54 GdPM1QtYGsQteSvT
36 79 56 57 66 73 73 70 72 6C 65 33 49 6B 74 45 6yVWfssprle3IktE
32 53 64 4B 4E 63 62 49 44 57 39 5A 56 67 46 2F 2SdKNcbIDW9ZVgF/
2B 4B 79 6B 4C 69 43 37 77 4F 72 4B 4E 46 4E 49 +KykLiC7wOrKNFNI
61 45 4C 4B 52 50 37 33 33 31 73 30 76 67 75 34 aELKRP7331s0vgu4
57 61 76 6F 39 39 66 79 69 43 66 41 48 65 64 45 Wavo99fyiCfAHedE
6E 42 45 7A 4B 41 50 52 53 42 36 6A 74 39 6D 34 nBEzKAPRSB6jt9m4
4B 67 73 4D 4F 54 48 67 45 37 45 7A 53 6D 42 4C KgsMOTHgE7EzSmBL
72 64 38 51 31 58 44 34 52 75 77 61 6F 76 37 47 rd8Q1XD4Ruwaov7G
46 2B 31 52 78 42 44 35 73 61 38 6B 6F 4B 6E 53 F+1RxBD5sa8koKnS
50 68 6E 75 35 48 78 76 71 57 73 49 78 38 70 54 Phnu5HxvqWsIx8pT
42 59 51 45 79 2B 57 33 51 71 2F 6E 66 32 6B 46 BYQEy+W3Qq/nf2kF
70 41 33 61 6D 45 63 2F 53 71 4E 2B 30 46 73 55 pA3amEc/SqN+0FsU
31 48 71 52 78 64 59 5A 68 68 62 66 58 69 52 70 1HqRxdYZhhbfXiRp
4B 67 74 4E 70 31 50 43 55 2B 46 2F 55 32 2B 67 KgtNp1PCU+F/U2+g
64 58 39 6C 68 38 52 31 6C 59 7A 79 38 2F 4A 4C dX9lh8R1lYzy8/JL
65 5A 6F 4F 37 34 31 47 34 34 54 75 38 4A 69 4F eZoO741G44Tu8JiO
69 59 74 6D 43 32 58 42 39 54 52 51 36 49 74 71 iYtmC2XB9TRQ6Itq
46 30 36 67 4D 53 32 78 71 6C 50 6B 54 58 4F 6D F06gMS2xqlPkTXOm
4E 4B 71 72 31 39 76 4C 79 64 66 34 42 5A 34 62 NKqr19vLydf4BZ4b
57 53 58 4A 71 54 39 66 2B 54 47 39 75 48 79 56 WSXJqT9f+TG9uHyV
31 77 35 6E 68 6F 34 44 7A 68 38 6D 65 73 44 55 1w5nho4Dzh8mesDU
6C 30 2F 71 72 71 48 45 4F 73 57 6C 4B 50 72 57 l0/qrqHEOsWlKPrW
4F 59 2F 5A 42 41 56 4B 4F 6A 6D 34 5A 57 67 5A OY/ZBAVKOjm4ZWgZ
76 71 42 56 61 35 71 57 67 4A 44 68 50 37 37 72 vqBVa5qWgJDhP77r
44 68 47 69 2B 50 34 4C 57 45 35 31 49 6A 44 44 DhGi+P4LWE51IjDD
43 48 6F 59 42 4B 4C 6E 5A 4A 41 42 37 55 39 61 CHoYBKLnZJAB7U9a
44 31 7A 54 4E 52 31 47 42 31 65 58 73 50 35 4D D1zTNR1GB1eXsP5M
77 42 66 70 30 34 4C 5A 47 71 77 70 30 52 36 59 wBfp04LZGqwp0R6Y
4E 31 54 79 4E 4B 5A 47 74 65 71 42 4E 4F 51 37 N1TyNKZGteqBNOQ7
62 5A 59 76 39 46 55 76 45 76 72 47 66 38 64 62 bZYv9FUvEvrGf8db
6D 31 73 69 63 77 45 41 41 51 41 3D 00 m1sicwEAAQA=.
Mine:
000061: Bulk or Interrupt Transfer (DOWN), 2019-08-16 08:27:20,0783520 +0,0001867 (1. Device: SAMSUNG_Android (Galaxy series, misc. (MTP mode)))
Pipe Handle: Control PipeSend 0x15d bytes to the device
41 51 41 42 70 47 65 58 30 4F 38 79 6A 37 63 6B AQABpGeX0O8yj7ck
62 42 68 79 75 75 4B 76 6E 79 4F 59 44 55 49 56 bBhyuuKvnyOYDUIV
31 66 39 6A 59 4B 52 78 78 6B 38 4D 68 39 38 34 1f9jYKRxxk8Mh984
4B 7A 2F 75 46 69 55 61 54 47 68 70 59 6A 6D 2F Kz/uFiUaTGhpYjm/
7A 79 42 4A 75 5A 68 71 6E 6C 43 34 4A 6E 65 62 zyBJuZhqnlC4Jneb
75 31 6B 75 79 48 61 4F 69 38 65 71 34 63 6A 4A u1kuyHaOi8eq4cjJ
4F 79 62 63 5A 2F 4A 7A 56 43 4B 75 61 78 4D 36 OybcZ/JzVCKuaxM6
48 58 47 4B 59 70 39 74 47 63 51 6C 59 6F 36 63 HXGKYp9tGcQlYo6c
64 53 61 53 59 33 50 6D 49 47 38 74 6D 69 56 69 dSaSY3PmIG8tmiVi
61 62 4C 30 35 56 78 70 51 65 50 48 4E 37 32 2F abL05VxpQePHN72/
2B 64 37 75 73 48 36 52 54 41 35 2F 66 58 79 52 +d7usH6RTA5/fXyR
6B 54 78 33 69 2B 73 36 73 34 35 58 65 50 43 77 kTx3i+s6s45XePCw
65 32 5A 71 63 4E 44 2F 5A 61 4B 73 73 43 38 6F e2ZqcND/ZaKssC8o
4C 74 5A 4F 6A 54 35 4E 6D 44 58 34 4E 30 38 49 LtZOjT5NmDX4N08I
6C 49 30 71 61 76 46 31 2F 4F 49 64 38 65 4D 70 lI0qavF1/OId8eMp
35 64 38 4F 71 61 31 48 74 47 70 64 4D 7A 68 43 5d8Oqa1HtGpdMzhC
47 6B 69 49 2F 33 63 77 6A 48 39 71 61 58 39 5A GkiI/3cwjH9qaX9Z
76 4D 38 4F 61 6D 76 6C 76 68 4B 36 44 47 5A 39 vM8OamvlvhK6DGZ9
62 47 74 61 73 50 76 59 78 46 69 51 73 51 6A 46 bGtasPvYxFiQsQjF
6B 56 72 79 75 76 6D 6D 43 74 70 61 32 42 2B 4A kVryuvmmCtpa2B+J
78 6F 38 6F 54 37 6B 70 4B 31 69 6D 6A 45 73 6E xo8oT7kpK1imjEsn
74 7A 6E 44 75 73 49 62 4B 77 3D 3D 00 tznDusIbKw==.
Here is how i generate the key pair and data signing using BouncyCastle library:
public static string GetBase64Key(AsymmetricKeyParameter key)
{
var publicKey = (RsaKeyParameters)key;
using (MemoryStream ms = new MemoryStream())
{
byte[] buff = publicKey.Exponent.ToByteArrayUnsigned();
ms.Write(buff, 0, buff.Length);
buff = publicKey.Modulus.ToByteArrayUnsigned();
ms.Write(buff, 0, buff.Length);
ms.Flush();
return Convert.ToBase64String(ms.ToArray());
}
}
public static AsymmetricCipherKeyPair LoadKeyPair(string file)
{
using (StreamReader reader = new StreamReader(file))
{
PemReader pem = new PemReader(reader);
return pem.ReadObject() as AsymmetricCipherKeyPair;
}
}
public static void SaveKeyPair(AsymmetricCipherKeyPair pair, string file)
{
using (StreamWriter writer = new StreamWriter(file))
{
PemWriter pem = new PemWriter(writer);
pem.WriteObject(pair.Private);
pem.WriteObject(pair.Public);
}
}
public static AsymmetricCipherKeyPair GenerateKeyPair(int keySize)
{
CryptoApiRandomGenerator randomGenerator = new CryptoApiRandomGenerator();
SecureRandom secureRandom = new SecureRandom(randomGenerator);
var keyGenerationParameters = new KeyGenerationParameters(secureRandom, keySize);
var keyPairGenerator = new RsaKeyPairGenerator();
keyPairGenerator.Init(keyGenerationParameters);
return keyPairGenerator.GenerateKeyPair();
}
public static byte[] SignDataSHA1(byte[] data, AsymmetricKeyParameter privateKey)
{
ISigner signer = SignerUtilities.GetSigner("SHA1WITHRSA");
signer.Init(true, privateKey);
signer.BlockUpdate(data, 0, data.Length);
return signer.GenerateSignature();
}
And this is how i used those helper functions:
// Initialiizing key pair.
AsymmetricCipherKeyPair pair;
if (File.Exists(RSA_CONTAINER_NAME))
{
pair = CryptoHelper.LoadKeyPair(RSA_CONTAINER_NAME);
}
else
{
pair = CryptoHelper.GenerateKeyPair(RSA_KEY_STRENGTH);
CryptoHelper.SaveKeyPair(pair, RSA_CONTAINER_NAME);
}
// Signing token.
byte[] signedData = CryptoHelper.SignDataSHA1(token, pair.Private);
// Writing data.
Message msg = new Message();
msg.Command = Message.A_AUTH;
msg.Arg0 = 2; // Private key signing.
msg.Data = signedData;
msg.DataLength = signedData.Length;
await SendMessageAsync(device, msg);
// Reading response.
Message reply = await ReadMessageAsync(device);
byte[] replyBody = new byte[reply.DataLength];
await device.ReadAsync(replyBody, 0, replyBody.Length);
if (reply.Command == Message.A_AUTH)
{
// Sending rsa public key.
string pem = CryptoHelper.GetBase64Key(pair.Public) + "\0";
byte[] publicKey = Encoding.ASCII.GetBytes(pem);
msg = new Message();
msg.Command = Message.A_AUTH;
msg.Arg0 = 3; // RSAPUBLICKEY
msg.Data = publicKey;
msg.DataLength = publicKey.Length;
await SendMessageAsync(device, msg);
// Reading response.
reply = await ReadMessageAsync(device);
replyBody = new byte[reply.DataLength];
await device.ReadAsync(replyBody, 0, replyBody.Length);
}
if (reply.Command != Message.A_CNXN)
{
// Something went wrong.
throw new Exception("Authentication error");
}
// Done.
Any help would be highly appreciated.
Note: I have completed the steps until i got the token to sign from the device for brevity. I also have tried PEM format (without including the BEGIN and END lines), didn't work either.

I finally managed to fix my authentication problem. I decided to include pre generated public and private key to my app (not generated automatically anymore). I also installed bouncy castle library for C# to parse private key provided by adb. Public key problem fixed.
Turned out that my signing method is also wrong. Here is the correct one (using bouncy castle):
public static byte[] SignDataSHA1(byte[] data, AsymmetricKeyParameter privateKey)
{
// Converting bouncy castle key to native csp.
RSAParameters rsaParam = DotNetUtilities.ToRSAParameters(privateKey as RsaPrivateCrtKeyParameters);
using (RSACryptoServiceProvider rsa = new RSACryptoServiceProvider())
{
rsa.ImportParameters(rsaParam);
// Signing data.
return rsa.SignHash(data, CryptoConfig.MapNameToOID("SHA1"));
}
}
I basically just tell the signer that the data already hashed and voila, it worked.

C# Converting to ?? from Wireshark Response

I am struggling a bit here. I am sending an tcp request to an endpoint.
client = new TcpClient(serverIP, serverPort);
tcpStream = client.GetStream();
var stringtosend = #"43 50 43 52 00 01 19 00 00 00 1a 00 00 00
01 00 00 00 00 00 00 00 32 00 00 01";
stringtosend = stringtosend.Replace(" ", "");
var requestbytes = ConvertToByteArray(stringtosend);
tcpStream.Write(requestbytes, 0, requestbytes.Length);
List<int> ResponseBytes = new List<int>();
var sb = new StringBuilder();
do
{
var byteRead = tcpStream.ReadByte();
if (byteRead != -1)
{
sb.Append(byteRead);
}
else
{
break;
}
} while (tcpStream.DataAvailable);
This is the response as read from wireshark:
43 50 43 52 00 01 19 00 00 00 34 01 00 00 1B 01 00 00 0B 01 0B 01 3C 00 00 32 00 01 FF FF 0C 00 48 56 41 43 20 43 54 52 4C 00 02 02 40 01 46 0B 01 00 00 00 00 00 00 45 4D 2D 4D 38 38 49 23 31 00 11 00 00 00 00 81 01 00 00 00 00 00 00 41 43 31 4D 38 38 49 23 32 00 11 00 00 00 00 81 02 00 00 00 00 00 00 45 4D 2D 4D 38 38 4F 23 31 00 12 00 00 00 00 82 01 00 00 00 00 00 00 41 43 31 4D 38 38 4F 23 32 00 12 00 00 00 00 82 02 00 00 00 00 00 00 41 43 32 4D 52 54 55 23 31 00 0F 03 0B 01 46 90 01 00 00 00 00 00 00 41 43 33 4D 52 54 55 23 32 00 0F 03 0B 01 46 90 02 00 00 00 00 00 00 41 43 34 4D 52 54 55 23 33 00 0F 03 0B 01 46 90 03 00 00 00 00 00 00 41 43 35 4D 52 54 55 23 34 00 0F 03 0B 01 46 90 04 00 00 00 00 00 00 41 43 36 4D 52 54 55 23 35 00 0F 03 0B 01 46 90 05 00 00 00 00 00 00 52 41 43 4B 20 31 26 32 00 00 03 02 40 01 46 01 01 00 00 00 00 00 00 52 41 43 4B 20 33 26 34 00 00 03 02 40 01 46 02 01 00 00 00 00 00 00
CPCR......4...........<..2..ÿÿ..HVAC CTRL...#.F........EM-M88I#1.............AC1M88I#2.............EM-M88O#1.............AC1M88O#2.............AC2MRTU#1.....F.......AC3MRTU#2.....F.......AC4MRTU#3.....F.......AC5MRTU#4.....F.......AC6MRTU#5.....F.......RACK 1&2....#.F........RACK 3&4....#.F........
I am interested in how to convert what I believe to be a hex response into what is bolded above.
I'm new at this so please be gentle.

At the simplest you can do the following which was taken directly from
https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/types/how-to-convert-between-hexadecimal-strings-and-numeric-types
// Convert the number expressed in base-16 to an integer.
int value = Convert.ToInt32(hex, 16);
// Get the character corresponding to the integral value.
string stringValue = Char.ConvertFromUtf32(value);
Also I have used websites like the following
http://www.rapidtables.com/convert/number/hex-to-ascii.htm

I've just knocked this up as a console application, so you should be able to copy and paste as your Program.cs:
using System;
using System.Linq;
using System.Text;
using System.Text.RegularExpressions;
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
const string inputString = "43 50 43 52 00 01 19 00 00 00 34 01 00 00 1B 01 00 00 0B 01 0B 01 3C 00 00 32 00 01 FF FF 0C 00 48 56 41 43 20 43 54 52 4C 00 02 02 40 01 46 0B 01 00 00 00 00 00 00 45 4D 2D 4D 38 38 49 23 31 00 11 00 00 00 00 81 01 00 00 00 00 00 00 41 43 31 4D 38 38 49 23 32 00 11 00 00 00 00 81 02 00 00 00 00 00 00 45 4D 2D 4D 38 38 4F 23 31 00 12 00 00 00 00 82 01 00 00 00 00 00 00 41 43 31 4D 38 38 4F 23 32 00 12 00 00 00 00 82 02 00 00 00 00 00 00 41 43 32 4D 52 54 55 23 31 00 0F 03 0B 01 46 90 01 00 00 00 00 00 00 41 43 33 4D 52 54 55 23 32 00 0F 03 0B 01 46 90 02 00 00 00 00 00 00 41 43 34 4D 52 54 55 23 33 00 0F 03 0B 01 46 90 03 00 00 00 00 00 00 41 43 35 4D 52 54 55 23 34 00 0F 03 0B 01 46 90 04 00 00 00 00 00 00 41 43 36 4D 52 54 55 23 35 00 0F 03 0B 01 46 90 05 00 00 00 00 00 00 52 41 43 4B 20 31 26 32 00 00 03 02 40 01 46 01 01 00 00 00 00 00 00 52 41 43 4B 20 33 26 34 00 00 03 02 40 01 46 02 01 00 00 00 00 00 00";
var bytes = HexStringToBytes(inputString);
var readableString = Encoding.UTF7.GetString(bytes).ToCharArray();
var paddedReadableString = readableString.Select(ch => char.IsControl(ch) ? '.' : ch).ToArray();
var outputString = new String(paddedReadableString);
Console.WriteLine(outputString);
Console.ReadLine();
}
private static byte[] HexStringToBytes(string hex)
{
var strippedHex = Regex.Replace(hex, #"\s+", "");
byte[] raw = new byte[strippedHex.Length / 2];
for (int i = 0; i < raw.Length; i++)
{
raw[i] = Convert.ToByte(strippedHex.Substring(i * 2, 2), 16);
}
return raw;
}
}
}
For info, your desired format is UTF7.

You are reading your bytes direct into a StringBuilder.
The key line here is:
var byteRead = tcpStream.ReadByte();
You have an opportunity to convert the encoding here using this static method which will look something like:
// Create two different encodings.
Encoding utf8 = Encoding.UTF8;
Encoding unicode = Encoding.Unicode;
// Perform the conversion from one encoding to the other.
byte[] unicodeBytes = Encoding.Convert(utf8, unicode, new byte[]{byteRead} );

C# RegEx a value between numbers and characters?

Hello I want to search for the first occurrence of a specific string and capture a value between a set length of numbers and characters that change.
Using Nate Barbettini’s https://dotnetfiddle.net/vhkUV5 example I butchered it into doing what I almost need, it won’t compile and from what I’ve seen my RegEx is way off so I defiantly need help with it.
In my example I want to find the first occurrence of the PID value “116c” for chrome.exe and not all three PID values. What’s the best way to get one PID value?
Code:
using System;
using System.Collections.Generic;
using System.Text.RegularExpressions;
public class Program
{
public static void Main()
{
var output = #"
0a80 6e 6f 74 65 70 61 64 2b 2b 2e 65 78 65 20 20 20 notepad++.exe
0a90 50 49 44 3d 31 64 38 63 7c 30 37 35 36 34 0d 0a PID=1d8c|07564..
0aa0 6a 68 69 5f 73 65 72 76 69 63 65 2e 65 78 20 20 jhi_service.ex
0ab0 50 49 44 3d 31 38 64 34 7c 30 36 33 35 36 0d 0a PID=18d4|06356..
0ac0 4c 4d 53 2e 65 78 65 20 20 20 20 20 20 20 20 20 LMS.exe
0ad0 50 49 44 3d 31 63 36 38 7c 30 37 32 37 32 0d 0a PID=1c68|07272..
0ae0 63 6d 64 2e 65 78 65 20 20 20 20 20 20 20 20 20 cmd.exe
0af0 50 49 44 3d 30 66 37 38 7c 30 33 39 36 30 0d 0a PID=0f78|03960..
0b00 63 6f 6e 68 6f 73 74 2e 65 78 65 20 20 20 20 20 conhost.exe
0b10 50 49 44 3d 30 62 64 30 7c 30 33 30 32 34 0d 0a PID=0bd0|03024..
0b20 76 63 74 69 70 2e 65 78 65 20 20 20 20 20 20 20 vctip.exe
0b30 50 49 44 3d 31 38 30 38 7c 30 36 31 35 32 0d 0a PID=1808|06152..
0b40 63 68 72 6f 6d 65 2e 65 78 65 20 20 20 20 20 20 chrome.exe
0b50 50 49 44 3d 31 31 36 63 7c 30 34 34 36 30 0d 0a PID=116c|04460..
0b60 63 68 72 6f 6d 65 2e 65 78 65 20 20 20 20 20 20 chrome.exe
0b70 50 49 44 3d 31 36 39 34 7c 30 35 37 38 30 0d 0a PID=1694|05780..
0b80 63 68 72 6f 6d 65 2e 65 78 65 20 20 20 20 20 20 chrome.exe
0b90 50 49 44 3d 31 30 62 30 7c 30 34 32 37 32 0d 0a PID=10b0|04272..";
var regex = new Regex(#"chrome.exe[\s].................................................................(.*)........");
var resultList = new List<string>();
foreach (Match match in regex.Matches(output))
{
resultList.Add(match.Groups[1].ToString());
}
var pid = string.Join(", ", resultList);
Console.WriteLine(pid);
}
}
Output:
116c, 1694, 10b0
I’m extremely new so any help or pointers are welcome.

Try this code:
var regex = new Regex("chrome\\.exe\\s*.*PID=(.*)\\|");
var pid = regex.Matches(output)
.Cast<Match>()
.Select(match => match.Groups[1].ToString())
.First();
You can test the regular expression here.

Encoding/Decoding hex packet

I want to send this hex packet:
00 38 60 dc 00 00 04 33 30 3c 00 00 00 20 63 62
39 62 33 61 36 37 34 64 31 36 66 32 31 39 30 64
30 34 30 63 30 39 32 66 34 66 38 38 32 62 00 06
35 2e 31 33 2e 31 00 00 02 3c
so i build the string:
string packet = "003860dc0000" + textbox1.text+ "00000020" + textbox2.text+ "0006" + textbox3.text;
then "convert" it to ascii:
conn_str = HexString2Ascii(packet);
then i send the packet... but i have this:
00 38 60 **c3 9c** 00 00 04 33 30 3c 00 00 00 20 63
62 39 62 33 61 36 37 34 64 31 36 66 32 31 39 30
64 30 34 30 63 30 39 32 66 34 66 38 38 32 62 00
06 35 2e 31 33 2e 31 00 00 02 3c **0a**
why??
Thank you!
P.S.
the function is:
private string HexString2Ascii(string hexString)
{
byte[] tmp;
int j = 0;
int lenght;
lenght=hexString.Length-2;
tmp = new byte[(hexString.Length)/2];
for (int i = 0; i <= lenght; i += 2)
{
tmp[j] =(byte)Convert.ToChar(Int32.Parse(hexString.Substring(i, 2), System.Globalization.NumberStyles.HexNumber));
j++;
}
return Encoding.GetEncoding(1252).GetString(tmp);
}
EDIT:
if i convert directly in byte, the hex packet in coded as string:
00000000 30 30 33 38 36 30 64 63 30 30 30 30 30 34 33 33 003860dc 00000433
00000010 33 30 33 43 30 30 30 30 30 30 32 30 33 34 33 32 303C0000 00203432
00000020 36 33 36 33 33 35 33 39 33 32 33 34 36 36 33 39 63633539 32346639
00000030 36 33 33 39 33 31 33 39 33 30 33 36 33 33 36 35 63393139 30363365
00000040 33 35 36 33 36 35 36 35 36 35 33 31 33 39 33 38 35636565 65313938
00000050 36 33 33 31 36 34 33 34 36 33 33 30 30 30 30 36 63316434 63300006
00000060 33 35 32 65 33 31 33 33 32 65 33 31 30 30 30 30 352e3133 2e310000
00000070 30 32 33 43 023C

You cannot convert raw binary data to string data and expect things to just work. They are not the same. This is especially true when you mix up your character encodings.
C# characters are not ASCII characters. They are Unicode characters, represented by Unicode code points. When you then turn around and write those characters out, you need to specify what kind of data to write out. When you read your byte array into a string, using Encoding.GetEncoding(1252), you are getting the characters corresponding to code page 1252, in which 0xdc is a Ü.
But when your string is being converted back into bytes to send over the network, it is being written out as UTF-8. In UTF-8, UTF-00DC cannot be encoded as a single byte, since that byte value is used to indicate the start of a multi-byte sequence. Instead, it's encoded as the multi-byte sequence 0xc3 0x9c. As far as C# is concerned, those two values are the same character. (I don't know where that extra 0x0a is coming from, but my guess is an errant line feed from one of your text boxes and/or some other part of your process).
Its not clear what exactly you're trying to do, but I suspect you are converting way too many times for it to work out correctly. If you know the byte sequence you want to send, why not just encode that as a byte[] directly? For example, use a MemoryStream and write the constant bytes you need into it.
To get the values out of your text boxes, your original code to "convert" the string of hex digits into a string of ASCII characters had the right idea. You just need to stop at the point where you have a byte array, since ultimately the byte array is what you want.
public byte[] GetBytesFrom(string hex)
{
var length = hex.Length / 2;
var result = new byte[length];
for (var i = 0; i < length; i++)
{
result[i] = byte.Parse(hex.Substring(i, 2), NumberStyles.HexNumber);
}
return result;
}
// Variable portions of packet structure.
var byte[] segment2 = GetBytesFrom(textbox1.Text);
var byte[] segment4 = GetBytesFrom(textbox2.Text);
var byte[] segment6 = GetBytesFrom(textbox3.Text);
MemoryStream output = new MemoryStream();
output.Write(new[] { 0x00, 0x38, 0x60, 0xdc, 0x00, 0x00 }, 0, 6);
output.Write(segment2, 0, segment2.Length);
output.Write(new[] { 0x00, 0x00, 0x00, 0x20 }, 0, 4);
output.Write(segment4, 0, segment4.Length);
output.Write(new[] { 0x00, 0x06 }, 0, 2);
output.Write(segment6, 0, segment6.Length);
From here, you could use MemoryStream.CopyTo() to copy it to another stream, or MemoryStream.Read() to read the entire packet into a new byte array, or MemoryStream.GetBuffer() to get the underlying buffer (though that last one is rarely what you want -- it includes unused padding bytes)