We Keep Coding

Using TcpClient on websocket receive data get \0\0\0\0

I have establish a connection with a websocket , i want to receive message from it. Following is my code for receiving message from the websocket.
//mClient is my TCP connection
byte[] bytes;
NetworkStream netStream;
string returndata;
while(true)
{
bytes = new byte[mClient.ReceiveBufferSize];
netStream = mClient.GetStream();
netStream.Read(bytes, 0, (int)mClient.ReceiveBufferSize);
returndata = Encoding.UTF8.GetString(bytes);
Console.WriteLine("This is what the host returned to you: " + returndata);
}
The data should be some json array when I open with browser , but i have receive weird data like
??\0\0\0\0\0\0\0\0\0\0\
And the second loop onwards is forever
\0\0\0\0\0\0\0\0\0\0\0
I have seen a Similar Question but i have no idea on his answer. May I know how to fix this thing and what is the problem ?

Just read the stream with a StreamReader instead of fiddling with array buffers and the encoding by yourself:
//mClient is my TCP connection
StringBuilder returndata = new StringBuilder();
Console.Write("This is what the host returned to you: ");
// the StreamReader handles the encoding for you
using(var sr = new StreamReader(mClient.GetStream(), Encoding.UTF8))
{
int value = sr.Read(); // read an int
while(value != -1) // -1 means, we're done
{
var ch = (char) value; // cast the int to a char
Console.Write(ch); // print it
returndata.Append(ch); // keep it
value = sr.Read(); // read next char
}
}
Console.WriteLine(" done.");
capture the result in a StringBuilder so you can convert that to a string if the loop ends (based on whatever condition that will be)

It won't work like that. WebSockets uses a framing protocol that you have to parse. Your JSON payload will be wrapped in one or multiple frames you need to read and parse.
https://www.rfc-editor.org/rfc/rfc6455#section-5.2

c# webClient Upload String cuts off

I need some help with the C# WebClient UploadString Method. I'm trying to upload a long string (that I read from a database) to a server (PHP) and I'm currently trying to do that with the UploadString Method because it seemed to be the easiest. The problem that I have is that the string that I upload gets cut off after about 4000 characters and I can't figure out why.
For Example:
data.length: 19000 (before Upload)
Post.length: 4000 (in PHP)
What I did to bypass this problem: I upload my string in pieces of less than 4000 characters. BUT I still face the problem! Every second upload gets cut off and I can't figure out why.
This is my C# Code:
WebClient client = new WebClient();
foreach (DataRow dr in dra)
{
foreach (int y in index)
{
data += dr[y] + ";";
Console.Write(".");
}
data += ":";
if (count1 > 50)
{
// Upload the data.
Console.WriteLine("Uploading Data.....");
Console.WriteLine("Länge des Strings:" + data.Length);
Console.WriteLine(data);
client.Dispose();
client.Encoding = System.Text.Encoding.UTF8;
client.Headers[HttpRequestHeader.ContentType] = "application/x-www-form-urlencoded";
string Ergebnis = client.UploadString(address, "POST", data);
Console.WriteLine(Ergebnis);
client.Dispose();
result.ErrorMessage += Ergebnis;
count1 = -1;
data = "table="+table+"&columns=continueUpload&values=";
}
++count1;
}
Does anyone have any idea where this comes from? Is there any string limit on the webclient method?

Alright, I found the solution, thanks Alex for the hint!
I had to urlencode all my values, than it worked!

How to cancel large file download yet still get page source in C#?

I'm working in C# on a program to list all course resources for a MOOC (e.g. Coursera). I don't want to download the content, just get a listing of all the resources (e.g. pdf, videos, text files, sample files, etc...) which are made available to the course.
My problem lies in parsing the html source (currently using HtmlAgilityPack) without downloading all the content.
For example, if you go to this intro video for a banking course on Coursera and check the source (F12 in Chrome for Developer Tools), you can see the page source. I can stop the video download which autoplays, but still see the source.
How can I get the source in C# without download all the content?
I've looked in the HttpWebRequest headers (problem: time out), and DownloadDataAsync with Cancel (problem: the Completed Result object is invalid when cancelling the async request). I've also tried various Loads from HtmlAgilityPack but with no success.
Time out:
HttpWebRequest postRequest = (HttpWebRequest)WebRequest.Create(url);
postRequest.Timeout = TIMEOUT * 1000000; //Really long
postRequest.Referer = "https://www.coursera.org";
if (headers != null)
{ //headers here }
//Deal with cookies
if (cookie != null)
{ cookieJar.Add(cookie); }
postRequest.CookieContainer = cookiejar;
postRequest.Method = "GET";
postRequest.AllowAutoRedirect = allowRedirect;
postRequest.ServicePoint.Expect100Continue = true;
HttpWebResponse postResponse = (HttpWebResponse)postRequest.GetResponse();
Any tips on how to proceed?

There are at least two ways to do what you're asking. The first is to use a range get. That is, specify the range of the file you want to read. You do that by calling AddRange on the HttpWebRequest. So if you want, say, the first 10 kilobytes of the file, you'd write:
request.AddRange(-10240);
Read carefully what the documentation says about the meaning of that parameter. If it's negative, it specifies the ending point of the range. There are also other overloads of AddRange that you might be interested in.
Not all servers support range gets, though. If that doesn't work, you'll have to do it another way.
What you can do is call GetResponse and then start reading data. Once you've read as much data as you want, you can stop reading and close the stream. I've modified your sample slightly to show what I mean.
string url = "https://www.coursera.org/course/money";
HttpWebRequest postRequest = (HttpWebRequest)WebRequest.Create(url);
postRequest.Method = "GET";
postRequest.AllowAutoRedirect = true; //allowRedirect;
postRequest.ServicePoint.Expect100Continue = true;
HttpWebResponse postResponse = (HttpWebResponse) postRequest.GetResponse();
int maxBytes = 1024*1024;
int totalBytesRead = 0;
var buffer = new byte[maxBytes];
using (var s = postResponse.GetResponseStream())
{
int bytesRead;
// read up to `maxBytes` bytes from the response
while (totalBytesRead < maxBytes && (bytesRead = s.Read(buffer, 0, maxBytes)) != 0)
{
// Here you can save the bytes read to a persistent buffer,
// or write them to a file.
Console.WriteLine("{0:N0} bytes read", bytesRead);
totalBytesRead += bytesRead;
}
}
Console.WriteLine("total bytes read = {0:N0}", totalBytesRead);
That said, I ran this sample and it downloaded about 6 kilobytes and stopped. I don't know why you're having trouble with timeouts or too much data.
Note that sometimes trying to close the stream before the entire response is read will cause the program to hang. I'm not sure why that happens at all, and I can't explain why it only happens sometimes. But you can solve it by calling request.Abort before closing the stream. That is:
using (var s = postResponse.GetResponseStream())
{
// do stuff here
// abort the request before continuing
postRequest.Abort();
}

HttpWebRequest Slows with multiple instances of application

trying to get to the bottom of this!
i have a very basic app that is using httpwebrequests to login, navigate to a page and then grab the html of that page. it then preforms another webrequest to a third page every 5 mins in a loop.
its all working fine and is single threaded (and fairly old), however circumstances have changed and i now need to run multiple instances of this app closely together (i have a .bat starting the app every 2seconds as a temporary measure until i am able to code a new multithreaded solution).
when the first instances of the app start everything is fine, first request is completed in ~2seconds. second one in about 3seconds.
however as more and more instances of this app are run concurrently (>100) something strange starts to happen.
the first web request still takes ~2 seconds, however the second request gets delayed much more >1min up to the point of timeout. i cant seem to think why this is. the second page is larger than the first, but nothing out of the ordinary that would take >1min to download.
The internet connection and hardware of this server is more than capable of handling these requests.
CookieContainer myContainer = new CookieContainer();
// first request is https
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(https://mysite.com/urlone);
request.CookieContainer = myContainer;
request.Proxy = proxy;
Console.WriteLine(System.DateTime.Now.ToLongTimeString() + " " + "Starting login request");
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream resStream = response.GetResponseStream();
string tempString = null;
int count = 0;
do
{
// fill the buffer with data
count = resStream.Read(buf, 0, buf.Length);
// make sure we read some data
if (count != 0)
{
// translate from bytes to ASCII text
tempString = Encoding.ASCII.GetString(buf, 0, count);
// continue building the string
sb.Append(tempString);
}
}
while (count > 0); // any more data to read?
sb.Clear();
response.Close();
resStream.Close();
string output6;
Console.WriteLine(System.DateTime.Now.ToLongTimeString() + " " + "login request comeplete");
HttpWebRequest request6 = (HttpWebRequest)WebRequest.Create(#"http://mysite.com/page2");
request6.CookieContainer = myContainer;
response = (HttpWebResponse)request6.GetResponse();
resStream = response.GetResponseStream();
tempString = null;
count = 0;
do
{
count = resStream.Read(buf, 0, buf.Length);
if (count != 0)
{
tempString = Encoding.ASCII.GetString(buf, 0, count);
sb.Append(tempString);
}
}
while (count > 0);
output6 = sb.ToString();
sb.Clear();
response.Close();
resStream.Close();
Any ideas? Im not very advanced with http web requests so if someone could check i haven't made any silly code mistakes above id appreciate it. Im at a loss as to what other information i may need to include here, if i have missed anything out please tell me and i will do my best to provide.
Thanks in advance.
EDIT 1:
I used fiddler to find out the source of the issue. It looks like the issue lies with the application (or windows) not sending the requests for some reason - the physical request actually takes < 1second according to fiddler.

Check out a few things
ServicePointManager.DefaultConnectionLimit : if you are planning to open more then 100 connection the set this value to something like 200-300.
If possible use HttpWebRequest.KeepALive = true

Try wrapping your request into a using directive to make sure it's always properly closed. If you hit the max number of connections, you otherwise have to wait for the earlier ones to time out before new ones connect.

Retrieve POST data sent from a C# application in PHP

Having stumbled upon a problem when doing this, I first searched SO to try and find if others were having similar problems, and found this question: POST data to a PHP page from C# WinForm
However, when I tried the code example given in the answer to this question, it does not work. The PHP script that I am making a request to responds with a message indicating that the POST variable that has to be set is not set. Here's my C# code:
HttpWebRequest POSTreq =
(HttpWebRequest)WebRequest.Create("http://82.41.255.140/api/post-ptr");
string POSTdata = "action=" + HttpUtility.UrlEncode("date");
byte[] data = Encoding.ASCII.GetBytes(POSTdata);
POSTreq.Method = "POST";
POSTreq.ContentType = "application/x-www-form-urlencoded";
POSTreq.ContentLength = data.LongLength;
POSTreq.GetRequestStream().Write(data, 0, data.Length);
HttpWebResponse POSTres = (HttpWebResponse)POSTreq.GetResponse();
Console.WriteLine("HTTP Status Code {0}", POSTres.StatusCode);
Console.WriteLine("Response Method: {0}", POSTres.Method);
Console.WriteLine("Response Text: {0}",
new StreamReader(POSTres.GetResponseStream()).ReadToEnd());
Console.ReadLine();
And this is the code inside the PHP script:
<?php
$A = strtolower($_POST["action"]);
if ($A == "date")
{
echo date("c");
}
else if ($A == "ip")
{
echo $_SERVER["REMOTE_ADDR"];
}
else if ($A == null || $A == "")
{
echo "bad_request:no_argument:POST_action";
}
else
{
echo "bad_request:invalid_argument:POST_action";
}
exit();
?>
When I make the POST request from my C# program, I see the following screen, indicating that the variable action has not been set. Am I missing the obvious in my code?
Thanks to those who reply.

You might need to flush the stream. I usually do it like this:
string POSTdata = "action=" + HttpUtility.UrlEncode("date");
byte[] data = Encoding.ASCII.GetBytes(POSTdata);
POSTreq.Method = "POST";
POSTreq.ContentType = "application/x-www-form-urlencoded";
POSTreq.ContentLength = data.LongLength;
using (Stream stream = POSTreq.GetRequestStream()) {
stream.Write(data, 0, data.Length);
stream.Flush();
}

You're not closing the request stream. See this example in the MSDN docs, which is very close to your code.
EDIT
Your PHP null check is incorrect also. See this article.

I've found the problem.
It turns out I'd left off the trailing / in the request, and the request was getting 301'd, which for whatever reason removed the post variables.
So http://82.41.255.140/api/post-ptr should have been http://82.41.255.140/api/post-ptr/.
Thanks to everyone who answered, I've +1'd you all.