We Keep Coding

C# (.NET), how to fix web response performance, too much elapsed time

I'm need to optimize my code, elapsed time for response 1sec, elapsed time for ReadToEnd() it - 0.5sec, all other code (with full web request) take only 0.1sec.
request2.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
//This 1sec+
HttpWebResponse response = (HttpWebResponse)request2.GetResponse();
StreamReader sr = new StreamReader(response.GetResponseStream(), Encoding.UTF8);
//This 0.5sec+
string mem = sr.ReadToEnd();
sr.Close();
P.S: Html code about 200k+ chars, but i need only 4-5.

Most likely, having not set the proxy settings for your HttpWebResponse is causing the delay. It is always recommended to set the proxy, even if its not used. Also try using the using clause:
request2.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
request2.Proxy = null;
using (HttpWebResponse response = (HttpWebResponse)request2.GetResponse())
{
StreamReader sr = new StreamReader(response.GetResponseStream(), Encoding.UTF8);
string mem = sr.ReadToEnd();
}

The initial GetResponse call generates a security negotiation, Get and excecute, then an HTTP response whose first few bytes include the Http Response Code (200,500,404 etc).
The body of the response may already have been received by your client Http buffer or still be being streamed into it - you can't really tell. Your second call (readToEnd) reads all the bytes in the receive buffer and waits until the server has sent all the bytes indicated in the Http Header.
Your code is very unlikely to be causing any appreciable cost to the execution time of the web service call, and I can't see any likely optmisation steps - you need to determine how long the call takes without your client code.
Use Telerik Fiddler to track the number of bytes being called from the destination web service, and the amount of time the raw transfer from sever to client takes - do this by simply calling the URL within Fiddler or on a web browser. This will isolate whether its your code, or the server, or the connection latency costing time.

To add on top of other's suggestions of using using and setting proxy to null, you might also want to try to only read a specific amount of chars, instead of reading everything.
var content = new char[10];
sr.Read(content, 0, content.Length);
string contentStr = new String(content);

Check your js scripts doesnt depend on another services, repositories etc. It was worked for me before in my project. Download them and move it to your local folders

Fastest Way To Download Website Data C#

I am writing an Rcon in Visual Studio for Black Ops. I know its an old game but I still have a server running.
I am trying to download the data from this link
Black Ops Log File
I am using this code.
System.Net.WebClient wc = new System.Net.WebClient();
string raw = wc.DownloadString(logFile);
Which take between 6441ms to 13741ms according to Visual Studio.
Another attempt was...
string html = null;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(logFile);
request.AutomaticDecompression = DecompressionMethods.GZip;
request.Proxy = null;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
using (Stream stream = response.GetResponseStream())
using (StreamReader reader = new StreamReader(stream))
{
html = reader.ReadToEnd();
}
Which also takes around 6133ms according to VS debugging.
I have seen other rcon respond to commands really quickly. Mine take on best 5000ms which is not really acceptable. How can I download this this information quicker. I am told it shouldn't take this long??? What am I doing wrong?

This is just how long the server takes to answer:
In the future you can debug such problems yourself using network tools such as Fiddler or by profiling your code to see what takes the longest amount of time.

Getting JSON data from a response stream and reading it as a string?

I am trying to read a response from a server that I receive when I send a POST request. Viewing fiddler, it says it is a JSON response. How do I decode it to a normal string using C# Winforms with preferably no outside APIs. I can provide additional code/fiddler results if you need them.
The fiddler and gibberish images:
The gibberish came from my attempts to read the stream in the code below:
Stream sw = requirejs.GetRequestStream();
sw.Write(logBytes, 0, logBytes.Length);
sw.Close();
response = (HttpWebResponse)requirejs.GetResponse();
Stream stream = response.GetResponseStream();
StreamReader sr = new StreamReader(stream);
MessageBox.Show(sr.ReadToEnd());

As mentioned in the comments, Newtonsoft.Json is really a good library and worth using -- very lightweight.
If you really want to only use Microsoft's .NET libraries, also consider System.Web.Script.Serialization.JavaScriptSerializer.
var serializer = new System.Web.Script.Serialization.JavaScriptSerializer();
var jsonObject = serializer.DeserializeObject(sr.ReadToEnd());

Going to assume (you haven't clarified yet) that you need to actually decode the stream, since A) retrieving a remote stream of text is well documented, and B) you can't do anything much with a non-decoded JSON stream.
Your best course of action is to implement System.Web.Helpers.Json:
using System.Web.Helpers.Json
...
var jsonObj = Json.Decode(jsonStream);

Reading information from a website c#

In the project I have in mind I want to be able to look at a website, retrieve text from that website, and do something with that information later.
My question is what is the best way to retrieve the data(text) from the website. I am unsure about how to do this when dealing with a static page vs dealing with a dynamic page.
From some searching I found this:
WebRequest request = WebRequest.Create("anysite.com");
// If required by the server, set the credentials.
request.Credentials = CredentialCache.DefaultCredentials;
// Get the response.
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
// Display the status.
Console.WriteLine(response.StatusDescription);
Console.WriteLine();
// Get the stream containing content returned by the server.
using (Stream dataStream = response.GetResponseStream())
{
// Open the stream using a StreamReader for easy access.
StreamReader reader = new StreamReader(dataStream, Encoding.UTF8);
// Read the content.
string responseString = reader.ReadToEnd();
// Display the content.
Console.WriteLine(responseString);
reader.Close();
}
response.Close();
So from running this on my own I can see it returns the html code from a website, not exactly what I'm looking for. I eventually want to be able to type in a site (such as a news article), and return the contents of the article. Is this possible in c# or Java?
Thanks

I hate to brake this to you but that's how webpages looks, it's a long stream of html markup/content. This gets rendered by the browser as what you see on your screen. The only way I can think of is to parse to html by yourself.
After a quick search on google I found this stack overflow article.
What is the best way to parse html in C#?
but I'm betting you figured this would be a bit easier than you expected, but that's the fun in programming always challenging problems

You can just use a WebClient:
using(var webClient = new WebClient())
{
string htmlFromPage = webClient.DownloadString("http://myurl.com");
}
In the above example htmlFromPage will contain the HTML which you can then parse to find the data you're looking for.

What you are describing is called web scraping, and there are plenty of libraries that do just that for both Java and C#. It doesn't really matter if the target site is static or dynamic since both output HTML in the end. JavaScript or Flash heavy sites on the other hand tend to be problematic.

Please try this,
System.Net.WebClient wc = new System.Net.WebClient();
string webData = wc.DownloadString("anysite.com");

"An error occurred while parsing EntityName" while Loading an XmlDocument

I have written some code to parse RSS feeds for a ASP.NET C# application and it works fine for all RSS feeds that I have tried, until I tried Facebook.
My code fails at the last line below...
WebRequest request = WebRequest.Create(url);
WebResponse response = request.GetResponse();
Stream rss = response.GetResponseStream();
XmlDocument xml = new XmlDocument();
xml.Load(rss);
...with the error "An error occurred while parsing EntityName. Line 12, position 53."
It is hard to work out what is at thhat position of the XML file as the entire file is all in one line, but it is straight from Facebook and all characters appear to be encoded properly except possibly one character (♥).
I don't particularly want to rewrite my RSS parser to use a different method. Any suggestions for how to bypass this error? Is there a way of turning off checking of the file?

Look at the downloaded stream. It doesn't contain the RSS feed, but a HTML page with message about incompatible browser. That's because when downloading the URL like this, the user agent header is not set. If you do that, your code should work:
var request = (HttpWebRequest)WebRequest.Create(url);
request.UserAgent = "MyApplication";
var xml = new XmlDocument();
using (var response = request.GetResponse())
using (var rss = response.GetResponseStream())
{
xml.Load(rss);
}