We Keep Coding

Getting the HTML code of a webpage

I am trying to get the HTML code of a webpage using it's url. I have written the following code, it works, but comparing the resulting string it doesn't match the code I see when I use google chrome's inspect. I am not an HTML gru, but it seems to be different.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://fantasy.premierleague.com/a/leagues/standings/517292/classic");
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
StreamReader stream = new StreamReader(response.GetResponseStream(), Encoding.GetEncoding(response.CharacterSet));
string PageScript = stream.ReadToEnd();
The resulting script is as follows: https://ideone.com/DXzfKy
I am using those two lines to set the security protocol
ServicePointManager.Expect100Continue = true;
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
If someone can tell me what am I looking at and what might be wrong, I will be grateful.

All you need to do is to create an instance of a WebClient and using that you can read the data from URI, than convert it into StreamReader and finally in Plain Text Format.
WebClient client = new WebClient();
Stream dataFromPage = client.OpenRead(new Uri("https://ideone.com/DXzfKy"));
StreamReader reader = new StreamReader(dataFromPage);
string htmlContent = reader.ReadToEnd();

C# Invalid URI: The URI is empty [closed]

Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 7 years ago.
Improve this question
Hello i'm trying to collect all data from website.
On start i collecting all links on that website. After that accessing them.
The problem is when i accessing multiple pages of website the "Invalid URI: The URI is empty" error is showing in some pages. These are some of those failed urls.
http://syayoyu.com/category/zensinyokudiet
http://syayoyu.com/fruitgranola-6759.html
http://syayoyu.com/category/diet/fruitgranola
Accessing from browser and accessing them one by one is working ok. But when i try to access in loop the error is occuring. Would you please tell me what is wrong with it.
This is my accessing code
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
request.UserAgent = "A .NET Web Crawler";
WebResponse response = request.GetResponse();
Stream stream = response.GetResponseStream();
StreamReader reader = new StreamReader(stream);
string htmlText = reader.ReadToEnd();

List<string> list = new List<string>(){
#"http://syayoyu.com/category/zensinyokudiet",
#"http://syayoyu.com/fruitgranola-6759.html",
#"http://syayoyu.com/category/diet/fruitgranola"
};
foreach (var url in list)
{
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
request.UserAgent = "A .NET Web Crawler";
WebResponse response = request.GetResponse();
//Stream stream = response.GetResponseStream();
//StreamReader reader = new StreamReader(stream);
//string htmlText = reader.ReadToEnd();
//Console.WriteLine(htmlText);
using (Stream stream = response.GetResponseStream())
{
using (StreamReader reader = new StreamReader(stream))
{
string htmlText = reader.ReadToEnd();
Console.WriteLine(htmlText);
}
}
}
In my solution, it works fine,so is there any questions you don't know?

How to handle and get the body from POST using .NET C#?

So, I know how to get form data out by using Request.Form["abc"] however how would I go by getting the body out?
I've used this snippet in the below link:
https://gist.github.com/leggetter/769688
But, I'm not sure what to pass in as the Response.
In PHP to do this: file_get_contents('php://input'); and it's as simple as that.
Notes: The content type of the POST is application/json and the body contains the json string.
Thanks in advance.

If I understood your question correctly, here's what you could do when posting to a resource for which you expect a JSON response:
HttpWebRequest httpWebRequest = (HttpWebRequest)WebRequest.Create("http://foo.com/bar/");
httpWebRequest.Method = WebRequestMethods.Http.Post;
httpWebRequest.Accept = "application/json";
HttpWebResponse response = (HttpWebResponse)httpWebRequest.GetResponse();
Stream receiveStream = response.GetResponseStream();
StreamReader readStream = new StreamReader (receiveStream, Encoding.UTF8);
//store json in a variable for later use
string json = readStream.ReadToEnd();
//make sure to close
response.Close();
readStream.Close();
Of course, this approach is synchronous; but, depending on your requirements, this might be just fine.
Since your question didn't specify whether you need to know how to parse the JSON itself, so I've left out an example of JSON parsing.

You'll need code similar to this to read the raw request body into a string variable.
using (StreamReader reader = new StreamReader(HttpContext.Current.Request.InputStream))
{
string text = reader.ReadToEnd();
}

webclient in c# not get correct result html [closed]

Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 7 years ago.
Improve this question
I used webclient in c# to fetch html result but it is incorrect.
This is my code snippet.
WebClient client = new WebClient();
string htmlCode = client.DownloadString("https://kat.cr/");
MessageBox.Show(htmlCode);
And the result is not what is seen on the HTML page.
‹ í}ksÛ8²èçɯÀjÏnì›#âS¢òÚ’_I6vâµ<ÉÌÉM¥ ’8¦H–•=ûáVÝ_zÉíI=lÓ&dFÊNmjÆ’H 
4ÝFãÅŸ>ìŸÿzzHÆÉÄ{õè~«‰çÇ/ã$ ŸµZÓé´9Õ›A4j©Ýn·u…eÄq£—
/‰X‹3çÕ#ÿ^LxÂÖ¤ü÷Ô½|ÙØü„û =Ÿ…¼Aìì×ËF¯’BzNì1‹bž¼L“!µ­û õ“™ÇogÇñjuŸMøËƥ˧a%Kŧ®“Œ_:üÒµ9?ž×w—y4¶™Ç_ªM¥AnæðØŽÜ0q ^Ÿ³Èæ;Ð/±Æ1ñùTü²Ë rNÎ?’xLcòWñ•G.Ÿ’Ip™}¦±k‹
À7ç„$nâñWïzç„’w®}Á öyE€7~ÑÊÞf%=׿ ÷^6bì¢xÌ9Ðœ#G-õG|ø²Ñj „%®Ý´ƒI‹yí¨ŠcèNS”*Ƥ‘
What should I do so that I can retrieve the actual page?

the html code is compressed.. set the AutomaticDecompression to GZip.
Just try this code and it will work (tested it with youe webpage)
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(urlAddress);
request.AutomaticDecompression = DecompressionMethods.GZip;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
if (response.StatusCode == HttpStatusCode.OK)
{
Stream receiveStream = response.GetResponseStream();
StreamReader readStream = null;
if (response.CharacterSet == null)
readStream = new StreamReader(receiveStream);
else
readStream = new StreamReader(receiveStream, Encoding.GetEncoding(response.CharacterSet));
string data = readStream.ReadToEnd();
response.Close();
readStream.Close();
}
Since you asked for a "webClient" Solution (see comment):
Because the webClient implementation is without decompression you will have to make your own DecompressedWebClient. Pretty easy if you inherit from the WebClient class:
public class AutomaticDecompressionWebClient : WebClient
{
protected override WebRequest GetWebRequest(Uri address)
{
var request = base.GetWebRequest(address) as HttpWebRequest;
if (request == null) throw new InvalidOperationException("You can not use this WebClient implementation with an address that is not an http uri.");
request.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
return request;
}
}

How to create JSON post to api using C# [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 9 years ago.
Improve this question
I'm in the process of creating a C# console application which reads text from a text file, turns it into a JSON formatted string (held in a string variable), and needs to POST the JSON request to a web api. I'm using .NET Framework 4.
My struggle is with creating the request and getting the response, using C#. What is the basic code that is necessary? Comments in the code would be helpful. What I've got so far is the below, but I'm not sure if I'm on the right track.
//POST JSON REQUEST TO API
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("POST URL GOES HERE?");
request.Method = "POST";
request.ContentType = "application/json";
System.Text.UTF8Encoding encoding = new System.Text.UTF8Encoding();
byte[] bytes = encoding.GetBytes(jsonPOSTString);
request.ContentLength = bytes.Length;
using (Stream requestStream = request.GetRequestStream())
{
// Send the data.
requestStream.Write(bytes, 0, bytes.Length);
}
//RESPONSE HERE

Have you tried using the WebClient class?
you should be able to use
string result = "";
using (var client = new WebClient())
{
client.Headers[HttpRequestHeader.ContentType] = "application/json";
result = client.UploadString(url, "POST", json);
}
Console.WriteLine(result);
Documentation at
http://msdn.microsoft.com/en-us/library/system.net.webclient%28v=vs.110%29.aspx
http://msdn.microsoft.com/en-us/library/d0d3595k%28v=vs.110%29.aspx

Try using Web API HttpClient
static async Task RunAsync()
{
using (var client = new HttpClient())
{
client.BaseAddress = new Uri("http://domain.com/");
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
// HTTP POST
var obj = new MyObject() { Str = "MyString"};
response = await client.PostAsJsonAsync("POST URL GOES HERE?", obj );
if (response.IsSuccessStatusCode)
{
response.//.. Contains the returned content.
}
}
}
You can find more details here Web API Clients