I want to get the HTML code from http://www.w3schools.com/
Here is my code:
static void Main(string[] args)
{
TcpClient client = new TcpClient("www.w3schools.com", 80);
client.SendTimeout = 3000;
client.ReceiveTimeout = 3000;
StreamWriter writer = new StreamWriter(client.GetStream());
StreamReader reader = new StreamReader(client.GetStream());
writer.WriteLine("GET www.w3schools.com HTTP/1.1");
writer.WriteLine("Host: www.w3schools.com");
writer.WriteLine();
writer.Flush();
string response = reader.ReadToEnd();
Console.WriteLine("Got Response: {0}", response);
Console.ReadLine();
}
But I get the following:
Where I'm wrong?
The second element of the GET line should be the query path, not the domain name. This should work:
writer.WriteLine("GET / HTTP/1.1");
writer.WriteLine("Host: www.w3schools.com");
For this TcpClient to work you need to have a Wporking TcpServer.
The correct uri should be like ("https://www.w3schools.com/html/default.asp") where after .com the file name will b provided.
The below code will work even without TcpServer.
public static void getSavedHtmlCode()
{
string html = string.Empty;
try
{
var request = System.Net.HttpWebRequest.Create(string.Format("{0}", "https://www.w3schools.com/html/default.asp"));
request.Method = "GET";
var response = (HttpWebResponse)request.GetResponse();
//prepare as html
//html = new StreamReader(response.GetResponseStream()).ReadToEnd();
// Get the stream associated with the response.
Stream receiveStream = response.GetResponseStream();
// Pipes the stream to a higher level stream reader with the required encoding format.
StreamReader readStream = new StreamReader(receiveStream, Encoding.UTF8);
//prepare as html
html = readStream.ReadToEnd();
Console.WriteLine("Response stream received.");
Console.WriteLine(html);
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
}
Related
I'm new to ASP.NET development and the use of webhooks.
I'm using ASP.NET and an Azure Automation Account which has a webhook. I can currently execute the webhook, however I would like to have my code wait until it receives the output of the webhook. How best to do this?
ASP.NET Code:
public ActionResult UpdateAll()
{
(random db calls)
string jsonList = JsonConvert.SerializeObject(userEnvironmentList);
try
{
string uri = "webhook_url";
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(uri);
string data = jsonList;
request.Method = "POST";
request.ContentType = "text/plain;charset=utf-8";
System.Text.UTF8Encoding encoding = new System.Text.UTF8Encoding();
byte[] bytes = encoding.GetBytes(data);
request.ContentLength = bytes.Length;
using (Stream requestStream = request.GetRequestStream())
{
requestStream.Write(bytes, 0, bytes.Length);
}
request.BeginGetResponse((x) =>
{
using (HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(x))
{
using (Stream stream = response.GetResponseStream())
{
StreamReader reader = new StreamReader(stream, Encoding.UTF8);
String responseString = reader.ReadToEnd();
}
}
}, null);
}
catch (Exception ex)
{
throw ex;
}
return View();
}
PS in Automation Account:
param
(
[Parameter (Mandatory = $false)]
[object] $WebhookData
)
if ($WebhookData) {
return "Finally this works"
}
Call GetResponse in a synchronized context then.
Change ...
request.BeginGetResponse((x) =>
{
using (HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(x))
{
using (Stream stream = response.GetResponseStream())
{
StreamReader reader = new StreamReader(stream, Encoding.UTF8);
String responseString = reader.ReadToEnd();
}
}
}, null);
To ...
var response = (HttpWebResponse)webRequest.GetResponse();
using (Stream stream = response.GetResponseStream())
{
StreamReader reader = new StreamReader(stream, Encoding.UTF8);
String responseString = reader.ReadToEnd();
}
note: GetResponse() waits till the underlying web request finishes, then it returns the result. no need to use BeginGetResponse() in your code context.
My remote server is throwing a web exception as bad request. But I know there is more information included in the error than what I get. If I look at the details from the exception it does not list the actual content of the response. I can see the content-type, content-length and content-encoding only. If I run this same message through another library (such as restsharp) I will see detailed exception information from the remote server. How can I get more details from the response since I know the remote server is sending them?
static string getXMLString(string xmlContent, string url)
{
//string Url;
string sResult;
//Url = ConfigurationManager.AppSettings["UserURl"] + url;
var httpWebRequest = (HttpWebRequest)WebRequest.Create(url);
httpWebRequest.ContentType = "application/xml";
httpWebRequest.Method = "POST";
using (var streamWriter = new StreamWriter(httpWebRequest.GetRequestStream()))
{
streamWriter.Write(xmlContent);
streamWriter.Flush();
streamWriter.Close();
var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
{
var result = streamReader.ReadToEnd();
sResult = result;
}
}
return sResult;
}
EDIT : Have you tried with a simple try-catch to see if you can get more details ?
try
{
var response = (HttpWebResponse)(request.GetResponse());
}
catch(Exception ex)
{
var response = (HttpWebResponse)ex.Response;
}
In my recherches in an answer for you, I noticed that in code there was something about encoding, that you didn't specified. Look here for exemple with such code.
var encoding = ASCIIEncoding.ASCII;
using (var reader = new System.IO.StreamReader(response.GetResponseStream(), encoding))
{
string responseText = reader.ReadToEnd();
}
Or here, in the doc, also.
// Creates an HttpWebRequest with the specified URL.
HttpWebRequest myHttpWebRequest = (HttpWebRequest)WebRequest.Create(url);
// Sends the HttpWebRequest and waits for the response.
HttpWebResponse myHttpWebResponse = (HttpWebResponse)myHttpWebRequest.GetResponse();
// Gets the stream associated with the response.
Stream receiveStream = myHttpWebResponse.GetResponseStream();
Encoding encode = System.Text.Encoding.GetEncoding("utf-8");
// Pipes the stream to a higher level stream reader with the required encoding format.
StreamReader readStream = new StreamReader( receiveStream, encode );
Console.WriteLine("\r\nResponse stream received.");
Have you tried with such ?
I have a small C# console app working that copies the results of a webrequest to a text file and then runs each command in that text file, saving the results to a separate text file.
Problem is, I have to make two requests to the same server, which I don't like doing. The problem is I can't seem to go to the beginning of the Stream/StreamReader after writing it to the text file, forcing me to make another request.
How do I do this with only one webrequest?
Thanks,
John
static void Main(string[] args)
{
// Set all variables
string epoUrl = "https://de-ser2012ecm:8443/remote/core.help.do";
string commandHelpPath = #"C:\Logs\AllCommandsHelp.txt";
string coreHelpPath = #"C:\Logs\CoreHelp.txt";
string epoUsername = "admin";
string epoPassword = "password";
string responseFromServer;
StringReader strReader;
try
{
// Get stream from webrequest
Stream coreStream = WebHelper.GetWebResponseStream(epoUrl, epoUsername, epoPassword);
StreamReader coreReader = new StreamReader(coreStream);
// Write core help page to text file
using (StreamWriter corefile = new StreamWriter(coreHelpPath, true, Encoding.UTF8))
{
responseFromServer = coreReader.ReadToEnd();
// Display the content.
corefile.Write(responseFromServer);
strReader = new StringReader(responseFromServer);
}
// Get new stream from webrequest
Stream commandStream = WebHelper.GetWebResponseStream(epoUrl, epoUsername, epoPassword);
StreamReader commandReader = new StreamReader(commandStream);
using (StreamWriter outfile = new StreamWriter(commandHelpPath, true, Encoding.UTF8))
{
while (!strReader.Peek().Equals(-1))
{
string streamLine = strReader.ReadLine();
string[] words = streamLine.Split(' ');
// Check if first string contains a period that's not at the end
if ((words[0].Contains(".")) & !(words[0].EndsWith(".")))
{
StreamReader helpReader = WebHelper.GetWebResponse(epoUrl + "?command=" + words[0], epoUsername, epoPassword);
string helpResponseFromServer = helpReader.ReadToEnd();
outfile.Write(helpResponseFromServer);
outfile.WriteLine("==============================");
}
}
}
}
catch (Exception ex)
{
Console.WriteLine("Main exception: " + ex.Message);
}
finally
{
// Close streams
//coreReader.Close();
//commandReader.Close();
Console.WriteLine("Press any key to continue");
Console.ReadKey();
}
}
And the GetWebResponseStream method:
public static Stream GetWebResponseStream(string url, string username, string password)
{
Stream dataStream = null;
try
{
// Set the credentials.
CredentialCache credentialCache = new CredentialCache();
credentialCache.Add(new System.Uri(url), "Basic", new System.Net.NetworkCredential(username, password));
ServicePointManager.ServerCertificateValidationCallback = (s, cert, chain, ssl) => true;
// Create a request for the URL.
WebRequest request = WebRequest.Create(url);
request.Credentials = credentialCache;
// Get the response.
WebResponse response = request.GetResponse();
// Get the stream containing content returned by the server.
dataStream = response.GetResponseStream();
return dataStream;
}
catch (Exception ex)
{
Console.WriteLine("GetWebResponse threw an exception: " + ex.Message);
return dataStream;
}
}
Thanks to Gildor for the suggestion that put me on the right track. I had the response, I just needed to copy that string into a StringReader. This automatically resets the cursor so you can read from the top! I updated the code to show the new fix. Thanks to everyone for the suggestions. John
what I want is simple, I want to read a text file from my website via my application, I managed to do this in C# but not in metro apps, here my code in C#
WebClient client = new WebClient();
Stream stream = client.OpenRead(strURL);
StreamReader reader = new StreamReader(stream);
String content = reader.ReadToEnd();
return content;
besides the above code I also tried the code below, but still failed
HttpWebRequest request = (HttpWebRequest)
WebRequest.Create(strURL);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream resStream = response.GetResponseStream();
string tempString = null;
int count = 0;
byte[] buf = new byte[1000];
StringBuilder sb = new StringBuilder();
do
{
count = resStream.Read(buf, 0, buf.Length);
if (count != 0)
{
tempString = Encoding.Unicode.GetString(buf, 0, count);
sb.Append(tempString);
}
}
return sb.ToString();
I think the problem is in the WebClient and GetResponse () which is not known in metro apps
You should be able to use System.Net.Http.HttpClient and HttpResponseMessage, as they are included on http://msdn.microsoft.com/en-us/library/windows/apps/hh454046.aspx.
There is an example on http://msdn.microsoft.com/en-us/library/system.net.http.httpclient.aspx:
static async void Main()
{
try
{
// Create a New HttpClient object.
HttpClient client = new HttpClient();
HttpResponseMessage response = await client.GetAsync("http://www.contoso.com/");
response.EnsureSuccessStatusCode();
string responseBody = await response.Content.ReadAsStringAsync();
// Above three lines can be replaced with new helper method in following line
// string body = await client.GetStringAsync(uri);
Console.WriteLine(responseBody);
}
catch(HttpRequestException e)
{
Console.WriteLine("\nException Caught!");
Console.WriteLine("Message :{0} ",e.Message);
}
}
var httpClient = new HttpClient();
var text = await httpClient.GetStringAsync(uri);
Wrapped up in an async method of course
After I make two C# HttpWebRequests that throw an exception because of "(500) Internal Server Error 500", the third attempt throws a time out exception. Why doesn't it throw another (500) Internal Server Error exception?
When I restart my application, it throws two 500 errors and then starts timing out again.
This is my code:
GetPages GetPages = new GetPages();
string test = GetPages.GetPage(); /* Exception: (500) Internal Server Error */
GetPages.Dispose();
GetPages GetPages = new GetPages();
string test = GetPages.GetPage(); /* Exception: (500) Internal Server Error */
GetPages.Dispose();
GetPages GetPages = new GetPages();
string test = GetPages.GetPage(); /* Exception: time out, why? */
GetPages.Dispose();
This is GetPages class and GetPage method:
namespace MyNamespace
{
class GetPages
{
public string GetPage()
{
this.httpRequest = (HttpWebRequest)WebRequest.Create("http://myurl");
try
{
StringBuilder postData = new StringBuilder(100);
postData.Append("test=test");
byte[] dataArray = Encoding.UTF8.GetBytes(postData.ToString());
httpRequest.CachePolicy = new HttpRequestCachePolicy(HttpRequestCacheLevel.NoCacheNoStore);
httpRequest.KeepAlive = false;
httpRequest.Proxy = null;
httpRequest.Method = "POST";
httpRequest.Timeout = 10;
httpRequest.ContentType = "application/x-www-form-urlencoded";
httpRequest.ContentLength = dataArray.Length;
using (this.requestStream = httpRequest.GetRequestStream())
{
requestStream.Write(dataArray, 0, dataArray.Length);
requestStream.Flush();
requestStream.Close();
this.webResponse = (HttpWebResponse)httpRequest.GetResponse();
Stream responseStream = webResponse.GetResponseStream();
StreamReader responseReader = new System.IO.StreamReader(responseStream, Encoding.UTF8);
String responseString = responseReader.ReadToEnd();
MessageBox.Show(responseString);
return responseString;
}
}
catch (Exception e)
{
MessageBox.Show(e.Message);
return "FAIL";
}
}
public void Dispose()
{
System.GC.SuppressFinalize(this);
}
}
}
UPDATE
Thanks you all for helping out. I have not been able to solve the issue.
The dispose method is gone now.
I have made HttpWebRequest httpRequest, HttpWebResponse webResponse and Stream requestStream local and am using the following using statements:
using (webResponse = (HttpWebResponse)httpRequest.GetResponse())
{
using (Stream responseStream = webResponse.GetResponseStream())
{
using (StreamReader responseReader = new System.IO.StreamReader(responseStream, Encoding.UTF8))
{
responseString = responseReader.ReadToEnd();
}
}
}
Another update
This is the entire GetPage method now:
public string GetPage()
{
HttpWebRequest httpRequest;
HttpWebResponse webResponse;
Stream requestStream;
try
{
StringBuilder postData = new StringBuilder(100);
postData.Append("test=test");
byte[] dataArray = Encoding.UTF8.GetBytes(postData.ToString());
httpRequest = (HttpWebRequest)WebRequest.Create("http://myurl");
httpRequest.Proxy = null;
httpRequest.Method = "POST";
httpRequest.Timeout = 10;
httpRequest.ContentType = "application/x-www-form-urlencoded";
httpRequest.ContentLength = dataArray.Length;
using (requestStream = httpRequest.GetRequestStream())
{
/* this is never reached when the time out exception starts
so the error seems to be related to too many GetRequestStreams */
requestStream.Write(dataArray, 0, dataArray.Length);
webResponse = (HttpWebResponse)httpRequest.GetResponse();
/* this is never reached when the 500 server error occurs */
String responseString = "";
using (Stream responseStream = webResponse.GetResponseStream())
{
using (StreamReader responseReader = new System.IO.StreamReader(responseStream, Encoding.UTF8))
{
responseString = responseReader.ReadToEnd();
return responseString;
}
}
}
}
catch (Exception e)
{
return "FAIL";
}
return "...";
}
** SOLVED!! **
httpRequest was not getting abort()'ed. In the catch() block I have added httpRequest.abort(), now it works correctly.
I suspect this is the problem:
this.webResponse = (HttpWebResponse)httpRequest.GetResponse();
Stream responseStream = webResponse.GetResponseStream();
StreamReader responseReader = new StreamReader(responseStream, Encoding.UTF8);
String responseString = responseReader.ReadToEnd();
You're not disposing of any of these disposable objects. That means the connection to the web server will be maintained until finalization, so the connection pool (of two connections per server) won't allow any other requests.
I suggest that:
You move the GetResponse call out of the using statement for the request stream
You remove the explicit calls to Flush and Close for the request stream (disposing of it is good enough)
You make webResponse and webRequest local variables instead of instance variables unless you really need them later, in which case you should dispose of them in your Dispose method.
You use using statements for the WebResponse, Stream, and StreamReader. (The last isn't strictly necessary, as disposing of the stream is good enough.)
You make GetPages not implement IDisposable unless you really need it to.
HTTP protocol defines that only two connection can be made at the same time to the same server. Close the responseStream after successful or unsuccessful reading.