We Keep Coding

How to get text in stream from WebRequest?

So, here's a scenario that I made up, in which I am making my own web browser and I want to make sure I'm sending correct POST text to web server.
For that to be achieved, I need to get the POST text that WebRequest creates for me before I invoke GetResponseStream().
I tried to read through the stream from WebRequest.GetRequestStream(), but I assume that isn't a way to do that.
I do NOT want plain HTML text responsed from web server.
The POST text that I need to gain must look like something like this as follows :
POST http://www.some_web_server.com/default.aspx HTTP/1.1
Cache-Control : max-age=0 Connection : keep-alive .....
Thanks in advance.
[UPDATE]
It's clear that I already have all fomulated request(POST) text in my WebRequest instance.
Is there any way conveniently to just get the whole plain text from it, rather than using separated get properties such as ContentType or Headers?
(Because I am lazy to 'assemble' all the headers that I specified into the whole complete POST text, which web server will eventually see.)
// This might be a very poor code to approach.
public void Show_Full_POST_Text(WebRequest req)
{
// For example.
String strRawText = req.GetRequestLiteral();
// Your POST text looks like this!
ShowToUser(strRawText);
}
public void Foo()
{
// ...
Show_Full_POST_Text(req);
var ResponseStream = req.GetResponseStream();
// Continue.
}

If your saying that you don't want to get some headers via the request properties e.g "request.ContentType" and other via the header collection, then you can just use the header collection as it already contains the key for ContentType.
using System;
using System.Collections.Generic;
using System.Net;
namespace Demo
{
class Program
{
static void Main(string[] args)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://www.example.com");
request.Method = "POST";
request.ContentType = "image/png";
Console.WriteLine(GetRequestAsString(request));
Console.ReadKey();
}
public static string GetRequestAsString(HttpWebRequest request)
{
string str = request.Method + " " + request.RequestUri + " HTTP/" + request.ProtocolVersion + Environment.NewLine;
string[] headerKeys = request.Headers.AllKeys;
foreach (string key in headerKeys)
{
str += key + ":" + request.Headers[key];
}
return str;
}
}
}

Your question is pretty vague, however, I believe what you're looking for is including the headers within the Webrequest. The MSDN has a good example. See below:
public static void Main (string[] args)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create (args[0]);
// Set some reasonable limits on resources used by this request
request.MaximumAutomaticRedirections = 4;
request.MaximumResponseHeadersLength = 4; //This sets the max headers coming back to your response.
// Set credentials to use for this request.
request.Credentials = CredentialCache.DefaultCredentials;
HttpWebResponse response = (HttpWebResponse)request.GetResponse ();
Console.WriteLine ("Content length is {0}", response.ContentLength);
Console.WriteLine ("Content type is {0}", response.ContentType);
// Get the stream associated with the response.
Stream receiveStream = response.GetResponseStream ();
// Pipes the stream to a higher level stream reader with the required encoding format.
StreamReader readStream = new StreamReader (receiveStream, Encoding.UTF8);
Console.WriteLine ("Response stream received.");
Console.WriteLine (readStream.ReadToEnd ());
response.Close ();
readStream.Close ();
}

Call webpage from program

I would like my c# program to call a webpage when it opens. The code in the webpage will increment a counter... thus I just need the program to call the page. Don't think that I need to post or get or anything else.
Thoughts?

for more information check out msdn
// Create a request for the URL.
WebRequest request = WebRequest.Create (
"http://www.contoso.com/default.html");
// If required by the server, set the credentials.
request.Credentials = CredentialCache.DefaultCredentials;
// Get the response.
WebResponse response = request.GetResponse ();
// Display the status.
Console.WriteLine (((HttpWebResponse)response).StatusDescription);
// Get the stream containing content returned by the server.
Stream dataStream = response.GetResponseStream ();
// Open the stream using a StreamReader for easy access.
StreamReader reader = new StreamReader (dataStream);
// Read the content.
string responseFromServer = reader.ReadToEnd ();
// Display the content.
Console.WriteLine (responseFromServer);
// Clean up the streams and the response.
reader.Close ();
response.Close ();
if you can't extract the code you need from the above.. here it is
WebRequest request = WebRequest.Create ("http://www.mysite.com/counter.php?YourId");
WebResponse response = request.GetResponse();
response.Close ();

I know its late but for future people who might fall onto this.
System.Net.WebClient client = new System.Net.WebClient();
client.DownloadDataAsync(new Uri("http://stackoverflow.com"));

GetRequestStream() is throwing time out exception when posting data to HTTPS url

I'm calling an API hosted on Apache server to post data. I'm using HttpWebRequest to perform POST in C#.
API has both normal HTTP and secure layer (HTTPS) PORT on the server. When I call HTTP URL it works perfectly fine. However, when I call HTTPS it gives me time-out exception (at GetRequestStream() function). Any insights? I'm using VS 2010, .Net framework 3.5 and C#. Here is the code block:
string json_value = jsonSerializer.Serialize(data);
HttpWebRequest request = (HttpWebRequest)System.Net.WebRequest.Create("https://server-url-xxxx.com");
request.Method = "POST";
request.ProtocolVersion = System.Net.HttpVersion.Version10;
request.ContentType = "application/x-www-form-urlencoded";
byte[] buffer = Encoding.ASCII.GetBytes(json_value);
request.ContentLength = buffer.Length;
System.IO.Stream reqStream = request.GetRequestStream();
reqStream.Write(buffer, 0, buffer.Length);
reqStream.Close();
EDIT:
The console program suggested by Peter works fine. But when I add data (in JSON format) that needs to be posted to the API, it throws out operation timed out exception. Here is the code that I add to console based application and it throws error.
byte[] buffer = Encoding.ASCII.GetBytes(json_value);
request.ContentLength = buffer.Length;

I ran into the same issue. It seems like it is solved for me. I went through all my code making sure to invoke webResponse.Close() and/or responseStream.Close() for all my HttpWebResponse objects. The documentation indicates that you can close the stream or the HttpWebResponse object. Calling both is not harmful, so I did. Not closing the responses may cause the application to run out of connections for reuse, and this seems to affect the HttpWebRequest.GetRequestStream as far as I can observe in my code.

I don't know if this will help you with your specific problem but you should consider Disposing some of those objects when you are finished with them. I was doing something like this recently and wrapping stuff up in using statements seems to clean up a bunch of timeout exceptions for me.
using (var reqStream = request.GetRequestStream())
{
if (reqStream == null)
{
return;
}
//do whatever
}
also check these things
Is the server serving https in your local dev environment?
Have you set up your bindings *.443 (https) properly?
Do you need to set credentials on the request?
Is it your application pool account accessing the https resources or is it your account being passed through?
Have you thought about using WebClient instead?
using (WebClient client = new WebClient())
{
using (Stream stream = client.OpenRead("https://server-url-xxxx.com"))
using (StreamReader reader = new StreamReader(stream))
{
MessageBox.Show(reader.ReadToEnd());
}
}
EDIT:
make a request from console.
internal class Program
{
private static void Main(string[] args)
{
new Program().Run();
Console.ReadLine();
}
public void Run()
{
var request = (HttpWebRequest)System.Net.WebRequest.Create("https://server-url-xxxx.com");
request.Method = "POST";
request.ProtocolVersion = System.Net.HttpVersion.Version10;
request.ContentType = "application/x-www-form-urlencoded";
using (var reqStream = request.GetRequestStream())
{
using(var response = new StreamReader(reqStream )
{
Console.WriteLine(response.ReadToEnd());
}
}
}
}

Try this:
WebRequest req = WebRequest.Create("https://server-url-xxxx.com");
req.Method = "POST";
string json_value = jsonSerializer.Serialize(data); //Body data
ServicePointManager.Expect100Continue = false;
using (var streamWriter = new StreamWriter(req.GetRequestStream()))
{
streamWriter.Write(json_value);
streamWriter.Flush();
streamWriter.Close();
}
HttpWebResponse resp = req.GetResponse() as HttpWebResponse;
Stream GETResponseStream = resp.GetResponseStream();
StreamReader sr = new StreamReader(GETResponseStream);
var response = sr.ReadToEnd(); //Response
resp.Close(); //Close response
sr.Close(); //Close StreamReader
And review the URI:
Reserved characters. Send reserved characters by the URI can bring
problems ! * ' ( ) ; : # & = + $ , / ? # [ ]
URI Length: You should not exceed 2000 characters

I ran into this, too. I wanted to simulate hundreds of users with a Console app. When simulating only one user, everything was fine. But with more users came the Timeout exception all the time.
Timeout occurs because by default the ConnectionLimit=2 to a ServicePoint (aka website).
Very good article to read: https://venkateshnarayanan.wordpress.com/2013/04/17/httpwebrequest-reuse-of-tcp-connections/
What you can do is:
1) make more ConnectionGroups within a servicePoint, because ConnectionLimit is per ConnectionGroups.
2) or you just simply increase the connection limit.
See my solution:
private HttpWebRequest CreateHttpWebRequest<U>(string userSessionID, string method, string fullUrl, U uploadData)
{
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(fullUrl);
req.Method = method; // GET PUT POST DELETE
req.ConnectionGroupName = userSessionID; // We make separate connection-groups for each user session. Within a group connections can be reused.
req.ServicePoint.ConnectionLimit = 10; // The default value of 2 within a ConnectionGroup caused me always a "Timeout exception" because a user's 1-3 concurrent WebRequests within a second.
req.ServicePoint.MaxIdleTime = 5 * 1000; // (5 sec) default was 100000 (100 sec). Max idle time for a connection within a ConnectionGroup for reuse before closing
Log("Statistics: The sum of connections of all connectiongroups within the ServicePoint: " + req.ServicePoint.CurrentConnections; // just for statistics
if (uploadData != null)
{
req.ContentType = "application/json";
SerializeToJson(uploadData, req.GetRequestStream());
}
return req;
}
/// <summary>Serializes and writes obj to the requestStream and closes the stream. Uses JSON serialization from System.Runtime.Serialization.</summary>
public void SerializeToJson(object obj, Stream requestStream)
{
DataContractJsonSerializer json = new DataContractJsonSerializer(obj.GetType());
json.WriteObject(requestStream, obj);
requestStream.Close();
}

You may want to set timeout property, check it here http://www.codeproject.com/Tips/69637/Setting-timeout-property-for-System-Net-WebClient

"Request timed out" when makinng two HttpWebRequests to the same Java server from C# over .NET Compact

I have a client application running on C# on a .NETCF 3.5 device POSTing to a Java servlet located remotely. I am receiving a "Request Timed out" during my third HTTP POST to the same servlet. For example, if the servlet manages login to our Java server, the first two login attempts from the client would go through (same client device) and when I attempt the third one, it will return in a "Request timed out" exception from the server. I have noticed this happen always and I can't figure out the problem. I read that C# by default sends the Request 100 continue in the HTTP headers so I used the ServicePointManager to set the request 100 to false to no avail.
Here is the code that is throwing this error:
serverUrl = url;
string responseFromServer = "";
try
{
System.Net.ServicePointManager.Expect100Continue = false;
int tmp = ServicePointManager.DefaultConnectionLimit;
// Create a request using a URL that can receive a post.
request = (HttpWebRequest)WebRequest.Create(url);
// Set the Method property of the request to POST.
request.Method = "POST";
// Create POST data and convert it to a byte array.
byte[] byteArray = System.Text.Encoding.UTF8.GetBytes(url);
// Set the ContentType property of the WebRequest.
request.ContentType = "application/x-www-form-urlencoded";
// Set the ContentLength property of the WebRequest.
request.ContentLength = byteArray.Length;
request.Timeout = (50 * 100);
request.Proxy = System.Net.GlobalProxySelection.GetEmptyWebProxy();
// Get the request stream.
Stream dataStream = request.GetRequestStream();
// Write the data to the request stream.
dataStream.Write(byteArray, 0, byteArray.Length);
// Close the Stream object.
dataStream.Close();
// Get the response.
response = (HttpWebResponse)request.GetResponse();
// Get the stream containing content returned by the server.
dataStream = response.GetResponseStream();
// Open the stream using a StreamReader for easy access.
StreamReader reader = new StreamReader(dataStream);
// Read the content.
responseFromServer = reader.ReadToEnd();
// Clean up the streams.
reader.Close();
dataStream.Close();
response.Close();
return responseFromServer;
}
catch (Exception WebExp)
{
Logging.Instance.Log(Logging.Levels.Error, "Error in DoPost while retrieving : "+url+ " " + WebExp.Message.ToString());
Logging.Instance.Log(Logging.Levels.Error, WebExp.StackTrace.ToString());
throw WebExp;
}
Any help would be much appreciated. Thanks!

This behaviour is due to wrong Exception handling regarding to the WebResponse. You have always to handle the response and close it. Otherwise the third attempt of HTTP Webrequest will fail with timeout which is limited by WinCE.
Following source code will be safe:
HttpWebResponse response = null;
try
{
//...
response = (HttpWebResponse)request.GetResponse();
//...
}
catch(Exception e)
{
// logging, etc.
throw e;
}
finally
{
if(response!=null)
{
response.Close();
}
}

Making http request from code behind

I am trying to implement Message API
I am not sure how I will be calling this from code-behind and in their snippet it says:
https://platform.3cinteractive.com/api/send_message.php
POST
username=aRDSe3vcaMzh06YrMcxcQw==&password=1BSvQc6lpNlnp4ufWgRLPHNJ7RMrL8CcaWCzL1Vtw+Y=&phone_number=+11234567890&trigger_id=1105&message=howdy

Use the WebRequest class below or use a library like RestSharp for more control or your HTTP request:
// Create a request for the URL.
WebRequest request = WebRequest.Create ("http://www.contoso.com/default.html");
// If required by the server, set the credentials.
request.Credentials = CredentialCache.DefaultCredentials;
// Get the response.
HttpWebResponse response = (HttpWebResponse)request.GetResponse ();
// Display the status.
Console.WriteLine (response.StatusDescription);
// Get the stream containing content returned by the server.
Stream dataStream = response.GetResponseStream ();
// Open the stream using a StreamReader for easy access.
StreamReader reader = new StreamReader (dataStream);
// Read the content.
string responseFromServer = reader.ReadToEnd ();
// Display the content.
Console.WriteLine (responseFromServer);
// Cleanup the streams and the response.
reader.Close ();
dataStream.Close ();
response.Close ();

You need to use a WebRequest and do an HTTP POST. See this article entitled How to: Send Data Using the WebRequest Class.