I would like to create 2 HTTP requests on the same connection (HTTP persistent connection).
I'm using HttpWebRequest:
WebRequest request = HttpWebRequest.Create("http://localhost:14890/Service1/3");
WebResponse response = request.GetResponse();
byte[] buffer = new byte[1024];
int x = response.GetResponseStream().Read(buffer, 0, 1024);
string str = System.Text.ASCIIEncoding.ASCII.GetString(buffer);
I think if I use request again it will create a whole new HTTP connection which I don't want to do.
Is there another class I can use isntead or is there something I'm missing?
I'm also not sure how the WebClient class works with respect to persistent connections.
Set the KeepAlive property.
For example:
string str;
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create("http://localhost:14890/Service1/3");
request.KeepAlive = true;
using (WebResponse response = request.GetResponse())
using (Stream responseStream = response.GetResponseStream())
using (StreamReader reader = new StreamReader(responseStream, Encoding.ASCII)) {
str = reader.ReadToEnd();
}
if you want to keep your server session from one httprequest to the next, you have explicitly store and send your session identifier, unlike with msinet.ocx which does it all for you. for instance when connecting to a php webserver, the session identifier is stored in a header labelled Set-Cookie/PHPSESSID=... and this has header has to be manually added to the next httprequest but renamed to Cookie/PHPSESSID=....
Related
We have a program that has been running for years making API calls to a web server using HttpWebRequest and yesterday it started giving an error (something like "connection forcibly closed by remote host"). The request works just fine when made through a web browser so I would love to be able to see the difference in requests. With the Firefox developer console, I can see the raw request that is made through the browser (that works) and I need to compare that to the http request that is made from our program. It seems like it should be simple (and very useful) to stream the request out to a string or a file so I can look at it (but I have not had any luck finding how to do that).
Can you tell me how to modify the below code to store the request that HttpWebRequest would send to a file or a string (instead of a network stream)?
public string Post(string uri, string data, string contentType, string method = "POST")
{
byte[] dataBytes = Encoding.UTF8.GetBytes(data);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
request.ContentLength = dataBytes.Length;
request.ContentType = contentType;
request.Method = method;
using(Stream requestBody = request.GetRequestStream())
{
requestBody.Write(dataBytes, 0, dataBytes.Length);
}
using(HttpWebResponse response = (HttpWebResponse)request.GetResponse())
using(Stream stream = response.GetResponseStream())
using(StreamReader reader = new StreamReader(stream))
{
return reader.ReadToEnd();
}
}
You can't simply log the httprequest. Because you should also consider logging all the headers.
I suggest you to use some http sniffer to log the traffic(if you can't debug or modify your code)
In addition you can catch the exceptions by using WebException and get the raw error message from the server. Maybe it'll give you idea what the problem is.
catch (WebException ex)
{
using (var stream = ex.Response.GetResponseStream())
using (var reader = new StreamReader(stream))
{
var responseString = reader.ReadToEnd();
}
}
I have to send a file since a cURL command :
curl -X POST -F "csv[file]=#/mypath.csv" https://mylogin:mypassword#the-server.net
Si i tried with an HttpClient :
var client = new HttpClient();
// Create the HttpContent for the form to be posted.
var requestContent = new FormUrlEncodedContent(new[] { new KeyValuePair<string, string>("csv[file]", $#"#/{this.pathFile}")});
// Get the response.
HttpResponseMessage response = await client.PostAsync($#"https://{this.login}:{this.password}#myserver.net",requestContent);
// Get the response content.
HttpContent responseContent = response.Content;
// Get the stream of the content.
using (var reader = new StreamReader(await responseContent.ReadAsStreamAsync()))
{
// Write the output.
var testResult= await reader.ReadToEndAsync();
}
Or with the following code :
WebRequest request = WebRequest.Create($#"https://{this.login}:{this.password}#myserver.net");
// Set the Network credentials
request.Credentials = CredentialCache.DefaultCredentials;
request.Method = "POST";
// Create POST data and convert it to a byte array.
string postData = $#"csv[file]=#/{this.pathFile}";
byte[] byteArray = Encoding.UTF8.GetBytes(postData);
request.ContentType = "application/x-www-form-urlencoded";
// Set the ContentLength property of the WebRequest.
request.ContentLength = byteArray.Length;
using (Stream dataStream = request.GetRequestStream())
{
// Write the data to the request stream.
dataStream.Write(byteArray, 0, byteArray.Length);
}
using (WebResponse response2 = request.GetResponse())
{
// Display the status.
Console.WriteLine(((HttpWebResponse)response2).StatusDescription);
// Get the stream containing content returned by the server.
using (StreamReader reader = new StreamReader(response2.GetResponseStream()))
{
Console.WriteLine(reader.ReadToEnd());
}
}
But each time, it's the result 401 Unauthorized. Of course, my credentials are the good ones...
[EDIT]
I work for a professional project. the server where to send the file belongs to a partner. The cURL command is imposed on me and i haven't got control of this server
[EDIT 2]
I did an analysis with wireshark
IP 229 is the partners server
IP 160 is my compuer
I just have an encrypted alert. I tested with HTTP and not https but i have the same message
It Looks like that the file you are accessing is placed at a point where the user did not have any right to access the file, Try to give permission to the folder of read and write access to use the file.
That could solve your issue.
I'm calling an API hosted on Apache server to post data. I'm using HttpWebRequest to perform POST in C#.
API has both normal HTTP and secure layer (HTTPS) PORT on the server. When I call HTTP URL it works perfectly fine. However, when I call HTTPS it gives me time-out exception (at GetRequestStream() function). Any insights? I'm using VS 2010, .Net framework 3.5 and C#. Here is the code block:
string json_value = jsonSerializer.Serialize(data);
HttpWebRequest request = (HttpWebRequest)System.Net.WebRequest.Create("https://server-url-xxxx.com");
request.Method = "POST";
request.ProtocolVersion = System.Net.HttpVersion.Version10;
request.ContentType = "application/x-www-form-urlencoded";
byte[] buffer = Encoding.ASCII.GetBytes(json_value);
request.ContentLength = buffer.Length;
System.IO.Stream reqStream = request.GetRequestStream();
reqStream.Write(buffer, 0, buffer.Length);
reqStream.Close();
EDIT:
The console program suggested by Peter works fine. But when I add data (in JSON format) that needs to be posted to the API, it throws out operation timed out exception. Here is the code that I add to console based application and it throws error.
byte[] buffer = Encoding.ASCII.GetBytes(json_value);
request.ContentLength = buffer.Length;
I ran into the same issue. It seems like it is solved for me. I went through all my code making sure to invoke webResponse.Close() and/or responseStream.Close() for all my HttpWebResponse objects. The documentation indicates that you can close the stream or the HttpWebResponse object. Calling both is not harmful, so I did. Not closing the responses may cause the application to run out of connections for reuse, and this seems to affect the HttpWebRequest.GetRequestStream as far as I can observe in my code.
I don't know if this will help you with your specific problem but you should consider Disposing some of those objects when you are finished with them. I was doing something like this recently and wrapping stuff up in using statements seems to clean up a bunch of timeout exceptions for me.
using (var reqStream = request.GetRequestStream())
{
if (reqStream == null)
{
return;
}
//do whatever
}
also check these things
Is the server serving https in your local dev environment?
Have you set up your bindings *.443 (https) properly?
Do you need to set credentials on the request?
Is it your application pool account accessing the https resources or is it your account being passed through?
Have you thought about using WebClient instead?
using (WebClient client = new WebClient())
{
using (Stream stream = client.OpenRead("https://server-url-xxxx.com"))
using (StreamReader reader = new StreamReader(stream))
{
MessageBox.Show(reader.ReadToEnd());
}
}
EDIT:
make a request from console.
internal class Program
{
private static void Main(string[] args)
{
new Program().Run();
Console.ReadLine();
}
public void Run()
{
var request = (HttpWebRequest)System.Net.WebRequest.Create("https://server-url-xxxx.com");
request.Method = "POST";
request.ProtocolVersion = System.Net.HttpVersion.Version10;
request.ContentType = "application/x-www-form-urlencoded";
using (var reqStream = request.GetRequestStream())
{
using(var response = new StreamReader(reqStream )
{
Console.WriteLine(response.ReadToEnd());
}
}
}
}
Try this:
WebRequest req = WebRequest.Create("https://server-url-xxxx.com");
req.Method = "POST";
string json_value = jsonSerializer.Serialize(data); //Body data
ServicePointManager.Expect100Continue = false;
using (var streamWriter = new StreamWriter(req.GetRequestStream()))
{
streamWriter.Write(json_value);
streamWriter.Flush();
streamWriter.Close();
}
HttpWebResponse resp = req.GetResponse() as HttpWebResponse;
Stream GETResponseStream = resp.GetResponseStream();
StreamReader sr = new StreamReader(GETResponseStream);
var response = sr.ReadToEnd(); //Response
resp.Close(); //Close response
sr.Close(); //Close StreamReader
And review the URI:
Reserved characters. Send reserved characters by the URI can bring
problems ! * ' ( ) ; : # & = + $ , / ? # [ ]
URI Length: You should not exceed 2000 characters
I ran into this, too. I wanted to simulate hundreds of users with a Console app. When simulating only one user, everything was fine. But with more users came the Timeout exception all the time.
Timeout occurs because by default the ConnectionLimit=2 to a ServicePoint (aka website).
Very good article to read: https://venkateshnarayanan.wordpress.com/2013/04/17/httpwebrequest-reuse-of-tcp-connections/
What you can do is:
1) make more ConnectionGroups within a servicePoint, because ConnectionLimit is per ConnectionGroups.
2) or you just simply increase the connection limit.
See my solution:
private HttpWebRequest CreateHttpWebRequest<U>(string userSessionID, string method, string fullUrl, U uploadData)
{
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(fullUrl);
req.Method = method; // GET PUT POST DELETE
req.ConnectionGroupName = userSessionID; // We make separate connection-groups for each user session. Within a group connections can be reused.
req.ServicePoint.ConnectionLimit = 10; // The default value of 2 within a ConnectionGroup caused me always a "Timeout exception" because a user's 1-3 concurrent WebRequests within a second.
req.ServicePoint.MaxIdleTime = 5 * 1000; // (5 sec) default was 100000 (100 sec). Max idle time for a connection within a ConnectionGroup for reuse before closing
Log("Statistics: The sum of connections of all connectiongroups within the ServicePoint: " + req.ServicePoint.CurrentConnections; // just for statistics
if (uploadData != null)
{
req.ContentType = "application/json";
SerializeToJson(uploadData, req.GetRequestStream());
}
return req;
}
/// <summary>Serializes and writes obj to the requestStream and closes the stream. Uses JSON serialization from System.Runtime.Serialization.</summary>
public void SerializeToJson(object obj, Stream requestStream)
{
DataContractJsonSerializer json = new DataContractJsonSerializer(obj.GetType());
json.WriteObject(requestStream, obj);
requestStream.Close();
}
You may want to set timeout property, check it here http://www.codeproject.com/Tips/69637/Setting-timeout-property-for-System-Net-WebClient
This is driving me a bit nuts. I am trying to do something quite simple, and I have done it many times before. Just trying to call a REST API.
I am trying to call GetMessage with endpoint = "http://feed.linksynergy.com/productsearch?token=717f8c8511725ea26fd5c3651f32ab187d8db9f4b208be781c292585400e682d&keyword=DVD", and it keeps returning empty string. If I pass it any other valid URL, it will work. But if I just copy and paste the original URL into the web browser, it returns fine!
Can any smart developer tell me what's going on?
Code below. Thanks in advance.
James
public string GetMessage(string endPoint)
{
HttpWebRequest request = CreateWebRequest(endPoint);
using (var response = (HttpWebResponse)request.GetResponse())
{
var responseValue = string.Empty;
if (response.StatusCode != HttpStatusCode.OK)
{
string message = String.Format("POST failed. Received HTTP {0}", response.StatusCode);
throw new ApplicationException(message);
}
// grab the response
using (var responseStream = response.GetResponseStream())
{
using (var reader = new StreamReader(responseStream))
{
responseValue = reader.ReadToEnd();
}
}
return responseValue;
}
}
private HttpWebRequest CreateWebRequest(string endPoint)
{
var request = (HttpWebRequest)WebRequest.Create(endPoint);
request.Method = "GET";
request.ContentLength = 0;
request.ContentType = "text/xml";
return request;
}
Not sure why your setting ContentLength/ContentType - that is generally for HTTP POST, where there is a request body for which you write data to via a stream.
This is a HTTP GET, so there is no request body. (just URI w/ query string)
This should work:
using System;
using System.IO;
using System.Net;
using System.Text;
// Create the web request
HttpWebRequest request = WebRequest.Create("http://www.someapi.com/") as HttpWebRequest;
// Get response
using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
{
// Get the response stream
StreamReader reader = new StreamReader(response.GetResponseStream());
// Console application output
Console.WriteLine(reader.ReadToEnd());
}
EDIT
#Gabe is also quite right - try this on another computer, that is isn't behind any kind of firewall/proxy server.
My work PC was behind a proxy server, so in order to make REST-based HTTP calls, i needed to do this:
var proxyObject = new System.Net.WebProxy("http://myDomain:8080/", true);
System.Net.WebRequest req = System.Net.WebRequest.Create("http://www.someapi.com/");
req.Proxy = proxyObject;
proxyObject.Credentials = New System.Net.NetworkCredential("domain\username","password")
This may be a pathetically simple problem, but I cannot seem to format the post webrequest/response to get data from the Wikipedia API. I have posted my code below if anyone can help me see my problem.
string pgTitle = txtPageTitle.Text;
Uri address = new Uri("http://en.wikipedia.org/w/api.php");
HttpWebRequest request = WebRequest.Create(address) as HttpWebRequest;
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
string action = "query";
string query = pgTitle;
StringBuilder data = new StringBuilder();
data.Append("action=" + HttpUtility.UrlEncode(action));
data.Append("&query=" + HttpUtility.UrlEncode(query));
byte[] byteData = UTF8Encoding.UTF8.GetBytes(data.ToString());
request.ContentLength = byteData.Length;
using (Stream postStream = request.GetRequestStream())
{
postStream.Write(byteData, 0, byteData.Length);
}
using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
{
// Get the response stream.
StreamReader reader = new StreamReader(response.GetResponseStream());
divWikiData.InnerText = reader.ReadToEnd();
}
You might want to try a GET request first because it's a little simpler (you will only need to POST for wikipedia login). For example, try to simulate this request:
http://en.wikipedia.org/w/api.php?action=query&prop=images&titles=Main%20Page
Here's the code:
HttpWebRequest myRequest =
(HttpWebRequest)WebRequest.Create("http://en.wikipedia.org/w/api.php?action=query&prop=images&titles=Main%20Page");
using (HttpWebResponse response = (HttpWebResponse)myRequest.GetResponse())
{
string ResponseText;
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
{
ResponseText = reader.ReadToEnd();
}
}
Edit: The other problem he was experiencing on the POST request was, The exception is : The remote server returned an error: (417) Expectation failed. It can be solved by setting:
System.Net.ServicePointManager.Expect100Continue = false;
(This is from: HTTP POST Returns Error: 417 "Expectation Failed.")
I'm currently in the final stages of implementing an C# MediaWiki API which allows the easy scripting of most MediaWiki viewing and editing actions.
The main API is here: http://o2platform.googlecode.com/svn/trunk/O2%20-%20All%20Active%20Projects/O2_XRules_Database/_Rules/APIs/OwaspAPI.cs and here is an example of the API in use:
var wiki = new O2MediaWikiAPI("http://www.o2platform.com/api.php");
wiki.login(userName, password);
var page = "Test"; // "Main_Page";
wiki.editPage(page,"Test content2");
var rawWikiText = wiki.raw(page);
var htmlText = wiki.html(page);
return rawWikiText.line().line() + htmlText;
You seem to be pushing the input data on HTTP POST, but it seems you should use HTTP GET.
From the MediaWiki API docs:
The API takes its input through
parameters in the query string. Every
module (and every action=query
submodule) has its own set of
parameters, which is listed in the
documentation and in action=help, and
can be retrieved through
action=paraminfo.
http://www.mediawiki.org/wiki/API:Data_formats