I am trying to use .Net WebRequest to POST a form. The form contains fields that are XML. (Among other things) I have tried the following code:
WebRequest req = WebRequest.Create(ctx.SvcUrl);
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
using (var writer = new StreamWriter(req.GetRequestStream(), System.Text.Encoding.ASCII))
{
string reqBody = "first=<bill/>&last=smith"; //(embedded <>) - 500 Internal Server Error
writer.Write(reqBody);
}
rsp = req.GetResponse();
var strm = rsp.GetResponseStream();
var rdr = new StreamReader(strm);
string input = rdr.ReadToEnd();
The <> in reqBody causes a 500 - Internal Server error.
What's the right way to encode this? Or are multi-part forms the answer??
Try using:
string reqBody = string.Format("first={0}&last={1}", HttpUtility.HtmlEncode("<bill/>"), "smith");
You need to encode the request. Use the HttpEncoder class.
using System.Web.Util;
WebRequest req = WebRequest.Create(ctx.SvcUrl);
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
using (var writer = new StreamWriter(req.GetRequestStream(),
System.Text.Encoding.ASCII))
{
var encoder = new HttpEncoder();
string reqBody = String.Format("first={0}&last={1}",
encoder.HtmlEncode("<bill/>"),
encoder.HtmlEncode("smith") );
writer.Write(reqBody);
}
rsp = req.GetResponse();
var strm = rsp.GetResponseStream();
var rdr = new StreamReader(strm);
string input = rdr.ReadToEnd();
I used String.Format() because I thought it looked nicer and made it clearer what I was doing, but it isn't necessary. You can build the string through string concatenation, too, as long as you pass it through HttpEncoder.HtmlEncode() first.
It turns out that UrlEncoding is being done automatically, so doing it myself can cause trouble. Also, the server I was connecting to couldn't handle any encoding. This muddied the water and made it difficult to see what was failing.
Bottom line solution was to get the server fixed to handle UrlEncoding.
As 'cheong00 on Microsoft's Forums' points out, to avoid the automatic, use TcpClient. But the encoding should be there.
Related
I spent all day trying to figure out what I was doing wrong yesterday.
Coming here to try and find some help.
The follow error is triggered when I run the actual GetResponse.
I am new to APIs so I am sure I am missing something real simple.
You must provide a request body if you set ContentLength>0 or SendChunked==true. Do this by calling [Begin]GetRequestStream before [Begin]GetResponse.
Here is my code I am using to try and send JSON to the API. Payment object just has the form values entered in and the credentials to use the correct account on the merchants end.
var json = JsonConvert.SerializeObject(payment);
var apiUrl = new Uri($"Removed endpoint URL");
var postBytes = Encoding.UTF8.GetBytes(json);
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
var httpWebRequest = (HttpWebRequest)WebRequest.Create(apiUrl);
httpWebRequest.ContentType = "application/json";
httpWebRequest.Accept = "application/json";
httpWebRequest.Method = "POST";
httpWebRequest.ContentLength = postBytes.Length;
httpWebRequest.AllowWriteStreamBuffering = false;
//This is where the error triggers and drops to the catch.
var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
{
var result = streamReader.ReadToEnd();
}
I appreciate any help in advance, I may be doing this completely wrong, its a series of things I threw together trying to fix issues with the call.
Unless I missed it, you're not actually writing your payload data to the HttpWebRequest body before you're sending it.
using (Stream _reqStrm = httpWebRequest.GetRequestStream())
{
_reqStrm.Write(postBytes, 0, postBytes.Length);
}
var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
....
Unrelated but if you can, consider HttpClient
Hth..
This is being done on a Windows Forms App. I've spent a ton of time stepping through this code with the debugger. What I've found are the following things and they all seem to be at this line:
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
1. If I include
request.SendChunked = true;
I get this error at the response line previously stated:
'System.Net.WebException: The remote server returned an error: (415) Unsupported Media Type.
2. If I comment out the code in #1, I receive this error at that main response line that I mentioned in the beginning:
'System.Net.WebException: The underlying connection was closed: The connection was closed unexpectedly.
3. If I go with route #1, the "Connection" of the request remains as "KeepAlive" all the way through. But if I go with route #2, the "Connection" of the request changes to "null" at the response line that I mentioned in the beginning.
private void HttpPost()
{
HttpWebRequest request = null;
Uri uri = new Uri("https://post.craigslist.org/bulk-rss/post");
request = (HttpWebRequest)WebRequest.Create(uri);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
XmlDocument doc = new XmlDocument();
doc.Load("XMLFile1.xml");
//request.ContentLength = doc.InnerXml.Length;
request.SendChunked = true;
using (Stream writeStream = request.GetRequestStream())
{
UTF8Encoding encoding = new UTF8Encoding();
byte[] bytes = encoding.GetBytes(doc.InnerXml);
//request.ContentLength = bytes.Length;
writeStream.Write(bytes, 0, bytes.Length);
}
string result = string.Empty;
request.ProtocolVersion = System.Net.HttpVersion.Version11;
request.KeepAlive = false;
try
{
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (Stream responseStream = response.GetResponseStream())
{
using (System.IO.StreamReader readStream = new System.IO.StreamReader(responseStream, Encoding.UTF8))
{
result = readStream.ReadToEnd();
}
}
}
}
catch (Exception e)
{
string innerException = String.Format("Inner exception: '{0}'", e.Data);
string exceptionCause = String.Format("An error occurred: '{0}'", e);
System.IO.File.WriteAllText(#"C:\Users\Nathan\Documents\DebugOutputFile\exception.txt", exceptionCause);
System.IO.File.WriteAllText(#"C:\Users\Nathan\Documents\DebugOutputFile\innerException.txt", innerException);
}
}
I feel like these things are adding up towards a solution, but I could really use some guidance.
Option 1: Change your content type to match the body encoding
request.ContentType = "application/xml";
Option 2: Change your body encoding to match the specified content-type
If your server expects only "application/x-www-form-urlencoded", then you need to change your body encoding to suit it, for example, like this:
using (Stream writeStream = request.GetRequestStream())
{
UTF8Encoding encoding = new UTF8Encoding();
string response = String.Concat("arg=", HttpUtility.UrlEncode(doc.InnerXml))
byte[] bytes = encoding.GetBytes(doc.InnerXml);
//request.ContentLength = bytes.Length;
writeStream.Write(bytes, 0, bytes.Length);
}
You need to know the parameter name (above was set to "arg") and add a reference to System.Web, if you haven't.
See following XML...
<?xml version="1.0" encoding="UTF-8"?><test></test>
and encoded string for reference (your request body should look similar to this):
arg=%3C%3Fxml%20version%3D%221.0%22%20encoding%3D%22UTF-8%22%3F%3E%3Ctest%3E%3C%2Ftest%3E
Explanation
If you look at the response you are getting with the first approach: 415 - Unsupported Media Type, you can notice that the content type you are specifying ("application/x-www-form-urlencoded") doesn't match what you are sending in the body (an XML document). Chunk encoding should be enabled when sending files.
Note
When you are having trouble with a request done in source code, try to test the request alone with a web debugging tool, like Fiddler. There you would compose and issue the request until you get the response you want. Then you can compare that with what you are sending from source code (again you should use the same tool for inspecting your request).
I'm not sure why this PayPal Pay operation is giving me this error even though I seem to have covered all the required fields:
Error Code: 81002 Severity: Error Message: Unspecified Method (Method Specified is not Supported)"
string postData = JsonConvert.SerializeObject(request);
value of postData:
"actionType=PAY
¤cyCode=USD
&cancelUrl=https%3a%2f%2fexample.com%2fcancel
&returnUrl=https%3a%2f%2fexample.com%2freturn
&requestenvelope.errorLanguage=en_US
&receiverList.receiver(0).email=recipientemail%40gmail.com
&receiverList.receiver(0).amount=0.05
&VERSION=94.0
&USER=bizemail-facilitator_api1.gmail.com
&PWD=xxxxx
&SIGNATURE=xxxxxxxxx"
Here's how I do the post:
SendRequest("https://api-3t.sandbox.paypal.com/nvp", postData);
public string SendRequest(string url, string postData)
{
var uri = new Uri(url);
var request = WebRequest.Create(uri);
var encoding = new UTF8Encoding();
var requestData = encoding.GetBytes(postData);
request.ContentType = "application/x-www-form-urlencoded";
request.Method = "POST";
request.Timeout = (300*1000); //TODO: Move timeout to config
request.ContentLength = requestData.Length;
using (var stream = request.GetRequestStream()) {
stream.Write(requestData, 0, requestData.Length);
}
var response = request.GetResponse();
string result;
using (var reader = new StreamReader(response.GetResponseStream(), Encoding.ASCII)) {
result = reader.ReadToEnd();
}
return result;
}
The method PAY doesn't exist on the NVP api. Full list of methods supported by that API is found here.
The PAY method, however, is defined in the Adaptive Payments API. Depending on your needs, you have two options:
Change the endpoint to https://svcs.paypal.com/AdaptivePayments/PAY and modify your values
Use another method, like DoDirectPayment, but I'm not sure it does what you want to.
I assume the value of postData is before you serialize it to a JSON string, instead of after as you say, since it is clearly not a JSON string.
If so, then it is already x-www-form-urlencoded, and you do not need to serialize it to a JSON string at all.
Or if you do after all, then change just this: request.ContentType = "application/json";
I'm calling an API hosted on Apache server to post data. I'm using HttpWebRequest to perform POST in C#.
API has both normal HTTP and secure layer (HTTPS) PORT on the server. When I call HTTP URL it works perfectly fine. However, when I call HTTPS it gives me time-out exception (at GetRequestStream() function). Any insights? I'm using VS 2010, .Net framework 3.5 and C#. Here is the code block:
string json_value = jsonSerializer.Serialize(data);
HttpWebRequest request = (HttpWebRequest)System.Net.WebRequest.Create("https://server-url-xxxx.com");
request.Method = "POST";
request.ProtocolVersion = System.Net.HttpVersion.Version10;
request.ContentType = "application/x-www-form-urlencoded";
byte[] buffer = Encoding.ASCII.GetBytes(json_value);
request.ContentLength = buffer.Length;
System.IO.Stream reqStream = request.GetRequestStream();
reqStream.Write(buffer, 0, buffer.Length);
reqStream.Close();
EDIT:
The console program suggested by Peter works fine. But when I add data (in JSON format) that needs to be posted to the API, it throws out operation timed out exception. Here is the code that I add to console based application and it throws error.
byte[] buffer = Encoding.ASCII.GetBytes(json_value);
request.ContentLength = buffer.Length;
I ran into the same issue. It seems like it is solved for me. I went through all my code making sure to invoke webResponse.Close() and/or responseStream.Close() for all my HttpWebResponse objects. The documentation indicates that you can close the stream or the HttpWebResponse object. Calling both is not harmful, so I did. Not closing the responses may cause the application to run out of connections for reuse, and this seems to affect the HttpWebRequest.GetRequestStream as far as I can observe in my code.
I don't know if this will help you with your specific problem but you should consider Disposing some of those objects when you are finished with them. I was doing something like this recently and wrapping stuff up in using statements seems to clean up a bunch of timeout exceptions for me.
using (var reqStream = request.GetRequestStream())
{
if (reqStream == null)
{
return;
}
//do whatever
}
also check these things
Is the server serving https in your local dev environment?
Have you set up your bindings *.443 (https) properly?
Do you need to set credentials on the request?
Is it your application pool account accessing the https resources or is it your account being passed through?
Have you thought about using WebClient instead?
using (WebClient client = new WebClient())
{
using (Stream stream = client.OpenRead("https://server-url-xxxx.com"))
using (StreamReader reader = new StreamReader(stream))
{
MessageBox.Show(reader.ReadToEnd());
}
}
EDIT:
make a request from console.
internal class Program
{
private static void Main(string[] args)
{
new Program().Run();
Console.ReadLine();
}
public void Run()
{
var request = (HttpWebRequest)System.Net.WebRequest.Create("https://server-url-xxxx.com");
request.Method = "POST";
request.ProtocolVersion = System.Net.HttpVersion.Version10;
request.ContentType = "application/x-www-form-urlencoded";
using (var reqStream = request.GetRequestStream())
{
using(var response = new StreamReader(reqStream )
{
Console.WriteLine(response.ReadToEnd());
}
}
}
}
Try this:
WebRequest req = WebRequest.Create("https://server-url-xxxx.com");
req.Method = "POST";
string json_value = jsonSerializer.Serialize(data); //Body data
ServicePointManager.Expect100Continue = false;
using (var streamWriter = new StreamWriter(req.GetRequestStream()))
{
streamWriter.Write(json_value);
streamWriter.Flush();
streamWriter.Close();
}
HttpWebResponse resp = req.GetResponse() as HttpWebResponse;
Stream GETResponseStream = resp.GetResponseStream();
StreamReader sr = new StreamReader(GETResponseStream);
var response = sr.ReadToEnd(); //Response
resp.Close(); //Close response
sr.Close(); //Close StreamReader
And review the URI:
Reserved characters. Send reserved characters by the URI can bring
problems ! * ' ( ) ; : # & = + $ , / ? # [ ]
URI Length: You should not exceed 2000 characters
I ran into this, too. I wanted to simulate hundreds of users with a Console app. When simulating only one user, everything was fine. But with more users came the Timeout exception all the time.
Timeout occurs because by default the ConnectionLimit=2 to a ServicePoint (aka website).
Very good article to read: https://venkateshnarayanan.wordpress.com/2013/04/17/httpwebrequest-reuse-of-tcp-connections/
What you can do is:
1) make more ConnectionGroups within a servicePoint, because ConnectionLimit is per ConnectionGroups.
2) or you just simply increase the connection limit.
See my solution:
private HttpWebRequest CreateHttpWebRequest<U>(string userSessionID, string method, string fullUrl, U uploadData)
{
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(fullUrl);
req.Method = method; // GET PUT POST DELETE
req.ConnectionGroupName = userSessionID; // We make separate connection-groups for each user session. Within a group connections can be reused.
req.ServicePoint.ConnectionLimit = 10; // The default value of 2 within a ConnectionGroup caused me always a "Timeout exception" because a user's 1-3 concurrent WebRequests within a second.
req.ServicePoint.MaxIdleTime = 5 * 1000; // (5 sec) default was 100000 (100 sec). Max idle time for a connection within a ConnectionGroup for reuse before closing
Log("Statistics: The sum of connections of all connectiongroups within the ServicePoint: " + req.ServicePoint.CurrentConnections; // just for statistics
if (uploadData != null)
{
req.ContentType = "application/json";
SerializeToJson(uploadData, req.GetRequestStream());
}
return req;
}
/// <summary>Serializes and writes obj to the requestStream and closes the stream. Uses JSON serialization from System.Runtime.Serialization.</summary>
public void SerializeToJson(object obj, Stream requestStream)
{
DataContractJsonSerializer json = new DataContractJsonSerializer(obj.GetType());
json.WriteObject(requestStream, obj);
requestStream.Close();
}
You may want to set timeout property, check it here http://www.codeproject.com/Tips/69637/Setting-timeout-property-for-System-Net-WebClient
This may be a pathetically simple problem, but I cannot seem to format the post webrequest/response to get data from the Wikipedia API. I have posted my code below if anyone can help me see my problem.
string pgTitle = txtPageTitle.Text;
Uri address = new Uri("http://en.wikipedia.org/w/api.php");
HttpWebRequest request = WebRequest.Create(address) as HttpWebRequest;
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
string action = "query";
string query = pgTitle;
StringBuilder data = new StringBuilder();
data.Append("action=" + HttpUtility.UrlEncode(action));
data.Append("&query=" + HttpUtility.UrlEncode(query));
byte[] byteData = UTF8Encoding.UTF8.GetBytes(data.ToString());
request.ContentLength = byteData.Length;
using (Stream postStream = request.GetRequestStream())
{
postStream.Write(byteData, 0, byteData.Length);
}
using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
{
// Get the response stream.
StreamReader reader = new StreamReader(response.GetResponseStream());
divWikiData.InnerText = reader.ReadToEnd();
}
You might want to try a GET request first because it's a little simpler (you will only need to POST for wikipedia login). For example, try to simulate this request:
http://en.wikipedia.org/w/api.php?action=query&prop=images&titles=Main%20Page
Here's the code:
HttpWebRequest myRequest =
(HttpWebRequest)WebRequest.Create("http://en.wikipedia.org/w/api.php?action=query&prop=images&titles=Main%20Page");
using (HttpWebResponse response = (HttpWebResponse)myRequest.GetResponse())
{
string ResponseText;
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
{
ResponseText = reader.ReadToEnd();
}
}
Edit: The other problem he was experiencing on the POST request was, The exception is : The remote server returned an error: (417) Expectation failed. It can be solved by setting:
System.Net.ServicePointManager.Expect100Continue = false;
(This is from: HTTP POST Returns Error: 417 "Expectation Failed.")
I'm currently in the final stages of implementing an C# MediaWiki API which allows the easy scripting of most MediaWiki viewing and editing actions.
The main API is here: http://o2platform.googlecode.com/svn/trunk/O2%20-%20All%20Active%20Projects/O2_XRules_Database/_Rules/APIs/OwaspAPI.cs and here is an example of the API in use:
var wiki = new O2MediaWikiAPI("http://www.o2platform.com/api.php");
wiki.login(userName, password);
var page = "Test"; // "Main_Page";
wiki.editPage(page,"Test content2");
var rawWikiText = wiki.raw(page);
var htmlText = wiki.html(page);
return rawWikiText.line().line() + htmlText;
You seem to be pushing the input data on HTTP POST, but it seems you should use HTTP GET.
From the MediaWiki API docs:
The API takes its input through
parameters in the query string. Every
module (and every action=query
submodule) has its own set of
parameters, which is listed in the
documentation and in action=help, and
can be retrieved through
action=paraminfo.
http://www.mediawiki.org/wiki/API:Data_formats