We Keep Coding

https request fails only in .net web app

I am trying to patch a .net web application that after years of working started failing to get UPS shipping quotes, which is impacting web business dramatically. After much trial and error, I found the following code that works just fine in a console application:
static string FindUPSPlease()
{
string post_data = "<xml data string>";
string uri = "https://onlinetools.ups.com/ups.app/xml/Rate";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.Method = "POST";
request.KeepAlive = false;
request.ProtocolVersion = HttpVersion.Version10;
byte[] postBytes = Encoding.ASCII.GetBytes(post_data);
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = postBytes.Length;
Stream requestStream = request.GetRequestStream();
requestStream.Write(postBytes, 0, postBytes.Length);
requestStream.Close();
// get response and send to console
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Console.WriteLine(new StreamReader(response.GetResponseStream()).ReadToEnd());
Console.WriteLine(response.StatusCode);
return "done";
}
This runs in Visual Studio just fine and gets a nice little response from UPS that the XML is, of course, malformed.
But, if I paste this function into the web application without changing a single character, an exception is thrown on request.GetRequestStream():
Authentication failed because the remote party has closed the transport stream.
I tried it in a couple of different place in the application with the same result.
What is there about the web application environment that would affect the request?

It turns out to be a TLS issue. I guess the console app uses a higher protocol by default than the web application, although none was specified. So, all you have to do is add the following line(s) of code sometime prior to making the request:
using System.Net;
...
System.Net.ServicePointManager.SecurityProtocol |= SecurityProtocolType.Tls11 | SecurityProtocolType.Tls12;
That was all it took, though I spent an enormous amount of getting there.
Here is the response from UPS on the issue:
Effective January 18, 2018, UPS will only accept TLS 1.1 and TLS 1.2 security protocols... 100% of requests from customers who are on TLS 1.0 while using production URLS (onlinetools.ups.com/tool name) will be rejected.
Anyway, hope this helps someone.
Jim

Can you try setting the Credentials to your request object like following.
request.Credentials = System.Net.CredentialCache.DefaultNetworkCredentials;

Try setting the default credentials or check if there is any proxy server set and pass it like in the example below.
The example is given for WebClient.
I was having problem with setting Default Credential, as proxy was enabled on the server. So i passed the proxy URL and port with credentials which can access it.
using (System.Net.WebClient web = new System.Net.WebClient())
{
//IWebProxy defaultWebProxy = WebRequest.DefaultWebProxy;
//defaultWebProxy.Credentials = CredentialCache.DefaultCredentials;
//web.Proxy = defaultWebProxy;
var proxyURI = new Uri(string.Format("{0}:{1}", proxyURL, proxyPort));
//Set credentials
System.Net.ICredentials credentials = new System.Net.NetworkCredential(proxyUserId, proxyPassword);
//Set proxy
web.Proxy = new System.Net.WebProxy(proxyURI, true, null, credentials);
web.Headers.Add("Content-Type", "application/x-www-form-urlencoded");
var result = web.UploadString(URL, "");
return result;
}

How do I specify a global HttpWebRequest timeout?

I'm using a twitter library that uses HttpWebRequest internally to make requests to the twitter API. For some odd reason, the requests sometimes take a lot of time to complete (~10 minutes).
The HttpWebRequest object is not exposed by the library.
Is there any way to specify a global timeout and readwritetimeout for the requests, perhaps via app.config?

Unfortunately not currently possible. The constructor of HttpWebRequest has this value hardcoded - reference source.

That timeout is in milliseconds - so 2000ms = only 2 seconds.
System.Net.HttpWebRequest req = (System.Net.HttpWebRequest)System.Net.WebRequest.Create("URL");
req.Timeout = Convert.ToInt32(ConfigurationManager.AppSettings["timeOut"]);
Req.ReadWriteTimeout = Convert.ToInt32(ConfigurationManager.AppSettings["readWriteTimeout "]);
App.config
<appSettings>
<add key="timeOut" value="200" />
<add key="readWriteTimeout " value="10000" />
</appSettings>
Timeout = time spent trying to establish a connection (not including lookup time)
ReadWriteTimeout = time spent trying to read or write data after connection established

How to use IE proxy server settings and its credentials in .Net Application

How to use the saved proxy settings and credentials as default for HttpWebRequests? The proxy settings are accessible and used but not the credentials:
IWebProxy proxy = WebRequest.GetSystemWebProxy();
proxy.Credentials = CredentialCache.DefaultNetworkCredentials;
WebRequest.DefaultWebProxy = proxy;
Are there any permissions on using the credentials?
It works with passing the credentials via NetworkCredential instance:
proxy.Credentials = new NetworkCredential("username", "password");
But I would like to use the saved ones from the operating system/IE.
Edit:
I am using a third party lib creating and calling the HttpWebRequests which should be passed through the proxy. Could that be part of the problem?
Using the App.Config file with this content doesn't work either
<configuration>
<system.net>
<defaultProxy enabled="true" useDefaultCredentials="true">
</defaultProxy>
</system.net>
</configuration>

I was having this exact same issue maybe two weeks ago!! HTTP 407 errors. I tried both suggestions, set network credentials and add the default proxy line in the app.config but neither worked. Strange too because I didn't have this issue on a vb application I wrote last year that did the exact same thing, go out to a web site and download the source as a string. When I did it then, I ran into this issue but adding the defaultProxy line in App.config worked! Weird.
This is how I was able to get around the issue.
I don't create a proxy, I assign the default one (pulled from app.config) to the webRequest's proxy attribute. I then ensure that "UseDefaultCredentials" is set to "true".
HttpWebRequest request = (HttpWebRequest.Create(m.Url) as HttpWebRequest);
request.Proxy = WebRequest.DefaultWebProxy;
request.UseDefaultCredentials = true;
I added this to the app.config (same as the above post).
<system.net>
<defaultProxy useDefaultCredentials="true"/>
</system.net>
Here's the kicker (and I wish I could explain why this solved my issue). Right after I create the request and assign the proxy with the correct credentials... I had to create a cookie container and assign it to my web request. Seriously, I don't know why this solved this issue because I didn't have to do it before.
CookieContainer cc = new CookieContainer();
request.CookieContainer = cc;
I hope this helps you.
Here's the complete code (minus the app.config lines):
HttpWebRequest request = (HttpWebRequest.Create(m.Url) as HttpWebRequest);
request.Proxy = WebRequest.DefaultWebProxy;
request.UseDefaultCredentials = true;
CookieContainer cc = new CookieContainer();
request.CookieContainer = cc;
HttpWebResponse response = (request.GetResponse() as HttpWebResponse);

As klugerama already suggested you should use CredentialCache.DefaultNetworkCredentials.
This contains the network credentials even when they seem empty.
I had like to paste another answer from this post on SO with an excellent explanation:
The NetworkCredential returned from CredentialCache.DefaultCredential is just a placeholder. ... Internal API check for this type to see if integrated authentication should be used or not.
So the things you should check is:
Is there really a problem or do you think there is because of the missing username and password?
Check which account is used as 'default network credentials' (check the Credential Manager control panel)

Check your app.config (or web.config) file, if it exists. If these proxy settings are going to be application-wide, then make sure you have the following in that file (create it if necessary):
<configuration>
<system.net>
<defaultProxy enabled="true" useDefaultCredentials="true">
</defaultProxy>
</system.net>
</configuration>
You won't need to have anything else in your code; all WebRequest instances will simply use the proxy.
If your proxy settings are not going to be application-wide, then make sure those settings are not in the .config file above, and be sure to set them in code as required.

This worked for me. Of course you will have to add the URL. The code snip-it was pulled from an app I made that sends XML Soap Requests and responses to one companies web-service that needed to be usable while on or off another companies Cisco Any-Connect network.
public HttpWebRequest CreateWebRequest()
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("url");
IWebProxy proxy = request.Proxy;
if (proxy != null)
{
string proxyuri = proxy.GetProxy(request.RequestUri).ToString();
request.UseDefaultCredentials = true;
request.Proxy = new WebProxy(proxyuri, false);
request.Proxy.Credentials = System.Net.CredentialCache.DefaultCredentials;
}
return request;
}
App.config file:
<system.net>
<defaultProxy enabled="true" useDefaultCredentials="true">
</defaultProxy>
</system.net>

WCF REST based GET request returing raw XML

What I am trying to accomplish is adding a GET method to my WCF REST based service and accessing it via the WebRequest class from a Silverlight 3 client application.
I am getting the error The remote server returned an error: NotFound. which, as I understand it, can be just a generic error for any 500 error encountered on the server.
WCF operation contract:
[OperationContract, WebGet(UriTemplate = "path/{id}")]
Stream Get(string id);
Operation implementation:
public Stream Get(string id)
{
WebOperationContext.Current.OutgoingResponse.ContentType = "application/xml; charset=utf-8";
return new MemoryStream(Encoding.UTF8.GetBytes("<xml><id>1</id><name>Some Name</name></xml>));
}
Client code that throws exception:
HttpWebRequest webRequest = WebRequest.CreateHttp("http://domain.com/my-service.svc/path/1");
webRequest.BeginGetResponse(
x =>
{
try
{
using (WebResponse webResponse = webRequest.EndGetResponse(x)) <--Exception thrown here
using (Stream stream = webResponse.GetResponseStream())
{
//do stuff here...eventually.
}
}
catch (Exception ex)
{
}
},
null);
I suspect that it has something to do with the return type and have also tried returning XmlElement to no avail. I am really stumped here, any ideas what I might be doing wrong?
Note that I can successfully hit the method via Fiddler and a web browser.

Try putting the code below into your web.config file(change the file name in the initializeData attribute appropriately.
If you are using full IIS, and not Casini or IIS Express (I use the latter), make sure to put the log file somewhere where your web application has write permissions). This will cause WCF to generate a fairly detailed log file. I've found the log to be pretty handy.
<system.diagnostics>
<sources>
<source name="System.ServiceModel"
switchValue="Information, ActivityTracing"
propagateActivity="true">
<listeners>
<add name="traceListener"
type="System.Diagnostics.XmlWriterTraceListener"
initializeData= "c:\temp\WEBTraces.log" />
</listeners>
</source>
</sources>
</system.diagnostics>

Here's another thing to check: Is domain.com exactly the same domain name as your silverlight app is running from (example -- is your SL app starting as localhost/xx and your web service call going to domain.com?
For security reasons, Silverlight will not make cross-domain web service calls unless the called domain grants it permission (same as Flash). If this is the case, you will need a clientaccesspolicy.xml file.
You can read about it here: http://weblogs.asp.net/jgalloway/archive/2008/12/12/silverlight-crossdomain-access-workarounds.aspx
There is a video here: http://www.silverlight.net/learn/data-networking/introduction-to-data-and-networking/how-to-use-cross-domain-policy-files-with-silverlight
There are some helpers here: http://timheuer.com/blog/archive/2008/04/06/silverlight-cross-domain-policy-file-snippet-intellisense.aspx

NotFound should mean 404 and not 500. A 404 error could be produced by a wrong URI.
Uri resturi = new Uri(String.Format("http://{0}:8080/MyService/", hostname)); // http
WebHttpBinding rest = new WebHttpBinding(WebHttpSecurityMode.TransportCredentialOnly); // WebHttpSecurityMode.Transport for ssl
host.AddServiceEndpoint(typeof(IMyService), rest, resturi);
In the code example above your service would be aviable via http://host:8080/MyService/path/1

.NET Web Service receive HTTP POST request (500) Internal Server Error

I am currently writing a C# web service which has several methods, one of which has to receive HTTP POST requests. The first thing i have done is alter the web.config file in the web service project as below.
<webServices>
<protocols>
<add name="HttpSoap"/>
<add name="HttpPost"/>
<add name="HttpPostLocalhost"/>
<add name="Documentation"/>
</protocols>
</webServices>
I can run the web service locally and when i click on the method in the browser, i can see it handles HTTP POST requests and accepts args=string, as my signature of the web method accepts one string parameter named args. I am then testing this via a test ASP.NET app using the code below to fire the HTTP POST request.
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(ConfigurationManager.AppSettings["PaymentHubURL"].ToString());
request.KeepAlive = false;
request.ContentType = "application/x-www-form-urlencoded";
request.Method = "POST";
StringBuilder sb = new StringBuilder();
sb.Append("message_type=");
sb.Append(HttpUtility.UrlEncode("Txn_Response"));
byte[] bytes = UTF8Encoding.UTF8.GetBytes(sb.ToString());
request.ContentLength = bytes.Length;
using (Stream postStream = request.GetRequestStream())
{
postStream.Write(bytes, 0, bytes.Length);
}
string test;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
StreamReader reader = new StreamReader(response.GetResponseStream());
test = reader.ReadToEnd();
}
But when i run this i get "The remote server returned an error: (500) Internal Server Error". If i remove the parameter, by removing the stringbuilder and byte code, as well as having no parameter in the web service, it works. So it is obviously a problem with the parameters. I actually want to send more data, and was using a string[] parameter in the web service, but this also failed.
Can anyone help??

I would suggest that you reconsider your approach. Microsoft has written pretty awesome libraries for consuming web services, but there are two ways to do it - "add web reference" and "add service reference".
In your case, it seems you have an "asmx web service" so I would recommend that you add a "web reference" to you project in visual studio. This is assuming you are using visual studio.
After you add this web reference, you can create your client by "new"-ing it. You can the execute any web method on this client. This is the easiest way to consume web services. You do not have to deal with any http complications.
Hope this helps.

I can guess you are building the HttpPost request wrongly.
Try to use the code showed at the link below to create your request:
http://msdn.microsoft.com/en-us/library/debx8sh9.aspx
Or probabily the response doesn't contain unicode char value
try to copy and paste this code to get the response
System.Text.Encoding encode = System.Text.Encoding.GetEncoding("utf-8");
StreamReader objSR;
webResponse = (HttpWebResponse)response.GetResponse();
StreamReader reader = webResponse.GetResponseStream();
objSR = new StreamReader(objStream, encode, true);
sResponse = objSR.ReadToEnd();