This is probably the weirdest problem I have run into. I have a piece of code to submit POST to a url. The code doesn't work neither throws any exceptions when fiddler isn't running, However, when fiddler is running, the code posts the data successfuly. I have access to the post page so I know if the data has been POSTED or not. This is probably very non-sense, But it's a situation I am running into and I am very confused.
byte[] postBytes = new ASCIIEncoding().GetBytes(postData);
HttpWebRequest req = (HttpWebRequest)WebRequest.Create("http://myURL);
req.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.224 Safari/534.10";
req.Accept = "application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
req.Headers.Add("Accept-Charset", "ISO-8859-1,utf-8;q=0.7,*;q=0.3");
req.Headers.Add("Accept-Language", "en-US,en;q=0.8");
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
req.ContentLength = postBytes.Length;
req.CookieContainer = cc;
Stream s = req.GetRequestStream();
s.Write(postBytes, 0, postBytes.Length);
s.Close();
If you don't call GetResponseStream() then you can't close the response. If you don't close the response, then you end up with a socket in a bad state in .NET. You MUST close the response to prevent interference with your later request.
Close the HttpWebResponse after getting it.
I had the same problem, then I started closing the responses after each request, and Boom, no need to have fiddler running.
This is a pseudo of a synchronous code:
request.create(url);
///codes
httpwebresponse response = (httpwebresponse)request.getresponse();
/// codes again like reading it to a stream
response.close();
I had a similar problem recently. Wireshark would show the HTTPWebRequest not leave the client machine unless Fiddler was running. I tried removing proxy settings, but that didn't fix the problem for me. I tried everything from setting the request to HttpVersion.Version10, enabling/disabling SendChuck, KeepAlive, and a host of other settings. None of which worked.
Ultimately, I just checked if .Net detected a proxy and had the request attempt to ignore it. That fixed my issue with request.GetResponse() throwing an immediate exception.
IWebProxy proxy = request.Proxy;
if (request.Proxy != null)
{
Console.WriteLine("Removing proxy: {0}", proxy.GetProxy(request.RequestUri));
request.Proxy = null;
}
In my case when I had the same situation (POST only works when Fiddler is running) the code was sending the POST from an application running on IISExpress in a development environment behind a proxy to an external server. Apparently even if you have proxy settings configured in Internet Options the environment IIS is running in may not have access to them. In my work environment I simply had to update web.config with the path to our proxy's configuration script. You may need to tweak other proxy settings. In that case your friend is this MSDN page that explains what they are: http://msdn.microsoft.com/en-us/library/sa91de1e.aspx.
Ultimately I included the following in the application's web.config and then the POST went through.
<configuration>
<system.net>
<defaultProxy>
<proxy scriptLocation="http://example.com:81/proxy.script" />
</defaultProxy>
</system.net>
</configuration>
Well i have faced similar problem few weeks back and the reason was that when fiddler is running it changes the proxy settings to pass the request through Fiddler but when its closed the proxy somehow still remains and thus doesn't allow your request to go ahead on internet.
I tried by setting the IE's and Firefox's network settings to not to take any proxy and it worked.
Try this, may it be the same problem...
I ran into the same problem with Python - requests to a local server were failing with a 404, but then when I ran them with Fiddler running they were working correctly.
The real clue to the problem here is that Fiddler works by acting as a proxy for HTTP traffic so that all requests from the local machine go through Fiddler rather than straight out into the network.
In the exact situation I was in, I was making requests to a local server, regular traffic passes through a proxy and in Local Area Network (LAN) Settings for the network connection the Proxy server pane the Bypass proxy server for local addresses option was checked.
My suspicion is that the "Bypass proxy server for local addresses" is not necessarily picked up by the programming language, but the proxy server details are. Fiddler is aware of that policy, so requests through Fiddler work but requests direct from the programming language don't.
By setting the proxy for the request for the local server to nothing, it worked correctly from code. Obviously, that could be a gotcha if you find yourself moving from an internal to external server during deployment.
I faced the same scenario : I was POSTing to an endpoint behind Windows Authentication.
Fiddler keeps a pool of open connections, but your C# test or powershell script does not when it runs without fiddler.
So you can make the test/script to also maintain a pool of open authenticated connections, by setting the property UnsafeAuthenticatedConnectionSharing to true on your HttpWebRequest. Read more about it here, microsoft KB. In both cases in that article, you can see that they are making two requests. The first one is a simple GET or HEAD to get the authentication header (to complete the handshake), and the second one is the POST, that will use the header obtained before.
Apparently you cannot (sadness) directly do the handshake with POST http requests.
Always use using construct. it make sure all resource release after call
using (HttpWebResponse responseClaimLines = (HttpWebResponse)requestClaimLines.GetResponse())
{
using (StreamReader reader = new StreamReader(responseClaimLines.GetResponseStream()))
{
responseEnvelop = reader.ReadToEnd();
}
}
add following entries to webconfig file
<system.net>
<connectionManagement>
<add address="*" maxconnection="30"/>
I found the solution in increasing the default number of connections
ServicePointManager.DefaultConnectionLimit = 10000;
Related
I have application which uses http to obtain data from my server like this:
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(requestString);
req.Timeout = 200 * 1000;
req.Headers.Add(String.Format("deleteme: {0}", content));
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
Stream resStream = resp.GetResponseStream();
StreamReader read = new StreamReader(resStream);
html = read.ReadToEnd();
Everything works fine, but how can I hide my requests from Fiddler (something similar to Wireshark)? I want to prevent users to see it.
Fiddler works by registering itself as the http proxy in Windows.
You can disable your application from using the default proxy by setting a specific proxy (like in the code below "no proxy") anywhere in your application before making web requests:
HttpWebRequest.DefaultWebProxy = new WebProxy();
Note that this will also prevent your application from using a configured proxy when one is set-up for legitimate reasons.
This will hide the requests from Fiddler or any other tool that traces web requests by registering itself as http proxy. This will not prevent tracing the request with other tools that operate on a different level in the stack (like Wireshark)
Security by obscurity does not really work. If you want to make it impossible to read the data transferred, use actual encryption.
I have App that makes use of some web service and acquire data via JSON, all was working fine for quite long time, up until latest discoveries about SSLv3 being vulnerable to man-in-the-middle attacks and server owners turning off SSLv3 for good. My application started to have problems connecting and returned error "Request was aborted: cannot establish secure SSL/TLS connection". I've tried to look for solution and found information i got to add this code before creating web request:
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
ServicePointManager.ServerCertificateValidationCallback = delegate{
return true;
};
Unfortunately no luck here, app acts the same as before, and I have no clue if this code does nothing or there is still some problem with server. Error information is pretty vague and i have problem figuring where things go wrong.
Here is my code
...
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.ContentType = GetRequestContentType();
request.Method = method.ToString();
request.Credentials = GetCredential(url);
request.PreAuthenticate = true;
CookieContainer cookieContainer = new CookieContainer();
request.CookieContainer = cookieContainer;
...
I want to ask how to set Tls12 to be used as default and ensure that at my end request I make is with desired protocol.
If I confirm that my app at my end works fine, is there way to get more detailed information from server response and pinpoint precise reason of error?
Thanks for all answers and suggestions.
EDIT
Second part of question is solved, I found this tool http://www.telerik.com/download/fiddler it pretty much allows to see what is going on with outgoing and incoming data. There is also thing that this tool allow to decode SSL connections, enabling this option makes that my application starts to work. I assume that this app does something that make communication between my app and destination host possible. But i do still have no idea what it could be. And how to make my app to handle these connections properly by itself.
Being desperate made me to inspect whole source code (part responsible for getting data of the internet was 3rd party and until it worked fine there was no reason to change it) and I discovered that line
request.Credentials = GetCredential(url);
called method that in its body had
ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3;
So all my attempts to change that value before creating httpwebrequest was overwritten. Changing SecurityProtocolType to Tls12 makes it all work now.
Issue:
Consider the following working code.
System.Net.WebProxy proxy = new System.Net.WebProxy(proxyServer[i]);
System.Net.HttpWebRequest objRequest = (System.Net.HttpWebRequest)System.Net.WebRequest.Create(https_url);
objRequest.Method = "GET";
objRequest.Proxy = proxy;
First, notice that proxyServer is an array so each request may use a different proxy.
If I comment out the last line thereby removing the use of any proxies, I can monitor requests in Fiddler just fine, but once I reinstate the line and start using them, Fiddler stops logging outbound requests from my app.
Question:
Do I need to configure something in Fiddler to see the requests or is there a change in .Net I can make?
Notes:
.Net 4.0
requests are sometimes https, but i don't think this is directly relevant to issue
all requests are outbound (not localhost/127.0.0.1)
Fiddler is a proxy itself. By assigning a different proxy to your request.. you're essentially taking Fiddler out of the equation.
If you're looking to capture traffic and use your own proxy.. you can't use a proxy (by definition that makes no sense).. you want a network analyzer, such as WireShark. This captures the traffic instead of having the traffic routed through it (as a proxy does), allowing you to have it monitor traffic and route your requests through your custom proxy.
Basically what I've been trying to do is download a file off a server. The server sends a redirect automatically which is fine, but through packet sniffing a program that does successfully download the file I've found that the Headers (for the second request) are:
GET /path/to/file.txt
...
Host: server.com
Rather than the current response being generated (what I thought was standard):
GET www.server.com/path/to/file.txt
Using the normal HttpWebRequest method results in a 500 server error, and I get exceptions thrown when trying to use just the relative path as one would expect.
Using AllowAutoRedirect does not work for this scenario as the cookies are not handled properly, but even if I handle it manually the same error occurs.
How does one go about doing this (preferably without sockets :D)?
To be honest, I'm really not sure what you are asking, but you mentioned cookie troubles. As a total shot in the dark guess, are you setting the CookieContainer on your WebRequest?
request.CookieContainer = new CookieContainer();
request.AllowAutoRedirect = true;
I am tring to debug whats wrong with my HTTP requests from another question here on SO. So i read a bit about Fiddler and wanted to use it to debug my problem. But I can't seem to get traffic from my WPF application to go through Fiddler. I believe I need to configure a proxy. I am using a WebClient for a basic example, but I think i will require a WebRequest later. But for now, with a simple WebClient, how can I get it to go through Fiddler (I believe I have to set proxy to localhost:8888)?
UPDATE:
I don't know if i did the right thing or not but I tried
var wc = new WebClient();
WebProxy proxy = new WebProxy();
proxy.Address = new Uri("http://localhost:8888");
wc.Proxy = proxy;
but failed - I don't see any traffic in Fiddler
I tried ...
var wc = new WebClient();
WebProxy proxy = new WebProxy("127.0.0.1", 8888);
wc.Proxy = proxy;
still nothing
I found the solution at this fiddler2.com page
Why don't I see traffic sent to
http://localhost or http://127.0.0.1?
Internet Explorer and the .NET Framework are hardcoded not to send
requests for Localhost through any
proxies, and as a proxy, Fiddler will
not receive such traffic.
The simplest workaround is to use your machine name as the hostname
instead of Localhost or 127.0.0.1. So,
for instance, rather than hitting
http://localhost:8081/mytestpage.aspx,
instead visit
http://machinename:8081/mytestpage.aspx.
Maybe a little late, but...
I get around this simply by appending a "dot" to localhost, so instead of accessing localhost, I try to access localhost. (notice the dot at the end of the hostname)
Credit where credit is due:
I got this unusual tip from this thread http://www.west-wind.com/weblog/posts/2009/Jan/14/Monitoring-HTTP-Output-with-Fiddler-in-NET-HTTP-Clients-and-WCF-Proxies#596591
Works fine!
You can find answer in below post
https://stackoverflow.com/a/7506427/471499
it lists that you need to add this in your web.config OR App.Config
<system.net>
<defaultProxy>
<proxy bypassonlocal="False" usesystemdefault="True" proxyaddress="http://127.0.0.1:8888" />
</defaultProxy>
</system.net>
then Start Fiddler on the same machine as the application running.
Click Tools | Fiddler Options => Connections => adjust the port as 8888.(allow remote if you need that)
Ok, then from file menu, capture the traffic.
That's all, but don't forget to remove the web.config lines after closing the fiddler, because if you don't it will make an error.
Run Fiddler for DotNet Core requests
RUN FIDDLER for .net core required "Netsh" tool https://learn.microsoft.com/en-us/windows-server/networking/technologies/netsh/netsh-contexts)
command to add proxy :
netsh winhttp set proxy 127.0.0.1:8880
After run the proxy, adjust Fiddler proxy to the same port, and enjoy
remove proxy
netsh winhttp reset proxy
Reference :
http://fiddler2.com/documentation/Configure-Fiddler/Tasks/UseFiddlerAsReverseProxy
https://docs.telerik.com/fiddler/configure-fiddler/tasks/configuredotnetapp
All the time I use below configuration to redirect the network HTTP calls to pass thru fiddler proxy from my applications.
This works in all kinds of .NET applications (which has either web.config or app.config file) and in fiddler its best to disable Capture Traffic option to avoid capturing general traffic from all the applications running. Shortcut key for this is F12.
<system.net>
<defaultProxy>
<proxy proxyaddress="http://localhost:8888/" />
</defaultProxy>
</system.net>
This is valuable configuration if you have third party assemblies in which you don't have chance of changing the code that calls URL.
I hope this helps someone.
"IIS Express won't receive traffic to machinename so instead route to localhost.fiddler fiddler2.com/documentation/Configure-Fiddler/Troubleshooting/… – robrich May 9 '13 at 6:02"
RobRich above got it right.
This is the only thing that worked as I can only use IIS Express.