We Keep Coding

Hide Http(s) traffic made by my application from Fiddler

I have application which uses http to obtain data from my server like this:
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(requestString);
req.Timeout = 200 * 1000;
req.Headers.Add(String.Format("deleteme: {0}", content));
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
Stream resStream = resp.GetResponseStream();
StreamReader read = new StreamReader(resStream);
html = read.ReadToEnd();
Everything works fine, but how can I hide my requests from Fiddler (something similar to Wireshark)? I want to prevent users to see it.

Fiddler works by registering itself as the http proxy in Windows.
You can disable your application from using the default proxy by setting a specific proxy (like in the code below "no proxy") anywhere in your application before making web requests:
HttpWebRequest.DefaultWebProxy = new WebProxy();
Note that this will also prevent your application from using a configured proxy when one is set-up for legitimate reasons.
This will hide the requests from Fiddler or any other tool that traces web requests by registering itself as http proxy. This will not prevent tracing the request with other tools that operate on a different level in the stack (like Wireshark)
Security by obscurity does not really work. If you want to make it impossible to read the data transferred, use actual encryption.

C# console application not getting intermediate redirects (http 302)

I am trying to get some data downloaded from a URL using HttpWebRequest from a C# console applicaiton. In browser and Postman, it works fine but not from the applicaiton - it does not return the expected data. Using Fiddler, I inspected the request and I figured out that the request initially sent is actually gets redirected thrice (I see three http 302 in Fiddler before the final Http 200 response) and eventually returns the data. However, from my C# console application I get only the final response - the HttpWebResponse status always gives"OK" (200).
I noticed in fiddler that the http 302 returns few cookies and the subsequent request sends the cookies in its header. This is handled correctly in browser/postman but I am not able to do this in my application. Any help will be highly appreciated.
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);

I believe that you are looking for AllowAutoRedirect property of HttpWebRequest class. Setting it to false should do the thing:
HttpWebRequest request = new HttpWebRequest(someUri);
request.AllowAutoRedirect = false;
You can read more info about HttpWebRequest here.

Sending post to creation factory on RTC but getting a 'GET' response

I've hit a strange bit of behaviour and I'm pretty sure is related to my code rather than the RTC instance I'm working with.
I've got a web request setup and configured:
var cookies = new CookieContainer();
var request = (HttpWebRequest)WebRequest.Create(getCreationFactoryUri);
var xmlString = getRDF.ToString();
request.CookieContainer = cookies;
request.Accept = "application/rdf+xml";
request.Method = "POST";
request.ContentType = "application/rdf+xml";
request.Headers.Add("OSLC-Core-Version", "2.0");
request.Timeout = 40000;
request.KeepAlive = true;
byte[] bytes = Encoding.ASCII.GetBytes(xmlString);
request.ContentLength = bytes.Length;
Stream dataStream = request.GetRequestStream();
dataStream.Write(bytes, 0, bytes.Length);
dataStream.Close();
This is passed to another method wrote based on an RTC example using forms authentication for RTC.
Under the OSLC v2 spec, I'm using a creation factory URL to post to. I know the URL is fine because I've setup a call using RESTClient in Firefox. Added the headers that are needed (Content-Type: application/rdf+xml, Accept: application/rdf+xml, OSLC-Core-Version: 2.0) and used the generated XML that my code is trying to pass. My manual call works perfectly and the ticket is created.
In my logs I captured the response from RTC, which is a list of tickets rather than a response showing my ticket as being created. I can re-create this behavior by doing a GET on the creation factory URL I'm using to create an event ticket.
So although I know I'm sending a POST to the creation factory (I debugged to check that my web request method was 100% set to 'POST') RTC instead returns a list of tickets and I can only conclude somewhere my request is treated as a 'GET'.
As a test I changed my request to use PUT instead of POST. This isn't permitted for use on the creation factory URL and in testing it indeed throws an error. So I'm totally miffed as to why RTC isn't creating my ticket, but instead treating my request as a GET and returning a list of tickets.
Anyone have any ideas?
Thanks.

If the server using form authentication, as you state, then I expect what is happening is that the POST is resulting in an HTTP redirection to the authentication form. Even if your other code is handling that authentication (which it sounds like it is), the result of that authentication will be an HTTP redirection to the URL of the original request. However, that redirection is likely to result in a GET to that URL, not the original POST. (Also, I don't believe the redirection after authentication is 100% reliable, if your requests are multi-threaded).
The jazz.net information on form authentication says "After authentication succeeded, you always have to replay the original request at least once to get to the protected resource. More replays may be required if the first replay led to another set of redirections and the original request had a non-GET method."
So if your code received an authentication challenge, you will need to re-send the original POST.
I believe the reason why the RESTClient plug-in in your browser is working first time is that it is sending the cookies from your previous log-in to the RTC web UI in the browser. (I had this experience recently, and also found it very confusing).
Also, if you are not preserving cookies between requests to RTC in your client app, then you will meet an auth challenge for every request. If you preserve cookies between calls from your client app (how you do that will depend on your client library - I'm not familiar with the code in your examples) then my experience is that you won't receive an auth challenge for every request. (However, you still need to be able to handle an auth challenge on every request - including POSTs - otherwise it may fail intermittently if the session times out just before you send a POST).

Windows Authentication for OData service returns "401 - Unauthorized" in C# app, but works in browser

I am trying to add functionality to my C# app, to test a connection to an OData service which is secured with only Windows Authentication. The following block of code is what I am using to perform this test:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(new Uri(SERVICE_NAME));
CredentialCache myCache = new CredentialCache();
myCache.Add(new Uri(SERVICE_NAME), "Negotiate", new NetworkCredential(user, password));
resolver.Credentials = myCache;
// Do a simple request
request.Credentials = myCache;
request.PreAuthenticate = true;
request.KeepAlive = true;
request.AllowAutoRedirect = true;
request.CookieContainer = new CookieContainer();
object response = request.GetResponse(); // This is where the exception is thrown
When I run the above code, I receive the 401 - Unauthorized error as previously stated. However, when I have Fiddler2 running, the code works fine. So I am using Wireshark instead. In addition, the service works perfect within my browser (Chrome), and if I use Wireshark to compare the HTTP requests/responses for the Authentication, I see that they are nearly identical, except that in Chrome I have: Accept, User-Agent, Accept-Encoding, and Accept-Language headers, while my C# app does not have these. The only other difference is that my C# app sets the "Negotiate Seal" flag in the NTLM header, while Chrome does not set this flag.
Despite these differences, the authentication phase seems to work fine in the C# app, up until the service returns a 302 - Redirection, at which point the app tries a GET on the newly redirected URI, which returns a 401 again (when Chrome does the analogous GET, it receives HTTP 200 - OK, and proceeds on its merry way).
So, any ideas what could cause this? Problem with the service? or my code?
Thanks a lot!
-Erik

Ok, two whole days of research and I found the answer. Line 3 in the code above was using the full URI of the service (".../Northwind/Northwind.svc"), when the request was redirected, the credentials no longer applied to the new URI. The solution was to pass in only the beginning part ("...") of the URI. Stupid mistake on my part.

HttpWebRequest: how get the session id

we are using a web service for a web site to communicate with an external server.
External server ask for a session id.
Our following code ask external server:
HttpWebRequest webRequest = WebRequest.Create(ExtUrl) as HttpWebRequest;
webRequest.Credentials = new NetworkCredential(ExtAccountToUse, ExtPassword);
HttpWebResponse webResponse;
webRequest.Method = "POST";
webRequest.ContentType = "application/x-www-form-urlencoded";
StreamWriter writer = new StreamWriter(webRequest.GetRequestStream());
writer.Write(xmlOutput);
writer.Close();
webResponse = webRequest.GetResponse() as HttpWebResponse;
Is it possible to get a session id to send to external server ?
Thanks for your time

That depends on the type of server you are sending the request to. For example, if you have an IIS hosted site, it expects a session id inside a cookie named ASP.NET_SessionId (or on the request string). If you have a Java servlet engine on the other side, it expects a cookie called JSESSIONID (or a request path parameter jsessionid).
So it depends. However, setting cookies inside a HttpWebRequest is not difficult. You can use the property CookieContainer:
CookieContainer cookies = new CookieContainer();
cookies.Add(new Cookie("ASP.NET_SessionId", sessionId));
request.CookieContainer = cookies;
The session identifier you store inside the cookie should have a particular format and again, this depends on the server type at the other end. In ASP.NET by default the class SessionIDManager is used to produce and validate session ids. This class is hard to reuse because it requires an HttpContext. However, you could check with Reflector how it generates a session id.

If you mean the external server requires an existing session ID that identifies a session created by prior requests sent to it then you need to maintain an instance of a CookieContainer for all of the requests involved.
CookieContainer myExternalServerCookies = new CookieContainer();
With every HttpWebRequest you use to talk to the external server include this line:-
request.CookieContainer = myExternalServerCookies;
Now when the external server sets a session cookie, it will see that cookie in subsequent requests.

as far as getting session ID is concerned you can get it using:
Session.SessionID
but I don't think session id on your server is of any interest to external server.