I'm trying to come up with the most appropriate way to make a two way socket connection through a HTTP proxy - lets say it's a telnet style protocol. Unfortunately I also need to support NTLM authentication (with the proxy) as well as Basic and Digest, in addition to any other future authentication mechanisms that I can't forecast.
If it was just basic and digest I'd handle the connection myself, but I really don't want to get stuck in the mire that is NTLM. Looking at the underlying AuthenticationManager API it looks very tied to HttpWebRequest so I can't leverage that functionality if I'm using a socket/tcpclient/whatever or even writing a new WebRequest derivation.
Playing around with HttpWebResponse yields a stream that can't be written, using the RequestStream after the response stream has been retrieved gives a concurrent io exception.
Having run through all the possibilities I can think of, I've come up with some nasty code that gets out the NetworkStream associated with a HttpWebRequest which allows two way communication:
.....
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
Stream str = resp.GetResponseStream();
System.Type type = str.GetType();
PropertyInfo info = type.GetProperty("Connection", BindingFlags.NonPublic|BindingFlags.Instance| BindingFlags.Public);
object obj = info.GetValue(str, null);
type = obj.GetType();
info = type.GetProperty("NetworkStream", BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.Public);
object obj2 = info.GetValue(obj, null);
NetworkStream networkStream = obj2 as NetworkStream;
Which I'm fairly repulsed by (it won't work with Mono for a start), so I'm wondering if there's a better way using public APIs which will allow me to leverage the built in runtime functionality of proxy authentication.
HTTP is two way. Clients can send a dataless request using HTTP GET (although even then data can be put in the URL or headers), or they can send data using HTTP POST, and the server gets to send a response with headers and data.
If when you say "two way" you were thinking something more like a simple TCP socket where client and server read and write at will, then sorry, but that's not what HTTP does. The client sends a request and the server sense a response. That's all. Technically, if you didn't have a client side API getting in the way, enforcing the intended constraints of HTTP, and you could cook up your own non-standard server, you could have multiple client<->server exchanges within a single HTTP request, but at that point it wouldn't really be HTTP anymore, it'd be a TCP connection with an HTTP like handshake, and your proxy might not even allow it.
That said, it sounds like you don't really need to write to the response stream at all, either you're quite confused and you just need to do a POST (see GetRequestStream), or you're just a little confused and you can just send a new request after you've processed the response. You can even reuse the same HttpWebRequest instance once you've called the .Close method on the WebResponse you got. And all this will happen on the same TCP socket (if your server and proxy support it).
Okay, I hope that all made sense. If it didn't answer your question one way or the other, just provide a little more detail about what you're trying to accomplish with regard to "two-way" communication. I understand that you have the constraint of going through an HTTP proxy w/ HTTP authentication requirements which limits things a lot.
Related
I'm using HttpClient to send an asynchronous POST request to a remote web server. That remote web server responds with the Connection header set to keep-alive. Eventually, it will close the connection.
What I can't figure out how to do is keep receiving data until the connection is set to close.
For example, consider the following code:
HttpClient client = new HttpClient();
string requestUri = ...; //My GET uri
string content = ...; //url-encoded contents of the request
HttpResponseMessage response = await client.PostAsync(requestUri, new StringContent(content));
In the code above, response will have the Connection header set to keep-alive. I couldn't find any members on HttpResponseMessage that seem to give me any means of continuing to receive information after the first keep-alive is received.
I can't figure out how to continue receiving responses until the Connection closes.
I was able to find all sorts of resources that dealt with sending keep-alive requests. That's not my issue. My issue is in receiving/handling a keep-alive response from the server.
Please let me know if there is anymore information that I could provide to improve this question.
Connection: keep-alive purpose is not to enable server to send multiple responses for the same single request - this would fundamentally violate how HTTP works.
Connection: keep-alive is server's way of signaling to the client that client can submit next request using the same, already established, TCP connection, thus help avoid the costs of establishing new connection (and possibly TLS handshake as well).
Reuse of the connection is (typically) handled by underlying library, and is abstracted away such that user (as in developer) of the library need not worry about handling.
All you should be doing is keep making as many client.Post/Get/etc requests, and the HttpClient library will take care of managing the underlying TCP connection and its reuse.
There is one area in HTTP that may look counter to the above and thus confusing, but is not. The case is when server returns HTTP Status 100 Continue. In this case you will see another response come from the server for still the same original request (and by "see" I mean not from HttpClient, but rather on the wire if you were to snoop with a tool like WireShark or a proxy). What is happening in this case is if a client is making a large request, the server simply signals to the client that it is continuing to accept and read request, and for the client to continue sending that original request. Once full request is received, the server will process and respond with a final response code and message. And again, HttpClient will abstract away this interim 100 Continue response, so as developer you need not worry about it (in most if not all cases).
If all you want is receive the response for a given request you don't need to deal with keep-alive at all. Keep-alive is purely a performance optimization. The framework transparently manages connections.
All you do is send a request and receive a response. There is exactly one response per request.
One side will eventually close the connection which is not of any concern to you. You don't notice any effect of that.
I'm doing research into web programming in ASP .NET and came across these two classes. I was wondering what might these be used for?
My first thought is that they could be used if you were setting up a proxy between the client and server, but I'm not sure if this is the main purpose or not.
Thanks
edit: classes not methods
They can indeed be used for that. This isn't specific to ASP.NET however.
You can create a HttpWebRequest object by doing:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://url.com");
And you invoke it to retrieve an HttpWebResponse:
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
There's a lot of customization you can do here, but hopefully this will give you a starting point for reading data off the web.
As their names imply, these are classes to either create a request (HttpWebRequest) or create a response (HttpWebResponse).
Using HttpWebRequest you define a request to a URI using the HTTP-protocol. Whereas the HttpWebResponse class delivers you an answer of an HTTP-server providing all the information like HTTP-headers and actual body of the request.
This is an example from MSDN
HttpWebRequest HttpWReq =
(HttpWebRequest)WebRequest.Create("http://www.contoso.com");
HttpWebResponse HttpWResp = (HttpWebResponse)HttpWReq.GetResponse();
// Insert code that uses the response object.
HttpWResp.Close();
Consider the casting from the base-class WebRequest.
See HttpWebRequest and HttpWebResponse.
They are used for communicating with another process using the HTTP protocol.
In the context of ASP.NET, your process could use them to talk to another service. Maybe your database uses the HTTP protocol, such as CouchDB. Perhaps you have a rest service that your ASP.NET application needs to talk to.
I have a Perl based REST service and I'm using C# and WCF to make a client to talk to the service. I have a few expensive calls and would like to construct a caching system. I need the ability to check and see if newer versions of the cached data exist on the server. I had the idea to use the standard "If-Modified-Since" request header and "304 Not Modified" response status code, however I'm having trouble catching the exception that is thrown on the response.
My client class derives from ClientBase<>. Here is the method that I use to call a service method:
private T RunMethod<T>(ReqHeaderType reqHeaders, ResHeaderType resHeaders, Func<T> meth)
{
//Get request and response headers
var reqProp = GetReqHeaders(reqHeaders);
var resProp = GetResHeaders(resHeaders);
using (var scope = new OperationContextScope(this.InnerChannel))
{
//Set headers
OperationContext
.Current
.OutgoingMessageProperties[HttpRequestMessageProperty.Name] = reqProp;
OperationContext
.Current
.OutgoingMessageProperties[HttpResponseMessageProperty.Name] = resProp;
//Return the result of the call
return meth();
}
}
The exception occurs when the call back, which runs the service method, is executed. Is there a way to catch the exception and check if it is a "Not Modified" response?
In my opinion, you really only want to use WCF channels on the client if you are using non-web WCF bindings on the server.
In your case you are not even using .Net on the server so I think WCF is going to cause you a whole lot of pain.
I suggest you simply use the HttpWebRequest and HttpWebResponse classes in System.Net. If you do that you can also take advantage of the built in caching that is provided by WinINet cache. If you set the caching policy in the client Http client you will get all the caching behaviour you need for free.
I'm narrowing in on an underlying problem related to two prior questions.
Basically, I've got a URL that when I fetch it manually (paste it into browser) works just fine, but when I run through some code (using the HttpWebRequest) has a different result.
The URL (example):
http://208.106.250.207:8192/announce?info_hash=-%CA8%C1%C9rDb%ADL%ED%B4%2A%15i%80Z%B8%F%C&peer_id=01234567890123456789&port=6881&uploaded=0&downloaded=0&left=0&compact=0&no_peer_id=0&event=started
The code:
String uri = BuildURI(); //Returns the above URL
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(uri);
req.Proxy = new WebProxy();
WebResponse resp = req.GetResponse();
Stream stream = resp.GetResponseStream();
... Parse the result (which is an error message from the server claiming the url is incorrect) ...
So, how can I GET from a server given a URL? I'm obviously doing something wrong here, but can't tell what.
Either a fix for my code, or an alternative approach that actually works would be fine. I'm not wed at all to the HttpWebRequest method.
I recommend you use Fiddler to trace both the "paste in web browser" call and the HttpWebRequest call.
Once traced you will be able to see any differences between them, whether they are differences in the request url, in the form headers, etc, etc.
It may actually be worth pasting the raw requests from both (obtained from Fiddler) here, if you can't see anything obvious.
Well, the only they might differ is in the HTTP headers that get transmitted. In particular the User-Agent.
Also, why are you using a WebProxy? That is not really necessary and it most likely is not used by your browser.
The rest of your code is fine.. Just make sure you set up the HTTP headers correctly. Check this link out:
I would suggest that you get yourself a copy of WireShark and examine the communication that happens between your browser and the server that you are trying to access. Doing so will be rather trivial using WireShark and it will show you the exact HTTP message that is being sent from the browser.
Then take a look at the communication that goes on between your C# application and the server (again using WireShark) and then compare the two to find out what exactly is different.
If the communication is a pure HTTP GET method (i.e. there is no HTTP message body involved), and the URL is correct then the only two things I could think of are:
make sure that your are send the right protocol (i.e. HTTP/1.0 or HTTP/1.1 or whatever it is that you should be sending)
make sure that you are sending all required HTTP headers correctly, and obviously that you are not sending any HTTP headers that you shouldn't be sending.
There could be something wrong with the URL. Instead of using a string, it's usually better to use an instance of System.Uri:
String url = BuildURI(); //Returns the above URL
Uri uri = new Uri(url);
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(url);
req.Proxy = new WebProxy();
using (WebResponse resp = req.GetResponse()) {
using (Stream stream = resp.GetResponseStream()) {
// whatever
}
}
I think you need to see exactly what's flowing to your server in the HTTP request. Does sound likely that the headers are interestingly different.
You can introduce a some kind of debugging proxy between your request and the server (for example RAD has such a capability in the box).
I have built a web proxy from scratch (using Socket and NetworkStream classes). I am now trying to implement SSL support for it so that it can handle HTTPS requests and responses. I have a good idea of what I need to do (using SslStream) but I don't know how to determine if the request I get from the client is SSL or not.
I have searched for hours on this subject and have been unable to find a suitable solution.
After I do this:
TcpListener pServer = new TcpListener(localIP, port);
pServer.Start(256);
Socket a_socket = pServer.AcceptSocket();
How do I know if I need to read the information using SslStream or NetworkStream?
Client will send you a CONNECT method request after this point you need to just redirect the traffic.
Sample Connect :
CONNECT www.google.com:443 HTTP/1.1
After seeing this just switch to data redirect mode. You can not intercept or read the data so you don't need to worry about SSLStream anyway, you won't touch it.
However if you want to MITM (man in the middle) then you need to switch to SSL otherwise just redirect whatever comes to the target URL and port, that's it.
Obviously client browser will popup with an SSL certificate exception if you intercept the request.
You need to add support for the CONNECT command.
http://www.codeproject.com/KB/IP/akashhttpproxy.aspx
This is why proxy clients use one proxy for HTTP and different one for HTTPS. You can't know what type of connection you're going to receive.