I'd like to determine whether the proxy at a given IP address is transparent or anonymous. Transparent proxies connect to websites with your real IP in headers like HTTP_X_FORWARDED_FOR or HTTP_VIA. I would like to check these proxies, but all solutions I found are developed to work on server side, to test incoming connections for proxyness. My plan is to make a web request to an example page via the proxy. How do I check the headers sent by the proxy, preferably using the WebRequest class?
EDIT: So is there some free web API that will allow me to do this? I'm not keen on setting up a script on my own small server that will be bombarded with requests.
Simply you don't need that headers. I could check transparency of a proxy by sending request to any get-my-IP site, if it returns my IP then it is transparent. If not then the proxy is anonymous. So steps are:
send request to any get-my-IP site without proxies
extract the IP from response as my local IP address
send new request to any get-my-IP site with the proxy
extract the IP from response and compare it with my local IP (step 2)
if(LocalIp==ProxyIp) then the proxy is transparent else it is anonymous
That is technically impossible since the client only sees what the proxy returns back to the client - the proxy can do whatever it wants when communicating with the target server and transform your request and the answer from the server anyway it wants...
To really know what the proxy does you NEED see what the server gets and sends back without any interference from the proxy...
The reason all solutions are server side is that the headers you're talking about are only passed from the proxy to the server and never back to the client again in the response.
In other words, if you plan to check for HTTP headers in the request from the proxy to the server, you either need to check them server side (as the solutions you found do) or actively pass them right back in the response to the client to check.
Either way, you can't just make a request to a random page and check the headers the server gets, the server needs to be involved in some way.
Related
I have simple http server based on HttpListener. Http server is running on local machine (IP: X.X.X.X:10101) behind the firewall. To make Http server visible for external request i have added port forwarding on firewall from external interface Y.Y.Y.Y:8080 to X.X.X.X:10101.
Now: When i receive request from external host i see that HttpListenerRequest.Url is set to Y.Y.Y.Y:10101 - so its kind of mix between internal and external interfaces (IP belongs to external interface, and port belongs to internal interface).
What is the reason of this behavior? I would expect that url should be Y.Y.Y.Y:8080. Is there any way to "reslove" original url that request was posted to?
Best Regards,
It Man
I seems that firewall modifies http request header, changing external port to local.
I'm creating a crawler which uses several IP Proxies. Whenever I tried to crawl the website without proxy, I'm able to get the html source, but when I tried to enable the ip proxy, it always fail and throws an exceptions (The remote server returned an error: (403) Forbidden.)
Upon looking at the fiddler, it seems the website stores cookies upon visit. But if the proxy is enabled, it fails at get response part.
I don't understand why the cookies was not set using a proxy? Is it the proxy server settings for cookies that cause it? or I can do something about it while still enabling proxy?
I'm using C# by the way, but the question doesn't seems language dependent.
Another thing to consider is that you set a cookie from the ip address of the non proxied machine (which worked), then when you sent another request with the same cookie from another ip address which might have gotten you blocked.
Some network level software looks at stuff like this which might have flagged you as a malicious crawler or annonymous tor browser.
I've build a proxy with some logic on request headers.
I use sockets.
Before even any connect occurs, inside my proxy I parse headers, add authentification
and forward the request to target host.
And now I've stumbled upon following issues:
I have to relay https requests
I don't need any body manipulations, I just need to modify headers the same way, forward CONNECT to target, receive an answer and return to source socket.
Questions:
Do I need CA on the proxy side?
Do I need to use SSlStream to forward and receive request?
If so, what could be the algorithm?
The SSL handshake comes after the CONNECT request and its response. If you only need to manipulate the CONNECT request you can simply forward the data after the handshake between the peers, e.g. no SSL interception which means no SSLStream and no CA. But, if you need to manipulate the HTTP headers of the requests inside the HTTPS connection (GET, POST...) you have to do SSL interception (e.g. man-in-the-middle with your own proxy CA and SSLStream).
i want to create a proxy server or (Socks host ??) to communicate with browsers (such as firefox) to forward http proxy.
i have a software that do this: bitvise tunnelier.
so, i need to write an application similiar to tunnelier proxy forwarder.
in other words, i want to enable firefox Socks Host and get browser request with c# application
Please help me lead the way
thanks a lot
Create a proxy means that your application need to act as an HTTP Server when talking to the clients (the browsers) and act as a browser when comunicating with the web sites.
The proxy opens a port for the server socket (for example port 90) and accepts the browser request on this port, than connect the web site (generally on port 80) and forward the request to the site server. The proxy waits for the server response, read it and after send the response to the client.
The proxy changes the messages header if needed ( for example change the port in the url).
In general proxy are multi thread applications, so they can manage more request in the same time.
What .NET class/methods could I use to write a simple HTTP(s) proxy that would run on my PC that would:
Proxy all HTTP(S) requests through it
Let me get bandwidth used per proxy'ed request (e.g. content length)
Let me get requesting application or process name per proxied request
Ability to proxy/stream the requests on through to my normal proxy server (server name, port, username, password)
You want to do this on the client side, right?
I think that WebRequest is probably what you are looking for. As far as how to hook it in to the OS... that's a different question.