How would I go about creating an application that can somehow see URLs I am viewing in most popular browsers?
Can it be done out of process?
In what ways can this be done?
Can it be done without browser plugins?
The simplest answer is use a proxy server.. prevent html access out except from the proxy server so they have to use it, then all url requests go to the proxy server, even if you choose to make it yourself, then you can collect the URLS either from logs or something and off you go.
You will have to install something in your network infrastructure and then collect data from it.
That could be a proxy server (eg Fiddler) or you can look at tools like WireShark (pcap).
Related
i need an open source app (c# or c++ ) that grab all windows http traffic and forward to an http proxy
there is an app name Proxifier do this but i want open source project to embed to my app or at least a free commandline program that do this
thanks so much
You can use Fiddler2 to first intercept web traffic and force it to your own proxy.
Tools > Options > Gateway and set your proxy.
Or consume FiddlerCore in your own app.
Commandline wise I can recommend proxychains.
http://proxychains.sourceforge.net/
For a full blown proxying solution you might want to look into Privoxy.
http://www.privoxy.org/
There is also squid proxy for instance and many other solutions. The solutions I mentioned are rather unix/linux centric so you might need cygwin or similar to drive this.
The last time I needed something like this I used the mentalis proxy, which has a BSD style license.
I am thinking an HttpModules for IIS, a Pre and Post request handler and some custom module for Apache.
Please only suggest Open Source, we will need to modify the projects.
We want to run these on our high volume production systems so tools that emphasise debugging is not what we are looking for. We want the simplest capture paradigm.
We want to capture data for particular web sites, not for interfaces so I am not looking at packet sniffers.
We would prefer an in-process solution which means avoiding wire-shark, squid or other proxy based solutions.
A nice to have would be the ability to replay a stream of requests in another environment, but I am confident we can build this ourselves if we can capture the requests in a sensible manner.
Any ideas, suggestions and questions will gladly be entertained.
A proxy server that logs is best, but you've already ruled that out. Outside of that, you need a product per web server. For IIS, yes, an HttpModule is the way to go. To replay them, that's likely a carefully crafted log viewer. This is typically a feature of a proxy server. Even Fiddler has the ability to capture and replay.
I've some fishy application that makes HTTP requests to a website, i would like to intersect that request and send other data to the server. Is that possible in C#,java or C++?
EDIT: The application isn't mine, i just know the endpoint that it sends http requests
Fiddler might provide the functionality you need. At the very least it may enable you to see what is being sent to the web site.
in Java You can intercept request from Filter
You may want to look into using an HttpModule, whose purpose is to intercept incoming HTTP requests.
The ASP Column: HTTP Modules
Firstly are you aware of how it is connecting to the internet? For example, is it using the settings from Internet Explorer, or is it establishing a direct connection? If the latter, this may be tricky, there is no direct port forwarding as there in Linux, so you'll need some third-party tools to redirect the traffic to a server (which you can write in Java, C++ or C#, I would go for C# if you know it for pure speed of development) In that server you can intercept the request, and then create your own to actually send to the real destination.
Sounds like a cludge, but I think you're stuck with this approach due to the lack of direct port forwarding. You'll have to configure the third-party tool that you use to forward someother well known port to 80, and your server should write to this.
I want to check which web sites is open in browsers ( IE, Firefox, Chrome ) to write a program in C# which can block web site which is in list of forbidden web site. Is there some API of browsers?
The better solution can be to write a TCP/IP filter, like most firewalls do.
UPD: this topic can be relevant: How do I hook the TCP stack in Windows to sniff and modify packets?
There is no generic "browser API" that allows access to this kind of information across all browsers.
I'm pretty sure the idea of doing this by accessing the browsers is doomed from the start. It is going to be almost impossible to implement, require frequent updates, and always extremely easy to circumvent (there are dozens and dozens of browsers that your program will not know.).
The only reliable way is to filter traffic on network level. I would recommend looking into using an existing proxy server or TCP filtering program. There are several Open Source ones that I'm sure you can use as a basis to rewrite or hook into.
The easier solution is to write an http listener that logs the requests.
Fiddler2 is one of these, you can check it out. it logs all incomming and outcomming http content.
On the client machine I need to be able to somehow detect which sites the current user are looking at right now.
I know the base URL of the sharepoint app, say sharepoint.thecompany.net but how the hack do I get the last requested url from the server?
I have hit a dead stop when trying to iterate the current processes and the casting the iexplorer process to something I can work with, I just don't know which kind of object to cast the process to :-(
I hope to implement this logic in a C# assembly what should run on the client box.
Any other approach that might work?
Thanks in advance
WatiN will allow you to attach to IE instances and get the current url from them. It will also allow you to do the same with Firefox instances.
It might be more efficient however to try to get requested urls at the network level using a wireshark type concept where you are just listening to http traffic on the computer and keeping track of the urls but a solution like that is a bit over my head.
EDIT: I came across this while looking for a solution: http://www.codeproject.com/KB/IP/networkmonitor.aspx
From what I can see I would think you could adapt the monitoring code to monitor and look for http request packets and parse the headers for the url information you need.