I would like to do as follows. What would be the best way? A general answer will also be fine.
I would like to intercept an HTTP request at the client end to alter some html content. For example, I go to CNN.com and rather than an article that's displaying "Two LA Dogs marry", it should say "Ridiculous title blocked".
It should be smooth that even a secure certificate won't be disturbed.
I am using C#.
Thanks!
UPDATE: Thank you all for your answers!
You can do this with Privoxy via their filter files. Their fun filter is a good example of exactly the sort of substitutions you want to do.
To replace "Two LA Dogs Marry" with "Ridiculous Title Blocked" on cnn.com your action file would look something like this:
{ +filter{ridiculous-title-censor} }
.cnn.com
and your filter file would look like
# FILTER: ridiculous-title-censor Remove ridiculous titles
# This keeps CNN from getting too ridiculous
#
s/Two LA Dogs Marry/Ridiculous Title Blocked/ig
Local HTTP proxy is possible and most generic approach.
I.e. you can use Fiddler to see if it works for you. I supports modifying requests/responses in addition to regular watching for traffic.
Another option if you're using Firefox is to use Greasemonkey scripts. Here's an example script which changes the main headline on cnn.com
If you're not familar with the Javascript coding needed to make Greasemoney scripts, you can use the Platypus add-on to edit the page in place and automatically generate a script file.
You could setup a proxy with HTTPListener. But I would think if you wanted to do it right, you'll need a program that is more low level.
Open 2 TCP Ports (80 & 443) and actively listen for incoming connections.
Once received
Go out and make the request on behalf of the requester
Retrieve HTTP Response
Inspect and change the HTTP Response (where appropriate)
Perhaps modifying headers (where appropriate)
Forward on the response to the requester
I'd start with a simple proxy that just forwards all requests and returns back all responses. Once that is in place you can start inspecting the responses.
That is a good place to start.
Such an approach is the least efficient method of doing what you want to achieve.
If this is a client side application, the client may disable it and thus render it useless.
It is also hard to maintain and requires more complex programming to ensure that it works with SSL.
If using a browser plugin, or toolbar, it would need to be made for a specific browser.
If using a listening server to intercept the HTTP request, this provides complexity and difficulty when the content is encrypted, also unnecessary overhead.
If using a local proxy (meaning that the client's browser needs to point to a local proxy service), maybe the most effective client side method, but still have the disadvantages mentioned above (hard to maintain etc.)
I believe that what you are looking to do is completely reinventing the wheel.
The fact that you have offered a bounty begs the question that you indeed need to do this in C# and client side, but 'censoring bad things' means you need to prohibit content, and any client side method would eventually give the power to the client to remove this limitation.
Personally, I have had great success with Squid and it's content adaptation features.
This means, that the clients need to have a controlled Internet source. Meaning that, if they are all in a LAN and sharing a common Internet gateway, this is easily feasible if you have spare a server to act as a proxy.
I recommend you get a small linux box, which can have a simple Ubuntu Server Edition, then add Squid. The net is full of tutorials, but the level of implementation has become easy enough to do even without them.
I may have gone completely off-topic, but I hope I could assist.
you can come to China ^_^
censorship like this is everywhere, you don't have to implement your own.
Ok, that is a joke, the answer is you can implement browser plugins for such kind of tasks. or maybe you need to implement a ROUTING filter ( GFW like ) on the router.
Taken from here.
It can be done via a Layered Service Provider on Windows.
From Wikipedia:
"A Layered Service Provider (LSP) is a feature of the Microsoft Windows Winsock 2 Service Provider Interface (SPI). A Layered Service Provider is a DLL that uses Winsock APIs to insert itself into the TCP/IP stack. Once in the stack, a Layered Service Provider can intercept and modify inbound and outbound Internet traffic. It allows processing all the TCP/IP traffic taking place between the Internet and the applications that are accessing the Internet (such as a web browser, the email client, etc). "
AdMuncher for example is intercepting and inserting http code to avoid ads. Another suggestion is to find an open-source ad blocking program and see how they've implemented it.
Are you saying you want to intercept this for your own site?
In any case, it needs to be done in javascript or jQuery as C# is not a client-side language.
or, you can code a toolbar, or maybe a simple chrome addon, it's really easy but its not C#
you can search for libraries to monitor browsing trough proxy, like this:
http://httpproxynet.codeplex.com/
The same concept used by java in this project:
http://www.charlesproxy.com/
sounds intresting, good luck :)
A long time ago I've implemented this feature for IE using Plugable Mime Filters so after I've searched about it in c# here in stackoverflow I've founded this post that should help you start with it.
Hope this is useful for you.
Asynchronous Pluggable Protocols can be used for this type of thing. Although, as stated here INFO: Implementing HTTP-like Asynchronous Pluggable Protocols: "For various reasons, Microsoft neither supports nor recommends that you replace or wrap the default HTTP protocol."
I need to write a fiddler "lite" Windows forms application to perform some HTTP tracing of messages. Specifically I need to show the request/response XMLs to a server as they are occurring, highlighting certain elements etc.
Could anyone point me to a F/OSS application doing something similar as a starter, or perhaps outline the classes I need to use? I am a bit of a newbie with networking classes in C# and appreciate all help.
p.s. I cannot use Fiddler directly for several reasons, so that's not a valid answer. Thanks.
SharpPcap might be what you want
Check out .NET Network Tracing. Then your project would become mostly an exercise in log parsing.
I am attempting to implement some additional statistics gathering in a C# server application - I have about 20 or so variables I'd like to be able to report to network monitoring tools; so I am assuming (hopefully correctly) that SNMP is the correct way to go.
There are however two problems:
The application is an open source server that cant include proprietary components, and
It needs to run on Mono under *nix environs as well as Windows via .NET.
The "#SNMP" library at http://sharpsnmplib.codeplex.com/ appears to be a promising solution - but there are no samples I can find; and my knowledge of SNMP is lacking.
Does anyone here have any (quick) examples of reporting data via SNMP? (ideally using a library under the LGPL, X/MIT, BSD or Zlib licenses)
Help much appreciated
Edit:
The problem appears to be a lack of libraries capable of acting as SNMP servers, if anyone is aware of any - this would probably solve the problem nicely.
Edit #2:
Basically I'm looking for a SNMP server library or sample reference implementation.
As the author :) I suggest you check out TestAgent sample. It can send out TRAP or INFORM messages to the IP address you specify.
Note that this is a tiny sample, which does not reveal all powers of SNMP.
#SNMP source comes with some samples (they're actually tests, but one can have an idea about how it works).
Another thing you could try is send your snmp queries calling directly the snmpget executable with the Process class. It's not the best way, but it could work (I've used this kind of approach with another language).
I want to write a small application that can:
1. monitor URLs requested via a web browser and/or
2. monitor incoming http responses on the local machine
I have been doing some Googling, but I am not finding any clear answers. I am thinking maybe System.Net.Sockets.TCPListener and messing around with it, but I am under the impression that it is either not what I"m looking for or can't handle both things.
I don't need a detailed step by step explanation. Just a small overview would be helpful (if this is even possible) such as (what classes to use, what events to subscribe to, any additional details needed to instantiate necessary objects) I can google the details.
Why not just extend Fiddler to suit your needs?
You may want to use a utility like Wireshark (graphical, but not scriptable from C#, AFAIK). Wireshare uses a library called WinPcap, which does the actual monitoring of the TCP streams on the computer. Decoding of the HTTP headers will need to be done in your application (this is what wireshark/tcpdump/windump should be able to do). You will need to use PInvoke to call the WinPcap DLL. You can find library wrappers by performing a web search for "WinPcap PInvoke".
This question sound very similar to the following:
Is there any way in .NET to programmatically listen to HTTP traffic?
EDIT: Also, Capturing HTTP requests may help
Is there a way to monitoring internet traffic programatically? I would like to log the pages users are visiting on the Internet. Can this be achieved with .NET code, is there a 3rd party .NET component that could be used to retrieved data.
Information about Internet traffic must be stored to a database so I cannot use a plugin or something for IE. We are also looking to include this code into our existing product so we cannot use a 3rd party product that cannot be redistributed.
It would be cool if this thing could monitor traffic for all browsers but monitoring IE traffic might also be sufficient.
Setting up a sniffer is doable via the WinPCap library, which has several projects wrapping it to .NET:
Link
http://www.codeproject.com/KB/IP/dotnetwinpcap.aspx
http://coolthingoftheday.blogspot.com/2006/02/sharppcap-net-winpcap-wrapper.html
and probably some others as well, just a matter of Googling.
You would need to build some software that acts as a proxy. You should start by looking at programs like "Fiddler" to understand the concepts and what you need to implement.
If you want my profesional opionon you should go to Server Fault and ask for opionons for low cost internet proxy solution. Writing this thing yourself while challenging and fun will not make good business sense.
If you want to monitor all traffic from everyone on your network consider a product like Microsoft's Internet Security and Acceleration (ISA) Server
While this is probably overkill for what you want, the point is that you need a way to have all traffic go through a single point (a proxy server) where the traffic can be logged. Since all traffic goes through this one point, users can't avoid detection by using an alternate browser etc.
You can also try Pcap.Net to easily use WinPcap in .NET.
It is a wrapper for WinPcap written in C++/CLI and C# and includes a packet interpretation and creation framework.
Like JD already mentioned you should take a look into Fiddler. With this proxy you are able to monitor all the web traffic (if your browser is configured to use fiddler as proxy).
If you don't want fiddler as a standalone application and need its functionality within your own app, you should take a look into FiddlerCore. It is a .Net assembly which encapsulates the core functionality of Fiddler.
If you need a more raw way to sniff into the data (maybe you don't want to [or can't] configure the proxy of the client) the answer from Vinko will help you further.