We Keep Coding

WebSockets on .Net / c# without IIS (self hosted)

This has been discussed here but without a conclusion.
I would like to set up a WebSocket server in a self hosted Windows Service where I currently have multiple REST web services running on WCF. My requirements are not that advanced, I just need to consume an incoming stream of data. The problem is that I cannot change the way the data is send, if I could I would just convert it to a streamed HTTP POST, but this is not possible as the protocol is fixed.
According to Microsoft the WebSocket Class should be available from .NET Framework 4.5 and I am using .NET Framework 4.8 that is also supported but nowhere can I find how to self host it. There are many examples using IIS which I do not use.
So what I want to know is, 1) can the WebSocket class be used to self host a WebSocket Server inside a Windows Service?
If so how is it done? Looking for some example code.
EDIT
I think this might be somewhat undocumented but I finally got it to work. The magic seem to be replacing localhost or * with +. The following seem to work hosting the WebSocket while maintaining working WCF web services.
WebSocketsServer.Start("http://+:80/Socket/");
Some of it was described here but I could not make out when to use + or * and what their actual uses are.
When a port is specified, the host element can be replaced with "*" to
indicate that the HttpListener accepts requests sent to the port if
the requested URI does not match any other prefix. For example, to
receive all requests sent to port 8080 when the requested URI is not
handled by any HttpListener, the prefix is http://*:8080/. Similarly,
to specify that the HttpListener accepts all requests sent to a port,
replace the host element with the "+" character. For example,
https://+:8080. The "*" and "+" characters can be present in prefixes
that include paths.
I still do not think this makes sense fully. Anyone have a reference to why this would work?

Alchemy Websockets: Can't host server on Azure

I'm currently using Alchemy Websockets as my WebSocket solution, however this is proving to be a pain as I've spent quite a number of hours trying to figure out why I can't get it up and going on my WebRole (both on Compute Emulator and cloudapp.net itself). It runs okay locally.
I'm aware of this question that was asked previously, and I've followed everything in the code and done everything suggested, so I can't get why I still can't connect to my WebSocket server. I've tried connecting using WebSocket.org's Echo Test as well as an Android client (leveraging on Autobahn). I'm quite sure it's not a client-side issue anymore considering that both ways don't work, but more of a server-side issue (what is stopping me from opening the WebSocket server on Azure?!).
In summary, what I've done so far:
Included TCP InputEndpoint 8080 in my Azure
Start the WebSocket server in either Global.asax or RoleEntryPoint with port 8080
Tried connecting using both Echo Test & Autobahn for Android
If it helps, I've previously tried SuperWebSocket Server. I understand that the creator Kerry Jiang has uploaded some Azure samples, but the Command Assembly baffles me and I don't want that kind of extra logic in my application (I just want something that I can plug and play), so I decided to ditch that altogether.
Both Alchemy Websockets and SuperWebSocket Server work perfectly fine locally, but when it's on Azure (either emulator on uploaded on cloudapp.net) I can't seem to connect at all. Any guidance please...?

When using WebSocket with cloud-hosted VMs, a number of things can go wrong:
The cloud service might have firewalls in place that block any (non-standard) ports by default. On Amazon EC2, you can configure open (unblocked) inbound ports via AWS management console and the respective configuration of the "security group" that applies to the EC2 instance your WS server is running on.
Even if you use a standard Web port (like 80) and this port is open in the firewall, there might be a proxy, load-balancer or other so-called intemediary in place that is not WebSocket aware.
When running over non-local networks, there are all kinds of intermediaries that might interfere. For example, mobile networkwork providers have intermediaries that are too stupid for WebSocket (which uses the HTTP standard Upgrade mechanism .. it is standard .. but there is a lot of broken stuff out there).
To work around for 2. and 3. (and also in general), you probably want to run WebSocket over TLS (secure WebSocket .. WSS) in production. Running over TLS, and terminating the TLS connection at your server, no intermediary can interfere (there are exceptions even here .. so called MITM HTTPS proxies that unwrap the TLS .. but that only on some corporate networks and relies on the proxy being able to dynamically installing certificates in the user's browser).
To make a robust WSS connection, you MUST make sure the server certificate used is accepted by browsers without any problems. If it is self-signed e.g., a browser would normally not accept, but ask the user if he nevertheless wants to accept. With WS, the browser will not render such dialogs, because WS is a so called "subresource". Hence: make sure your cert is 100% ok.
With 5. being done, WebSocket (WSS then), will work (almost) all the time. Mobile, enterprise, public internet.

Load Testing TCP with IP spoofing

I'm trying to come up with a way to Load Test the following architecture:
We have an Application Server that recieves data from multiple servers with data-collecting agents on them.
However, our testing environment only has ONE server with an agent, and we wish to emulate multiple (100) connections from the same server outgoing to the Application Server.
I have a wide-open range of IPs I have already assigned to the NIS card on the agent machine, but I am unable to find a way to make each outgoing connection from the machine to the app server use a different IP and thus open a new connection.
I know that web tests on HTTP protocols can use Virtual Users and IP Spoofing tools easily when facing thi sproblem, but since the agent-server connection uses TCP - I cannot find ANY tool that is capable of fulfilling this function.
I also thought about possibly editing each packet by code and replacing it's header with a different IP each time - but this method seems both too convuluted and impractical as it would delay each request for too long to make the load testing relevant.
Any ideas and solutions are welcome!
thanks in advance :)

You can set the IP adress by using Socket.Bind before connecting to the server.
It's described here: http://msdn.microsoft.com/en-us/library/system.net.sockets.socket.localendpoint(v=vs.100).aspx

You can use winpcap or its .net wrapper pcap.Net (http://pcapdotnet.codeplex.com/) to get complete control over the IP/TCP packets sent.

Multi Client/Server Discovery in C#

I'm developing a multiple client / multiple server program in C#, and before I got down to the nitty gritty, I was wondering if anyone has ever worked on a similar project and might be able to share their tips / ideas for implementation.
The servers will sit on many PCs, and listen for incoming connections from clients (Or should the Servers broadcast, and the clients listen?).
When a client starts, it should populate a list of potential server IP addresses automatically.
When a server closes, the client should remove that server from it's list.
When a new server starts, the clients should be notified and have it added to their list.
A server may also act as a client, and should be able to see itself, as well as all other servers.
A message sent from a client to the server, that affects the server, should broadcast the change to all connected clients.
Should my server be a Windows Service? What advantages/disadvantages does that present?
Any ideas on how I might go about getting started on this? I've been looking into UDP Multicast, and LAN Scans. I'm using C# and .NET 4.0
EDIT: Found this: http://code.google.com/p/lidgren-network-gen3/ Does anyone have any experience with it and can recommend/not recommend it?

I would suggest NetPeerTcpBinding WCF communications to create a Peer Mesh. Clients and Servers would all join a mesh using a Peer resolver. You can use PNRP or create a custom peer resolver (.Net actually provides you with an implementation called CustomPeerResolverService). See Peer To Peer Networking documentation.
Also you can implement a Discovery service using DiscoveryProxy. With a discovery service, services can announce their endpoints. The discovery service can then service find requests (see FindCriteria) to return endpoints that match the requests. This is referred to as Managed Discovery. Another mode is Ad Hoc Discovery. Each service will announce their endpoints via UDP and discovery clients will probe the network for these endpoints.
I have actually implemented a Managed Discovery service in combination with Peer 2 Peer WCF networking to provide a redundant mesh of discovery services that all share published service endpoints via P2P. Using Managed Discovery I have found performs far better as Ad Hoc Discovery using UDP probing is slower and has some limitations crossing some network boundaries while Managed Discovery leverages a centralized repository of announced service endpoints.
Either/both technologies I think can lead to your solution.

So is this effectively a peer to peer style network (almost like bittorrent), where all servers are clients, but not all clients are servers.
and the requirements are every client should hold a list of all other servers (which are, in turn, clients).
The problem lies in getting the server IPs to the clients in the first place. You can use a master server that has a fixed DNS to act as a kind of tracker, which all of the servers check in to, and the clients check periodically.
Another option (or an additional method) is to use a peer exchange style system, where each of the clients and servers use UDP broadcast packets over a local network to discover each other and then transfer the servers they know of, kind of like a routing protocol. However if the PCs are spread out over a non local network such as the internet, there's little chance that they will ever discover each other on their own, making this method only useful when used in conjunction with other methods of finding servers. Also, you will probably have to deal with router UPnP to allow clients to connect to each other through each others router NAT, so this method is probably too complex for the gains you get. (However, if you're just on a LAN, this is all you need!)
A third option (and again, this sounds a lot like torrent technology), is to use Distributed Hash Tables to store information about the IPs of your servers in the cloud, without having to rely on a central master server.
I have had a shot at a project like this before (a pure P2P, server-less messaging system), but could never get it to work. Without a huge amount of peers, or a master server to track all of the other servers, it is very difficult to reliably retrieve the IPs of all the servers.

How are server side applications created, how is client - server communication done?

I would like to have a client-server application written in .NET which would do following:
server is running Linux
on the server there is SQL database (mySQL) containing document URLs
What we want:
- server side would regularly crawl all URLs and create a full text index for them
- client side would be able to perform a query into this index using GUI
The client application is written in .NET using C#. Besides of searching in documents it will be able to do a lot of other things which are not described here and which are done client-side very well.
We would like to use C# for the server side as well, but we have no experience in this area. How are things like this usually done?
Clarifying question now based on some answers:
The thing which is most unclear to me is how client-server communication is usually handled. Is client and server usually using sockets, caring about details like IP addresses, ports or NAT traversal? Or are there some common frameworks and patters, which would make this transparent, and make client-server messaging or procedure calling easy? Any examples or good starting points for this? Are there some common techniques how to handle the fact a single server is required to server multiple clients at the same time?

To use c# on Linux you will need to use Mono. This is an open source implementation of the CLR specification.
Next you need to decide on how to communicate between server and client, from the lowest level of just opening a TCP/IP socket and sending bits up and down, to .Net remoting, to WCF, to exposing webservices on the server. I do not know how compleat WCF implementation is on mono, also I think you may have issue with binary remoting between mono and MS .Net .
I would suggest RPC style WebServices offer a very good solution. WebServices also have the advantage of alowing clients from other platforms to connect easily.
EDIT
In response to the clarification of the question.
I would suggest using mono/ASP.NET/WebServices on the server, if you wish to use c# on both server and client.
One assumption I have made is that you can do a client pull model, where every message is initiated by the client. Using another approach could allow the server to push events to the client. Given the client has the ability to pole the server regularly I don't consider this much of a draw back but it may be depending on the type of application you are developing.
Mono allow execution of c# (compiled to IL) on a Linux box. Mono ASP.NET allows you to use the standard ASP.NET and integrate into Apache see http://www.mono-project.com/ASP.NET and finally WebServices allow you to communicate robustly in a strongly typed manner between you client and your server.
Using this approach negates most of the issues raised in your clarification and makes them someone else's problem.
Sockets/SSL - is taken care of by standard .Net runtime on the client and Apache on the server.
IPAddress/ports/NAT traversal - Is all taken care of. DNS look up will get the servers IP. Open socket will allow the server to respond through any firewall and NAT setup.
Multiple Clients - Apache is built to handle multiple clients processing at the same time as is ASP.NET, so you should not encounter any problems there.

As many have already mentioned there are a number of thing that you have mentioned which are going to cause you pain. I'm not going to go into those, instead I will answer your original question about communication.
The current popular choice in this kind of communication is web services. These allow you to make remote calls using the HTTP protocol, and encoding the requests and responses in XML. While this method has its critics I have found it incredibly simple to get up and running, and works fine for nearly all applications.
The .NET framework has built in support for web services which can definitely be called by your client. A brief look at the mono website indicates that it has support for web services also, so writing your server in C# and running it under mono should be fine. Googling for "C# Web Service Tutorial" shows many sites which have information about how to get started, here is a random pick from those results:
http://www.codeguru.com/Csharp/Csharp/cs_webservices/tutorials/article.php/c5477

have a look at Grasshopper:
"With Grasshopper, you can use your favorite development environment from Microsoft® to deploy applications on Java-enabled platforms such as Linux"
Or see here
The ideea is to convert your app to Java and then run it on Tomcat or JBoss.

Another approach: use the Mod_AspDotNet module for Apache, as described here.

This Basic Client/Server Chat Application in C# looks like a kind of example which might be a starting point for me. Relevant .NET classes are TcpClient and TcpListener