I'm attempting to write (in C#) a piece of software that communicates with another piece of software, built with MSYS, over (MSYS emulated) Unix domain sockets. I've learned that the "socket server" (I'm not clear on what the proper terminology is) creates a temporary file with contents such as this:
!<socket >59108 282F93E1-9E2D051A-46B57EFC-64A1852F
The 59108 corresponds to a TCP port, which the "socket server" is listening on on the loopback interface. Using a packet capture tool, I've been able to determine that the "socket client" connects to this port, and information is exchanged over the loopback interface.
I replicated this behavior in my software, and the "socket client" connects to my listening port, but no information is transmitted. I believe there's another step here, one most likely involving the GUID in the "socket" file, but I've been unable to determine what it is. What do I need to do to trigger the communication from the client?
It seems that MSYS is using Cygwin's mechanism, which involves a named event, that is (probably?) created by the "server", and signaled (apparently) by the "server", but my naive attempt at an implementation doesn't seem to be working.
I've located an email written by Conrad Scott which describes various shortcomings in the "handshaking" process, and proposes a patch which allegedly solves them. In this email, Conrad describes somewhat the process used, and he indicates that there are actually TWO events, one managed by the "server" and one managed by the "client". I've used API Monitor to look for calls to CreateEvent(), and while there are several, I cannot find one that looks like the "smoking gun" here. There are no interesting calls to CreateSemaphore() either, so it seems like Conrad's patch was never applied (or, at least, it was applied some time AFTER MSYS forked Cygwin).
It appears that both the answers from divB and Mark are correct, but they both leave out some details, so this is hopefully a bit more complete.
There are 2 different implementations here. I have not done an exhaustive investigation of who implements which implementation, but as of this writing, the current version of cygwin uses the implementation described by divB and MsysGit uses the implementation described by Mark.
Initializing the server:
Create a socket (AddressFamily = IPv4, Type = Stream, Protocol = TCP). (.NET/MFC)
Bind it to loopback (127.0.0.1). (.NET/MFC)
Tell the socket to listen. (.NET/MFC)
Generate a random 16-byte GUID.
Create a file with the following contents based on the TCP port and the GUID. Using the original example where 59108 is the TCP port and 282F93E1-9E2D051A-46B57EFC-64A1852F is the GUID.
In the cygwin implementation, the socket file contents are:
!<socket >59108 s 282F93E1-9E2D051A-46B57EFC-64A1852F
And in the msysgit implementation, the socket file contents are:
!<socket >59108 282F93E1-9E2D051A-46B57EFC-64A1852F
The difference being the extra "s" between the port and the GUID.
Set the System attribute on this file.
In msysgit implementation only, Create a named wait handle with the name cygwin.local_socket.secret.58598.282F93E1-9E2D051A-46B57EFC-64A1852F (InitalState = False, Reset = AutoReset). (.NET/MFC)
58598 is derived by using a HostToNetworkOrder function on the port (as 16-bit unsigned integer). i.e. 59108 == 0xE6E4 and 58598 == 0xE4E6.
Handling connections:
Accept the incoming socket. (.NET/MFC).
In the case of the cygwin implementation only, do handshake that consists of:
Read 16 bytes. If these do not match the GUID, then fail.
Send the same 16 bytes.
Read 12 bytes as 3 32-bit integers. They are the pid, uid and gid of the calling process.
Send 12 bytes (3 32-bit integers) back. Use the pid of the server and the uid and gid that were received.
In the case of the msysgit implementation only, synchronize with the client by:
Get the port of the incoming socket. For this example, we'll say it is 63524.
Open existing wait handle for the client. (.NET/MFC). You need to convert the port to network byte order just like we did for the server. So, for this example, the name is cygwin.local_socket.secret.9464.282F93E1-9E2D051A-46B57EFC-64A1852F
Signal the server and wait for the client (ToSignal = server, WaitOn = client, Timeout = 10000 msec, ExitContext/Alertable = False). (.NET/MFC). Not 100% sure about the ExitContext/Alertable parameter, but False seems to work.
Hand off the socket to the (hopefully already existing) code for whatever it is you are doing (which in the case of all three of us, seems to be an ssh agent).
So at least for cygwin I can answer your question now: I just implemented a cygwin compatible socket server using MFC. I did it by looking into cygwin source.
It seems that there are not even events. So the patch you mentioned does not seem to have been implemented.
All that happens is:
1.) The socket file is created, the GUID ("shared key") are just random numbers.
2.) The file MUST have "system" attribute. The cygwin code does some weird permission stuff if it's on NTFS, haven't looked into that.
3.) a network socket on localhost is created with the port indicated in the socket file.
So, then, when a client connects to the socket (via TCP/IP):
4.) It first sends the 4 random numbers to the server; the server checks if they are valid
5.) The server sends them back
6.) The client sends 3 32 bit numbers: The pid, the uid and gid
7.) The server sends back his own version of these numbers.
I don't understand what's the purpose of this handshake because from a security point of view it's completely worthless.
I've worked out something that functions correctly for the build of OpenSSH (ssh-agent.exe) that comes with Git:
Setup on the server side consists of these steps:
1. Create a "secret string" that consists of four groups of eight hex digits separated by a dash ("-")
2. Listen on a local port
3. Create an EventWaitHandle with mode EventResetMode.AutoReset named cygwin.local_socket.secret.[secret string].[listen port number here with byte order reversed]
4. Write out the "socket" file, which consists of the string ![port number here, byte order NOT reversed] [secret string]
When a connection comes in, the following steps must be undertaken:
1. Open the client's event handle with EventWaitHandle.OpenExisting(), using the event name cygwin.local_socket.secret.[remote port number with byte order reversed].[secret string]
2. Signal the server's event handle and wait for the client's wait handle to be signaled with `EventWaitHandle.SignalAndWait()
I agree that it looks like the patch discussed on the mailing list was never applied. The sequence I worked out seems closer to the sequence discussed on that list, and as well, it matches the code I dug up from Cygwin.
I don't understand the disparity between what I found to work vs what divB found to work, but I did confirm that it functioned with the software I was using (Git's OpenSSH)
Related
A. Description
I am using ZeroMQ monitor and I find that it works when logic disconnection but does not work when network broken down (unplug my cable).
For example:
I launch client app on an android pad, then launch a server app on my windows laptop. They are connected through a router with cables.
Everything will be OK with monitor if I close or open either client app or server app manually. Namely, the monitor on both sides can receive a 'Connect' or an 'Accept' and a 'Disconnect' event.
But If I unplug cable on the server side, while the client and server are connected and running, the monitors on both sides can not detect the 'Disconnect' event.
Is the monitor designed like this?
If so, are there any solutions to detect network broken down ( a cable unplug event ) except heartbeats?
If not, how to use the ZeroMQ's original monitor mechanism to solve this problem? Can a setTCPKeepAlive() interface be useful?
B. System environment
My scenario is as below.
Client
OS: Android, running on a pad, IDE: Android studio 2.3, lib:jeromq-0.4.3
// Java Code
String monitorAddr = "inproc://client.req";
ZContext ctx = new ZContext();
ZMQ.Socket clientSocket = ctx.createSocket(ZMQ.REQ);
clientSocket.monitor(monitorAddr,ZMQ.EVENT_ALL);
// Then start a montitor thread which is implemented by my own.
Server
OS: Windows 7 ( 64 bit ), running on my laptop, IDE: VS2013, lib: Clrzmq4
// C# Code
const string MonitorEndpoint = "inproc://server.rep";
var ctx = new ZContext();
var serverSocket = new ZSocket(ctx,ZSocketType.REP);
ZError error;
// Create serverSocket pair socket
if (!serverSocket.Monitor(MonitorEndpoint, ZMonitorEvents.AllEvents, out error))
{
if (error == ZError.ETERM)
return ; // Interrupted
throw new ZException(error);
}
// Create a monitor
ZMonitor _monitor = ZMonitor.Create(ctx, MonitorEndpoint);
_monitor.AllEvents += _monitor_AllEvents;
_monitor.Start();
AFAIK there is no built in heartbeat within ZeroMQ. I know there was some discussion on the topic within the ZMQ community some years ago, and that discussion may still be going on.
It is comparatively simple to incorporate your own heartbeat messaging in your application's use of ZeroMQ, especially if you use something like Google Protocol Buffers to encode different message types; the heartbeat is just another message.
Doing heartbeats in your application (rather than relying on some inbuilt mechanism) is ultimately more flexible; you can choose the heartbeat rate, you can choose what to do if the heartbeat fails, you can decide when heartbeating is important and not important, etc.
Consider heartbeats within a PUB/SUB pattern; it's a bit difficult for the ZMQ authors to decide on your behalf what connection / disconnection / connection-break events matter to you. And if they do build in a mechanism, but an application developer didn't want it, then it is a waste of bandwidth.
It's far easier for the ZMQ authors to leave that kind of application architectural issue to the application author (that's you!) to deal with.
With your specific example, an unplugged network cable simply looks (so far as any software can determine) like no traffic is flowing; it's the same as the application not sending anything. ZMQ doesn't send anything if the application hasn't sent anything.
If you look at the events that the socket monitor can report on, they're all the consequence of something flowing over the network connection, or something done to the socket by the application.
Trying to go lower than ZMQ protocol itself and access the TCP connection that specific ZeroMQ sockets use (while others do not) doesn't sound like a good idea; it would required to break encapsulation in multiple classes.
The answer #bazza gave in 2017 was entirely correct at the time.
However, newer versions of ZMQ (specifically ZMTP) include an heartbeat functionality.
Check ZMQ documentation for
socketOpt
Java functions
name
purpose
ZMQ_HEARTBEAT_IVL
get/setHeartbeatLvl()
heartbeat interval
milliseconds between ZMPT PINGs
ZMQ_HEARTBEAT_TIMEOUT
get/setHeartbeatTimeout()
local heartbeat timeout
how long the local socket waits between received packets until it considers the connection timed out
ZMQ_HEARTBEAT_TTL
get/setHeartbeatTtl()
remote heartbeat timeout
if and when remote side shall consider the connection timed out
ZMQ_HEARTBEAT_CONTEXT is still in draft state as of 2022. It is supposed to send an byte[] context with every ping.
Now, by design of ZMQ, quoting from chapter 2 of its documentation,
The network connection itself happens in the background, and ZeroMQ
will automatically reconnect if the network connection is broken
(e.g., if the peer disappears and then comes back).
Thus, answering your main question, I'd expect the monitor to give you ZMQ_EVENT_CONNECT_RETRIED / ZMQ_EVENT_CONNECTED events after the underlying connection was detected as disrupted.
I'm attempting to connect to a remote server using a specific local interface. My logs tell me everything is working as intended, but checking with netstat, every connection is using the default interface.
I'm using the following code to bind a TcpClient to a specific Local Endpoint
Console.WriteLine("Binding to {0}", connectionArgs.LocalBindingInterface.ToString());
client = new TcpClient(connectionArgs.LocalBindingInterface);
Console.WriteLine("Bound to {0}", client.Client.LocalEndPoint.ToString());
Where connectionArgs.LocalBindingInterface is an IPEndPoint specified as such
IPEndPoint[] localEndPoints = new IPEndPoint[2];
localEndPoints[0] = new IPEndPoint(IPAddress.Parse("192.168.0.99"), 0);
localEndPoints[1] = new IPEndPoint(IPAddress.Parse("192.168.0.100"), 0);
The IP addresses listed here are not the actual addresses.
When i check my logs, this is the info I get
Binding to 192.168.0.99:0
Bound to 192.168.0.99:59252
Binding to 192.168.0.100:0
Bound to 192.168.0.100:53527
But when i netstat -n -p --tcp -a I get
tcp 0 0 192.168.0.98:39948 remote_addr_here:443 ESTABLISHED 17857/mono
tcp 0 0 192.168.0.98:60009 remote_addr_here:443 ESTABLISHED 17857/mono
Clearly something's wrong here. None of the ports, nor the interfaces match. Netstat is run as sudo so I can't assume it's wrong. I also tried to manually create a socket, call it's bind method, and set the TcpClient's Client property to the manually bound socket, but I get the same result.
Is there something i'm doing wrong here? Is there a different way to force a Socket to use a specific Local EndPoint on mono?
I'm running this app as a non-root user, mono --version is Mono JIT compiler version 3.2.8 (Debian 3.2.8+dfsg-4ubuntu1.1), server's ubuntu version is Ubuntu 14.04.3 LTS
Edit 1:
Added an extra logging call after calling TcpClient.Connect()
Binding to 192.168.0.100:59000
Bound to 192.168.0.100:59000
After connect bound to 192.168.0.98:55484
Bottom line: you can't do this, not at the socket level.
The routing of outbound traffic is determined by the network routing configuration. You would have to create an explicit routing table entry for your destination to force a specific adapter to be used.
You can bind to a specific IP address, but this only causes inbound traffic to be filtered, i.e. you'll only receive traffic sent to that IP address.
There are related questions you may want to read as well:
How to stop behaviour: C++ Socket sendto changes interface — context is C++ and not constrained to Windows, but it has what is IMHO the most direct, most relevant notes on the topic.
Using a specific network interface for a socket in windows — fairly poor question and answer both, frankly. But it does contain some quotes and links that you might find useful anyway.
Arguably, this question might have been closed as a duplicate of one of those, or perhaps even another similar question. But those two don't really answer the question in an accurate, C#/.NET-specific way, and I didn't actually find any others that seemed any better.
This is the first time I'm working with IPEndPoint, so pardon my lack of experience.
If I create an IPEndPoint like this:
IPEndPoint localEndPoint = new IPEndPoint(IPAddress.Parse("192.168.1.25"), 0);
Is it possible that port 0 will already be in-use such that this will produce an exception? Is there any type of convention here for which local port number to use? Should I just use a randomly generated number that's less than 65,536? Is there a best or safest approach to choosing a port and does it even matter?
The use case in this particular instance is just for me to learn more. I'm fiddling with sending UDP broadcast WOL packets at the moment, but later I expect to get into a bit more complicated code. I just want to make sure I understand best practice.
According to IANA
Port numbers are assigned in various ways, based on three ranges:
System Ports (0-1023), User Ports (1024-49151), and the Dynamic and/or
Private Ports (49152-65535); the difference uses of these ranges is
described in [RFC6335]. System Ports are assigned by IETF process for
standards-track protocols, as per [RFC6335]. User Ports are assigned
by IANA using the "IETF Review" process, the "IESG Approval" process,
or the "Expert Review" process, as per [RFC6335]. Dynamic Ports are
not assigned.
Also from the UdpClient(port) ctor MSDN documentation:
If you pass 0 to the constructor, the underlying service provider will assign a port number.
First, you would want to check to see if you are interfering with any standard windows service ports
Random in that case would be a very bad way to go, the port may be in use by some other app.
But you can always go here and choose something that is unassigned.
There, the IANA says
The Dynamic and/or Private Ports are those from 49152 through 65535
In my Client Server Application i wondered how to make a packet and send it to the server via the Client
Then on the server i recognize which packet is this and send the proper replay
but suddenly i got across this topic and it make me worry if i may fall in this problem
The Problem
One of the most common beginner mistakes for people designing
protocols for TCP/IP is that they assume that message boundaries are
preserved. For example, they assume a single "Send" will result in a
single "Receive".
Some TCP/IP documentation is partially to blame. Many people read
about how TCP/IP preserves packets - splitting them up when necessary
and re-ordering and re-assembling them on the receiving side. This is
perfectly true; however, a single "Send" does not send a single
packet.
Local machine (loopback) testing confirms this misunderstanding,
because usually when client and server are on the same machine they
communicate quickly enough that single "sends" do in fact correspond
to single "receives". Unfortunately, this is only a coincidence.
This problem usually manifests itself when attempting to deploy a
solution to the Internet (increasing latency between client and
server) or when trying to send larger amounts of data (requiring
fragmentation). Unfortunately, at this point, the project is usually
in its final stages, and sometimes the application protocol has even
been published!
True story: I once worked for a company that developed custom client/server software.
The original communications code had made this
common mistake. However, they were all on dedicated networks with
high-end hardware, so the underlying problem only happened very
rarely. When it did, the operators would just chalk it up to "that
buggy Windows OS" or "another network glitch" and reboot. One of my
tasks at this company was to change the communication to include a lot
more information; of course, this caused the problem to manifest
regularly, and the entire application protocol had to be changed to
fix it. The truly amazing thing is that this software had been used in
countless 24x7 automation systems for 20 years; it was fundamentally
broken and no one noticed.
So how could i send something like AUTH_CALC,VALUE1=10,VALUE2=12 packet and receive it from the server in a safe way...
And if you wanna an example of what i am doing here it is below
[CLIENT]
Send(Encoding.ASCII.GetBytes("1001:UN=user123&PW=123456")) //1001 is the ID
[SERVER]
private void OnReceivePacket(byte[] arg1, Wrapper Client)
{
try
{
int ID;
string V = Encoding.ASCII.GetString(arg1).Split(':')[0];
int.TryParse(V, out ID);
switch (ID)
{
case 1001://Login Packet
AppendToRichEditControl("LOGIN PACKET RECEIVED");
break;
case 1002:
//OTHER IDs
break;
default:
break;
}
}
catch { }
}
So is this is a good way to structure a Message and handling it on the server ?
Also which is better encoding to use ASCII or UTF8 ?
The best way you can do is by using length indicator. Suppose you are sending a file of 10000 bytes, first send the length of the file and receive the ack i.e "OK" string from other side, then keep on sending 10,000 bytes chunk by chunk(may be u can take 4096 bytes). Send 4096 bytes each time for two time and send 2000 odd bytes on the last chunk. On the receiver side there is no gauranty that for one send you will receive the whole 4096 bytes, so you need to wait until u get 4096 bytes and then proceed with next 4096 bytes.
My situation: I have an xspeech for voip, when there is an action about phone, the messages(packets) are sent to the xspeech interface. And from this interface the logs of these action are sent to specific IP address which is my IP address. And also these logs are sent from a specific port.
I am trying to develop an application that reads these logs(not a file). I tried TCPListener and TCPClient, they worked but not in my situation. I thought these logs are not sent from TCP port. This is the problem.
From your post I suspect it is using UDP... this would mean that you need to use UDPClient class to receive and process...
Depending on whether they offer a TAPI provider you can use:
http://www.codeproject.com/KB/IP/devangpro.aspx
http://www.codeproject.com/KB/dotnet/CShart_TAPI_3x.aspx
http://www.codeproject.com/KB/IP/Video_Voice_Conferencing.aspx
http://msdn.microsoft.com/en-us/library/ms734214.aspx
http://msdn.microsoft.com/en-us/library/windows/desktop/ms734257%28v=vs.85%29.aspx
It may be necessary to build a SIP proxy (this would be able to generate for example "Pickup event"):
http://www.codeproject.com/KB/cs/SIP_stack_with_SIP_proxy.aspx
http://sipsorcery.codeplex.com/
http://www.independentsoft.de/sip/index.html
http://www.konnetic.com/products/products_sip_sdk_std.aspx
http://www.voiceelements.com/Products/VEToolkit.aspx
EDIT - as per comments other possibilities:
If what you try to catch is sent as syslog message then see http://michael.chanceyjr.com/useful-code/syslogd-class-for-sending-and-receiving-syslogd-events/ .
Another possibility is SNMP - for this see http://www.snmpsharpnet.com/
SIP uses UDP per default.
It's not as simple as being able to take packets from one endpoint, log them and then send them to the real destination. You'll break things like NAT handling if you do so.
You need to act as a full SIP proxy (stateful) or as a B2BUA.
The easier way is to download an existing sip-proxy and just make it log to a file and read that file.
If you're recieving the information on at your IP address then I too would have thought that TCP/IP was being used. Have you checked that your firewall isn't blovking the port in question.
I would suggest that you have another look at what protocol is being used (it may be UDP) and use the Socket class instead.