I have a Visual C++ programm with a proprietary point-to-point protocol built on top of TCP/IP sockets that allows a set of messages to be flow between a third party software.
There is a note in documentation to that protocol:
IP logical packets do not necessarily map directly to physical packets on the underlying network socket, they may be broken apart or aggregated by the TCP/IP stack.
What does this mean?
I've write my C# application to connect and due to technical restriction it is able to run and communicate only locally. Plus every millisecond is critical.
Seems this is not about named pipes: pipelist.exe doesn't show any specific entries.
If you are just using loopback there may be no IP packets at all, and in any case (a) the implementor of your protocol should have already taken all that into account and (b) TCP hides all that from you too - it just provides a byte stream interface.
When TCP/IP packets go out over, say, Ethernet, the packets are repackaged as Ethernet frames. This may include breaking up the original packets.
When the frames arrive at their destination, the Ethernet header information is removed and the original packet (reassembled if necessary) is presented to the TCP/IP layer on the destination machine.
But this repackaging can also happen within the TCP/IP stack. TCP and IP are actually separate protocols; IP is responsible for routing, TCP does the "handshaking" (maintains session state, guarantees delivery [or tries to], etc.)
Named pipes are a completely different interprocess communication mechanism. Usually faster than TCP/IP, but typically restricted to use on a single machine, I believe.
IP logical packets do not necessarily map directly to physical packets on the underlying network socket, they may be broken apart or aggregated by the TCP/IP stack.
TCP/IP is not the lowest level network protocol that exists. There are others: the Ethernet protocol that connects ethernet devices, the 802.11x wireless protocols, and others. All this statement says is that a single IP packet may correspond to multiple packets in the lower-level protocols, and that the IP networking layer is responsible for buffering or joining these packets.
Your application shouldn't need to worry about this at all. TCP/IP networking is handled very efficiently by all modern OS kernels, and unless your requirements are very unusual you should never have to worry about the way your application protocol is broken up into packets by TCP/IP or the lower-level protocols.
Related
I'm looking to enhance a WPF app I have that currently receives/broadcasts UDP datagrams within its network with a bridge functionality to share packets with other instances of the app on different subnets.
I essentially want one instance to connect with another and then be broadcast UDP packets while a connection is maintained. To do this, I figure a TCP connection will work at least for the handshake and periodic acks. However, I do not require the TCP overhead for the data itself, which could be transmitted multiple times per second for hours or longer (however long the two are connected).
Given these requirements does it make sense to do a hybrid where the TCP is used for handshaking/acks but data is sent via UDP (I assume the TCP connection can be configured to be kept open indefinitely), or would the additional overhead of sending the datagrams as TCP payloads be negligible? Or should I implement a syn/ack functionality within the UDP transmission? Is there an established standard for this sort of connection?
I had an argument with a colleague on the selection.
We have two processes running on the same machine.
=> NamedPipe and UDP are KERNEL OBJECT so as far as i understand this is same overhead.
The advantage of UDP is that if tomorrow we will separate those two processes and they will run on two different computers so I do not have to change anything.
I think that the NamedPipe performance are better since there is no need to use a network card to send the information to the same machine (am I right .. sending localhost will use the network card - right ?)
Can anyone advise us please ??
Thanks
Before Implementation , you can care below points :
Named pipes:
Named pipes provide interprocess communication between a pipe server and one or more pipe clients.
They support message-based communication and allow multiple clients to connect simultaneously to the server process using the same pipe name.
Named pipes also support impersonation, which enables connecting processes to use their own permissions on remote servers.
User Datagram Protocol :
User Datagram Protocol (UDP) is a simple protocol that makes a best effort to deliver data to a remote host.
The UDP protocol is a connectionless protocol, UDP datagrams sent to the remote endpoint are not guaranteed to arrive, nor are they guaranteed to arrive in the same sequence in which they are sent.
Applications that use UDP must be prepared to handle missing, duplicate, and out-of-sequence datagrams.
I am trying to send messages over tcp/ip between two servers.
I want to send a message that is 30KB.
But I want to send it with as a whole.
I don't want tcp protocol to break it into segments.
I am using communication between 2 Windows Server 2008 R2.
The client and the server are coded using c#.
I tryed using
tcpclnt.SendBufferSize = 100000;
tcpclnt.Client.DontFragment = true;
and the same at the server.
I also tried configuring the window size of the server(editing the registry).
I would strongly suggest that you need to carry out further research into IPv4 and TCP, as well as Ethernet and Gigabit Ethernet (particularly Jumbo Frames).
Essentially, the short answer to your question is that you cannot send a single IP datagram containing a TCP payload of 30kb, despite the IP header permitting a maximum size of 64kb for the complete datagram.
The reason for this is that the underlying network (most likely Ethernet or Gigabit Ethernet) will have smaller frame sizes, and will therefore require the IP datagram to be fragmented in order to transmit that datagram over the physical network within the frame size limitations of that network.
The TCP protocol does guarantee successful delivery of a complete, uncorrupted datagram (via automatic reassembly, automatic detection of corrupted datagrams and automatic retransmissions of lost or corrupted datagrams), so unless you have a highly specialised requirement, you should be able to just let the TCP stack fragment your message and reassemble it on your behalf.
Altering buffersize will have the sideeffect of ramping up ram usage - not recommended...
As TCP actually deals with streams and not packets (UDP uses packets), I believe your answer lies within framing the message, see message framing
see also code
Found this possible solution somewhat later but thought it should be included here:
SetTcpWindowSize
Search towards the bottom for a VB example entitled "Setting the TCP Window Size for All Network Adapters"
Alternatively there is a buffer handler here which looks like it will do the job of allowing you to read a message in one part even if it is in multiple packets it will allow you to reassemble them via buffer management. See this link
What is the data actual path if I run 2 applications sending/receiving data using sockets through TCP loopback? Will this data go through Windows network driver and/or NIC hardware?
EDIT. Please consider this is not Windows 'loopback' but physical loopback made on the NIC with 2 ports connected with a cable to each other making it physical loopback.
Loopback is a virtual interface, there's no hardware corresponding to it. Data copy happens inside the kernel even above the driver level.
Edit 0:
You are mixing two different things - loopback in TCP/IP is a virtual network interface with a special address in subnet 127/8, usually 127.0.0.1, which is created for you by default, and ethernet crossover cable looping between two cards on the same machine.
I addressed the first case in my initial answer.
The behavior of the TCP/IP stack in the second case depends on the implementation and the transport options. For a TCP connection and for unicast UDP the kernel might just realize that both source and target addresses are local to the machine and short-circuit data transfer through memory. You'll find that most modern network stacks behave this way. For broadcast or multicast UDP there's no other option but to send packets through physical card since there might be other listeners on the network.
I am trying to develop a client/server application in which client will send images to the server.
Case 1:
Server is running on one machine that is behind a router and client is running on another machine that is behind some different router. As this communication will be on WAN (public IPs), a port is forwarded on server side router so that server can easily receive incoming UDP datagrams on that port.
UDP's maximum transmission unit (MTU) size is 64KB. It means a UDP socket should be able to transmit anything thing of size less than or equal to 65,536 bytes. When in case of the application i am developing the client is only able to send an image(UDP datagram) of 10-13k. If i try to transfer an image of the size greater than 10Kb, server is unable to receive it and server side UDP socket will be always in (receive) blocking mode.
Case 2:
Server is running on a machine that is behind a router and client is running on a machine that is behind the same router. It means client & server are on the same local area network. Even client and server are sharing the same local area network client is sending the images (UDP datagrams) on server's public IP. In this case server is able to receive any size of UDP datagram upto 64K, which is what i am expecting from my application.
I tried to run my client on different remote PCs but the result is same. Server is not able to receive a UDP datagram of bigger than 10-13Kb. If anyone can help me to deal with this situation, he would be much appreciated.
Link to the code:
http://pastebin.com/f644fee71
Thanks and goodluck with your projects.
Regards,
Atif
Although the IP layer may allow UDP packets of up to 64k in size I think you will find that the maximum "in the wild" UDP packet size will be limited to the smallest MTU of the devices in between your source and destination.
The standard Ethernet MTU is ~1500 bytes. Some devices support "jumbo" frames of up to ~10kbytes to improve performance. But this sort of thing isn't generally supported over the public internet, only on LANs.
The IP layer may fragment the UDP packet (unless the no-fragment bit is set in the packet). But the recipient will only receive and defragment the packet if every fragment is received in order (or out of order within a specific time limit). Otherwise it will discard the packet.
It may also be the case that not all the devices in between your source and destination support the frame size of the sending device. I've encountered situations where I needed to lower the MTU on my routers to ~1450 bytes because intermediate routers were discarding packets at 1500. This is due to MTU discovery not working reliably. Ie, the sending device has no way of determinging what the MTU is of devices on its path to the destination. Somewhere in that path a device will be discarding packets it considers too large.
UDP is a very bad idea for what you are doing. You would be better off using TCP.
If you are concerned about the performance of TCP connection setup/tear down then keep the connection up for as long as possible.
UDP is only a good protocol for delivering data when you don't care too much about whether the target receives the packet or not. Delivery is not guaranteed. In all other cases use TCP.
If you are determined to use UDP you will have to implement path MTU discovery in your protocol and prey that the routers/firewalls don't block the "fragmentation needed" ICMP packets. Which they shouldn't otherwise TCP wouldn't work either. But like I said, I've seen cases in the past where fragmentation needed ICMP packets are blocked or discarded and I had to manually tweak my own MTU.
UDP is a unreliable datagram protocol. The packets are not guaranteed to arrive at their destination, or in the order that you sent them either.
In addition, when sending packets larger than around 1500 bytes you'll find they get fragmented, or worse, dropped. The receiver will try to piece it together best it can.
But if anything is missing, goodbye packet. So really, the limit is actually around that 1500 byte mark but sometimes much less to ensure no fragmentation and that they arrive. To send data larger than that a higher level protocol is going to have to put them all back together for you and request anything thats missing.
How big will the images be? 64K might be too small anyway.
You have two or three options.
1) Use TCP - it's reliable and stream orientated. You don't have to worry about how big or small or fragmentation as this is taken care of for you.
2) Use UDP but develop a higher level application protocol on top. Depending on your protocol that could be quite a bit of work. I'm doing this right now though.
3) Have a look at using the UDT library. It's designed for transferring bulk data over a WAN and has better performance than TCP.
However, I would like to suggest that TCP will likely suit your needs just fine.