We Keep Coding

TCP sender machine receiver-window-size shrinking to 0 on windows machine after sending tiny amounts of data for a few minutes

I'm writing an app that works on a windows laptop that sends TCP bytes in a loop as a "homemade keep-alive" message to keep a connection active (the server machine will disconnect after 15 seconds of no TCP data received). The "server machine" will send the laptop small chunks of data (about .5K bytes/second) as long as a connection is alive (according to the server documentation, this is ideally this is an "echo" packet, but I was unable to find how this is accomplished in .NET). My problem is that when I view this data in Wireshark I can see good network activity, then, after a few minutes, the "win" (receive window size available on the laptop) shrinks from 65K to 0 in increments of about 240 bytes each packet. Why is this happening and how can I prevent it? I can't seem to get the "keep-alive" flags in .Net to work, so this was supposed to be my workaround. I do not see any missed ACK messages, and my data rate is about 2Kb/sec, so I don't understand why the laptop window size is dropping. I definitely assume there is a misconception on my part about TCP and or windows/.NET use of TCPsockets since I have no experience with TCP (I've always used UDP).
TcpClient client = new TcpClient(iPEndpoint);
//Socket s = client.Client; none of these flags actually work on the keep alive feature
//s.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.KeepAlive, true);
//s.SetSocketOption(SocketOptionLevel.Tcp, SocketOptionName.TcpKeepAliveInterval, 10);
//s.SetSocketOption(SocketOptionLevel.Tcp, SocketOptionName.TcpKeepAliveInterval, 10);
// Translate the passed message into ASCII and store it as a Byte array.
// Byte[] data = System.Text.Encoding.ASCII.GetBytes(message);
IPAddress ipAdd = IPAddress.Parse("192.168.1.10");
IPEndPoint ipEndPoin = new IPEndPoint(ipAdd, 13000);
client.Connect(ipEndPoin);
NetworkStream stream = client.GetStream();
// Send the message to the connected TcpServer.
bool finished = false;
while (!finished)
{
try
{
stream.Write(data, 0, data.Length);
Thread.Sleep(5000);
}
catch (System.IO.IOException ioe)
{
if (ioe.InnerException is System.Net.Sockets.SocketException)
{
client.Dispose();
client = new TcpClient(iPEndpoint);
client.Connect(ipEndPoin);
stream = client.GetStream();
Console.Write("reconnected");
// this imediately fails again after exiting this catch to go back to the while loop because send window is still 0
}
}
}

You should really be familiar with RFC 793, Transmission Control Protocol, which is the definition of TCP. It explains the wondow and how it is used for flow control:
Flow Control:
TCP provides a means for the receiver to govern the amount of data
sent by the sender. This is achieved by returning a "window" with
every ACK indicating a range of acceptable sequence numbers beyond the
last segment successfully received. The window indicates an allowed
number of octets that the sender may transmit before receiving further
permission.
-and-
To govern the flow of data between TCPs, a flow control mechanism is
employed. The receiving TCP reports a "window" to the sending TCP.
This window specifies the number of octets, starting with the
acknowledgment number, that the receiving TCP is currently prepared to
receive.
-and-
Window: 16 bits
The number of data octets beginning with the one indicated in the
acknowledgment field which the sender of this segment is willing to
accept.
The window size is dictated by the receiver of the data in its ACK segments where it acknowledges receipt of the data. If your laptop receive window shrinks to 0, it is setting the window to that because it has no more space to receive, and it needs time to process and free up space in the receive buffer. When it has more space, it will send an ACK segment with a larger window.
Segment Receive Test
Length Window
------- ------- -------------------------------------------
0 0 SEG.SEQ = RCV.NXT
0 >0 RCV.NXT =< SEG.SEQ < RCV.NXT+RCV.WND
>0 0 not acceptable
>0 >0 RCV.NXT =< SEG.SEQ < RCV.NXT+RCV.WND
or RCV.NXT =< SEG.SEQ+SEG.LEN-1 < RCV.NXT+RCV.WND
Note that when the receive window is zero no segments should be
acceptable except ACK segments. Thus, it is be possible for a TCP to
maintain a zero receive window while transmitting data and receiving
ACKs. However, even when the receive window is zero, a TCP must
process the RST and URG fields of all incoming segments.

Can't get data out of FTDI FT201X using i2c

I have a project using the FTDI FT201X as a USB to i2c slave and the i2c master is an AVR microcontroller. I'm using WPF Core 3.1 C# on a Windows 10 machine. Basically, everything with the FTDI chip works fine except I can't successfully get data sent from the PC to the FTDI chip no matter what I try. The D2XX Write function says it was successful and returns no error, but there is never any data in the buffer when I try to read.
I've since written a small test program in an attempt to isolate the issue but the problem remains. Basically, when a button is clicked we open the device by serial number, we write a command to the device's buffers, handshake with the AVR to let it know to read and then wait for the AVR to drive a handshake pin low meaning it has received the data.
public class USBLibrary
{
byte targetDeviceCount = 0;
FTDI.FT_STATUS ftStatus = FTDI.FT_STATUS.FT_OK;
public FTDI connectedUSBDevice;
// Called from button click event
public void ConnectUSB()
{
bool isOK = true;
byte numOfBytes = 1;
uint bytesWritten = 0;
bool usbInPinIsHigh = false; // Tracks USB In Pin
byte lowMask = 0b00010000; // CBUS 0 is output (4-7), all pins low (0-3) (Default Setting)
byte highMask = 0b00010001; // CBUS 0 is output (4-7), CBUS 3 is high
byte inPinMask = 0b00001000; // AND with pin states to get input pin value (Bus3)
byte pinStates = 0; // Used to get the current pin values
double timeout = 0;
// Create new instance of the FTDI device class
connectedUSBDevice = new FTDI();
// Determine the number of FTDI devices connected to the machine
ftStatus = connectedUSBDevice.OpenBySerialNumber("P00001");
/*** Write to Device ***/
byte[] firmwareCmd = new byte[numOfBytes];
firmwareCmd[0] = 128; // 128 is Get Firmware Command
// firmwareCmd[1] = 61; // Just Testing
// Write Firmware Command to Tx buffer
ftStatus = connectedUSBDevice.Write(firmwareCmd, numOfBytes, ref bytesWritten);
Trace.WriteLine(bytesWritten);
// Handshake with Device
isOK = DeviceHandshake(lowMask, highMask, inPinMask);
// Check if handshake failed
if (isOK == false)
{
return;
}
Task.Delay(10);
// Wait until message is sent
while ((usbInPinIsHigh == false) && (timeout <= 1000))
{
Task.Delay(1);
// Check for USB In pin to go high. Signals FW transfer is complete and to retrieve.
ftStatus = connectedUSBDevice.GetPinStates(ref pinStates);
// Is input pin high or low?
if ((pinStates & inPinMask) == inPinMask) // In pin high
{
usbInPinIsHigh = true; // Means uC finished sending data
}
timeout++;
}
// TEST: displays timeout amount for testing
Trace.WriteLine("Timeout=" + timeout);
ftStatus = connectedUSBDevice.Close();
}
}
NOTE: For this code, I've taken out a lot of the error checking code for clarity. Also, the handshake code is not shown because it shouldn't be relevant: raise output pin, listen for AVR to raise output pin, lower output pin, listen for AVR to lower output pin.
On the AVR side, we simply poll for the FT201X's pin to go high and then handshake with the chip. Then we simply read. The read function always returns 0.
I doubt the problem is with i2c as there are 3 IO Expander chips controlling LEDs and buttons and we can read and write to those fine. Further, the FT chip has a function called Get USB State where you can check to see the device's status by sending the command and reading the result via i2c. When I do this, I always get back the correct 0x03 "Configured" state. So we can read from the chip via i2c.
There's also a function that will return the # of bytes in the buffer waiting to be read...when I do this, it always says 0 bytes.
And for good measure I replaced the chip with a new one in case it was bad and again we had the same results.
Is there anything I'm missing in terms of setting up the chip beyond using FT_Prog, like an initialization procedure or setting registers or something? Or do I need to somehow push the byte I write to the front of the queue or something before it can be read? Anybody seen anything like this before?
Given that I haven't affected the results, I'm either missing a key part in the process or something is wrong with their driver/version of the chip. It's been 3 weeks, I'm out of ideas, and my hair is patchy from ripping out large chunks. Please save my hair.

Check by oscilloscope that your I2C master gives clock for your slave (FT201x). Try to control only I2C (rip off GPIO controls) and check if you can isolate problem that way. I suppose you are very familiar with FT201X datasheet. Good luck!

Check the latency timer setting. It’s described in this document, https://www.ftdichip.com/Support/Documents/AppNotes/AN232B-04_DataLatencyFlow.pdf. In section 3.3, you’ll find a section describing a scenario in which no data will be made available to the app.
“While the host controller is waiting for one of the above conditions to occur, NO data is received by our driver and hence the user's application. The data, if there is any, is only finally transferred after one of the above conditions has occurred.”
You can use the latency timer to work around it, if you’re hitting this. Try setting it to 1ms, its lowest value. If your data has a delimiter character, consider setting that as an event character and you might get even better results.

Did this issue ever get resolved?
Experiencing the same issues with an FT200X except the function "bytes available" (0x0C) returns the correct byte count sent from the host PC, but can't read the actual bytes using the read procedure mentioned in the datasheet.
I have also several other I2C devices on the bus, working fine.

C# Socket Receive messes up data

I am trying to create a client/server application which communicates through Sockets. But when I receive data it randomly messed up and does not always (sometimes) show the correct data. The code I use to receive data:
private static void ReceiveCallback(IAsyncResult AR)
{
try
{
Socket socket = (Socket)AR.AsyncState;
if (SocketConnected(socket))
{
int received = socket.EndReceive(AR);
byte[] dataBuf = new byte[received];
Array.Copy(_buffer, dataBuf, received);
string text = Encoding.ASCII.GetString(dataBuf);
if (!text.Contains("GET") && !text.Contains("HTTP") && text != null)
{
Console.WriteLine(DateTime.Now.ToString("h:mm:ss tt") + ":" + text);
}
socket.BeginReceive(_buffer, 0, _buffer.Length, SocketFlags.None, new AsyncCallback(ReceiveCallback), socket);
}
}
catch
{
}
}
can someone explain me why this happends? Thanks in advance!
Also, I have tried to check untill it finds a character (which is on the end of every response '-' but that did not work). Unfortunatley this was the result:

I assume your socket is a TCP socket, since you're dealing with something like HTTP.
TCP deals with streams, not messages. So when you do your read, at most _buffer.Length bytes are read from the stream. This means that you can easily read just a part of the response - or rather, the response will be split between multiple callbacks. When you read enough by chance, everything works as expected. This is especially true when testing on localhost, because Windows treats localhost TCP very differently from the non-localhost case. But when it so happens that you don't read enough, text will not contain the whole response - and your data is broken.
Additionally, when received is zero, it means the other side has closed the stream and you should stop doing BeginReceive.

You have to loop on the socket receive until you get your entire message.
This also means that you must have an application level protocol. What defines a message?
A termination character? Then loop on the read until you get that character.
A count of characters? Then loop on the read until you get that many characters.
Is the length of the message at the front of the message? Then loop until you get enough bytes to get the length, then loop until you get the entire message.

Your socket code looks fine to me. Characters encoding format in the .Net world is "Unicode", you should convert the bytes you receive to Unicode instead of ASCII :
string text = Encoding.Unicode.GetString(dataBuf);

issue with getting TcpClient server response as string

so long story short, i am trying to develop an application which requires having Tcp Connection to a server. having no previous experiance in this field i ended up with having none of my functions working. so i decided to make a simple console app just to check my commands and their responses. the part about setting up the connections went well, so does the part about wrting into network stream(i think) but i hit a wall when trying to display server respons:
each time my porgram reaches the line where it has to Encode server respons and WriteLine, my console application goes all black and all texts goes away. can you tell what's wrong with the code i am using?everything up until the part where console trys to display response goes well, i checked. i want to have server's respones as normal strings(if possible i guess...)
static void ServerTalk(TcpClient tcpClient,NetworkStream networkStream, StreamWriter streamWriter, StreamReader streamReader)
{
//Messaging methods:
Console.WriteLine("Client: ");
string message = Console.ReadLine();
streamWriter.WriteLine(message);
streamWriter.Flush();
//Response methode:
byte[] data = new byte[tcpClient.ReceiveBufferSize];
int ret = networkStream.Read(data, 0, data.Length);
string respond = Encoding.ASCII.GetString(data).TrimEnd();
Console.WriteLine("Server: ");
Console.WriteLine(respond);
ServerTalk(tcpClient, networkStream, streamWriter, streamReader);
}

you need to process only ret bytes, you are translating all bytes in the array.
copy to a new array that is 'ret' long and then decode.
var retarr = new byte[ret];
Array.Copy(data,retarr,ret);
string respond = Encoding.ASCII.GetString(retarr);
More importantly note that this method of tcp communication is not going to work. You are assuming that when you send x bytes in one block you will receive x bytes in one block, this is not so, you can send 1 100 byte message and receive 100 1 byte messages
This means that you task is much more complex. YOu need to devise a way of sending self describing data - so that you know its completely received. classically you send a length followed by the data. You then keep looping in the receive code till you have received that many bytes

How to ask the socket to wait for more data to come

I am playing with RserveCLI project, which is a .net client communicating with the statistical environment R. The basic idea is sending data/commends between this .NET client and an R session via TCP protocol.
One bug that others and I found is that big data trunk, say over 10k bytes, cannot get transfer successfully. I found the but in the following code snippet:
// send the commend to R, then R will do some computation and get the data ready to send back
int toConsume = this.SubmitCommand(cmd, data);
var res = new List<object>();
while (toConsume > 0)
{
var dhbuf = new byte[4];
if (this.socket.Receive(dhbuf) != 4)
{
throw new WebException("Didn't receive a header.");
}
byte typ = dhbuf[0];
// ReSharper disable RedundantCast
int dlength = dhbuf[1] + (((int)dhbuf[2]) << 8) + (((int)dhbuf[3]) << 16);
// ReSharper restore RedundantCast
var dvbuf = new byte[dlength];
// BUG: I added this sleep line, without this line, bug occures
System.Threading.Thread.Sleep(500);
// this line cannot receive the whole data at once
var received = this.socket.Receive(dvbuf);
// so the exception throws
if (received != dvbuf.Length)
{
var tempR = this.socket.Receive(dvbuf);
throw new WebException("Expected " + dvbuf.Length + " bytes of data, but received " + received + ".");
}
The reason is that the .NET code runs too fast and the R side cannot send the data that fast. So the receive line after my inserted Thread.Sleep(500) does not get all the data. If I wait some time there, then it can get all the data. But I don't know how long.
I have some basic idea to deal with the problem, for example, continuously use this.socket.Receive() to get data, but if there is no data there .Receive will block there.
I have little experience in socket programming, so I am asking the best practice for this kind of problem. Thanks!

According to the docs:
If you are using a connection-oriented Socket, the Receive method will read as much data as is available, up to the size of the buffer.
So you are never guranteed to get all the data asked for in the receive call. You need to check how many bytes were actually returned by the Receive, then issue another receive call for the remaining bytes. Continue that loop until you get all the bytes you were looking for.

By definition, TCP is a streaming protocol, whereas UDP is message based. If the data you are trying to receive does not contain a byte count for the entire message, or some sort of end-of-message indicator, you will just have to loop on the socket.receive until some arbitrary timeout has expired. At that point, check the accumulated received data for completeness.