I have this code inside of a controller that is called from a view by multiple users. The purpose is to read data from a socket while the socket has information to deliver. Multiple calls can happen simultaneously. The problem is, for a same call the Socket will sometimes cut the message. We will not read it entirely. Is there something wrong in my call that could lead to an incomplete reading of the Socket ? I am not able to reproduce it on my dev environment I just know that on the production we have responses from the Socket being randomly cut / incomplete.
int byteCount = 1;
string response = "";
int total = 0;
bytesReceived = new Byte[8000];
bool bytesAreAvailable = true;
while (byteCount > 0 || bytesAreAvailable)
{
try {
byteCount = m_oSocket.Receive(bytesReceived, bytesReceived.Length, 0);
total += byteCount;
Debug.WriteLine("Bytecount: " + byteCount.ToString());
Debug.WriteLine("Avalable: " + m_oSocket.Available.ToString());
if (m_oSocket.Available > 0)
bytesAreAvailable = true;
else
bytesAreAvailable = false; //Everything was read
response += Encoding.UTF8.GetString(bytesReceived, 0, byteCount);
}
catch(Exception e)
{
Debug.WriteLine(e.Message);
}
}
Debug.WriteLine(response.ToString());
Thanks a lot for any help / tips you could have on solving my issue
Related
My dotnet/c# server is running very well for my Unity game on my local machine.
But when I host it on an online cloud server (AWS EC2 instance) it will result in not all TCP messages arriving at the client side.
Server side method:
git: github.com/IRiViI/TCPmreServer
public void SendData(Packet _packet)
{
try
{
if (socket != null)
{
stream.BeginWrite(_packet.ToArray(), 0, _packet.Length(), null, null);
}
}
catch (Exception _ex)
{
Console.WriteLine($"Error sending data to player {id} via TCP: {_ex}");
}
}
Client side method:git: github.com/IRiViI/TCPmreClient
private void ReceiveCallback(IAsyncResult _result){
try {
int _byteLength = stream.EndRead(_result);
if (_byteLength <= 0){
client.Disconnect();
return;
}
byte[] _data = new byte[_byteLength];
Array.Copy(receiveBuffer, _data, _byteLength);
receivedData.Reset(HandleData(_data));
stream.BeginRead(receiveBuffer, 0, dataBufferSize, ReceiveCallback, null);
} catch(Exception _ex) {
Disconnect();
Debug.Log(_ex);
}
}
private bool HandleData(byte[] _data){
int _packetLength = 0;
receivedData.SetBytes(_data);
if(receivedData.UnreadLength() >= 4){
_packetLength = receivedData.ReadInt();
if (_packetLength <= 0){
return true;
}
}
while (_packetLength > 0 && _packetLength <= receivedData.UnreadLength()){
byte[] _packetBytes = receivedData.ReadBytes(_packetLength);
ThreadManager.ExecuteOnMainThread(() => {
using (Packet _packet = new Packet(_packetBytes)){
int _packetId = _packet.ReadInt();
client.packetHandlers[_packetId](_packet);
}
});
_packetLength = 0;
if(receivedData.UnreadLength() >= 4){
_packetLength = receivedData.ReadInt();
if (_packetLength <= 0){
return true;
}
}
}
if (_packetLength <= 1){
return true;
}
return false;
}
It only happens when I send many (100+) messages at the same time. below 100 it usally goes well.
I think the issue might be AWS related (firewall and such). But I would expect to get some error messages when a TCP message could not be send. I'm not able to find any errors. I would have expect this behaviour of UDP but not of TCP.
My question is: How can I increase the amount of messages that can be send or have some message not received callback inorder to handle failed TCP messages.
Update --
The problem seems to be on the client side. I tested my server code on my server at home and the same problem occures as when running my server code on an EC2 AWS instance.
The problem occures when there are multiple tcp packets handled.
At the moment when the tcp handler breaks, I get a packetsLength of 16777216 (which equals 2^24).
if(receivedData.UnreadLength() >= 4){
_packetLength = receivedData.ReadInt();
if (_packetLength <= 0){
return true;
}
Console.WriteLine($"_{_packetLength}");
}
I am writing C# code to make data acquire system from FPGA with USB communication and not that familiar with C#.
The system received data continuously at regular intervals through USB.
Data is continuously received only when DOPPLER_NUM = 1, and while (DOPPLER_NUM == 1) is that process.
And there are two problems in operation.
When I operate code without Delay(1) in while loop, the program is completely stopped and none of the buttons in the program works.
I used Delay function to solve the problem, and it works.
But, the delay I want is 1ms and it actually varies from 1-15ms when I measured it with Stopwatch function.
public void Delay(int MS)
{
DateTime ThisMoment = DateTime.Now;
TimeSpan Duration = new TimeSpan(0, 0, 0, 0, MS);
DateTime AfterWards = DateTime.Now.Add(Duration);
while (AfterWards >= ThisMoment)
{
System.Windows.Forms.Application.DoEvents();
ThisMoment = DateTime.Now;
}
return;
}
int DOPPLER_NUM = 0;
private void Doppler_Start_Click(object sender, EventArgs e)
{
int transmit_mode = 1;
if (DOPPLER_NUM == 0)
{
DOPPLER_NUM = 1;
transmit_mode = 1;
}
else
{
DOPPLER_NUM = 0;
transmit_mode = 0;
cnt4GS = 0;
}
CyBulkEndPoint Inednpt1;
Inednpt1 = MyDevice.EndPointOf(0x86) as CyBulkEndPoint;
int bytes1 = Convert.ToInt32(256);
int bytes2 = bytes1 * 2;
bool bXferCompleted = false;
bool IsPkt = false;
byte[] buffer4GS = new byte[400 * 256];
byte[] buffer1 = new byte[bytes2];
byte[] Data_buf = new byte[bytes1];
while (DOPPLER_NUM == 1)
{
Stopwatch clk1 = new Stopwatch();
clk1.Start();
Delay(1);
clk1.Stop();
opertime.Text = (clk1.ElapsedTicks * 100 / 1000).ToString() + " us";
if (MyDevice != null)
{
if (Inednpt1 != null)
{
bXferCompleted = Inednpt1.XferData(ref buffer1, ref bytes2, IsPkt); // Data receiving from USB
Data_buf = Doppler_Processing(buffer1, bytes2);
if (cnt4GS >= 0 && cnt4GS <= 399)
{
Buffer.BlockCopy(Data_buf, 0, buffer4GS, cnt4GS * 256, 256);
cnt4GS++;
}
else if (cnt4GS >= 400)
{
Buffer.BlockCopy(buffer4GS, 256, buffer4GS, 0, 102144);
Buffer.BlockCopy(Data_buf, 0, buffer4GS, 102144, 256);
}
Grayscale(buffer4GS);
}
}
else if (MyDevice == null)
{
MessageBox.Show("ERROR. NODEVICE.", "Error Message");
break;
}
}
Your Delay method gives the UI thread to handle events, by calling System.Windows.Forms.Application.DoEvents();
If you have an intensive operation bound to your UI thread (which explains the behaviour) you'll end up with a blocked UI. The usage of System.Windows.Forms.Application.DoEvents(); can resolve this, but it would be better to just execute your DOPPLER_NUM loop in another thread - not bothering the UI thread with it and keeping your program responsive.
Alternatively, you can just call System.Windows.Forms.Application.DoEvents(); from your loop itself and it would work just as good, maybe increasing performance a bit.
Also:
Alternatives like Task.Delay and Thread.Sleep will not have a better accuracy as about 5ms. This is by design - because measuring the exact time will cost CPU power, same as the stopwatch does.
I have got an embedded debian board with mono running an .NET 4.0 application with a fixed number of threads (no actions, no tasks). Because of memory issues I used CLR-Profiler in Windows to analyse memory heap.
Following diagram shows now, that IThreadPoolWorkItems are not (at least not in generation 0) collected:
Now, I really dont have any idea where this objects are possibly used and why they arent collected.
Where could the issue be for this behaviour and where would the IThreadPoolWorkItem being used?
What can I do to find out where they are being used (I couldnt find them through searching the code or looking in CLR-Profiler yet).
Edit
...
private Dictionary<byte, Telegram> _incoming = new Dictionary<byte, Telegram>();
private Queue<byte> _serialDataQueue;
private byte[] _receiveBuffer = new byte[2048];
private Dictionary<Telegram, Telegram> _resultQueue = new Dictionary<Telegram, Telegram>();
private static Telegram _currentTelegram;
ManualResetEvent _manualReset = new ManualResetEvent(false);
...
// Called from other thread (class) to send new telegrams
public bool Send(Dictionary<byte, Telegram> telegrams, out IDictionary<Telegram, Telegram> received)
{
try
{
_manualReset.Reset();
_incoming.Clear(); // clear all prev sending telegrams
_resultQueue.Clear(); // clear the receive queue
using (token = new CancellationTokenSource())
{
foreach (KeyValuePair<byte, Telegram> pair in telegrams)
{
_incoming.Add(pair.Key, pair.Value);
}
int result = WaitHandle.WaitAny(new[] { token.Token.WaitHandle, _manualReset });
received = _resultQueue.Clone<Telegram, Telegram>();
_resultQueue.Clear();
return result == 1;
}
}
catch (Exception err)
{
...
return false;
}
}
// Communication-Thread
public void Run()
{
while(true)
{
...
GetNextTelegram(); // _currentTelegram is set there and _incoming Queue is dequeued
byte[] telegramArray = GenerateTelegram(_currentTelegram, ... );
bool telegramReceived = SendReceiveTelegram(3000, telegramArray);
...
}
}
// Helper method to send and receive telegrams
private bool SendReceiveTelegram(int timeOut, byte[] telegram)
{
// send telegram
try
{
// check if serial port is open
if (_serialPort != null && !_serialPort.IsOpen)
{
_serialPort.Open();
}
Thread.Sleep(10);
_serialPort.Write(telegram, 0, telegram.Length);
}
catch (Exception err)
{
log.ErrorFormat(err.Message, err);
return false;
}
// receive telegram
int offset = 0, bytesRead;
_serialPort.ReadTimeout = timeOut;
int bytesExpected = GetExpectedBytes(_currentTelegram);
if (bytesExpected == -1)
return false;
try
{
while (bytesExpected > 0 &&
(bytesRead = _serialPort.Read(_receiveBuffer, offset, bytesExpected)) > 0)
{
offset += bytesRead;
bytesExpected -= bytesRead;
}
for (int index = 0; index < offset; index++)
_serialDataQueue.Enqueue(_receiveBuffer[index]);
List<byte> resultList;
// looks if telegram is valid and removes bytes from _serialDataQueue
bool isValid = IsValid(_serialDataQueue, out resultList, currentTelegram);
if (isValid && resultList != null)
{
// only add to queue if its really needed!!
byte[] receiveArray = resultList.ToArray();
_resultQueue.Add((Telegram)currentTelegram.Clone(), respTelegram);
}
if (!isValid)
{
Clear();
}
return isValid;
}
catch (TimeOutException err) // Timeout exception
{
log.ErrorFormat(err.Message, err);
Clear();
return false;
} catch (Exception err)
{
log.ErrorFormat(err.Message, err);
Clear();
return false;
}
}
Thx for you help!
I found out, like spender mentioned already, the "issue" is the communication over SerialPort. I found an interesting topic here:
SerialPort has a background thread that's waiting for events (via WaitCommEvent). Whenever an event arrives, it queues a threadpool work
item that may result in a call to your event handler. Let's focus on
one of these threadpool threads. It tries to take a lock (quick
reason: this exists to synchronize event raising with closing; for
more details see the end) and once it gets the lock it checks whether
the number of bytes available to read is above the threshold. If so,
it calls your handler.
So this lock is the reason your handler won't be called in separate
threadpool threads at the same time.
Thats most certainly the reason why they arent collected immediatly. I also tried not using the blocking Read in my SendReceiveTelegram method, but using SerialDataReceivedEventHandler instead led to the same result.
So for me, I will leave things now as they are, unless you bring me a better solution, where these ThreadPoolWorkitems arent kept that long in the Queue anymore.
Thx for your help and also your negative assessment :-D
So I got a server that shoots off the below task in a while loop it's functioning as my client listener. The issue seems like it wants to loop as fast as possible through the clients which is great! But it fires tasks off too fast before the previous task is done a new one happens ( for the same client).
I don't want it to wait for the task to complete! I want it to shoot more tasks but just not create anymore tasks for a specific client that already has an existing task.
What is the best way to go about this... I see a lot of people using WhenAll or something but I don't care about All the tasks.
//the below is in a while loop which goes through the clients that are connected.
if (stream.DataAvailable)
{
// the below task is the one that is firing before the pervious fired one completes.
Task DataInterpetTask = Task.Factory.StartNew(() =>
{
int totalBuffer, totalRecieved = 0;
byte[] totalBufferByte = new byte[4];
byte[] buffer = new byte[0];
byte[] tbuffer;
int rLength, prevLength;
stream.Read(totalBufferByte, 0, 4);
totalBuffer = BitConverter.ToInt32(totalBufferByte, 0);
Console.WriteLine("got stuff: " + totalBuffer);
byte[] buf = new byte[c.getClientSocket().ReceiveBufferSize];
while (totalBuffer > totalRecieved)
{
rLength = stream.Read(buf, 0, buf.Length);
totalRecieved = rLength + totalRecieved;
if (rLength < buf.Length)
{
byte[] temp = new byte[rLength];
Array.Copy(buf, temp, rLength);
buf = temp;
}
prevLength = buffer.Length;
tbuffer = buffer;
buffer = new byte[buffer.Length + rLength];
Array.Copy(tbuffer, buffer, tbuffer.Length);
buf.CopyTo(buffer, prevLength);
}
String msg = Encoding.ASCII.GetString(buffer);
if (msg.Contains("PNG") || msg.Contains("RBG") || msg.Contains("HDR"))
{
Console.WriteLine("Receiving Picture");
RowContainer tempContainer;
if ((tempContainer = MainWindow.mainWindow.RowExists(c)) != null)
{
tempContainer.Image = buffer;
Console.WriteLine("Updating row: " + tempContainer.rowNumber);
MainWindow.mainWindowDispacter.BeginInvoke(new Action(() =>
MainWindow.mainWindow.UpdateRowContainer(tempContainer, 0)));
}
else
{
Console.WriteLine("Adding row for Picture");
MainWindow.mainWindowDispacter.BeginInvoke(new Action(() =>
MainWindow.mainWindow.CreateClientRowContainer(c, buffer)));
}
return;
}
String switchString = msg.Substring(0, 4);
if (msg.Length > 4)
msg = msg.Substring(4);
MainWindow.mainWindowDispacter.BeginInvoke(new Action(() =>
{
if (MainWindow.debugWindow != null)
MainWindow.debugWindow.LogTextBox.AppendText("Received message " + msg + " from client: " + c.getClientIp() + " as a " + switchString + " type" + Environment.NewLine);
}));
RowContainer tContain = MainWindow.mainWindow.RowExists(c);
if(tContain == null)
return;
switch (switchString)
{
case "pong":
c.RespondedPong();
break;
case "stat":
tContain.SetState(msg);
MainWindow.mainWindow.UpdateRowContainer(tContain, 1);
break;
case "osif":
tContain.Os = msg;
MainWindow.mainWindowDispacter.BeginInvoke(new Action(() =>
MainWindow.mainWindow.UpdateRowContainer(tContain, 2)));
break;
case "idle":
tContain.idle = msg;
MainWindow.mainWindowDispacter.BeginInvoke(new Action(() =>
MainWindow.mainWindow.UpdateRowContainer(tContain, 3)));
break;
}
});
}
If you don't want to start a new operation for a given client until the previous one has completed, then just keep track of that. Either move the client object to some "in progress" list, so that it's not even in the list of clients that you are processing in your loop, or add a flag to the client object class that indicates that the operation is "in progress", so that your loop can ignore the client until the current operation has completed (e.g. if (c.OperationInProgress) continue;).
That said, it appears that you are using polling (i.e. checking DataAvailable) to manage your I/O. This is a really inefficient approach, and on top of that is the root cause of your current problem. If you were using the much better asynchronous model, your code would be more efficient and you wouldn't even have this problem, because the asynchronous API would provide all of the state management you need in order to avoid overlapping operations for a given client.
Unfortunately, your code example is very sparse, missing all of the detail that would be required to offer specific advice on how to change the implementation to use the asynchronous model (and what little code there is, includes a lot of extra code that would not be found in a good code example). So hopefully the above is enough to get you on track to the right solution.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question appears to be off-topic because it lacks sufficient information to diagnose the problem. Describe your problem in more detail or include a minimal example in the question itself.
Closed 8 years ago.
Improve this question
I have developed one application in which i am polling the serial port(sending request and getting response) every after 5 sec. everything was working very fine and i tested it thoroughly.
but when i introduced threading i get the blue screen error regarding physical memory or something as attached below in threading i just separated GUI updating task and polling task.
can anyone suggest me what should i do to remove the same, just stuck and not understanding what to do.
This is happening every time i run the application, i am using profic driver serial to usb driver.
MAIN APPLICATION CODE
private void timer_Elapsed(object sender, ElapsedEventArgs e)
{
PollingAllMeters();
}
public void PollingAllMeters()
{
int isdbError = 0;
PollingUtility.MeterClass mData = new PollingUtility.MeterClass();
for (int index = 0; index < m_slaveID.Length; index++)
{
try
{
mData = poll.MeterOne(m_slaveID[index]);
Thread processData = new Thread(() => this.updateData(data));
processData.Start();
}
catch { MessageBox.Show("Inside polling box");
}
}
private void updateData(MeterValues data)
{
mainFormRef.updateData(data);
}
DLL CODE IN WHICH ACTUAL REQUEST AND RESPONSE ARE HAPPENING
enter code here
public MeterClass MeterOne(byte meterID)
{
MeterClass data1 = new MeterClass();
//Voltage
mb.SendFc3(meterID, startAdd, register, ref value_meter); //modbus call is made here for sending and receiving response at serial port
if (mb.modbusStatus == "Read successful")
{
//do some calculation
}
else
{
// error handling code to create log file
if (mb.modbusStatus == "Error in read event")
{
isPollError = 21;
}
else if (mb.modbusStatus == "CRC error")
{
isPollError = 22;
}
else if (mb.modbusStatus == "Serial port not open")
{
isPollError = 23;
}
}
startAdd = 360;
register = 2;
value_meter = new byte[register * 2];
meterData = new byte[register * 2];
//Max demand
mb.SendFc3(meterID, startAdd, register, ref value_meter);
if (mb.modbusStatus == "Read successful")
{
//do some calculation
}
else
{
// error handling code to create log file
if (mb.modbusStatus == "Error in read event")
{
isPollError = 21;
}
else if (mb.modbusStatus == "CRC error")
{
isPollError = 22;
}
else if (mb.modbusStatus == "Serial port not open")
{
isPollError = 23;
}
}
startAdd = 496;
register = 16;
value_meter = new byte[register * 2];
meterData = new byte[register * 2];
mb.SendFc3(meterID, startAdd, register, ref value_meter);
if (mb.modbusStatus == "Read successful")
{
//do some calcualtion
}
else
{
// error handling code to create log file
if (mb.modbusStatus == "Error in read event")
{
isPollError = 21;
}
else if (mb.modbusStatus == "CRC error")
{
isPollError = 22;
}
else if (mb.modbusStatus == "Serial port not open")
{
isPollError = 23;
}
}
data1.date = DateTime.Now;
data1.Status = isPollError;
return data1; // return result to the main program
}
USB and BlueTooth device drivers that emulate a serial port are notoriously unreliable. It is a cut-throat business that doesn't leave a lot money to pay a good programmer a decent wage, I guess. The only thing you can do about it if you can't get a driver update is to destroy the thing so it stops making your life miserable. Take it out to the parking lot and run over it with your car a couple of times.
Do take a good look at your code before you spin up the engine. "Multi-threading" is a red flag, the driver is oblivious to your program using more than one thread. Avoid a scenario where you can call Read or Write simultaneously from more than one thread. A driver should always be resilient to that but this doesn't get put to the test very often. Rejigger the code if necessary so only a single thread ever reads or writes or use the lock statement.