In the below code state is an object which has a property msgLength which is the total message length.The msgLength is the first four bytes of the message.
I am getting unexpected results in the state.msgLength propery when more than one message is received in "quick" succesion.When the state.msglength is not as expected
I get an error trying to receive. Specified argument was out of the range of valid values.
StateObj state = (StateObj)ar.AsyncState;
try
{
state.read += state.socket.EndReceive(ar);
if (state.read == 0)
{
state.socket.Close();
state.socket.Dispose();
state.socket = null;
this.Invoke((MethodInvoker)delegate { listBox1.Items.Add(DateTime.Now.ToString("HH:mm ss") + " socket closed"); });
return;
}
}
catch (Exception)
{
this.Invoke((MethodInvoker)delegate { listBox1.Items.Add(DateTime.Now.ToString("HH:mm ss") + " socket closed"); });
return;
}
if (state.read < 4)
{
//read again you dont have state,msglength
state.socket.BeginReceive(state.dataBuffer, state.read, 4 - state.read, 0, cbReceive, state);
}
else
{
state.msglength = BitConverter.ToInt32(state.dataBuffer, 0);
}
if (state.read < state.msglength)
{
//read again you dont have the complete message
//here is the error Specified argument was out of the range of valid values.
state.socket.BeginReceive(state.dataBuffer, state.read, state.msglength - state.read, 0, cbReceive, state);
}
else
{
//process here
if (state.read > state.msglength )
{
state.read = state.read - state.msglength;
}
else
{
state.read = 0;
}
state.socket.BeginReceive(state.dataBuffer, 0, state.dataBuffer.Length, 0, cbReceive, state);
}
I think that state.socket.BeginReceive is executed twice.
Because when it is reading it's header, both if (state.read < 4) and if (state.read < state.msglength) are true. Return after the first state.socket.BeginReceive(.. could solve the issue.
if (state.read < 4)
{
//read again you dont have state,msglength
state.socket.BeginReceive(state.dataBuffer, state.read, 4 - state.read, 0, cbReceive, state);
return; // <------
}
else
{
state.msglength = BitConverter.ToInt32(state.dataBuffer, 0);
}
if (state.read < state.msglength)
{
//read again you dont have the complete message
//here is the error Specified argument was out of the range of valid values.
state.socket.BeginReceive(state.dataBuffer, state.read, state.msglength - state.read, 0, cbReceive, state);
}
else
{
state.read = 0;
//process correctly expected message
}
I think I found another issue, If the messageLength specified within the message is only the size of the data. You should read if (state.read < state.msglength+4) Because the length header is included in your buffer.
I mostly split-up reading header/data. Like: (PSEUDO)
byte[] header;
byte[] buffer;
int bytesRead;
public void ReadHeader()
{
bytesNeeded = 4;
bytesRead = 0;
BeginReadHeader();
}
public void BeginReadHeader()
{
BeginReceive(header, bytesRead, bytesNeeded-bytesRead, EndReadHeader);
}
public void EndReadHeader()
{
int read = EndReceive();
if(read == 0)
{
CloseSocket();
return;
}
bytesRead += read;
if(bytesRead == byteNeeded)
ReadData();
else
BeginReadHeader();
}
public void ReadData()
{
bytesNeeded = BitConverter.ToInt32(header, 0);
bytesRead = 0;
if(buffer.Length < bytesNeeded)
buffer = new byte[bytesNeeded];
BeginReadData();
}
public void BeginReadData()
{
BeginReceive(buffer, bytesRead, bytesNeeded-bytesRead, EndReadData);
}
public void EndReadData()
{
int read = EndReceive();
if(read == 0)
{
CloseSocket();
}
bytesRead += read;
if(bytesRead == byteNeeded)
{
HandleData();
}
else
BeginReadData();
}
public void HandleData()
{
// handle data in buffer. BinaryReader
ReadHeader();
}
Related
I have a namedpipeServerStream and I want to stop Reading its pipe when I meet the timeout.
these are my Read and Read callback function , and I call Read in my main program loop repeatedly,
Actually my question is how can I call my callback function manually(or any other way)when timeout happened for endread.
private void Read()
{
tmpBuff = new byte[600];
inRead = true;
ReadTimer.Enabled = true;
len = 0;
try
{
_namedpipeserver.BeginRead(tmpBuff, 0, 600, ReadCallback, null);
}
catch (Exception ex) //disconnected/disposed
{
return;
}
}
static void ReadCallback(IAsyncResult ar)
{
int readbyte;
try
{
readbyte = _namedpipeserver.EndRead(ar);
if (readbyte > 0)
{
len = readbyte;
int packetLen = tmpBuff[0] + (tmpBuff[1] * 256);
Console.WriteLine(" packet len: " + packetLen + " bytes ");
readbyte = 0;
array = null;
array = new byte[packetLen];
Array.Copy(tmpBuff, 2, array, 0, packetLen);
}
}
catch (IOException) //closed
{
return;
}
inRead = false;
}
Any help Appreciated.Thanks
I resolved this problem by disposing connection after a defined timeout and waiting for other side of the pipe to connect again.
Task.Run(() =>
{
tmpBuff = new byte[600];
readbyte = _namedpipeserver.Read(tmpBuff, 0, 600);
}).Wait(10000);
if (readbyte <= 0)
return null;
In case null returned try to reinitiate pipe.
I am stuck with this serious problem.
I have async TcpListener. Sometimes there are several connected clients and everything is fine. Sometimes even if there is one person. The server starts usage 50-60% of my CPU.
I think the problem could be in handling of reading exceptions but it's my only tip and I don't know how to test it.
Here is code for server:
class Server
{
private TcpListener server;
public Server(string hostname, int port = 25000)
{
server = new TcpListener(IPAddress.Parse(hostname), port);
}
public void ServerStart()
{
server.Start();
WaitForClientConnect();
}
private async void WaitForClientConnect()
{
TcpClient client = await server.AcceptTcpClientAsync();
Console.WriteLine("The async connection created for: " + ((IPEndPoint)client.Client.RemoteEndPoint).Address.ToString());
OnClientConnect(client);
}
private void OnClientConnect(TcpClient client)
{
ClientLowAPI clientReq = new ClientLowAPI(client);
WaitForClientConnect();
}
}
Here is the code for handling a single client:
class ClientLowAPI
{
private TcpClient client;
private NetworkStream stream;
public ClientLowAPI(TcpClient clientConnected)
{
client = clientConnected;
stream = client.GetStream();
WaitForHeader();
}
private async void WaitForHeader()
{
byte[] buffer = new byte[4];
int bytesRead = 0;
while (bytesRead < 4)
{
try
{
bytesRead += await stream.ReadAsync(buffer, bytesRead, buffer.Length - bytesRead);
}
catch
{
stream.Close();
client.Close();
return;
}
}
WaitForData(FourBytesToInt(buffer));
}
private async void WaitForData(int length)
{
byte[] buffer = new byte[length];
int bytesRead = 0;
while (bytesRead < length)
{
try
{
bytesRead += await stream.ReadAsync(buffer, bytesRead, buffer.Length - bytesRead);
}
catch
{
stream.Close();
client.Close();
return;
}
}
ExecuteMessage(buffer);
}
private void ExecuteMessage(byte[] binaryData)
{
// Do something with message
WaitForHeader();
}
public async void SendMessage(byte[] message)
{
byte[] buffer = new byte[message.Length + 4];
byte[] length = IntToFourBytes(message.Length);
length.CopyTo(buffer, 0);
message.CopyTo(buffer, 4);
try
{
await stream.WriteAsync(buffer, 0, buffer.Length);
}
catch
{
stream.Close();
client.Close();
return;
}
}
private int FourBytesToInt(byte[] array)
{
int res = 0;
res += array[0] * 256 * 256 * 256;
res += array[1] * 256 * 256;
res += array[2] * 256;
res += array[3];
return res;
}
private byte[] IntToFourBytes(int intValue)
{
byte[] array = new byte[4];
array[0] = (byte)(intValue >> 24);
array[1] = (byte)(intValue >> 16);
array[2] = (byte)(intValue >> 8);
array[3] = (byte)intValue;
return array;
}
}
The problem was caused by this loop:
while (bytesRead < 4)
{
try
{
bytesRead += await stream.ReadAsync(buffer, bytesRead, buffer.Length - bytesRead);
}
catch
{
stream.Close();
client.Close();
return;
}
}
Normally was connection killed by force and server has raised exception but sometimes the connection was properly closed by client by TcpClient.Close() then the exception was not raised and the stream.ReadAsync method start returns 0 which caused infinty loop with huge CPU usage.
for fast fix i used the following condition:
if (bytesRead == 0)
{
throw new System.IO.IOException();
}
I am working on calculation of MD5, MD4, SHA1, SHA256, SHA512, RIPEMD160 etc. for selected file.
I have created the following algorithm but it has problem.
string finalHash;
byte[] buffer;
byte[] oldBuffer;
int bytesRead;
int oldBytesRead;
long streamSize;
long totalBytesRead = 0;
try
{
if (!String.IsNullOrEmpty(selectedFile))
{
_dataStream = File.OpenRead(selectedFile);
selectedFile = string.Empty;
}
foreach (var hashObject in from Control ctrl in Controls where ctrl is CheckBox && ((CheckBox)ctrl).Checked select HashObject)
{
//totalBytesRead = 0;
streamSize = _dataStream.Length;
buffer = new byte[4096];
bytesRead = _dataStream.Read(buffer, 0, buffer.Length);
totalBytesRead += bytesRead;
do
{
oldBytesRead = bytesRead;
oldBuffer = buffer;
buffer = new byte[4096];
bytesRead = _dataStream.Read(buffer, 0, buffer.Length);
totalBytesRead += bytesRead;
if (bytesRead == 0)
{
hashObject.TransformFinalBlock(oldBuffer, 0, oldBytesRead);
}
else
{
hashObject.TransformBlock(oldBuffer, 0, oldBytesRead, oldBuffer, 0);
}
hashCalculationWorker.ReportProgress((int)((double)totalBytesRead * 100 / streamSize));
} while (bytesRead != 0);
e.Result = hashObject.Hash;
finalHash = GenerateHex(hashObject.Hash);
Invoke(new MethodInvoker(() =>
{
// Get finalHash
}));
hashObject.Dispose();
}
}
catch (Exception)
{
}
private HashAlgorithm HashObject
{
get
{
if (isMD5Selected)
{
_hashObject = MD5.Create();
isMD5Selected = false;
}
else if (isMD4Selected)
{
_hashObject = MD4.Create();
isMD4Selected = false;
}
else if (isSHA1Selected)
{
_hashObject = SHA1.Create();
isSHA1Selected = false;
}
...
return _hashObject;
}
}
In the above code, foreach statement depends on number of selected hash algorithms. It calculates first selected hash correctly but on the second and other every next iteration it gives wrong values.
Whats wrong. Can anybody help me?
Thanks a lot in advance.
You are not resetting the stream so that you can re-read its contents for each iteration through the loop. Your buffer management logic can be simplified quite a bit, and it would be desireable to call hashObject.Dispose in a finally block so resources are released in case an exception is thrown.
streamSize = _dataStream.Length;
buffer = new byte[4096];
foreach (var hashObject in from Control ctrl in Controls where ctrl is CheckBox && ((CheckBox)ctrl).Checked select HashObject)
{
try
{
// reset stream position, progress
_dataStream.Position = 0;
_totalBytesRead = 0;
do
{
bytesRead = _dataStream.Read(buffer, 0, buffer.Length);
totalBytesRead += bytesRead;
if (_dataStream.Position == _dataStream.Length)
{
hashObject.TransformFinalBlock(buffer, 0, bytesRead);
}
else
{
hashObject.TransformBlock(buffer, 0, bytesRead, buffer, 0);
}
hashCalculationWorker.ReportProgress((int)((double)totalBytesRead * 100 / streamSize));
} while (_dataStream.Position < _dataStream.Length);
e.Result = hashObject.Hash;
finalHash = GenerateHex(hashObject.Hash);
Invoke(new MethodInvoker(() =>
{
// Get finalHash
}));
}
finally
{
hashObject.Dispose();
}
}
Better solution if the files are not large:
It may be more performant to read all of the data from the stream into a buffer once, and then re-use it:
if (!String.IsNullOrEmpty(selectedFile))
{
buffer = File.ReadAllBytes(selectedFile);
streamSize = buffer.Length;
selectedFile = string.Empty;
}
foreach (var hashObject in from Control ctrl in Controls where ctrl is CheckBox && ((CheckBox)ctrl).Checked select HashObject)
{
int offset = 0;
while (buffer.Length - offset >= streamSize)
{
offset += hashObject.TransformBlock(buffer, offset, streamSize, buffer, offset);
hashCalculationWorker.ReportProgress((int)((double)offset * 100 / streamSize));
}
hashObject.TransformFinalBlock(buffer, offset, buffer.Length - offset);
hashCalculationWorker.ReportProgress(100);
e.Result = hashObject.Hash;
finalHash = GenerateHex(hashObject.Hash);
Invoke(new MethodInvoker(() =>
{
// Get finalHash
}));
hashObject.Dispose();
}
i code socket application using blocing socket and use multithreading..my problem is i want to receive two large packet..as we know tcp must be split large data into multiple packet..this the scenario:
i send 6 mb text file and tcp split packet into 2 separate packet
at the same time i send 26 mb video file and tcp also split packet into 2 separate packet
at my application i recive first part of text file let say it 3 mb of 6 mb
and then i receive first part of video let say it 13 mb of 26 mb..
the question is how i know that first packet of text file and first packet of video file is a different data and should handle in different way..(different buffer maybe??)
sorry for my bad english..
thanks in advance..
this some part of my code
ns = client.GetStream();
while (isListen == true && client.Connected)
{
while (!ns.DataAvailable)
{
try
{
Thread.Sleep(1);
}
catch (Exception ex)
{
}
}
data = new byte[client.ReceiveBufferSize];
//client.Client.Receive(data);
int indx = ns.Read(data, 0, data.Length);
string message = Encoding.ASCII.GetString(data, 0, indx);
if (message == GetEnumDescription(TypeData.Disconnect))
{
isListen = false;
server.ClientKeluar = objClient;
if (ClientDisconnected != null)
{
ClientDisconnected(objClient);
}
thisThread.Abort();
Server.kumpulanThread.Remove(thisThread);
Server._serverConnections.Remove(this);
client.Close();
}
else if (message.Contains(GetEnumDescription(TypeData.GetFile)))
{
//jalankan proses pengambilan data
}
else if (message.Contains(GetEnumDescription(TypeData.ByteLength)))
{
string length = message.Substring(6, message.Length - 6);
int len = int.Parse(length);
expectedLength = client.ReceiveBufferSize = len;
data = new byte[len];
}
else if (message.Contains(GetEnumDescription(TypeData.Image)))
{
typeData = "Image";
dat1 = new byte[client.ReceiveBufferSize];
index = 0;
}
else if (message.Contains(GetEnumDescription(TypeData.Video)))
{
typeData = "Video";
dat2 = new byte[client.ReceiveBufferSize];
index = 0;
}
else
{
if (typeData == "Image")
{
expectedLength = expectedLength - message.Length;
if (expectedLength == 0)
{
Array.Copy(data, 0, dat1, index, message.Length);
if (ImageDelivered != null)
{
ImageDelivered(dat1);
}
}
else
{
Array.Copy(data, 0, dat1, index, message.Length);
index = message.Length;
}
}
else if (typeData == "Video")
{
expectedLength = expectedLength - message.Length;
if (expectedLength == 0)
{
Array.Copy(data, 0, dat2, index, message.Length);
if (VideoDelivered != null)
{
VideoDelivered(dat2);
}
}
else
{
Array.Copy(data, 0, dat2, index, message.Length);
index = message.Length;
}
}
else
{
expectedLength = expectedLength - message.Length;
if (expectedLength == 0)
{
dataToWrite = dataToWrite + message;
string text = dataToWrite;
if (MessageDelivered != null)
{
MessageDelivered(text);
}
dataToWrite = "";
}
else
{
dataToWrite += message;
}
}
}
}
may anyone give sample code so i can get inspiration to solve this problem?
TCP protocol take cares of making segments of files and later joining them. You will get complete data in receive.
I'm making a simple file transfer sender and receiver app through the wire. What I have so far is that the sender converts the file into a byte array and sends chunks of that array to the receiver.
This works with file of up to 256mb, but this line throws a "System out of memory" exception for anything above:
byte[] buffer = StreamFile(fileName); //This is where I convert the file
I'm looking for a way to read the file in chunks then write that chunk instead of loading the whole file into a byte. How can I do this with a FileStream?
EDIT:
Sorry, heres my crappy code so far:
private void btnSend(object sender, EventArgs e)
{
Socket clientSock = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
byte[] fileName = Encoding.UTF8.GetBytes(fName); //file name
byte[] fileData = null;
try
{
fileData = StreamFile(textBox1.Text); //file
}
catch (OutOfMemoryException ex)
{
MessageBox.Show("Out of memory");
return;
}
byte[] fileNameLen = BitConverter.GetBytes(fileName.Length); //length of file name
clientData = new byte[4 + fileName.Length + fileData.Length];
fileNameLen.CopyTo(clientData, 0);
fileName.CopyTo(clientData, 4);
fileData.CopyTo(clientData, 4 + fileName.Length);
clientSock.Connect("172.16.12.91", 9050);
clientSock.Send(clientData, 0, 4 + fileName.Length, SocketFlags.None);
for (int i = 4 + fileName.Length; i < clientData.Length; i++)
{
clientSock.Send(clientData, i, 1 , SocketFlags.None);
}
clientSock.Close();
}
And here's how I receive (the code was from a tutorial)
public void ReadCallback(IAsyncResult ar)
{
int fileNameLen = 1;
String content = String.Empty;
StateObject state = (StateObject)ar.AsyncState;
Socket handler = state.workSocket;
int bytesRead = handler.EndReceive(ar);
if (bytesRead > 0)
{
if (flag == 0)
{
Thread.Sleep(1000);
fileNameLen = BitConverter.ToInt32(state.buffer, 0);
string fileName = Encoding.UTF8.GetString(state.buffer, 4, fileNameLen);
receivedPath = fileName;
flag++;
}
if (flag >= 1)
{
BinaryWriter writer = new BinaryWriter(File.Open(receivedPath, FileMode.Append));
if (flag == 1)
{
writer.Write(state.buffer, 4 + fileNameLen, bytesRead - (4 + fileNameLen));
flag++;
}
else
writer.Write(state.buffer, 0, bytesRead);
writer.Close();
handler.BeginReceive(state.buffer, 0, StateObject.BufferSize, 0,
new AsyncCallback(ReadCallback), state);
}
}
else
{
Invoke(new MyDelegate(LabelWriter));
}
}
I just really want to know how I can read the file in chunks so that I dont need to convert it to a byte.
Thanks for the responses so far, I think I'm starting to get it :D
Just call Read repeatedly with a small buffer (I tend to use something like 16K). Note that the call to Read may end up reading a smaller amount than you request. If you're using a fixed chunk size and need the whole chunk in memory, you could just use an array of that size of course.
Without knowing how you're sending the file, it's hard to give much advice about how to structure your code, but it could be something like this:
byte[] chunk = new byte[MaxChunkSize];
while (true)
{
int index = 0;
// There are various different ways of structuring this bit of code.
// Fundamentally we're trying to keep reading in to our chunk until
// either we reach the end of the stream, or we've read everything we need.
while (index < chunk.Length)
{
int bytesRead = stream.Read(chunk, index, chunk.Length - index);
if (bytesRead == 0)
{
break;
}
index += bytesRead;
}
if (index != 0) // Our previous chunk may have been the last one
{
SendChunk(chunk, index); // index is the number of bytes in the chunk
}
if (index != chunk.Length) // We didn't read a full chunk: we're done
{
return;
}
}
If I was more awake I'd probably find a more readable way of writing this, but it'll do for now. One option is to extract another method from the middle section:
// Attempts to read an entire chunk into the given array; returns the size of
// chunk actually read.
int ReadChunk(Stream stream, byte[] chunk)
{
int index = 0;
while (index < chunk.Length)
{
int bytesRead = stream.Read(chunk, index, chunk.Length - index);
if (bytesRead == 0)
{
break;
}
index += bytesRead;
}
return index;
}
var b = new byte[1<<15]; // 32k
while((count = inStream.Read(b, 0, b.Length)) > 0)
{
outStream.Write(b, 0, count);
}
public static IEnumerable<byte[]> SplitStreamIntoChunks(Stream stream, int chunkSize)
{
var bytesRemaining = stream.Length;
while (bytesRemaining > 0)
{
var size = Math.Min((int) bytesRemaining, chunkSize);
var buffer = new byte[size];
var bytesRead = stream.Read(buffer, 0, size);
if (bytesRead <= 0)
break;
yield return buffer;
bytesRemaining -= bytesRead;
}
}