I'm trying to handle my incoming buffer and ensure that I got all the 125 bytes of the data at each transmission. I've created a byte array. How can I know 125 bytes of data is being received. I tried displaying the number of bytes but it displayed different number and I'm unsure if it's the right coding to get the number of bytes received.
Here's my code:
void datareceived(object sender, SerialDataReceivedEventArgs e)
{
myDelegate d = new myDelegate(update);
listBox1.Invoke(d, new object[] { });
}
public void update()
{
Console.WriteLine("Number of bytes:" + serialPort.BytesToRead); // it shows 155
while (serialPort.BytesToRead > 0)
bBuffer.Add((byte)serialPort.ReadByte());
ProcessBuffer(bBuffer);
}
private void ProcessBuffer(List<byte> bBuffer)
{
// Create a byte array buffer to hold the incoming data
byte[] buffer = bBuffer.ToArray();
// Show the user the incoming data // Display mode
for (int i = 0; i < buffer.Length; i++)
{
listBox1.Items.Add("SP: " + (bBuffer[43].ToString()) + " " + " HR: " + (bBuffer[103].ToString()) + " Time: ");
}
}
At the moment you are reading until the local receive buffer (BytesToRead) is empty, however, a better approach is to keep a buffer and offset, and loop until you have what you need, even if that means waiting - i.e.
byte[] buffer = new byte[125]
int offset = 0, toRead = 125;
...
int read;
while(toRead > 0 && (read = serialPort.Read(buffer, offset, toRead)) > 0) {
offset += read;
toRead -= read;
}
if(toRead > 0) throw new EndOfStreamException();
// you now have all the data you requested
Related
Could someone be kind enough to explain how I get my files the same size after copying it using a chunked stream? I presume it is because the last chunk still has a buffer size of 2048 so it is putting empty bytes at the end, but I am unsure how I would adjust the last read?
Original size: 15.1 MB (15,835,745 bytes)
New size: 15.1 MB (15,837,184 bytes)
static FileStream incomingFile;
static void Main(string[] args)
{
incomingFile = new FileStream(
#"D:\temp\" + Guid.NewGuid().ToString() + ".png",
FileMode.Create,
FileAccess.Write);
FileCopy();
}
private static void FileCopy()
{
using (Stream source = File.OpenRead(#"D:\temp\test.png"))
{
byte[] buffer = new byte[2048];
var chunkCount = source.Length;
for (int i = 0; i < (chunkCount / 2048) + 1; i++)
{
source.Position = i * 2048;
source.Read(buffer, 0, 2048);
WriteFile(buffer);
}
incomingFile.Close();
}
}
private static void WriteFile(byte[] buffer)
{
incomingFile.Write(buffer, 0, buffer.Length);
}
The last buffer read does not necessary contain exactly 2048 bytes (it can well be incomplete). Imagine, we have a file of 5000 bytes; in this case will read 3 chunks: 2 complete and 1 incomplete
2048
2048
904 the last incomplete buffer
Code:
using (Stream source = File.OpenRead(#"D:\temp\test.png"))
{
byte[] buffer = new byte[2048];
var chunkCount = source.Length;
for (int i = 0; i < (chunkCount / 2048) + 1; i++)
{
source.Position = i * 2048;
// size - number of actually bytes read
int size = source.Read(buffer, 0, 2048);
// if we bytes to write, do it
if (size > 0)
WriteFile(buffer, size);
}
incomingFile.Close();
}
...
private static void WriteFile(byte[] buffer, int size = -1)
{
incomingFile.Write(buffer, 0, size < 0 ? buffer.Length : size);
}
In your case you write 15837184 == 7733 * 2048 bytes (7733 complete chunks) when you should write 15835745 == 7732 * 2048 + 609 bytes - 7732 complete chunks and the last incomplete one of 609 bytes
I have spent a lot of team researching the proper ways to use the serial port in C# such that you don't have problems reading in data. I think I have a solution which is pretty close to functional, but I have some glitches every once in a while which I cannot seem to figure out.
My goal: Read formatted binary messages from the serial port, and pass them along to a processor.
The message format looks something like this:
(MSG-HEADER)(MSG-ID)(MSG-LENGTH)(DATA0)(DATA1)(DATA2)...(DATA-N)
Each "word" in the data is 16 bits (2-bytes). My basic approach is to start in a "read message header" state, where each time the serial data received event occurs, I read from the serial port, store the data in a buffer, and then check to see if I detect the message header. If I detect the message header, I move into a "read data" state, where I keep reading data into a data buffer until I have read bytes.
This seems to work pretty well, except occasionally I see "data glitches". Where I end up storing a message that looks something like this:
(MSG1-HEADER)(MSG1-ID)(MSG1-LENGTH)(DATA0)(DATA1)(DATA2)(MSG2-HEADER)(MSG2-ID)..etc
Basically, every so often I get a proper message header, message ID, message length, then the data starts (typically around 200 bytes), and right in the middle of that data I see another message header, message id, and message length, and presumably the start of another message data section. And I can't seem to figure out why.
Here is the code in the serial port data received event I am using:
public byte[] headerBuff = new byte[500];
public byte[] dataBuff = new byte[500];
public byte[] tempBuff = new byte[500];
public int bytesRead;
public int dataPos;
public int dataMsgLen;
public int dataBytesRead = 0;
public bool READ_HEADER = true;
ConcurrentQueue<byte[]> serialQ = new ConcurrentQueue<byte[]>();
//private void si_DataReceived(byte[] data)
private void si_DataReceived(object s, EventArgs e)
{
//If we're supposed to be reading the header, read some bytes and look
// For the header identification sequence (0xF989)
if (READ_HEADER)
{
//Read some bytes, save how many we read
bytesRead = comport.Read(headerBuff, 0, comport.BytesToRead);
//Any time we call comport.REad, we automatically log those bytes to a file
using (BinaryWriter writer = new BinaryWriter(File.Open(defDataDir, FileMode.Append)))
writer.Write(headerBuff.Skip(0).Take(bytesRead).ToArray());
//Loop through the bytes we just read and look for sequence
for (int i = 0; i < (bytesRead-1); i++)
{
if (headerBuff[i] == 0xF9 && headerBuff[i + 1] == 0x89)
{
//We have identified a header
// Lets copy it into a new array
dataPos = bytesRead-i;
Array.Copy(headerBuff, i, dataBuff, 0, dataPos);
dataMsgLen = dataBuff[4];
//Now we can switch to message logging
READ_HEADER = !READ_HEADER;
Array.Clear(headerBuff, 0, headerBuff.Length); //clear the buffer for next time
break; // don't need to look for headers anymore
}
}
}
//If we are done reading the header, let's wait until we get
// enough bytes to store the data message
else if (!READ_HEADER)
{
// Read some bytes into temp array
var tempNumBytes = comport.Read(tempBuff, 0, comport.BytesToRead);
//ADD this into data buffer
Array.Copy(tempBuff, 0, dataBuff, dataPos + dataBytesRead, tempNumBytes);
//Increment our counter
dataBytesRead += tempNumBytes;
//Save to stream
using (BinaryWriter writer = new BinaryWriter(File.Open(defDataDir, FileMode.Append)))
writer.Write(tempBuff.Skip(0).Take(tempNumBytes).ToArray());
//Add to FIFO if we have read enough bytes
if (dataBytesRead >= (dataMsgLen * 2))
{
//Debug.Print(BitConverter.ToString(dataBuff));
serialQ.Enqueue(dataBuff.Select(x => x).ToArray()); // Add to queue for processing
READ_HEADER = !READ_HEADER; // Go back to looking for headers
dataBytesRead = 0;
}
}
}
I appreciate any help, let me know if you need any clarifications.
Thank you in advance.
All,
Thank you for your comments. Based on what I read, I re-wrote the serial data handler (see code below) and it seems to be working much better. I have had it running for about ten minutes now and I haven't seen this glitch at all.
//Declare some public variables for serial port reading
public byte[] headerBuff = new byte[500];
public byte[] dataBuff = new byte[500];
public byte[] tempBuff = new byte[500];
public int headerBytesRead = 0;
public int dataBytesRead = 0;
public const int HEADER_LENGTH = 10;
public int dataInd;
public int fullMsgLen;
public byte[] queuePop;
//Declare some states
public bool READ_HEADER = true;
//Where will we store the data log?
public string defDataDir;
//Declare a public queue as a FIFO for incoming serial data once the
// buffer is full
ConcurrentQueue<byte[]> serialQ = new ConcurrentQueue<byte[]>();
//private void si_DataReceived(byte[] data)
private void si_DataReceived(object s, EventArgs e)
{
//If we're supposed to read the headers, do that
if(READ_HEADER)
{
//Read some bytes
var numBytesRead = comport.Read(tempBuff, 0, comport.BytesToRead);
//Any time we call comport.Read, we automatically log those bytes to a file
using (BinaryWriter writer = new BinaryWriter(File.Open(defDataDir, FileMode.Append)))
writer.Write(tempBuff.Skip(0).Take(numBytesRead).ToArray());
//Add these bytes to a header array
Array.Copy(tempBuff, 0, headerBuff, headerBytesRead, numBytesRead);
//Increment headerBytesRead counter
headerBytesRead += numBytesRead;
//Loop through header and see if we have a header
if(headerBytesRead>=HEADER_LENGTH)
{
//Loop through all the header bytes read so far
for(int i=0; i<headerBytesRead;i++)
{
//Look for the header start word. Note, 3rd bool statement
// here is to make sure we have enough bytes left to identify a header
// e.g. read 12 bytes, and bytes 11 and 12 are 0xF9 and 0x89, then we
// clearly don't have the rest of the header (since it is length 10)
if(headerBuff[i]==0xF9 && headerBuff[i+1]==0x89 && (headerBytesRead-i-1)>=9)
{
//We have identified a header, and have enough following characters to save it
//Copy the header into the data array
Array.Copy(headerBuff, i, dataBuff, 0, headerBytesRead - i);
dataInd = headerBytesRead - i;
//Save the message length
fullMsgLen = dataBuff[4]*2 + HEADER_LENGTH;
//Switch over to reading data
READ_HEADER = !READ_HEADER;
//Reset our header length counter
headerBytesRead = 0;
//Clear the header buffer for next time
Array.Clear(headerBuff, 0, headerBuff.Length);
break; // don't need to look for headers anymore
}
}
}
}
//Handle reading data into buffer here
else if (!READ_HEADER)
{
//We've just been told to start reading data bytes, and we know how many
var numBytesRead = comport.Read(tempBuff, 0, comport.BytesToRead);
//Any time we call comport.Read, we automatically log those bytes to a file
using (BinaryWriter writer = new BinaryWriter(File.Open(defDataDir, FileMode.Append)))
writer.Write(tempBuff.Skip(0).Take(numBytesRead).ToArray());
//Add these bytes into the data array
Array.Copy(tempBuff, 0, dataBuff, dataInd+dataBytesRead, numBytesRead);
//Increment our data array counter
dataBytesRead += numBytesRead;
//Check to see if we have saved enough
if((dataInd+dataBytesRead) >= fullMsgLen)
{
//Copy the header+msg into the queue
serialQ.Enqueue(dataBuff.Skip(0).Take(fullMsgLen).ToArray());
//Copy the remaining bytes back into the header buffer
Array.Copy(dataBuff, fullMsgLen, headerBuff, 0, dataInd + dataBytesRead - fullMsgLen);
headerBytesRead = dataInd + dataBytesRead - fullMsgLen;
//Reset data bytes read countery
dataBytesRead = 0;
//Switch back to looking for headers
READ_HEADER = !READ_HEADER;
}
}
}
Im building an application where i need to reed 15 byes from a serial device. (ScaleXtric c7042 powerbase) The bytes need to come in the right order, and the last one is a crc.
Using this code in an backgroundworker, I get the bytes:
byte[] data = new byte[_APB.ReadBufferSize];
_APB.Read(data, 0, data.Length);
The problem is that I don't get the first bytes first, Its like it stores some of the bytes in the buffer, so next time the DataRecieved event fires, I get the last x bytes from the previous message, and only the 15-x byte from the new. I write the bytes to a text box, and its all over the place, so some bytes are missing somewhere.
I have tried to clear the buffer after each read, but no luck.
_APB = new SerialPort(comboBoxCommAPB.SelectedItem.ToString());
_APB.BaudRate = 19200;
_APB.DataReceived += new SerialDataReceivedEventHandler(DataReceivedHandlerDataFromAPB);
_APB.Open();
_APB.DiscardInBuffer();
Hope any one can help me here
Use this Method to read fixed amout of bytes from serial port, for your case toread = 15;
public byte[] ReadFromSerialPort(SerialPort serialPort, int toRead)
{
byte[] buffer = new byte[toRead];
int offset = 0;
int read;
while (toRead > 0 && (read = serialPort.Read(buffer, offset, toRead)) > 0)
{
offset += read;
toRead -= read;
}
if (toRead > 0) throw new EndOfStreamException();
return buffer;
}
I have a C# console app that I'm trying to create that processes all the files in a given directory and writes output to another given directory. I want to process the input files X bytes at a time.
namespace FileConverter
{
class Program
{
static void Main(string[] args)
{
string srcFolder = args[0];
string destFolder = args[1];
string[] srcFiles = Directory.GetFiles(srcFolder);
for (int s = 0; s < srcFiles.Length; s++)
{
byte[] fileBuffer;
int numBytesRead = 0;
int readBuffer = 10000;
FileStream srcStream = new FileStream(srcFiles[s], FileMode.Open, FileAccess.Read);
int fileLength = (int)srcStream.Length;
string destFile = destFolder + "\\" + Path.GetFileName(srcFiles[s]) + "-processed";
FileStream destStream = new FileStream(destFile, FileMode.OpenOrCreate, FileAccess.Write);
//Read and process the source file by some chunk of bytes at a time
while (numBytesRead < fileLength)
{
fileBuffer = new byte[readBuffer];
//Read some bytes into the fileBuffer
//TODO: This doesn't work on subsequent blocks
int n = srcStream.Read(fileBuffer, numBytesRead, readBuffer);
//If we didn't read anything, there's no more to process
if (n == 0)
break;
//Process the fileBuffer
for (int i = 0; i < fileBuffer.Length; i++)
{
//Process each byte in the array here
}
//Write data
destStream.Write(fileBuffer, numBytesRead, readBuffer);
numBytesRead += readBuffer;
}
srcStream.Close();
destStream.Close();
}
}
}
}
I'm running into an error at execution time at:
//Read some bytes into the fileBuffer
//TODO: This doesn't work on subsequent blocks
int n = srcStream.Read(fileBuffer, numBytesRead, readBuffer);
I don't want to load the entire file into memory, as it could possibly be many gigabytes in size. I really want to be able to read some number of bytes, process them, write them out to a file, and then read in the next X bytes and repeat.
It gets through one iteration of the loop, and then dies on the second. The error I get is:
"Offset and length were out of bounds for the array or count is greater than the number of elements from index to the end of the source collection."
The sample file I'm working with is about 32k.
Can anyone tell me what I'm doing wrong here?
The second parameter to Read is not the offset into the file - it is the offset into the buffer at which to start writing data. So just pass 0.
Also, don't assume the buffer is filled each time: you should only process "n" bytes from the buffer. And the buffer should be reused between iterations.
If you need to read exactly a number of bytes:
static void ReadOrThrow(Stream source, byte[] buffer, int count) {
int read, offset = 0;
while(count > 0 && (read = source.Read(buffer, offset, count)) > 0) {
offset += read;
count -= read;
}
if(count != 0) throw new EndOfStreamException();
}
Note that Write works similarly, so you need to pass 0 as the offset and n as the count.
It should be
destStream.Write(fileBuffer, numBytesRead, n);
numBytesRead += n;
because n is the actual number of bytes that was read
I'm making a simple file transfer sender and receiver app through the wire. What I have so far is that the sender converts the file into a byte array and sends chunks of that array to the receiver.
This works with file of up to 256mb, but this line throws a "System out of memory" exception for anything above:
byte[] buffer = StreamFile(fileName); //This is where I convert the file
I'm looking for a way to read the file in chunks then write that chunk instead of loading the whole file into a byte. How can I do this with a FileStream?
EDIT:
Sorry, heres my crappy code so far:
private void btnSend(object sender, EventArgs e)
{
Socket clientSock = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
byte[] fileName = Encoding.UTF8.GetBytes(fName); //file name
byte[] fileData = null;
try
{
fileData = StreamFile(textBox1.Text); //file
}
catch (OutOfMemoryException ex)
{
MessageBox.Show("Out of memory");
return;
}
byte[] fileNameLen = BitConverter.GetBytes(fileName.Length); //length of file name
clientData = new byte[4 + fileName.Length + fileData.Length];
fileNameLen.CopyTo(clientData, 0);
fileName.CopyTo(clientData, 4);
fileData.CopyTo(clientData, 4 + fileName.Length);
clientSock.Connect("172.16.12.91", 9050);
clientSock.Send(clientData, 0, 4 + fileName.Length, SocketFlags.None);
for (int i = 4 + fileName.Length; i < clientData.Length; i++)
{
clientSock.Send(clientData, i, 1 , SocketFlags.None);
}
clientSock.Close();
}
And here's how I receive (the code was from a tutorial)
public void ReadCallback(IAsyncResult ar)
{
int fileNameLen = 1;
String content = String.Empty;
StateObject state = (StateObject)ar.AsyncState;
Socket handler = state.workSocket;
int bytesRead = handler.EndReceive(ar);
if (bytesRead > 0)
{
if (flag == 0)
{
Thread.Sleep(1000);
fileNameLen = BitConverter.ToInt32(state.buffer, 0);
string fileName = Encoding.UTF8.GetString(state.buffer, 4, fileNameLen);
receivedPath = fileName;
flag++;
}
if (flag >= 1)
{
BinaryWriter writer = new BinaryWriter(File.Open(receivedPath, FileMode.Append));
if (flag == 1)
{
writer.Write(state.buffer, 4 + fileNameLen, bytesRead - (4 + fileNameLen));
flag++;
}
else
writer.Write(state.buffer, 0, bytesRead);
writer.Close();
handler.BeginReceive(state.buffer, 0, StateObject.BufferSize, 0,
new AsyncCallback(ReadCallback), state);
}
}
else
{
Invoke(new MyDelegate(LabelWriter));
}
}
I just really want to know how I can read the file in chunks so that I dont need to convert it to a byte.
Thanks for the responses so far, I think I'm starting to get it :D
Just call Read repeatedly with a small buffer (I tend to use something like 16K). Note that the call to Read may end up reading a smaller amount than you request. If you're using a fixed chunk size and need the whole chunk in memory, you could just use an array of that size of course.
Without knowing how you're sending the file, it's hard to give much advice about how to structure your code, but it could be something like this:
byte[] chunk = new byte[MaxChunkSize];
while (true)
{
int index = 0;
// There are various different ways of structuring this bit of code.
// Fundamentally we're trying to keep reading in to our chunk until
// either we reach the end of the stream, or we've read everything we need.
while (index < chunk.Length)
{
int bytesRead = stream.Read(chunk, index, chunk.Length - index);
if (bytesRead == 0)
{
break;
}
index += bytesRead;
}
if (index != 0) // Our previous chunk may have been the last one
{
SendChunk(chunk, index); // index is the number of bytes in the chunk
}
if (index != chunk.Length) // We didn't read a full chunk: we're done
{
return;
}
}
If I was more awake I'd probably find a more readable way of writing this, but it'll do for now. One option is to extract another method from the middle section:
// Attempts to read an entire chunk into the given array; returns the size of
// chunk actually read.
int ReadChunk(Stream stream, byte[] chunk)
{
int index = 0;
while (index < chunk.Length)
{
int bytesRead = stream.Read(chunk, index, chunk.Length - index);
if (bytesRead == 0)
{
break;
}
index += bytesRead;
}
return index;
}
var b = new byte[1<<15]; // 32k
while((count = inStream.Read(b, 0, b.Length)) > 0)
{
outStream.Write(b, 0, count);
}
public static IEnumerable<byte[]> SplitStreamIntoChunks(Stream stream, int chunkSize)
{
var bytesRemaining = stream.Length;
while (bytesRemaining > 0)
{
var size = Math.Min((int) bytesRemaining, chunkSize);
var buffer = new byte[size];
var bytesRead = stream.Read(buffer, 0, size);
if (bytesRead <= 0)
break;
yield return buffer;
bytesRemaining -= bytesRead;
}
}