I have a requirement to programmatically read the buffer of tokbox audio bus. As per their documentation [here][1]
they are asking you to continually call ReadRenderData method with the amount of buffer you need to retrieve. Here is my code
private int SAMPLING_RATE = 48000;
private int NUM_CHANNELS_RENDERING = 1;
IntPtr pointer = Marshal.AllocHGlobal(SAMPLING_RATE);
int returnBytes = this.audioBus.ReadRenderData(pointer, SAMPLING_RATE / 2);
byte[] cbuff = new byte[returnBytes];
Console.WriteLine(returnBytes.ToString());
Marshal.Copy(pointer, cbuff, 0, returnBytes);
do
{
if (cbuff[n] == 0) count++;
n++;
} while (n < returnBytes);
if (count >= (returnBytes - 1))
{
Console.WriteLine("Here");
}
The do-while loop above is only a try to remove the blank data which does not do the job.
The problem I am getting is, the audio buffer has a really high excess of blank buffer because ReadRenderData never returns zero buffer however it sends blank buffers.
Is there anything I am doing wrong above?
Related
Sorry for such a vague title, I really dont know what to title this issue.
Basically when I get a stream thats chunked as told by Transfer-Encoding, I then do the following code:
private IEnumerable<byte[]> ReceiveMessageBodyChunked() {
readChunk:
#region Read a line from the Stream which should be a Block Length (Chunk Body Length)
string blockLength = _receiverHelper.ReadLine();
#endregion
#region If the end of the block is reached, re-read from the stream
if (blockLength == Http.NewLine) {
goto readChunk;
}
#endregion
#region Trim it so it should end up with JUST the number
blockLength = blockLength.Trim(' ', '\r', '\n');
#endregion
#region If the end of the message body is reached
if (blockLength == string.Empty) {
yield break;
}
#endregion
int blockLengthInt = 0;
#region Convert the Block Length String to an Int32 base16 (hex)
try {
blockLengthInt = Convert.ToInt32(blockLength, 16);
} catch (Exception ex) {
if (ex is FormatException || ex is OverflowException) {
throw new Exception(string.Format(ExceptionValues.HttpException_WrongChunkedBlockLength, blockLength), ex);
}
throw;
}
#endregion
// If the end of the message body is reached.
if (blockLengthInt == 0) {
yield break;
}
byte[] buffer = new byte[blockLengthInt];
int totalBytesRead = 0;
while (totalBytesRead != blockLengthInt) {
int length = blockLengthInt - totalBytesRead;
int bytesRead = _receiverHelper.HasData ? _receiverHelper.Read(buffer, 0, length) : _request.ClientStream.Read(buffer, 0, length);
if (bytesRead == 0) {
WaitData();
continue;
}
totalBytesRead += bytesRead;
System.Windows.Forms.MessageBox.Show("Chunk Length: " + blockLengthInt + "\nBytes Read/Total:" + bytesRead + "/" + totalBytesRead + "\n\n" + Encoding.ASCII.GetString(buffer));
yield return buffer;
}
goto readChunk;
}
What this is doing is reading 1 line of data from the stream which should be the Chunk's Length, does some checks here and there but eventually converts that to a Int32 Radix16 integer.
From there it essentially creates a byte buffer of that int32 as its length size.
It then just keeps reading from the stream until its read the same amount as the Int32 we converted.
This works splendid, however, for whatever reason, its responding incorrectly on the last read.
It will read the exact amount of bytes as the chunk length perfectly fine, and all data I expect is read. BUT it's ALSO reading again another small chunk of data that was ALREADY read at the very end, resulting in lets say all data from <!DOCTYPE html> down to </html> ASWELL as some data from inside somewhere like <form> e.t.c
Here's an example of what occured:
As you can see, the highlighted red text should NOT have been returned from the read! It should have ended at </html>.
Why is the Chunk's Length lying to me and how can I find the proper size to read at?
I'm not familiar with C# but if I understand your code and the semantics of Read in C# correctly (which seem to be similar to read in C) then the problem is that you are using the same buffer again and again without resetting it first:
byte[] buffer = new byte[blockLengthInt];
int totalBytesRead = 0;
while (totalBytesRead != blockLengthInt) {
int length = blockLengthInt - totalBytesRead;
int bytesRead = _receiverHelper.HasData ? _receiverHelper.Read(buffer, 0, length) : _request.ClientStream.Read(buffer, 0, length);
...
totalBytesRead += bytesRead;
...
yield return buffer;
}
To make some example of what goes wrong here: assume that the chunk size is 10, the content you read is 0123456789 and the first read will return 6 bytes and the second read the remaining 4 bytes. In this case your buffer will be 012345 after the first read and 567845 after the second read. These 45 at the end of the buffer remain from the previous read since you only replaced the first 4 bytes in the buffer but kept the rest.
Whats odd AF is that if I hand the request to another TCPStream proxied (127.0.0.1:8888 as a proxy which is fiddler) it works perfectly fine...
Fiddler is a proxy and might change how the response gets transferred. For example it might use Content-length instead of chunked encoding or it might use smaller chunks so that you always get the full chunk with the first read.
I have spent a lot of team researching the proper ways to use the serial port in C# such that you don't have problems reading in data. I think I have a solution which is pretty close to functional, but I have some glitches every once in a while which I cannot seem to figure out.
My goal: Read formatted binary messages from the serial port, and pass them along to a processor.
The message format looks something like this:
(MSG-HEADER)(MSG-ID)(MSG-LENGTH)(DATA0)(DATA1)(DATA2)...(DATA-N)
Each "word" in the data is 16 bits (2-bytes). My basic approach is to start in a "read message header" state, where each time the serial data received event occurs, I read from the serial port, store the data in a buffer, and then check to see if I detect the message header. If I detect the message header, I move into a "read data" state, where I keep reading data into a data buffer until I have read bytes.
This seems to work pretty well, except occasionally I see "data glitches". Where I end up storing a message that looks something like this:
(MSG1-HEADER)(MSG1-ID)(MSG1-LENGTH)(DATA0)(DATA1)(DATA2)(MSG2-HEADER)(MSG2-ID)..etc
Basically, every so often I get a proper message header, message ID, message length, then the data starts (typically around 200 bytes), and right in the middle of that data I see another message header, message id, and message length, and presumably the start of another message data section. And I can't seem to figure out why.
Here is the code in the serial port data received event I am using:
public byte[] headerBuff = new byte[500];
public byte[] dataBuff = new byte[500];
public byte[] tempBuff = new byte[500];
public int bytesRead;
public int dataPos;
public int dataMsgLen;
public int dataBytesRead = 0;
public bool READ_HEADER = true;
ConcurrentQueue<byte[]> serialQ = new ConcurrentQueue<byte[]>();
//private void si_DataReceived(byte[] data)
private void si_DataReceived(object s, EventArgs e)
{
//If we're supposed to be reading the header, read some bytes and look
// For the header identification sequence (0xF989)
if (READ_HEADER)
{
//Read some bytes, save how many we read
bytesRead = comport.Read(headerBuff, 0, comport.BytesToRead);
//Any time we call comport.REad, we automatically log those bytes to a file
using (BinaryWriter writer = new BinaryWriter(File.Open(defDataDir, FileMode.Append)))
writer.Write(headerBuff.Skip(0).Take(bytesRead).ToArray());
//Loop through the bytes we just read and look for sequence
for (int i = 0; i < (bytesRead-1); i++)
{
if (headerBuff[i] == 0xF9 && headerBuff[i + 1] == 0x89)
{
//We have identified a header
// Lets copy it into a new array
dataPos = bytesRead-i;
Array.Copy(headerBuff, i, dataBuff, 0, dataPos);
dataMsgLen = dataBuff[4];
//Now we can switch to message logging
READ_HEADER = !READ_HEADER;
Array.Clear(headerBuff, 0, headerBuff.Length); //clear the buffer for next time
break; // don't need to look for headers anymore
}
}
}
//If we are done reading the header, let's wait until we get
// enough bytes to store the data message
else if (!READ_HEADER)
{
// Read some bytes into temp array
var tempNumBytes = comport.Read(tempBuff, 0, comport.BytesToRead);
//ADD this into data buffer
Array.Copy(tempBuff, 0, dataBuff, dataPos + dataBytesRead, tempNumBytes);
//Increment our counter
dataBytesRead += tempNumBytes;
//Save to stream
using (BinaryWriter writer = new BinaryWriter(File.Open(defDataDir, FileMode.Append)))
writer.Write(tempBuff.Skip(0).Take(tempNumBytes).ToArray());
//Add to FIFO if we have read enough bytes
if (dataBytesRead >= (dataMsgLen * 2))
{
//Debug.Print(BitConverter.ToString(dataBuff));
serialQ.Enqueue(dataBuff.Select(x => x).ToArray()); // Add to queue for processing
READ_HEADER = !READ_HEADER; // Go back to looking for headers
dataBytesRead = 0;
}
}
}
I appreciate any help, let me know if you need any clarifications.
Thank you in advance.
All,
Thank you for your comments. Based on what I read, I re-wrote the serial data handler (see code below) and it seems to be working much better. I have had it running for about ten minutes now and I haven't seen this glitch at all.
//Declare some public variables for serial port reading
public byte[] headerBuff = new byte[500];
public byte[] dataBuff = new byte[500];
public byte[] tempBuff = new byte[500];
public int headerBytesRead = 0;
public int dataBytesRead = 0;
public const int HEADER_LENGTH = 10;
public int dataInd;
public int fullMsgLen;
public byte[] queuePop;
//Declare some states
public bool READ_HEADER = true;
//Where will we store the data log?
public string defDataDir;
//Declare a public queue as a FIFO for incoming serial data once the
// buffer is full
ConcurrentQueue<byte[]> serialQ = new ConcurrentQueue<byte[]>();
//private void si_DataReceived(byte[] data)
private void si_DataReceived(object s, EventArgs e)
{
//If we're supposed to read the headers, do that
if(READ_HEADER)
{
//Read some bytes
var numBytesRead = comport.Read(tempBuff, 0, comport.BytesToRead);
//Any time we call comport.Read, we automatically log those bytes to a file
using (BinaryWriter writer = new BinaryWriter(File.Open(defDataDir, FileMode.Append)))
writer.Write(tempBuff.Skip(0).Take(numBytesRead).ToArray());
//Add these bytes to a header array
Array.Copy(tempBuff, 0, headerBuff, headerBytesRead, numBytesRead);
//Increment headerBytesRead counter
headerBytesRead += numBytesRead;
//Loop through header and see if we have a header
if(headerBytesRead>=HEADER_LENGTH)
{
//Loop through all the header bytes read so far
for(int i=0; i<headerBytesRead;i++)
{
//Look for the header start word. Note, 3rd bool statement
// here is to make sure we have enough bytes left to identify a header
// e.g. read 12 bytes, and bytes 11 and 12 are 0xF9 and 0x89, then we
// clearly don't have the rest of the header (since it is length 10)
if(headerBuff[i]==0xF9 && headerBuff[i+1]==0x89 && (headerBytesRead-i-1)>=9)
{
//We have identified a header, and have enough following characters to save it
//Copy the header into the data array
Array.Copy(headerBuff, i, dataBuff, 0, headerBytesRead - i);
dataInd = headerBytesRead - i;
//Save the message length
fullMsgLen = dataBuff[4]*2 + HEADER_LENGTH;
//Switch over to reading data
READ_HEADER = !READ_HEADER;
//Reset our header length counter
headerBytesRead = 0;
//Clear the header buffer for next time
Array.Clear(headerBuff, 0, headerBuff.Length);
break; // don't need to look for headers anymore
}
}
}
}
//Handle reading data into buffer here
else if (!READ_HEADER)
{
//We've just been told to start reading data bytes, and we know how many
var numBytesRead = comport.Read(tempBuff, 0, comport.BytesToRead);
//Any time we call comport.Read, we automatically log those bytes to a file
using (BinaryWriter writer = new BinaryWriter(File.Open(defDataDir, FileMode.Append)))
writer.Write(tempBuff.Skip(0).Take(numBytesRead).ToArray());
//Add these bytes into the data array
Array.Copy(tempBuff, 0, dataBuff, dataInd+dataBytesRead, numBytesRead);
//Increment our data array counter
dataBytesRead += numBytesRead;
//Check to see if we have saved enough
if((dataInd+dataBytesRead) >= fullMsgLen)
{
//Copy the header+msg into the queue
serialQ.Enqueue(dataBuff.Skip(0).Take(fullMsgLen).ToArray());
//Copy the remaining bytes back into the header buffer
Array.Copy(dataBuff, fullMsgLen, headerBuff, 0, dataInd + dataBytesRead - fullMsgLen);
headerBytesRead = dataInd + dataBytesRead - fullMsgLen;
//Reset data bytes read countery
dataBytesRead = 0;
//Switch back to looking for headers
READ_HEADER = !READ_HEADER;
}
}
}
Im building an application where i need to reed 15 byes from a serial device. (ScaleXtric c7042 powerbase) The bytes need to come in the right order, and the last one is a crc.
Using this code in an backgroundworker, I get the bytes:
byte[] data = new byte[_APB.ReadBufferSize];
_APB.Read(data, 0, data.Length);
The problem is that I don't get the first bytes first, Its like it stores some of the bytes in the buffer, so next time the DataRecieved event fires, I get the last x bytes from the previous message, and only the 15-x byte from the new. I write the bytes to a text box, and its all over the place, so some bytes are missing somewhere.
I have tried to clear the buffer after each read, but no luck.
_APB = new SerialPort(comboBoxCommAPB.SelectedItem.ToString());
_APB.BaudRate = 19200;
_APB.DataReceived += new SerialDataReceivedEventHandler(DataReceivedHandlerDataFromAPB);
_APB.Open();
_APB.DiscardInBuffer();
Hope any one can help me here
Use this Method to read fixed amout of bytes from serial port, for your case toread = 15;
public byte[] ReadFromSerialPort(SerialPort serialPort, int toRead)
{
byte[] buffer = new byte[toRead];
int offset = 0;
int read;
while (toRead > 0 && (read = serialPort.Read(buffer, offset, toRead)) > 0)
{
offset += read;
toRead -= read;
}
if (toRead > 0) throw new EndOfStreamException();
return buffer;
}
I have been trying for a couple of days now to load a file in chunks to allow the user to use very large (GB) files and still keep the speed of the program. Currently i have the following code:
using (FileStream filereader = new FileStream(filename, FileMode.Open, FileAccess.Read))
{
using (StreamReader reader = new StreamReader(filereader))
{
while (toRead > 0 && (bytesread = reader.Read(buffer, offset, toRead)) > 0)
{
toRead -= bytesread;
offset += bytesread;
}
if (toRead > 0) throw new EndOfStreamException();
foreach (var item in buffer)
{
temporary = temporary += item.ToString();
}
temporary.Replace("\n", "\n" + System.Environment.NewLine);
Below are the declarations to avoid any confusion (hopefully):
const int Max_Buffer = 5000;
char[] buffer = new char[Max_Buffer];
int bytesread;
int toRead = 5000;
int offset = 0;
At the moment the program reads in 5000 bytes of the text file, then processes the bytes into a string which i then pass into a stringreader so i can take the information i want.
My problem at the moment is that the buffer can stop halfway through a line so when I take my data in the stringreader class it brings up index/length errors.
What i need is to know how to either seek back in the array to find a certain set of characters that signify the start of a line and then only return the data before that point for processing to a string.
Another issue after sorting the seeking back problem is how would i keep the data i didnt want to process and bring in more data to fill the buffer.
I hope this is explained well, i know i can sometimes be confusing hope someone can help.
I would suggest the use of reader.ReadLine() instead of reader.Read() in your loop
buffer=reader.ReadLine();
bytesread = buffer.Length*2;//Each charcter is unicode and equal to 2 bytes
You can then check on the whether (toRead - bytesread)<0.
I have a TCP socket application where I have to read several types of replies being the max size of a buffer 8192 some replies are splitted in more packets.
Currently I receive a list of members at reply 44, so the first idea I had to be able to deal with splitted packets was to define a stream out side of it to stored the incoming data until it is complete with a bool and current size variable.
Once it hits the reply 44 it will check if extraList is true or false, if false it means it is an initial request to incoming members list.
If the 4 initial bytes of the packet is bigger then bytes.Legth which is 8192 it will trigger the extraList to true and fill the initial data to the buffer I had previously set with the with the total packet size as it is size.
Since extraList has been trigged and turned into true, the packet reading will fall into it until the data is complete, which will then set it back to false and trigger the MemberList function with the complete list.
Would like some advices, suggestions, etc to improve this code.
int storedCurSize = 0;
MemoryStream stored = null;
bool extraList = false;
while (roomSocket.Connected)
{
byte[] bytes = new byte[roomSocket.ReceiveBufferSize];
roomSocket.Receive(bytes);
MemoryStream bufferReceived = new MemoryStream(bytes, 0, bytes.Length);
using (var reader = new BinaryReader(bufferReceived))
{
int packetSize = (int)reader.ReadInt32() + 9;
int reply = (int)reader.ReadByte();
if (reply == 44 || extraList)
{
if (!extraList && packetSize <= bytes.Length)
{
MemberList(bytes);
}
else
{
if (!extraList)
{
stored = new MemoryStream(new byte[packetSize], 0, packetSize);
stored.Write(bytes, 0, bytes.Length);
storedCurSize = bytes.Length;
extraList = true;
}
else
{
if (storedCurSize < stored.Length)
{
int storedLeftSize = (int)stored.Length - storedCurSize;
stored.Write(bytes, 0, (storedLeftSzie < bytes.Length) ? storedLeftSize : bytes.Length);
storedCurSize += (storedLeftSize < bytes.Length) ? storedLeftSize : bytes.Length;
if (storedCurSize >= stored.Length)
{
extraList = false;
MemberList(stored.ToArray());
stored.Close();
}
}
}
}
}
}
}
While reading code briefly what is flaring is magic numbers (9, 44) and very deep nesting.
Replace numbers with good-named constants and move out some parts of code as methods.
In case they are tightly twisted by used local variables - probably all this method worth moving out to worker class with single responsibility - to read the message. Thus local variables becomes class fields and methods wouldn't be so inflexible to refactoring.
Also MemberList(...) is poor name for a method as for me. Make it a verb that will describe what method is doing.
To merge bytes that aren't together, you can use Buffer.BlockCopy().
byte[] buf1;
byte[] buf2;
byte[] concatenated = new byte[buf1.Length + buf2.Length];
Buffer.BlockCopy(buf1, 0, concatenated, 0, buf1.Length);
Buffer.BlockCopy(buf2, 0, concatenated, buf1.ength, buf2.Length);