My code
private static int readinput(byte[] buff, BinaryReader reader)
{
int size = reader.ReadInt32();
reader.Read(buff, 0, size);
return size;
}
Exception in reader.Read(buff, 0, size);
The exception is offset and length were out of bounds for the array or count is greater than the number of elements from index to the end of source collection
Take a step back and think about your code
You've written a method that takes an array of bytes. We don't know how big this array is, but it's controlled by the code calling the method. Let's assume it is 1000 bytes long
Then you read an int from somewhere else, let's assume 2000 is read
Then you attempt to read 2000 bytes into an array that can only hold 1000 bytes, you perform no checking to make sure your array is big enough, nor do you attempt to read in chunks and concatenate if it's not big enough
That's why you get the error you're getting, but as to what you should be coding, I think you need to think about that some more- maybe make the size to the buffer in response to the size int you read, or read in chunks..
The buffer buff that you passed into your function to read the data is too small. buff.Length should be bigger than or equal to your variable called size.
Set a breakpoint on "reader.Read(buff, 0, size);" and hover over buff and size and you'll see what I mean.
Make sure when you call your function, the buff you pass in is sufficient size. If you don't know what size to create a buffer for ahead of time, then change your function to look something like this:
private static byte[] ReadInput(BinaryReader reader)
{
int size = reader.ReadInt32();
return reader.ReadBytes(size);
}
Especially since you're just reading it into the beginning of a provided buffer anyways.
Summary to frame what you're currently doing:
You provided us a function which takes a binary reader (whatever position it's already at, if it's new, then position 0), it reads a 32-bit integer (4 bytes) to figure out the size of some data after it. Then you read that data of that size into a buffer. You do all of this with a buffer provided called buff. You need to be sure that whatever size data you're going to read in, the buffer provided to the function is of sufficient size. If you make the size of the buffer too large, then "reader.Read(buff, 0, size)" is only reading it into the beginning of the buffer. So if your intention was just to read the data the way you coded that function into a perfectly sized buffer, I suggest using the code above.
Just thought I'd explain it a bit more in case that helps you understand what's going on.
Related
i'm trying to learn to use sockets in c# and i've a doubt, i'm using a code something like this:
byte[] data = new byte[64];
int length = 0;
length = sock.Receive(data);
//more code...
So, the byte[] data is filled with the recived data and the left space in the array is filled with 0s, Is the byte[] allocated into memory completely (all 64bytes)? If it's so, is there a way to make the byte[] the same size as the actual sent data?
You can check sock.Available to see what has already come in. (so far)
byte[] data = new byte[sock.Available];
int length = sock.Receive(data);
//more code...
Note: Since you may or may not know what is coming in next on the network it usually makes more sense to read only a header (with size info) first or to allocate more space than necessary, and call .Recieve() multiple times until the end of a record is reached.
Note: This code assumes you already know there is some data to receive and you've waited long enough for some useful amount of data to be ready.
If you do choose to use length headers, .Available can help you avoid reading a partial header and having to re-assemble it which is nice. (Only large messages may need manual reassembly in that case)
You simply need to use the return value from Receive to understand how much data has arrived. You can shorten the buffer using Array.Resize if you want but that normally would be a sign that something is wrong.
Also note, that TCP is a stream of bytes and does not preserve message boundaries.
As noted normally read may return fewer bytes then it was told. See a workaround function below which ensures it reads as many bytes as it was told - basically size of the passed buffer. Function is from here.
/// Reads data into a complete array, throwing an EndOfStreamException
/// if the stream runs out of data first, or if an IOException
/// naturally occurs.
/// </summary>
/// <param name="stream">The stream to read data from</param>
/// <param name="data">The array to read bytes into. The array
/// will be completely filled from the stream, so an appropriate
/// size must be given.</param>
public static void ReadWholeArray (Stream stream, byte[] data)
{
int offset=0;
int remaining = data.Length;
while (remaining > 0)
{
int read = stream.Read(data, offset, remaining);
if (read <= 0)
throw new EndOfStreamException
(String.Format("End of stream reached with {0} bytes left to read", remaining));
remaining -= read;
offset += read;
}
}
You can use this method first to read say a 2 byte integer which should represent the number of bytes that will follow. Then you read once again however now read as many bytes as specified in that two byte integer.
But for this to work, clearly the sender first has to send a two byte integer which represents length of data that will follow - and then the data itself.
So basically you call above function on a byte array of size two first (to get data length), and then on a byte array with size as indicated in that 2 byte integer (to get data).
You can use this to read from NetworkStream. Some more reading on this topic.
I am having trouble keeping the size of an array fixed. I have to read data from a file. when the number of bytes read is less then 160 the array MyPCMBuf changes to the size of bytes read.
byte[] MyPCMBuf;
MyPCMBuf = new byte[160];
BinaryReader reader = new BinaryReader(File.Open(fileNamein, FileMode.Open)) ;
Array.Clear (MyPCMBuf,0,160);
MyPCMBuf = reader.ReadBytes(160); //if bytes read =20 , size of MyPCMBuf becomes 20
What is going on and how to avoid it?
That's because ReadBytes returns a new byte array.
If you want to read bytes into an existing array, call Read.
That is:
int bytesRead = reader.Read(myBuffer, 0, 160);
You have no problem keeing the size fixed.
ReadBytes returns a new array. That simple. The old one never changes size.
If you want to use your buffer, user another method, for example:
public virtual int Read(
byte[] buffer,
int index,
int count
)
on that class. Then you keep your array.
Don't use BinaryReader at all. Use the FileStream returned by Open and call its Read method. If you don't assign to MyPCMBuf its length cannot possibly change.
Unrelated: Use Array.Clear (MyPCMBuf,0,MyPCMBuf.Length);. Less redundancy. Less potential for errors. Use using. Don't initialize MyPCMBuf if you always overwrite it. Don't clear it redundantly. I see a lot of misunderstandings here. Be more rigorous in your approach to programming. It appears you don't really understand all the language features and APIs you are using. That's dangerous.
You are overwriting MyPCMBuf in line
MyPCMBuf = reader.ReadBytes(160);
Thus, line MyPCMBuf = new byte[160]; is irrelevant in your code.
You are not really doing what you think you are.
reader.ReadBytes(160) will create a new byte array of size at most 160. It's not storing values in the array you already had. That one is send to Garbage Collection.
I want to read very large file (4GBish) chunk by chunk.
I am currently trying to use a StreamReader and the Read() read method. The syntax is:
sr.Read(char[] buffer, int index, int count)
Because the index is an int it will overflow in my case. What should i use instead?
The index is the starting index of buffer not the index of file pointer, usually it would be zero. On each Read call you will read characters equal to the count parameter of Read method. You would not read all the file at once rather read in chunks and use that chunk.
The index of buffer at which to begin writing, reference.
char[] c = null;
while (sr.Peek() >= 0)
{
c = new char[1024];
sr.Read(c, 0, c.Length);
//The output will look odd, because
//only five characters are read at a time.
Console.WriteLine(c);
}
The above example will ready 1024 bytes and will write to console. You can use these bytes, for instance sending these bytes to other application using TCP connection.
When using the Read method, it is more efficient to use a buffer that
is the same size as the internal buffer of the stream, where the
internal buffer is set to your desired block size, and to always read
less than the block size. If the size of the internal buffer was
unspecified when the stream was constructed, its default size is 4
kilobytes (4096 bytes), MSDN.
You could try the simpler version of Read which doesn't chunk the stream but instead reads it character by character. You would have to implement the chunking your self, but it would give you more control allowing you to use a Long instead.
http://msdn.microsoft.com/en-us/library/ath1fht8(v=vs.110).aspx
I am trying to read data from a file stream as shown below:
fileStream.Read(byteArray, offset, length);
The problem is that my offset and length are Unsigned Ints and above function accepts only ints. If I typecast to int, I am getting a negative value for offset which is meaningless and not acceptable by the function.
The offset and length are originally taken from another byte array as shown below:
BitConverter.ToUInt32(length, 0); //length is a 4 byte long byte-array
What is the right way to read from arbitrary locations of a file stream.
I am not sure if this is the best way to handle it, but you can change the position of the stream and use offset 0. The Position is of type long.
fileStream.Position = (long)length;
fileStream.Read(byteArray, 0, sizeToRead);
For such a filesize you should read your file in small blocks, proccess the block and read the next. int.MaxValue is about ~2GB, uint.MaxValue ~4GB. Such a size doesn't fit in most computers ram ;)
if you are having problems with conversion, something similar might help:
uint myUInt;
int i = (int)myUInt; or
int i = Convert.ToInt32(myUInt);
When reading data over the network, you specify a buffer to receive the data into:
byte[] b = new byte[4096];
socket.Receive(b);
Now my first thought is of course to reuse the receive buffer by declaring it as a member variable of the class. My next issue is that I have not received all of the data that I am expecting, so I need to buffer my data. This is easy to accomplish by keeping track of the count of bytes received, and specifying the offset:
socket.Receive(m_ReceiveBuffer, count, m_ReceiveBuffer.Length - count);
Now, the issue here is that if it is still not enough, I am guessing that I need to grow the buffer, which means copying memory, and continue to receive into this buffer. Assuming that something went wrong, this buffer would continue to grow, and if big enough messages are received, would run the system out of memory.
Any ideas how to properly handle this? Is there a better way of receiving the data than just fill, copy, grow, fill, copy, grow that I am talking about?
read in chunks:
const int ChunkSize = 4096;
int bytesRead;
byte[] buffer = new byte[ChunkSize];
while ((bytesRead = socket.Receive(buffer, 0, ChunkSize, SocketFlags.None)) > 0)
{
byte[] actualBytesRead = new byte[bytesRead];
Buffer.BlockCopy(buffer, 0, actualBytesRead, 0, bytesRead);
// Do something with actualBytesRead,
// maybe add it to a list or write it to a stream somewhere
}
Before starting with SYstem.Net.Sockets.Socket, are you sure that you can't use System.Net.Sockets.TcpClient (or UdpClient) that does all the messy buffer work for you and transforms it to an easily managed stream?
If not, remember that the amount of data you recieve doesn't have to be equal to what you request, so you should always look at the return value from the recieve function. And, the only way to not run out of memory is by actually processing what you recieve.
First, separate the code into receiving the data and processing the data. The receive buffer should only hold the data until the code gets a chance to copy it out to the processing area. The processing area is where you will determine if you have received enough data to do something useful. Don't leave the data in the network receive buffer until this happens. For the network receive buffer, I think using a circular buffer will help you with your idea of reusing a buffer. Hopefully, you have an idea of the message sizes. That will help in determining the size of the buffer. If you assert (or something similar) when the read and write pointers of the circular buffer meet, then increase the size of the buffer. Once the size of the buffer is large enough, you should be able to read enough data out of the buffer into the processing code at a rate fast enough that the circular buffer doesn't overflow.