I am trying to read the byte[] for each file and adding it to MemoryStream. Below is the code which throws error. What I am missing in appending?
byte[] ba = null;
List<string> fileNames = new List<string>();
int startPosition = 0;
using (MemoryStream allFrameStream = new MemoryStream())
{
foreach (string jpegFileName in fileNames)
{
ba = GetFileAsPDF(jpegFileName);
allFrameStream.Write(ba, startPosition, ba.Length); //Error here
startPosition = ba.Length - 1;
}
allFrameStream.Position = 0;
ba = allFrameStream.GetBuffer();
Response.ClearContent();
Response.AppendHeader("content-length", ba.Length.ToString());
Response.ContentType = "application/pdf";
Response.BinaryWrite(ba);
Response.End();
Response.Close();
}
Error:
Offset and length were out of bounds for the array or count is greater
than the number of elements from index to the end of the source
collection
startPosition is not offset to MemoryStream, instead to ba. Change it as
allFrameStream.Write(ba, 0, ba.Length);
All byte arrays will be appended to allFrameStream
BTW: Don't use ba = allFrameStream.GetBuffer(); instead use ba = allFrameStream.ToArray(); (You actually don't want internal buffer of MemoryStream).
The MSDN documentation on Stream.Write might help clarify the problem.
Streams are modelled as a continuous sequence of bytes. Reading or writing to a stream moves your position in the stream by the number of bytes read or written.
The second argument to Write is the index in the source array at which to start copying bytes from. In your case this is 0, since you want to read from the start of the array.
Maybe this is a simple solution, not the best but is easy
List<byte> list = new List<byte>();
list.AddRange(Encoding.UTF8.GetBytes("aaaaaaaaaaaaa"));
list.AddRange(Encoding.UTF8.GetBytes("bbbbbbbbbbbbbbbbbb"));
list.AddRange(Encoding.UTF8.GetBytes("cccccccc"));
byte[] c = list.ToArray();
Related
I have
[13,132,32,75,22,61,50,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
I want
[13,132,32,75,22,61,50]
I have an array of bytes size 1048576 that I have written to using a file stream. Starting at a particular index in this array until the end of the array are all null bytes. There might be 100000 bytes with values and 948576 null bytes at the end of the array. When I don't know the size of a file how do I efficiently create a new array of size 100000 (i.e. same as total bytes in unknown file) and write all bytes from that file to the byte array?
byte[] buffer = new byte[0x100000];
int numRead = await fileStream.ReadAsync(buffer, 0, buffer.length); // byte array is padded with null bytes at the end
You're stating in the comments that you're just decoding the byte array into a string, so why not read the file contents as a string, such as:
var contents = File.ReadAllText(filePath, Encoding.UTF8);
// contents holds all the text in the file at filePath and no more
or if you want to use a stream:
using (var sr = new StreamReader(path))
{
// Read one character at a time:
var c = sr.Read();
// Read one line at a time:
var line = sr.ReadLine();
// Read the whole file
var contents = sr.ReadToEnd();
}
If you, however, insist on going through a buffer you cannot avoid part of the buffer being empty (having null-bytes) when you reach the end of the file but that's where the return value of ReadAsync saves the day:
byte[] buffer = new byte[0x100000];
int numRead = await fileStream.ReadAsync(buffer, 0, buffer.length);
var sectionToDecode = new byte[numRead];
Array.Copy(buffer, 0, sectionToDecode, 0, numRead);
// Now sectionToDecode has all the bytes that were actually read from the file
I'm tring to translate C code to C# and I stumbled upon a line of code which I'm having problems translating.
sprintf((char*)&u8FirmareBuffer[0x1C0] + strlen((char*)&u8FirmareBuffer[0x1C0]), ".B%s", argv[3]);
specifically this line.
u8FirmwareBuffer is a unsigned char array in C, a byte array in C# I would guess.
argv[3] is a string.
How can I translate this line to C#.
Thank you for your help.
Edit: This has been marked as a duplicate, but I think they differenciate because I am using pointers which don't work with the solutions presented on the marked post.
You could do something like:
string myString = "This is my string";
byte[] buffer = new byte[1024];
int offset = 0;
// if you pass a byte buffer to the constructor of a memorystream, it will use that, don't forget that it cannot grow the buffer.
using (var memStream = new MemoryStream(buffer))
{
// you can even seek to a specific position
memStream.Seek(offset, SeekOrigin.Begin);
// check your encoding..
var data = Encoding.UTF8.GetBytes(myString);
// write it on the current offset in the memory stream
memStream.Write(data, 0, data.Length);
}
It's also possible with a StreamWriter
string myString = "This is my string";
byte[] buffer = new byte[1024];
int offset = 0;
// if you pass a byte buffer to the constructor.....(see above)
using (var memStream = new MemoryStream(buffer))
using (var streamWriter = new StreamWriter(memStream))
{
// you can even seek to a specific position
memStream.Seek(offset, SeekOrigin.Begin);
streamWriter.Write(myString);
streamWriter.Flush();
// don't forget to flush before you seek again
}
public static byte[] ReadMemoryMappedFile(string fileName)
{
long length = new FileInfo(fileName).Length;
using (var stream = File.Open(fileName, FileMode.OpenOrCreate, FileAccess.Read, FileShare.ReadWrite))
{
using (var mmf = MemoryMappedFile.CreateFromFile(stream, null, length, MemoryMappedFileAccess.Read, null, HandleInheritability.Inheritable, false))
{
using (var viewStream = mmf.CreateViewStream(0, length, MemoryMappedFileAccess.Read))
{
using (BinaryReader binReader = new BinaryReader(viewStream))
{
var result = binReader.ReadBytes((int)length);
return result;
}
}
}
}
}
OpenFileDialog openfile = new OpenFileDialog();
openfile.Filter = "All Files (*.*)|*.*";
openfile.ShowDialog();
byte[] buff = ReadMemoryMappedFile(openfile.FileName);
texteditor.Text = BitConverter.ToString(buff).Replace("-"," "); <----A first chance exception of type 'System.OutOfMemoryException' occurred in mscorlib.dll
I get a System.OutOfMemory exception when trying to read large files.
I've read a lot for 4 weeks in all the web... and tried a lot!!! But still, I can't seem to find a good solution to my problem.
Please help me..
Update
public byte[] FileToByteArray(string fileName)
{
byte[] buff = null;
FileStream fs = new FileStream(fileName,
FileMode.Open,
FileAccess.Read);
BinaryReader br = new BinaryReader(fs);
long numBytes = new FileInfo(fileName).Length;
buff = br.ReadBytes((int)numBytes);
//return buff;
return File.ReadAllBytes(fileName);
}
OR
public static byte[] FileToByteArray(FileStream stream, int initialLength)
{
// If we've been passed an unhelpful initial length, just
// use 32K.
if (initialLength < 1)
{
initialLength = 32768;
}
BinaryReader br = new BinaryReader(stream);
byte[] buffer = new byte[initialLength];
int read = 0;
int chunk;
while ((chunk = br.Read(buffer, read, buffer.Length - read)) > 0)
{
read += chunk;
// If we've reached the end of our buffer, check to see if there's
// any more information
if (read == buffer.Length)
{
int nextByte = br.ReadByte();
// End of stream? If so, we're done
if (nextByte == -1)
{
return buffer;
}
// Nope. Resize the buffer, put in the byte we've just
// read, and continue
byte[] newBuffer = new byte[buffer.Length * 2];
Array.Copy(buffer, newBuffer, buffer.Length);
newBuffer[read] = (byte)nextByte;
buffer = newBuffer;
read++;
}
}
// Buffer is now too big. Shrink it.
byte[] ret = new byte[read];
Array.Copy(buffer, ret, read);
return ret;
}
I still get a System.OutOfMemory exception when trying to read large files.
If your file is 4GB, then BitConverter will turn each byte into XX- string, each char in string is 2 bytes * 3 chars per byte * 4 294 967 295 bytes = 25 769 803 770. You need +25Gb of free memory to fit entire string, plus you already have your file in memory as byte array.
Besides, no single object in a .Net program may be over 2GB. Theoretical limit for a string length would be 1,073,741,823 chars, but you also need to have a 64-bit process.
So solution in your case - open FileStream. Read first 16384 bytes (or how much can fit on your screen), convert to hex and display, and remember file offset. When user wants to navigate to next or previous page - seek to that position in file on disk, read and display again, etc.
You need to read the file in chunks, keep track of where you are in the file, page the contents on screen and use seek and position to move up and down in the file stream.
You will not be able to display 4Gb file reading all of it in memory first by any approach.
The approach is to virtualize the data, reading only the visible lines when user scrolls. If you need to do a read-only text viewer then you can use WPF ItemsControl with virtulizing stack panel and bind to custom IList collection which will lazily fetch lines from the file calculating file offset by for the line index.
I'm trying to zip and unzip data in memory (so, I cannot use FileSystem), and in my sample below when the data is unzipped it has a kind of padding ('\0' chars) at the end of my original data.
What am I doing wrong ?
[Test]
public void Zip_and_Unzip_from_memory_buffer() {
byte[] originalData = Encoding.UTF8.GetBytes("My string");
byte[] zipped;
using (MemoryStream stream = new MemoryStream()) {
using (ZipFile zip = new ZipFile()) {
//zip.CompressionMethod = CompressionMethod.BZip2;
//zip.CompressionLevel = Ionic.Zlib.CompressionLevel.BestSpeed;
zip.AddEntry("data", originalData);
zip.Save(stream);
zipped = stream.GetBuffer();
}
}
Assert.AreEqual(256, zipped.Length); // Just to show that the zip has 256 bytes which match with the length unzipped below
byte[] unzippedData;
using (MemoryStream mem = new MemoryStream(zipped)) {
using (ZipFile unzip = ZipFile.Read(mem)) {
//ZipEntry zipEntry = unzip.Entries.FirstOrDefault();
ZipEntry zipEntry = unzip["data"];
using (MemoryStream readStream = new MemoryStream()) {
zipEntry.Extract(readStream);
unzippedData = readStream.GetBuffer();
}
}
}
Assert.AreEqual(256, unzippedData.Length); // WHY my data has trailing '\0' chars like a padding to 256 module ?
Assert.AreEqual(originalData.Length, unzippedData.Length); // FAIL ! The unzipped data has 256 bytes
//Assert.AreEqual(originalData, unzippedData); // FAIL at index 9
}
From MSDN
"Note that the buffer contains allocated bytes which might be unused. For example, if the string "test" is written into the MemoryStream object, the length of the buffer returned from GetBuffer is 256, not 4, with 252 bytes unused. To obtain only the data in the buffer, use the ToArray method;
So you actually want to change the line:
zipped = stream.GetBuffer();
To the line: zipped = stream.ToArray();
I suspect it is from 'MemoryStream.GetBuffer()'
http://msdn.microsoft.com/en-us/library/system.io.memorystream.getbuffer.aspx
Note that the buffer contains allocated bytes which might be unused. For example, if the string "test" is written into the MemoryStream object, the length of the buffer returned from GetBuffer is 256, not 4, with 252 bytes unused. To obtain only the data in the buffer, use the ToArray method; however, ToArray creates a copy of the data in memory.
I'm needing to create a zip file containing documents that exist on the server. I am using the .Net Package class to do so, and to create a new Package (which is the zip file) I have to have either a path to a physical file or a stream. I am trying to not create an actual file that would be the zip file, instead just create a stream that would exist in memory or something.
My question is how do you instantiate a new Stream (i.e. FileStream, MemoryStream, etc) without having a physical file to instantiate from.
MemoryStream has several constructor overloads, none of which require a file.
There is an example of how to do this on the MSDN page for MemoryStream:
using System;
using System.IO;
using System.Text;
class MemStream
{
static void Main()
{
int count;
byte[] byteArray;
char[] charArray;
UnicodeEncoding uniEncoding = new UnicodeEncoding();
// Create the data to write to the stream.
byte[] firstString = uniEncoding.GetBytes(
"Invalid file path characters are: ");
byte[] secondString = uniEncoding.GetBytes(
Path.GetInvalidPathChars());
using(MemoryStream memStream = new MemoryStream(100))
{
// Write the first string to the stream.
memStream.Write(firstString, 0 , firstString.Length);
// Write the second string to the stream, byte by byte.
count = 0;
while(count < secondString.Length)
{
memStream.WriteByte(secondString[count++]);
}
// Write the stream properties to the console.
Console.WriteLine(
"Capacity = {0}, Length = {1}, Position = {2}\n",
memStream.Capacity.ToString(),
memStream.Length.ToString(),
memStream.Position.ToString());
// Set the position to the beginning of the stream.
memStream.Seek(0, SeekOrigin.Begin);
// Read the first 20 bytes from the stream.
byteArray = new byte[memStream.Length];
count = memStream.Read(byteArray, 0, 20);
// Read the remaining bytes, byte by byte.
while(count < memStream.Length)
{
byteArray[count++] =
Convert.ToByte(memStream.ReadByte());
}
// Decode the byte array into a char array
// and write it to the console.
charArray = new char[uniEncoding.GetCharCount(
byteArray, 0, count)];
uniEncoding.GetDecoder().GetChars(
byteArray, 0, count, charArray, 0);
Console.WriteLine(charArray);
}
}
}
Is this what you are looking for?
You can create a new stream and write to it. You don't need a file to construct the object.
http://msdn.microsoft.com/en-us/library/system.io.memorystream.aspx
Write Method:
http://msdn.microsoft.com/en-us/library/system.io.memorystream.write.aspx
Constructors for Memory Stream:
http://msdn.microsoft.com/en-us/library/system.io.memorystream.memorystream.aspx