We Keep Coding

The stream is invalid or no corresponding signature was found

I am trying to use SevenZipSharp to compress and decompress a memory stream. Compression is working fine but decompression is not. I think SevenZipSharp is not able to figure the archive type from the stream.
SevenZipCompressor compress = new SevenZip.SevenZipCompressor();
compress.CompressionLevel = CompressionLevel.Normal;
compress.CompressionMethod = CompressionMethod.Lzma
using (MemoryStream memStream = new MemoryStream())
{
compress.CompressFiles(memStream, #"d:\Temp1\MyFile.bmp");
using (FileStream file = new FileStream(#"d:\arch.7z", FileMode.Create, System.IO.FileAccess.Write))
{
memStream.CopyTo(file);
}
}
//works till here, file is created
Console.Read();
using (FileStream file = new FileStream(#"d:\arch.7z", FileMode.Open, System.IO.FileAccess.Read))
{
using (MemoryStream memStream = new MemoryStream())
{
file.CopyTo(memStream);
//throws exception here on this line
using (var extractor = new SevenZipExtractor(memStream))
{
extractor.ExtractFiles(#"d:\x", 0);
}
}
}

Try to see if your output file can be loaded using the 7Zip client. I'm guessing that it will fail.
The problem lies in the writing to the memorystream. Say, you write 100 bytes to the stream, it will be on position 100. When you use CopyTo, the stream will be copied from the current position, not the start of the stream.
So you'll have to reset the position to 0 after reading/writing to allow the next reader to read all the data. For instance when creating the 7Zip file:
using (MemoryStream memStream = new MemoryStream())
{
// Position starts at 0
compress.CompressFiles(memStream, #"d:\Temp1\MyFile.bmp");
// Position is now N
memStream.Position = 0; // <-- Reset the position to 0.
using (FileStream file = new FileStream(#"d:\arch.7z", FileMode.Create, System.IO.FileAccess.Write))
{
// Will copy all data in the stream from current position till the end of the stream.
memStream.CopyTo(file);
}
}

DeflateStream doesnt work on MemoryStream?

I have the following piece of code:
MemoryStream resultStream = new MemoryStream();
string users = ""//Really long string goes here
BinaryFormatter bFormatter = new BinaryFormatter();
using (MemoryStream assignedUsersStream = new MemoryStream())
{
bFormatter.Serialize(assignedUsersStream, users);
assignedUsersStream.Position = 0;
using (var compressionStream =
new DeflateStream(resultStream, CompressionLevel.Optimal))
{
assignedUsersStream.CopyTo(compressionStream);
Console.WriteLine("Compressed from {0} to {1} bytes.",
assignedUsersStream.Length.ToString(),
resultStream.Length.ToString());
}
}
the thing is that resultStream is always empty!
What am I doing wrong here?

Put your verification WriteLine outside of the using. The buffers haven't been flushed yet.
using (DeflateStream compressionStream = new DeflateStream(resultStream, CompressionLevel.Optimal))
{
assignedUsersStream.CopyTo(compressionStream);
//Console.WriteLine("Compressed from {0} to {1} bytes.",
// assignedUsersStream.Length.ToString(), resultStream.Length.ToString());
}
Console.WriteLine("Compressed from {0} to {1} bytes.",
assignedUsersStream.Length, resultStream.ToArray().Length);
And aside, you don't need all those ToString()s in a writeline.
PS: All a BinaryFormatter does with a string is write the bytes with length prefix. If you don't need the prefix (my guess), it could become:
string users = "";//Really long string goes here
byte[] result;
using (MemoryStream resultStream = new MemoryStream())
{
using (DeflateStream compressionStream = new DeflateStream(resultStream,
CompressionLevel.Optimal))
{
byte[] inBuffer = Encoding.UTF8.GetBytes(users);
compressionStream.Write(inBuffer, 0, inBuffer.Length);
}
result = resultStream.ToArray();
}
The reverse is just as easy but you'll need an estimate of the maximum length to create the read-buffer:
string users2 = null;
using (MemoryStream resultStream = new MemoryStream(result))
{
using (DeflateStream compressionStream = new DeflateStream(resultStream,
CompressionMode.Decompress))
{
byte[] outBuffer = new byte[2048]; // need an estimate here
int length = compressionStream.Read(outBuffer, 0, outBuffer.Length);
users2 = Encoding.UTF8.GetString(outBuffer, 0, length);
}
}

That is because the DeflateStream doesn't flush the data to the underlying stream until it is closed. After it is closed, resultStream will contain the compressed data. Note that by default, DeflateStream closes the underlying stream when it's closed, but you don't want that, so you need to pass true for the leaveOpen parameter. Also, you don't need 2 memory streams, you can just serialize directly to the compressionStream:
string users = ""; //Really long string goes here
BinaryFormatter bFormatter = new BinaryFormatter();
using (MemoryStream resultStream = new MemoryStream())
{
using (DeflateStream compressionStream = new DeflateStream(resultStream, CompressionLevel.Optimal, true))
{
bFormatter.Serialize(compressionStream, users);
Console.WriteLine(resultStream.Length); // 0 at this point
}
Console.WriteLine(resultStream.Length); // now contains the actual length
}

From the original answer (I don't have enough credits to vote down)
Put your control WriteLine outside of the using
This is incomplete and IMO therefore misleading. DeflateStream's Dispose(bool) implementation Closes the underlying resultStream when the DeflateStream is being Finalized after it's been Garbage Collected. When this happens, resultStream.Length will throw:
Unhandled Exception: System.ObjectDisposedException: Cannot access a closed Stream.
In other words, Thomas Levesque's note is critical: also set leaveOpen to true.
An interesting question with some good points raised by HH and TL.

I've come in late to this as I ran into this same problem and reading the conflicting answers. The initial response works! as does this test (over simplified to highlight the answer):
var inStream = new MemoryStream(data);
var outStream = new MemoryStream();
using (var compressor = new DeflateStream(outStream, CompressionLevel.Optimal))
{
inStream.CopyTo(compressor);
}
return outStream;
where the using block needs to complete, triggering the compressor's Dispose, which internally Flush()es so that outStream will be guaranteed to contain the complete compressed data.

How to read byte array into FileStream

I have an byte array and I want to read the byte array into a FileStream. Below is my sample of code:
string fileName = "test.txt";
byte[] file = File.ReadAllBytes(Server.MapPath("~/Files/" + fileName));
FileStream fs = new FileStream();
fs.ReadByte(file);
object obj = LoadFile<object>(fs);
public static T LoadFile<T>(FileStream fs)
{
using (GZipStream gzip = new GZipStream(fs, CompressionMode.Decompress))
{
BinaryFormatter bf = new BinaryFormatter();
return (T)bf.Deserialize(gzip);
}
}
In the method above, I have use FileStream to read byte array, but unlucky fs.ReadByte cannot read byte array. Any help please focus on how to Read byte array into a FileStream for using as a parameter in method "LoadFile". Please do not read directly the file into FileStream because the file here is loaded from somewhere else like from database or other source.

string fileName = "test.txt";
byte[] file = File.ReadAllBytes(Server.MapPath("~/Files/" + fileName));
MemoryStream memStream = new MemoryStream();
BinaryFormatter binForm = new BinaryFormatter();
memStream.Write(file, 0, file.Length);
memStream.Seek(0, SeekOrigin.Begin);
Object obj = (Object)binForm.Deserialize(memStream);

I'm not sure where the misunderstanding is. FileStream represents a file on disk. You cannot "read bytes into it" without writing them to disk and you cannot read from it without reading from disk.
Maybe what you want is a MemoryStream which can contain arbitrary contents.
Both derive from Stream.

Yeah! Now I got a good solution after doing some more research. As the topic I have posted "How to read byte array into FileStream". We cannot read byte array into FileStream, it just use to read a file on driver to byte array. So I have change a little bit on my code, and now I have a file to read it using FileStream. How I made a file?
In this context I have an object. The object is anything as you want!
I use a collection as a samble object.
Collection<object> list = new Collection<object>();
//Now I will write this list to a file. fileName is what you want and be sure that folder Files is exist on server or at the root folder of your project
WriteFile(list, Server.MapPath("~/Files/" + fileName));
//The method to write object to file is here
public static void WriteFile<T>(T obj, string path)
{
FileStream serializeStream = new FileStream(path, FileMode.Create);
BinaryFormatter bf = new BinaryFormatter();
bf.Serialize(serializeStream, obj);
serializeStream.Flush();
serializeStream.Close();
}
After I have wrote my object to a file, I need a method to read it back to object. So I do write this method:
public static Collection<object> ReatFile(string fileName){
//I have to read the file which I have wrote to an byte array
byte[] file;
using (var stream = new FileStream(Server.MapPath("~/Files/" + fileName), FileMode.Open, FileAccess.Read))
{
using (var reader = new BinaryReader(stream))
{
file = reader.ReadBytes((int)stream.Length);
}
}
//And now is what I have to do with the byte array of file is to convert it back to object which I have wrote it into a file
//I am using MemoryStream to convert byte array back to the original object.
MemoryStream memStream = new MemoryStream();
BinaryFormatter binForm = new BinaryFormatter();
memStream.Write(file, 0, file.Length);
memStream.Seek(0, SeekOrigin.Begin);
Object obj = (Object)binForm.Deserialize(memStream);
Collection<object> list = (Collection<object>)obj;
return list;
}
After doing some steps above, I am now can write any type object to file and then read it back to original object. Thank too much for any help I have got there.

Why do you run File.ReadAllBytes prior to the usage of your FileStream?
string fileName = "test.txt";
using(FileStream fs = new FileStream(Server.MapPath("~/Files/" + fileName), FileMode.Open, FileAccess.Read))
{
object obj = LoadFile<object>(fs);
fs.Close();
}

Correct terminology for streaming a file into bytes and back

I have the below methods:
public static byte[] ConvertFileToBytes(string filePath)
{
var fInfo = new FileInfo(filePath);
var numBytes = fInfo.Length;
var dLen = Convert.ToDouble(fInfo.Length / 1000000);
var fStream = new FileStream(filePath, FileMode.Open, FileAccess.Read);
var br = new BinaryReader(fStream);
var data = br.ReadBytes((int)numBytes);
br.Close();
fStream.Close();
fStream.Dispose();
return data;
}
public static void ConvertBytesToFile(byte[] file, string filePath)
{
var ms = new MemoryStream(file);
var fs = new FileStream(filePath, FileMode.Create);
ms.WriteTo(fs);
ms.Close();
fs.Close();
fs.Dispose();
}
What is the correct to name these methods? (because ConvertXXXtoYYY just doesn't cut it in a Utilities library)

How about File.ReadAllBytes and File.WriteAllBytes ;)

The terms usually used are "serialize" and "deserialize" (or sometimes "marshal" and "demarshal").

Marshalling/Unmarshalling might be the appropriate term.
http://en.wikipedia.org/wiki/Marshalling_(computer_science)

In C++ they would be called read and write.

The WriteAllBytes and ReadAllBytes are a good suggestion, but to answer your Question ...
Save() would be a good choice for renaming of ConvertToFile() and Object.CreateFromFile() for the reverse.

End of Stream encountered before parsing was completed?

I am trying to deserialize a stream but I always get this error "End of Stream encountered before parsing was completed"?
Here is the code:
//Some code here
BinaryFormatter b = new BinaryFormatter();
return (myObject)b.Deserialize(s);//s---> is a Stream object that has been fill up with data some line over here
Any one have ideas?

Try to set the position to 0 of your stream and do not use your object but the object type.
BinaryFormatter b = new BinaryFormatter();
s.Position = 0;
return (YourObjectType)b.Deserialize(s);

Make sure the serialization completed, and that the serialization type matches the de-serialization type (i.e., make sure you're serializing with a BinaryFormatter if you're de-serializing with one). Also, make sure that the stream you serialized to really finished serializing, with a Stream.Flush() or something to that effect.

I had the same exception thrown, until I added the [Serializable] tag to the class I was Serializing :)
Then it all worked perfectly.

In my case I used:
stream.Seek(0, SeekOrigin.Begin);
after i serialized the stream, and before i deserialized the stream works charm. hope this helps!

I have spent 5 hourse and have got end of stream error and lost data (Not obvious feature in GzipStream: you should use underlying stream only after flush GzipStream).
Full example of working code:
using System;
using System.IO;
using System.IO.Compression;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Formatters.Binary;
namespace ConsoleApp3
{
class Program
{
static void Main(string[] args)
{
string large = LargeJsonContent.GetBigObject();
string base64;
using (var readStream = new MemoryStream())
using (var writeStream = new MemoryStream())
{
using (GZipStream compressor = new GZipStream(writeStream, CompressionMode.Compress, true)) //pay attention to leaveOpen = true
{
var formatter = new BinaryFormatter();
formatter.Serialize(readStream, large);
Console.WriteLine($"After binary serialization of JsonString: {readStream.Length} bytes");
readStream.Position = 0;
readStream.CopyTo(compressor);
}
Console.WriteLine($"Compressed stream size: {writeStream.Length} bytes");
writeStream.Position = 0;
byte[] writeBytes = writeStream.ToArray();
base64 = Convert.ToBase64String(writeBytes);
}
////
using (var stream = new MemoryStream())
{
var formatter = new BinaryFormatter();
formatter.Serialize(stream, base64);
Console.WriteLine($"Size of base64: {stream.Length} bytes");
}
Console.WriteLine("---------------------");
////
string large2;
var bytes = Convert.FromBase64String(base64);
using (var readStream = new MemoryStream())
{
readStream.Write(bytes, 0, bytes.Length);
readStream.Position = 0;
Console.WriteLine($"Compressed stream size: {readStream.Length} bytes");
using (var writeStream = new MemoryStream())
{
using (GZipStream decompressor = new GZipStream(readStream, CompressionMode.Decompress, true)) //pay attention to leaveOpen = true
{
decompressor.CopyTo(writeStream);
writeStream.Position = 0;
}
var formatter = new BinaryFormatter();
large2 = (string)formatter.Deserialize(writeStream);
}
}
Console.WriteLine(large == large2);
Console.WriteLine($"large:{large.Length} | large2:{large2.Length}");
}
}
}

Check in your sender code if you are not doing the following
NetworkStream strm = client.GetStream(); // the stream
formatter.Serialize(strm, status); // the serialization process
strm.Close();// Remove this code, this was the culprit in my case

the class which you created must has [Serializable].