We Keep Coding

uncompressed file is bigger than original file in GZIP

i'm using the following function to compress(thanks to http://www.dotnetperls.com/):
public static void CompressStringToFile(string fileName, string value)
{
// A.
// Write string to temporary file.
string temp = Path.GetTempFileName();
File.WriteAllText(temp, value);
// B.
// Read file into byte array buffer.
byte[] b;
using (FileStream f = new FileStream(temp, FileMode.Open))
{
b = new byte[f.Length];
f.Read(b, 0, (int)f.Length);
}
// C.
// Use GZipStream to write compressed bytes to target file.
using (FileStream f2 = new FileStream(fileName, FileMode.Create))
using (GZipStream gz = new GZipStream(f2, CompressionMode.Compress, false))
{
gz.Write(b, 0, b.Length);
}
}
and for decompress:
static byte[] Decompress(byte[] gzip)
{
// Create a GZIP stream with decompression mode.
// ... Then create a buffer and write into while reading from the GZIP stream.
using (GZipStream stream = new GZipStream(new MemoryStream(gzip), CompressionMode.Decompress))
{
const int size = 4096;
byte[] buffer = new byte[size];
using (MemoryStream memory = new MemoryStream())
{
int count = 0;
do
{
count = stream.Read(buffer, 0, size);
if (count > 0)
{
memory.Write(buffer, 0, count);
}
}
while (count > 0);
return memory.ToArray();
}
}
}
so my goal is actually compress log files and than to decompress them in memory and compare the uncompressed file to the original file in order to check that the compression succeeded and i'm able to open the compressed file successfuly.
the problem is that the uncompressed file is most of the time bigger than the original file and my compare check is failing altough the compression probably succeeded.
any idea why ?
btw here how i compare the uncompressed file to the original file:
static bool FileEquals(byte[] file1, byte[] file2)
{
if (file1.Length == file2.Length)
{
for (int i = 0; i < file1.Length; i++)
{
if (file1[i] != file2[i])
{
return false;
}
}
return true;
}
return false;
}

Try this method to compress a file:
public static byte[] Compress(byte[] raw)
{
using (MemoryStream memory = new MemoryStream())
{
using (GZipStream gzip = new GZipStream(memory,
CompressionMode.Compress, true))
{
gzip.Write(raw, 0, raw.Length);
}
return memory.ToArray();
}
}
}
And this to decompress :
static byte[] Decompress(byte[] gzip)
{
// Create a GZIP stream with decompression mode.
// ... Then create a buffer and write into while reading from the GZIP stream.
using (GZipStream stream = new GZipStream(new MemoryStream(gzip), CompressionMode.Decompress))
{
const int size = 4096;
byte[] buffer = new byte[size];
using (MemoryStream memory = new MemoryStream())
{
int count = 0;
do
{
count = stream.Read(buffer, 0, size);
if (count > 0)
{
memory.Write(buffer, 0, count);
}
}
while (count > 0);
return memory.ToArray();
}
}
}
}
Tell me if it worked.
Goodluck.

Think you'd be better off with the simplest API call, try Stream.CopyTo(). I can't find the error in your code. If I was working on it, I'd probably make sure everything is getting flushed properly.. can't recall if GZipStream is going to flush its output to FileStream when the using block closes.. but then you are also saying that the final file is larger, not smaller.
Anyhow, best policy in my experience.. don't rewrite gotcha prone code when you don't need to. At least you tested it ;)

How to overwrite an existing image with c#?

To save an image I use the following code:
string filenamewithpath =
System.Web.HttpContext.Current.Server.MapPath(
#"~/userimages/" + incID + ".jpg");
System.IO.File.WriteAllBytes(filenamewithpath, Util.ReadFully(image));
public class Util
{
public static byte[] ReadFully(Stream stream)
{
byte[] buffer = new byte[32768];
using (MemoryStream ms = new MemoryStream())
{
while (true)
{
int read = stream.Read(buffer, 0, buffer.Length);
if (read <= 0)
return ms.ToArray();
ms.Write(buffer, 0, read);
}
}
}
}
The above works for saving the image with an ID. When updating, I need to overwrite the existing image, and would need some advice on how to do this.

If you just need to get rid of the old image file before writing the new one, why not just call
if (System.IO.File.Exists(filenamewithpath)
{
System.IO.File.Delete(filenamewithpath);
}
Although, the description of System.IO.File.WriteAllBytes already says that "if the file exists, it is overwritten".

System.IO.File.WriteAllBytes(filenamewithpath, Util.ReadFully(image));
replace this line with:
using (FileStream fs = new FileStream(filenamewithpath, FileMode.OpenOrCreate))
{
var bytes=Util.ReadFully(image);
fs.Write(bytes, 0, bytes.Length);
}

C# decode (decompress) Deflate data of PDF File

I would like to decompress in C# some DeflateCoded data (PDF extracted).
Unfortunately I got every time the exception "Found invalid data while decoding.".
But the data are valid.
private void Decompress()
{
FileStream fs = new FileStream(#"S:\Temp\myFile.bin", FileMode.Open);
//First two bytes are irrelevant
fs.ReadByte();
fs.ReadByte();
DeflateStream d_Stream = new DeflateStream(fs, CompressionMode.Decompress);
StreamToFile(d_Stream, #"S:\Temp\myFile1.txt", FileMode.OpenOrCreate);
d_Stream.Close();
fs.Close();
}
private static void StreamToFile(Stream inputStream, string outputFile, FileMode fileMode)
{
if (inputStream == null)
throw new ArgumentNullException("inputStream");
if (String.IsNullOrEmpty(outputFile))
throw new ArgumentException("Argument null or empty.", "outputFile");
using (FileStream outputStream = new FileStream(outputFile, fileMode, FileAccess.Write))
{
int cnt = 0;
const int LEN = 4096;
byte[] buffer = new byte[LEN];
while ((cnt = inputStream.Read(buffer, 0, LEN)) != 0)
outputStream.Write(buffer, 0, cnt);
}
}
Does anyone has some ideas?
Thanks.

I added this for test data:-
private static void Compress()
{
FileStream fs = new FileStream(#"C:\Temp\myFile.bin", FileMode.Create);
DeflateStream d_Stream = new DeflateStream(fs, CompressionMode.Compress);
for (byte n = 0; n < 255; n++)
d_Stream.WriteByte(n);
d_Stream.Close();
fs.Close();
}
Modified Decompress like this:-
private static void Decompress()
{
FileStream fs = new FileStream(#"C:\Temp\myFile.bin", FileMode.Open);
//First two bytes are irrelevant
// fs.ReadByte();
// fs.ReadByte();
DeflateStream d_Stream = new DeflateStream(fs, CompressionMode.Decompress);
StreamToFile(d_Stream, #"C:\Temp\myFile1.txt", FileMode.OpenOrCreate);
d_Stream.Close();
fs.Close();
}
Ran it like this:-
static void Main(string[] args)
{
Compress();
Decompress();
}
And got no errors.
I conclude that either the first two bytes are relevant (Obviously they are with my particular test data.) or
that your data has a problem.
Can we have some of your test data to play with?
(Obviously don't if it's sensitive)

private static string decompress(byte[] input)
{
byte[] cutinput = new byte[input.Length - 2];
Array.Copy(input, 2, cutinput, 0, cutinput.Length);
var stream = new MemoryStream();
using (var compressStream = new MemoryStream(cutinput))
using (var decompressor = new DeflateStream(compressStream, CompressionMode.Decompress))
decompressor.CopyTo(stream);
return Encoding.Default.GetString(stream.ToArray());
}
Thank you user159335 and user1011394 for bringing me on the right track! Just pass all bytes of the stream to input of above function. Make sure the bytecount is the same as the length specified.

All you need to do is use GZip instead of Deflate. Below is the code I use for the content of the stream… endstream section in a PDF document:
using System.IO.Compression;
public void DecompressStreamData(byte[] data)
{
int start = 0;
while ((this.data[start] == 0x0a) | (this.data[start] == 0x0d)) start++; // skip trailling cr, lf
byte[] tempdata = new byte[this.data.Length - start];
Array.Copy(data, start, tempdata, 0, data.Length - start);
MemoryStream msInput = new MemoryStream(tempdata);
MemoryStream msOutput = new MemoryStream();
try
{
GZipStream decomp = new GZipStream(msInput, CompressionMode.Decompress);
decomp.CopyTo(msOutput);
}
catch (Exception e)
{
MessageBox.Show(e.Message);
}
}

None of the solutions worked for me on Deflate attachments in a PDF/A-3 document. Some research showed that .NET DeflateStream does not support compressed streams with a header and trailer as per RFC1950.
Error message for reference: The archive entry was compressed using an unsupported compression method.
The solution is to use an alternative library SharpZipLib
Here is a simple method that successfully decoded a Deflate attachment from a PDF/A-3 file for me:
public static string SZLDecompress(byte[] data) {
var outputStream = new MemoryStream();
using var compressedStream = new MemoryStream(data);
using var inputStream = new InflaterInputStream(compressedStream);
inputStream.CopyTo(outputStream);
outputStream.Position = 0;
return Encoding.Default.GetString(outputStream.ToArray());
}

C# Trying to replace a byte while using MemoryStream class

I get a text file from a mainframe and sometimes there are some 0x0D injected into the middle of the text lines.
The previos programmer created a method using the FileStream class. This method works fine but is taking around 30 minutes to go thru the entire file.
My thought was to pass the text lines that are needed (about 25 lines) to a method to decrease the processing time.
I've been working with the MemoryStream class but am having issue where it does not find the 0x0D control code.
Here is the current FileStream method:
private void ReplaceFileStream(string strInputFile)
{
FileStream fileStream = new FileStream(strInputFile, FileMode.Open, FileAccess.ReadWrite);
byte filebyte;
while (fileStream.Position < fileStream.Length)
{
filebyte = (byte)fileStream.ReadByte();
if (filebyte == 0x0D)
{
filebyte = 0x20;
fileStream.Position = fileStream.Position - 1;
fileStream.WriteByte(filebyte);
}
}
fileStream.Close();
}
and here is the MemoryStream method:
private void ReplaceMemoryStream(string strInputLine)
{
byte[] byteArray = Encoding.ASCII.GetBytes(strInputLine);
MemoryStream fileStream = new MemoryStream(byteArray);
byte filebyte;
while (fileStream.Position < fileStream.Length)
{
filebyte = (byte)fileStream.ReadByte();
if (filebyte == 0x0D)
{
filebyte = 0x20;
fileStream.Position = fileStream.Position - 1;
fileStream.WriteByte(filebyte);
}
}
fileStream.Close();
}
As I have not used the MemoryStream class before am not that familar with it. Any tips or ideas?

I don't know the size of your files, but if they are small enough that you can load the whole thing in memory at once, then you could do something like this:
private void ReplaceFileStream(string strInputFile)
{
byte[] fileBytes = File.ReadAllBytes(strInputFile);
bool modified = false;
for(int i=0; i < fileBytes.Length; ++i)
{
if (fileByte[i] == 0x0D)
{
fileBytes[i] = 0x20;
modified = true;
}
}
if (modified)
{
File.WriteAllBytes(strInputFile, fileBytes);
}
}
If you can't read the whole file in at once, then you should switch to a buffered reading type of setup, here is an example that reads from the file, writes to a temp file, then in the end copies the temp file over the original file. This should yield better performance then reading a file one byte at a time:
private void ReplaceFileStream(string strInputFile)
{
string tempFile = Path.GetTempFileName();
try
{
using(FileStream input = new FileStream(strInputFile,
FileMode.Open, FileAccess.Read))
using(FileStream output = new FileStream(tempFile,
FileMode.Create, FileAccess.Write))
{
byte[] buffer = new byte[4096];
bytesRead = input.Read(buffer, 0, 4096);
while(bytesRead > 0)
{
for(int i=0; i < bytesRead; ++i)
{
if (buffer[i] == 0x0D)
{
buffer[i] = 0x20;
}
}
output.Write(buffer, 0, bytesRead);
bytesRead = input.Read(buffer, 0, 4096);
}
output.Flush();
}
File.Copy(tempFile, strInputFile);
}
finally
{
if (File.Exists(tempFile))
{
File.Delete(tempFile);
}
}
}

if your replacement code does not find the 0x0D in the stream and the previous method with the FileStream does it, I think it could be because of the Encoding you are using to get the bytes of the file, you can try with some other encoding types.
otherwise your code seems to be fine, I would use a using around the MemoryStream to be sure it gets closed and disposed, something like this:
using(var fileStream = new MemoryStream(byteArray))
{
byte filebyte;
// your while loop...
}
looking at your code I am not 100% sure the changes you make to the memory stream will be persisted; Actually I think that if you do not save it after the changes, your changes will be lost. I can be wrong in this but you should test and see, if it does not save you should use StreamWriter to save it after the changes.

How to get a MemoryStream from a Stream in .NET?

I have the following constructor method which opens a MemoryStream from a file path:
MemoryStream _ms;
public MyClass(string filePath)
{
byte[] docBytes = File.ReadAllBytes(filePath);
_ms = new MemoryStream();
_ms.Write(docBytes, 0, docBytes.Length);
}
I need to change this to accept a Stream instead of a file path. Whats the easiest/most efficient way to get a MemoryStream from the Stream object?

In .NET 4, you can use Stream.CopyTo to copy a stream, instead of the home-brew methods listed in the other answers.
MemoryStream _ms;
public MyClass(Stream sourceStream)
_ms = new MemoryStream();
sourceStream.CopyTo(_ms);
}

Use this:
var memoryStream = new MemoryStream();
stream.CopyTo(memoryStream);
This will convert Stream to MemoryStream.

If you're modifying your class to accept a Stream instead of a filename, don't bother converting to a MemoryStream. Let the underlying Stream handle the operations:
public class MyClass
{
Stream _s;
public MyClass(Stream s) { _s = s; }
}
But if you really need a MemoryStream for internal operations, you'll have to copy the data out of the source Stream into the MemoryStream:
public MyClass(Stream stream)
{
_ms = new MemoryStream();
CopyStream(stream, _ms);
}
// Merged From linked CopyStream below and Jon Skeet's ReadFully example
public static void CopyStream(Stream input, Stream output)
{
byte[] buffer = new byte[16*1024];
int read;
while((read = input.Read (buffer, 0, buffer.Length)) > 0)
{
output.Write (buffer, 0, read);
}
}

You can simply do:
var ms = new MemoryStream(File.ReadAllBytes(filePath));
Stream position is 0 and ready to use.

You will have to read in all the data from the Stream object into a byte[] buffer and then pass that into the MemoryStream via its constructor. It may be better to be more specific about the type of stream object you are using. Stream is very generic and may not implement the Length attribute, which is rather useful when reading in data.
Here's some code for you:
public MyClass(Stream inputStream) {
byte[] inputBuffer = new byte[inputStream.Length];
inputStream.Read(inputBuffer, 0, inputBuffer.Length);
_ms = new MemoryStream(inputBuffer);
}
If the Stream object doesn't implement the Length attribute, you will have to implement something like this:
public MyClass(Stream inputStream) {
MemoryStream outputStream = new MemoryStream();
byte[] inputBuffer = new byte[65535];
int readAmount;
while((readAmount = inputStream.Read(inputBuffer, 0, inputBuffer.Length)) > 0)
outputStream.Write(inputBuffer, 0, readAmount);
_ms = outputStream;
}

I use this combination of extension methods:
public static Stream Copy(this Stream source)
{
if (source == null)
return null;
long originalPosition = -1;
if (source.CanSeek)
originalPosition = source.Position;
MemoryStream ms = new MemoryStream();
try
{
Copy(source, ms);
if (originalPosition > -1)
ms.Seek(originalPosition, SeekOrigin.Begin);
else
ms.Seek(0, SeekOrigin.Begin);
return ms;
}
catch
{
ms.Dispose();
throw;
}
}
public static void Copy(this Stream source, Stream target)
{
if (source == null)
throw new ArgumentNullException("source");
if (target == null)
throw new ArgumentNullException("target");
long originalSourcePosition = -1;
int count = 0;
byte[] buffer = new byte[0x1000];
if (source.CanSeek)
{
originalSourcePosition = source.Position;
source.Seek(0, SeekOrigin.Begin);
}
while ((count = source.Read(buffer, 0, buffer.Length)) > 0)
target.Write(buffer, 0, count);
if (originalSourcePosition > -1)
{
source.Seek(originalSourcePosition, SeekOrigin.Begin);
}
}

How do I copy the contents of one stream to another?
see that. accept a stream and copy to memory. you should not use .Length for just Stream because it is not necessarily implemented in every concrete Stream.

public static void Do(Stream in)
{
_ms = new MemoryStream();
byte[] buffer = new byte[65536];
while ((int read = input.Read(buffer, 0, buffer.Length))>=0)
_ms.Write (buffer, 0, read);
}

byte[] fileData = null;
using (var binaryReader = new BinaryReader(Request.Files[0].InputStream))
{
fileData = binaryReader.ReadBytes(Request.Files[0].ContentLength);
}