We Keep Coding

Retrieving only half the Image when converting to Base64

I have an API which reads an uploaded Image and changes it to a byte[], however in the database the field for where I have to save the image is a string instead of varbinary(MAX), and I cannot change the field type of the database.
I thought about converting the image to base64 and then storing it but this might cause unnecessary strain on the database.
I have found online the following way but this method can be inconsistent based on the server as the encoding might change:
var str = System.Text.Encoding.Default.GetString(result);
And if I were to use the above method I would need to know what type of encoding does ReadBytes use.
Below is my code:
byte[] fileData = null;
using (var binaryReader = new BinaryReader(image.InputStream))
{
binaryReader.BaseStream.Position = 0;
fileData = binaryReader.ReadBytes(image.ContentLength);
}
Furthermore, when I converted the image to a base64 and viewed it, only half the image was visible:
var base64String = Convert.ToBase64String(fileData);

use Convert.toBase64String() method or just using MemoryStream class->
// 1.toBase64String()
string str = Convert.ToBase64String(bytes);
// 2.MemoryStream class
using (MemoryStream stream = new MemoryStream(bytes))
using (StreamReader streamReader = new StreamReader(stream))
{
return streamReader.ReadToEnd();
}

Initialize the BinaryReader with this overload that specifies the encoding.
var binaryReader = new BinaryReader(System.IO.Stream input, System.Text.Encoding encoding);

Why Image.Save() in C# does not always give the same serialized result and how to serialize an image in a deterministic way?

I serialize images using the following code:
public static string SerializeImage(Image image)
{
using (MemoryStream memoryStream = new MemoryStream())
{
image.Save(memoryStream, image.RawFormat);
return Convert.ToBase64String(memoryStream.ToArray());
}
}
and deserialize the images by doing the following
public static Image DeserializeImage(string serializedImage)
{
byte[] imageAsBytes = Convert.FromBase64String(serializedImage);
using (MemoryStream memoryStream = new MemoryStream(imageAsBytes, 0, imageAsBytes.Length))
{
memoryStream.Write(imageAsBytes, 0, imageAsBytes.Length);
return Image.FromStream(memoryStream, true);
}
}
If I have an image and does
string serializedImage1 = SerializeImage(image);
Image deserializedImage = DeserializeImage(serializedImage1);
string serializedImage2 = SerializeImage(deserializedImage );
Then
serializedImage1 == serializedImage2;
as expected. But it is not always the case.
If I serialize an image on Process 1, and then redeserialize and reserialize it on Process 2, then the result of the reserialization on Process 2 is not the same as on the Process 1. Everything works, but a few bytes in the beginning of the serialization are different.
Worst, if I do the same thing on 2 different dll (or thread, I'm not sure), it seems the serialization result is not the same too. Again, the serialization/deserialization works, but a few bytes are different.
The image the first time is loaded with the following function :
public static Image GetImageFromFilePath(string filePath)
{
var uri = new Uri(filePath);
var bitmapImage = new BitmapImage(uri);
bitmapImage.Freeze();
using (var memoryStream = new MemoryStream())
{
var pngBitmapEncoder = new PngBitmapEncoder();
pngBitmapEncoder.Frames.Add(BitmapFrame.Create(bitmapImage));
pngBitmapEncoder.Save(memoryStream);
Image image = Image.FromStream(memoryStream);
return image;
}
}
Note however that it happens even if the image is loaded twice with the DeserializeImage() function.
The tests I have done are with ImageFormat.Jpeg and ImageFormat.Png.
First question, why it does this ? I would have expected the result to be always the same, but I suppose some salt is used when doing the Image.Save().
Second question : I want to have a deterministic way to serialize an image, keeping the image format intact. The goal is to save the image in a DB and also to compare serialized images to know if it already exists in the system where this function is used.

Well, I discovered while trying to solve this that a png or jpeg Image object inside C# has some metadata associated to it and doing what I was doing is just not a reliable way to compare images.
The solution I used was derived from this link
https://insertcode.wordpress.com/2014/05/13/compare-content-of-two-files-images-in-c/
So what I do finally is save the images inside the system with the SerializeImage(Image image) function previously described, and when I want to consume it I deserialize it with the DeserializeImage(string serializedImage) function previously described. But when I want to compare images I use the following functions
public static bool ImagesAreEqual(Image image1, Image image2)
{
string image1Base64Bitmap = GetImageAsBase64Bitmap(image1);
string image2Base64Bitmap = GetImageAsBase64Bitmap(image2);
return image1Base64Bitmap.Equals(image2Base64Bitmap);
}
public static string GetImageAsBase64Bitmap(Image image)
{
using (var memoryStream = new MemoryStream())
{
using (var bitmap = new Bitmap(image))
{
bitmap.Save(memoryStream, System.Drawing.Imaging.ImageFormat.Bmp);
}
return Convert.ToBase64String(memoryStream.ToArray());
}
}
That convert the image to raw bitmap before comparing them.
This does a perfect job for me in all my needed cases : the formats of the images are saved/restored correctly, and I can compare them between them to check if they are the same without having to bother with the possibly different serialization.

Converting a byte[] string back to byte[] array

I have one scenario with class like this.
Class Document
{
public string Name {get;set;}
public byte[] Contents {get;set;}
}
Now I am trying to implement the import export functionality where I keep the document in binary so the document will be in json file with other fields and the document will be something in this format.
UEsDBBQABgAIAAAAIQCitGbRsgEAALEHAAATAAgCW0NvbnRlbnRfVHlwZXNdLnhtbCCiBAIooAACAAAAAAA==
Now when I upload this file back, I get this file as a string and I get the same data but when I try to convert this in binary bytes[] the file become corrupt.
How can I achieve this ?
I use something like this to convert
var ss = sr.ReadToEnd();
MemoryStream stream = new MemoryStream();
StreamWriter writer = new StreamWriter(stream);
writer.Write(ss);
writer.Flush();
stream.Position = 0;
var bytes = default(byte[]);
bytes = stream.ToArray();

This looks like base 64. Use:
System.Convert.ToBase64String(b)
https://msdn.microsoft.com/en-us/library/dhx0d524%28v=vs.110%29.aspx
And
System.Convert.FromBase64String(s)
https://msdn.microsoft.com/en-us/library/system.convert.frombase64string%28v=vs.110%29.aspx

You need to de-code it from base64, like this:
Assuming you've read the file into ss as a string.
var bytes = Convert.FromBase64String(ss);

There are several things going on here. You need to know the encoding for the default StreamWriter, if it is not specified it defaults to UTF-8 encoding. However, .NET strings are always either UNICODE or UTF-16.
MemoryStream from string - confusion about Encoding to use
I would suggest using System.Convert.ToBase64String(someByteArray) and its counterpart System.Convert.FromBase64String(someString) to handle this for you.

No map for object error when deserializing object

I have the following C# code which is supposed to serialize arbitrary objects to a string, and then of course deserialize it.
public static string Pack(Message _message)
{
BinaryFormatter formatter = new BinaryFormatter();
MemoryStream original = new MemoryStream();
MemoryStream outputStream = new MemoryStream();
formatter.Serialize(original, _message);
original.Seek(0, SeekOrigin.Begin);
DeflateStream deflateStream = new DeflateStream(outputStream, CompressionMode.Compress);
original.CopyTo(deflateStream);
byte[] bytearray = outputStream.ToArray();
UTF8Encoding encoder = new UTF8Encoding();
string packed = encoder.GetString(bytearray);
return packed;
}
public static Message Unpack(string _packed_message)
{
UTF8Encoding encoder = new UTF8Encoding();
byte[] bytearray = encoder.GetBytes(_packed_message);
BinaryFormatter formatter = new BinaryFormatter();
MemoryStream input = new MemoryStream(bytearray);
MemoryStream decompressed = new MemoryStream();
DeflateStream deflateStream = new DeflateStream(input, CompressionMode.Decompress);
deflateStream.CopyTo(decompressed); // EXCEPTION
decompressed.Seek(0, SeekOrigin.Begin);
var message = (Message)formatter.Deserialize(decompressed); // EXCEPTION 2
return message;
}
But the problem is that any time the code is ran, I am experiencing an exception. Using the above code and invoking it as shown below, I am receiving InvalidDataException: Unknown block type. Stream might be corrupted. at the marked // EXCEPTION line.
After searching for this issue I have attempted to ditch the deflation. This was only a small change: in Pack, bytearray gets created from original.ToArray() and in Unpack, I Seek() input instead of decompressed and use Deserialize(input) instead of decompressed too. The only result which changed: the exception position and body is different, yet it still happens. I receive a SerializationException: No map for object '201326592'. at // EXCEPTION 2.
I don't seem to see what is the problem. Maybe it is the whole serialization idea... the problem is that somehow managing to pack the Message instances is necessary because these objects hold the information that travel between the server and the client application. (Serialization logic is in a .Shared DLL project which is referenced on both ends, however, right now, I'm only developing the server-side first.) It also has to be told, that I am only using string outputs because right now, the TCP connection between the servers and clients are based on string read-write on the ends. So somehow it has to be brought down to the level of strings.
This is how the Message object looks like:
[Serializable]
public class Message
{
public MessageType type;
public Client from;
public Client to;
public string content;
}
(Client right now is an empty class only having the Serializable attribute, no properties or methods.)
This is how the pack-unpack gets invoked (from Main()...):
Shared.Message msg = Shared.MessageFactory.Build(Shared.MessageType.DEFAULT, new Shared.Client(), new Shared.Client(), "foobar");
string message1 = Shared.MessageFactory.Pack(msg);
Console.WriteLine(message1);
Shared.Message mess2 = Shared.MessageFactory.Unpack(message1); // Step into... here be exceptions
Console.Write(mess2.content);
Here is an image showing what happens in the IDE. The output in the console window is the value of message1.
Some investigation unfortunately also revealed that the problem could lie around the bytearray variable. When running Pack(), after the encoder creates the string, the array contains 152 values, however, after it gets decoded in Unpack(), the array has 160 values instead.
I am appreciating any help as I am really out of ideas and having this problem the progress is crippled. Thank you.
(Update) The final solution:
I would like to thank everyone answering and commenting, as I have reached the solution. Thank you.
Marc Gravell was right, I missed the closing of deflateStream and because of this, the result was either empty or corrupted. I have taken my time and rethought and rewrote the methods and now it works flawlessly. And even the purpose of sending these bytes over the networked stream is working too.
Also, as Eric J. suggested, I have switched to using ASCIIEnconding for the change between string and byte[] when the data is flowing in the Stream.
The fixed code lies below:
public static string Pack(Message _message)
{
using (MemoryStream input = new MemoryStream())
{
BinaryFormatter bformatter = new BinaryFormatter();
bformatter.Serialize(input, _message);
input.Seek(0, SeekOrigin.Begin);
using (MemoryStream output = new MemoryStream())
using (DeflateStream deflateStream = new DeflateStream(output, CompressionMode.Compress))
{
input.CopyTo(deflateStream);
deflateStream.Close();
return Convert.ToBase64String(output.ToArray());
}
}
}
public static Message Unpack(string _packed)
{
using (MemoryStream input = new MemoryStream(Convert.FromBase64String(_packed)))
using (DeflateStream deflateStream = new DeflateStream(input, CompressionMode.Decompress))
using (MemoryStream output = new MemoryStream())
{
deflateStream.CopyTo(output);
deflateStream.Close();
output.Seek(0, SeekOrigin.Begin);
BinaryFormatter bformatter = new BinaryFormatter();
Message message = (Message)bformatter.Deserialize(output);
return message;
}
}
Now everything happens just right, as the screenshot proves below. This was the expected output from the first place. The Server and Client executables communicate with each other and the message travels... and it gets serialized and unserialized properly.

In addition to the existing observations about Encoding vs base-64, note you haven't closed the deflate stream. This is important because compression-streams buffer: if you don't close, it may not write the end. For a short stream, that may mean it writes nothing at all.
using(DeflateStream deflateStream = new DeflateStream(
outputStream, CompressionMode.Compress))
{
original.CopyTo(deflateStream);
}
return Convert.ToBase64String(outputStream.GetBuffer(), 0,
(int)outputStream.Length);

Your problem is most probably in the UTF8 encoding. Your bytes are not really a character string and UTF-8 is a encoding with different byte lengths for characters.
This means the byte array may not correspond to a correctly encoded UTF-8 string (there may be some bytes missing at the end for instance.)
Try using UTF16 or ASCII which are constant length encodings (the resulting string will likely contain control characters so it won't be printable or transmitable through something like HTTP or email.)
But if you want to encode as a string it is customary to use UUEncoding to convert the byte array into a real printable string, then you can use any encoding you want.

When I run the following Main() code against your Pack() and Unpack():
static void Main(string[] args)
{
Message msg = new Message() { content = "The quick brown fox" };
string message1 = Pack(msg);
Console.WriteLine(message1);
Message mess2 = Unpack(message1); // Step into... here be exceptions
Console.Write(mess2.content);
}
I see that the bytearray
byte[] bytearray = outputStream.ToArray();
is empty.
I did modify your serialized class slightly since you did not post code for the included classes
public enum MessageType
{
DEFAULT = 0
}
[Serializable]
public class Message
{
public MessageType type;
public string from;
public string to;
public string content;
}
I suggest the following steps to resolve this:
Check the intermediate results along the way. Do you also see 0 bytes in the array? What is the string value returned by Pack()?
Dispose of your streams once you are done with them. The easiest way to do that is with the using keyword.
Edit
As Eli and Marc correctly pointed out, you cannot store arbitrary bytes in a UTF8 string. The mapping is not bijective (you can't go back and forth without loss/distortion of information). You will need a mapping that is bijective, such as the Convert.ToBase64String() approach Marc suggests.

OutOfMemory exception when loading an image in .Net

Im loading an image from a SQL CE db and then trying to load that into a PictureBox.
I am saving the image like this:
if (ofd.ShowDialog() == DialogResult.OK)
{
picArtwork.ImageLocation = ofd.FileName;
using (System.IO.FileStream fs = new System.IO.FileStream(ofd.FileName, System.IO.FileMode.Open))
{
byte[] imageAsBytes = new byte[fs.Length];
fs.Read(imageAsBytes, 0, imageAsBytes.Length);
thisItem.Artwork = imageAsBytes;
fs.Close();
}
}
and then saving to the Db using LINQ To SQL.
I load the image back like so:
using (FileStream fs = new FileStream(#"C:\Temp\img.jpg", FileMode.CreateNew ,FileAccess.Write ))
{
byte[] img = (byte[])encoding.GetBytes(ThisFilm.Artwork.ToString());
fs.Write(img, 0, img.Length);
}
but am getting an OutOfMemoryException. I have read that this is a slight red herring and that there is probably something wrong with the filetype, but i cant figure what.
Any ideas?
Thanks
picArtwork.Image = System.Drawing.Bitmap.FromFile(#"C:\Temp\img.jpg");

Based on the code you provided it seems like you are treating the image as a string, it might be that data is being lost with the conversion from byte[] to string and string to byte[].
I am not familiar with SQL CE, but if you can you should consider treating the data as a byte[] and not encoding to and from string.
I base my assumption in this line of code
byte[] img = (byte[])encoding.GetBytes(ThisFilm.Artwork.ToString());

The GDI has the bad behavior of throwing an OOM exception whenever it is not capable of understanding a certain image format. Use Irfanview to see what kind of image that really is, it is likely GDI can't handle it.

My advice would be to not use a FileStream to load images unless you need access to the raw bytes. Instead, use the static methods provided in Bitmap or Image objects:
Image img = Image.FromFile(#"c:\temp\img.jpg")
Bitmap bmp = Bitmap.FromFile(#"c:\temp\img.jpg")
To save the file use .Save method of the Image or Bitmap object:
img.Save(#"c:\temp\img.jpg")
bmp.Save(#"c:\temp\img.jpg")
Of course we don't know what type ArtWork is. Would you care to share that information with us?