We Keep Coding

Sending GZipped data via TcpClient [duplicate]

I've got a pesky problem with gzipstream targeting .Net 3.5. This is my first time working with gzipstream, however I have modeled after a number of tutorials including here and I'm still stuck.
My app serializes a datatable to xml and inserts into a database, storing the compressed data into a varbinary(max) field as well as the original length of the uncompressed buffer. Then, when I need it, I retrieve this data and decompress it and recreates the datatable. The decompress is what seems to fail.
EDIT: Sadly after changing the GetBuffer to ToArray as suggested, my issue remains. Code Updated below
Compress code:
DataTable dt = new DataTable("MyUnit");
//do stuff with dt
//okay... now compress the table
using (MemoryStream xmlstream = new MemoryStream())
{
//instead of stream, use xmlwriter?
System.Xml.XmlWriterSettings settings = new System.Xml.XmlWriterSettings();
settings.Encoding = Encoding.GetEncoding(1252);
settings.Indent = false;
System.Xml.XmlWriter writer = System.Xml.XmlWriter.Create(xmlstream, settings);
try
{
dt.WriteXml(writer);
writer.Flush();
}
catch (ArgumentException)
{
//likely an encoding issue... okay, base64 encode it
var base64 = Convert.ToBase64String(xmlstream.ToArray());
xmlstream.Write(Encoding.GetEncoding(1252).GetBytes(base64), 0, Encoding.GetEncoding(1252).GetBytes(base64).Length);
}
using (MemoryStream zipstream = new MemoryStream())
{
GZipStream zip = new GZipStream(zipstream, CompressionMode.Compress);
log.DebugFormat("Compressing commands...");
zip.Write(xmlstream.GetBuffer(), 0, xmlstream.ToArray().Length);
zip.Flush();
float ratio = (float)zipstream.ToArray().Length / (float)xmlstream.ToArray().Length;
log.InfoFormat("Resulting compressed size is {0:P2} of original", ratio);
using (SqlCommand cmd = new SqlCommand())
{
cmd.CommandText = "INSERT INTO tinydup (lastid, command, compressedlength) VALUES (#lastid,#compressed,#length)";
cmd.Connection = db;
cmd.Parameters.Add("#lastid", SqlDbType.Int).Value = lastid;
cmd.Parameters.Add("#compressed", SqlDbType.VarBinary).Value = zipstream.ToArray();
cmd.Parameters.Add("#length", SqlDbType.Int).Value = xmlstream.ToArray().Length;
cmd.ExecuteNonQuery();
}
}
Decompress Code:
/* This is an encapsulation of what I get from the database
public class DupUnit{
public uint lastid;
public uint complength;
public byte[] compressed;
}*/
//I have already retrieved my list of work to do from the database in a List<Dupunit> dupunits
foreach (DupUnit unit in dupunits)
{
DataSet ds = new DataSet();
//DataTable dt = new DataTable();
//uncompress and extract to original datatable
try
{
using (MemoryStream zipstream = new MemoryStream(unit.compressed))
{
GZipStream zip = new GZipStream(zipstream, CompressionMode.Decompress);
byte[] xmlbits = new byte[unit.complength];
//WHY ARE YOU ALWAYS 0!!!!!!!!
int bytesdecompressed = zip.Read(xmlbits, 0, unit.compressed.Length);
MemoryStream xmlstream = new MemoryStream(xmlbits);
log.DebugFormat("Uncompressed XML against {0} is: {1}", m_source.DSN, Encoding.GetEncoding(1252).GetString(xmlstream.ToArray()));
try{
ds.ReadXml(xmlstream);
}catch(Exception)
{
//it may have been base64 encoded... decode first.
ds.ReadXml(Encoding.GetEncoding(1254).GetString(
Convert.FromBase64String(
Encoding.GetEncoding(1254).GetString(xmlstream.ToArray())))
);
}
xmlstream.Dispose();
}
}
catch (Exception e)
{
log.Error(e);
Thread.Sleep(1000);//sleep a sec!
continue;
}
Note the comment above... bytesdecompressed is always 0. Any ideas? Am I doing it wrong?
EDIT 2:
So this is weird. I added the following debug code to the decompression routine:
GZipStream zip = new GZipStream(zipstream, CompressionMode.Decompress);
byte[] xmlbits = new byte[unit.complength];
int offset = 0;
while (zip.CanRead && offset < xmlbits.Length)
{
while (zip.Read(xmlbits, offset, 1) == 0) ;
offset++;
}
When debugging, sometimes that loop would complete, but other times it would hang. When I'd stop the debugging, it would be at byte 1600 out of 1616. I'd continue, but it wouldn't move at all.
EDIT 3: The bug appears to be in the compress code. For whatever reason, it is not saving all of the data. When I try to decompress the data using a third party gzip mechanism, I only get part of the original data.
I'd start a bounty, but I really don't have much reputation to give as of now :-(

Finally found the answer. The compressed data wasn't complete because GZipStream.Flush() does absolutely nothing to ensure that all of the data is out of the buffer - you need to use GZipStream.Close() as pointed out here. Of course, if you get a bad compress, it all goes downhill - if you try to decompress it, you will always get 0 returned from the Read().

I'd say this line, at least, is the most wrong:
cmd.Parameters.Add("#compressed", SqlDbType.VarBinary).Value = zipstream.GetBuffer();
MemoryStream.GetBuffer:
Note that the buffer contains allocated bytes which might be unused. For example, if the string "test" is written into the MemoryStream object, the length of the buffer returned from GetBuffer is 256, not 4, with 252 bytes unused. To obtain only the data in the buffer, use the ToArray method.
It should be noted that in the zip format, it first works by locating data stored at the end of the file - so if you've stored more data than was required, the required entries at the "end" of the file don't exist.
As an aside, I'd also recommend a different name for your compressedlength column - I'd initially taken it (despite your narrative) as being intended to store, well, the length of the compressed data (and written part of my answer to address that). Maybe originalLength would be a better name?

FileStream error - The process cannot access the file...used by another process

I have created a method to read from my database, the database has a blob field that stores images.
This method is used to read the Image as well as every other field.
When i run my Application it works and the form displays all the details, but if i close the form and reopen it again, i end up with this error.
IO Exception was unhandled
An unhandled exception of type 'System.IO.IOException' occurred in mscorlib.dll
Additional information: The process cannot access the file 'C:\Users\MyName\Documents\Visual Studio 2013\Projects\MyApp\MyApp\bin\Debug\pic0jpg' because it is being used by another process."
I've tried putting the filestream object in a using statement and other solutions but the error still exists.
Any help would be appreciated, thanks.
public void loadPlane()
{
//convert picture to jpg and load other details
MySqlDataAdapter sqladapt;
DataSet dataSet;
DatabaseConnector dbcon = new DatabaseConnector();
dbcon.OpenConnection();
sqladapt = new MySqlDataAdapter();
sqladapt.SelectCommand = dbcon.InitSqlCommand("SELECT * FROM plane");
dataSet = new DataSet("dst");
sqladapt.Fill(dataSet);
dbcon.CloseConnection();
DataTable tableData = dataSet.Tables[0];
//use from iplane class
IPlane.countPlane = tableData.Rows.Count;
for (int i = 0; i < IPlane.countPlane; i++)
{
IPlane.planeObj[i] = new NormalPlane();
DataRow drow = tableData.Rows[i];
string plName = "pic" + Convert.ToString(i);
FileStream FSNew = new FileStream(plName+"jpg",FileMode.Create);
byte[] blob = (byte[])drow[5];
FSNew.Write(blob,0,blob.Length);
FSNew.Close();
FSNew = null;
IPlane.planeObj[i].PlaneImage=(Image.FromFile(plName + "jpg"));
IPlane.planeObj[i].ModelNumber = drow[0].ToString();
IPlane.planeObj[i].EngineType = drow[1].ToString();
IPlane.planeObj[i].FlySpeed = Convert.ToInt32( drow[2]);
IPlane.planeObj[i].SpecialFeature = drow[3].ToString();
IPlane.planeObj[i].Price = Convert.ToDouble( drow[4]);
IPlane.planeObj[i].EconSeats = Convert.ToInt32(drow[6]);
IPlane.planeObj[i].BussSeats = Convert.ToInt32(drow[7]);
IPlane.planeObj[i].FirstSeats = Convert.ToInt32(drow[8]);
//drow 5 is the image
}
}

this is the main problem:
IPlane.planeObj[i].PlaneImage=(Image.FromFile(plName + "jpg"));
you are opening the image and leaving it opened.
You have 2 way to go:
create the image with a memory stream from the blob and not as a file
create the image as file and use the path of the image as your property IPlane.planeObj[i].PlaneImage and load the file when you need it. (I don't understad why you save an image as blob on db and then create the file anyway)
If you don't have many image or they are not so big or you don't have memory problems I think the first is better.
by the way your code has a lots of resources not disposed (connection, streams, etc etc)

You really should be disposing of your DatabaseConnector after closing it. The best practice for this would be to use a Using Statement:
using (DatabaseConnector dbcon = new DatabaseConnector)
{
// Code that uses connection.
}
Alternatively, just call dbcon.Dispose() after you call .CloseConnection()
This will ensure ALL resources used by DatabaseConnection are released.

As #giammin noted, you are opening a file but never close it. The code also creates a lot of resources that should be disposed.
Moreover, you don't need to save the file at all unless you want to use it at a later time. You can create an Image object from a stream using Image.FromStream. You can wrap the buffer in a MemoryStream and pass it to Image.FromStream, e.g:
using(var stream=new MemoryStream(drow[5]))
{
var image=Image.FromStream(stream);
Plane.planeObj[i].PlaneImage = image;
}

Saving and retrieving of images from SQL Server database using c#

I have created a web page where I have used Image control to display the image. Along with this control I have some labels and input box also
Below is the code which I have used to save the image to database
byte[] imageSize = new byte[FileUpload1.PostedFile.ContentLength];
HttpPostedFile uploadedImage = FileUpload1.PostedFile;
uploadedImage.InputStream.Read(imageSize, 0, (int)FileUpload1.PostedFile.ContentLength);
SqlParameter UploadedImage = new SqlParameter("#image", SqlDbType.Image, imageSize.Length);
UploadedImage.Value = imageSize;
string sql = "insert into imageDB(Image) values (#image)";
using (SqlCommand cmd = new SqlCommand(sql, conn))
{
cmd.Parameters.Add(UploadedImage);
cmd.ExecuteNonQuery();
}
and below is the code I have used to retrieve the image from database
byte[] rawImg = (byte[])rdr["image"];
MemoryStream Stream = new MemoryStream();
Stream.Write(rawImg, 0, rawImg.Length);
Bitmap Display_Picture = new Bitmap(Stream);
//after this no idea how to proceed
I have read some links all are suggesting we can not set this byte information to Image control.
Let me know if my way of retrieving the image from data base is right, if its right, what type of control I should use, so that image which has been retrieved from database can be displayed on web page

Is my way of retrieving the image from database right?
I would use the following code instead (sorry not tested):
byte[] imageBytes = Convert.FromBase64String(rdr["image"]);
MemoryStream stream = new MemoryStream(imageBytes);
Image image = Image.FromStream(stream);
How do I bind the image to an asp.net image control?
I would create an HttpHandler which gets and returns the image. Then bind the ImageUrl property on the asp:Image to the url of the HttpHandler.
How you can do this you can see in the answer of Dale Ragan over here: How to bind a MemoryStream to asp:image control?

First, in your example your have to reset the Position inside your stream to 0: Stream.Position = 0;
Second, you have to write your image to a local cache folder of your web service that is part of your web application. You image control can have a Source-reference (img.src) to the address of that image.
Or... you can create an AXD-file / HttpHandler. An AXD file is a file, just like an ASPX, but specialized in returning any type of data, like an image.
For more info, see: http://blog.kurtschindler.net/post/using-httphandlers-to-serve-image-files

To retrieve image, and show on asp.net Image control, you can
cn.Open();
SqlCommand cm = new SqlCommand("select * from ImageCollection where img_id='" + DropDownList1.SelectedItem.ToString() + "'", cn);
SqlDataAdapter da = new SqlDataAdapter(cm);
SqlDataReader dr = cm.ExecuteReader();
try
{
if (dr.Read())
{
string image1 = Convert.ToString(DateTime.Now.ToFileTime());
FileStream fs1 = new FileStream(image1, FileMode.CreateNew, FileAccess.Write);
byte[] bimage1 = (byte[])dr["passport_photo"];
fs1.Write(bimage1, 0, bimage1.Length - 1);
fs1.Flush();
Image1.ImageUrl = "~/images/"+DropDownList1.SelectedItem.ToString();
}
dr.Close();
cn.Close();
}
catch (Exception ex)
{
throw ex;
}
Alternative solution:
Implement a handler to serve the image. Have a look at the System.Web.IHttpHandler interface.
The idea would be to write an image tag in your page along the lines of:
<asp:image runat="server" imageurl="~/Image.ashx?ID=xxx" />
Then in your handler implementation, get the id from the querystring, retrieve the image as you're already doing and then write directly to the response stream.

Load or convert varbinary from sql to textbox text?

I'm new one in C# programming.
At present I try to save and load a text file to sql server. I watched and searched some videos and document on youtube. They stored image file to sql. I try and already saved the text file to server (binary data). Now I stuck how to load or convert it:
When I save it, I used memorystream:
MemoryStream ms = new MemoryStream(Encoding.UTF8.GetBytes(textreview.Text);
byte[] theBytes = Encoding.UTF8.GetBytes(textreview.Text);
ms.Position = 0;
ms.Read(theBytes, 0, theBytes.Length);
//insert value to server
SqlCommand cmdb = new SqlCommand("insert into Assignment(text) values (#text)", con);
cmdb.Parameters.AddWithValue("#textname", textPathLabel.Text);
cmdb.Parameters.AddWithValue("#text", theBytes);
Now I try to load it but I stuck here!
//load from sql
byte[] picarr = (byte[])dr["text"];
ms = new MemoryStream(picarr);
ms.Seek(0, SeekOrigin.Begin);
StreamReader reader = new StreamReader(ms);
string text = reader.ReadToEnd();
textreview.Text = text;

Here's an example on how you convert a text to bytes and then back again:
var bytes = Encoding.UTF8.GetBytes("test");
This will give you a byte array that looks like this:
{ 116, 101, 115, 116 }
Now to get the text again, you can use Encoding.UTF8.GetString() like this:
Encoding.UTF8.GetString(bytes);
This means that you could simple do this when saving the value:
cmdb.Parameters.AddWithValue("#text", Encoding.UTF8.GetBytes(textreview.Text));
And then when loading it back you simply do this:
textreview.Text = Encoding.GetString((byte[])dr["text"]);

OutOfMemory exception when loading an image in .Net

Im loading an image from a SQL CE db and then trying to load that into a PictureBox.
I am saving the image like this:
if (ofd.ShowDialog() == DialogResult.OK)
{
picArtwork.ImageLocation = ofd.FileName;
using (System.IO.FileStream fs = new System.IO.FileStream(ofd.FileName, System.IO.FileMode.Open))
{
byte[] imageAsBytes = new byte[fs.Length];
fs.Read(imageAsBytes, 0, imageAsBytes.Length);
thisItem.Artwork = imageAsBytes;
fs.Close();
}
}
and then saving to the Db using LINQ To SQL.
I load the image back like so:
using (FileStream fs = new FileStream(#"C:\Temp\img.jpg", FileMode.CreateNew ,FileAccess.Write ))
{
byte[] img = (byte[])encoding.GetBytes(ThisFilm.Artwork.ToString());
fs.Write(img, 0, img.Length);
}
but am getting an OutOfMemoryException. I have read that this is a slight red herring and that there is probably something wrong with the filetype, but i cant figure what.
Any ideas?
Thanks
picArtwork.Image = System.Drawing.Bitmap.FromFile(#"C:\Temp\img.jpg");

Based on the code you provided it seems like you are treating the image as a string, it might be that data is being lost with the conversion from byte[] to string and string to byte[].
I am not familiar with SQL CE, but if you can you should consider treating the data as a byte[] and not encoding to and from string.
I base my assumption in this line of code
byte[] img = (byte[])encoding.GetBytes(ThisFilm.Artwork.ToString());

The GDI has the bad behavior of throwing an OOM exception whenever it is not capable of understanding a certain image format. Use Irfanview to see what kind of image that really is, it is likely GDI can't handle it.

My advice would be to not use a FileStream to load images unless you need access to the raw bytes. Instead, use the static methods provided in Bitmap or Image objects:
Image img = Image.FromFile(#"c:\temp\img.jpg")
Bitmap bmp = Bitmap.FromFile(#"c:\temp\img.jpg")
To save the file use .Save method of the Image or Bitmap object:
img.Save(#"c:\temp\img.jpg")
bmp.Save(#"c:\temp\img.jpg")
Of course we don't know what type ArtWork is. Would you care to share that information with us?