We Keep Coding

Problems uploading a binary serialized file

The idea of my program is to allow the user to save his data in a cloud. The data is stored in binary serialized files.
Serialization Code looks like so:
FileStream Stream = new FileStream(..., FileMode.Create); // create stream
BinaryFormatter Formatter = new BinaryFormatter(); // create formatter
Formatter.Serialize(Stream, ObjectToSerialize); // serialize
Stream.Close();
The Problem is that when I upload the serialized file, with the following code:
FtpWebRequest Request = (FtpWebRequest)WebRequest.Create(new Uri(...));
Request.Method = WebRequestMethods.Ftp.UploadFile;
Request.Credentials = new NetworkCredential(..., ...);
string Path = System.IO.Path.GetDirectoryName(...;
StreamReader SourceStream = new StreamReader(Path);
Request.UseBinary = true;
byte[] FileContents = Encoding.Default.GetBytes(SourceStream.ReadToEnd());
SourceStream.Close();
Request.ContentLength = FileContents.Length;
Stream RequestStream = Request.GetRequestStream();
RequestStream.Write(FileContents, 0, FileContents.Length);
RequestStream.Close();
FtpWebResponse Response = (FtpWebResponse)Request.GetResponse();
Response.Close();
the file encoding changes (I am not sure about that.. if the encoding is the problem but I think so) and I am not able to deserialize the file anymore, exceptions occure. Do you have a idea how to do not damage the file?
Best Regards!

Encoding only applies to text, and you're dealing with a binary file here. You should not attempt to read it as a string. The "encoding changed" because you are using Encoding.Default which depends on the current configuration of your operating system.
Just change the code to:
string Path = System.IO.Path.GetDirectoryName(...);
// StreamReader SourceStream = new StreamReader(Path);
// Request.UseBinary = true;
byte[] FileContents = File.ReadAllBytes(Path);
// SourceStream.Close();
if you want to read an entire binary file in memory.

JSON truncated when compressing HTTP responses

When I apply gzip or deflate compression to my HTTP responses, I seem to be losing the last bracket in my JSON structures. For example:
Result without compression:
{"alist":{"P_1":0,"P_2":0,"P_3":0}}
Result with compression as received by the browser:
{"alist":{"P_1":0,"P_2":0,"P_3":0}
When writing the response without compression I am doing the following:
byte[] buffer = Encoding.UTF8.GetBytes(responseContent);
context.Response.ContentLength64 = buffer.Length;
context.Response.ContentType = ContentTypeJson;
Stream outputStream = context.Response.OutputStream;
outputStream.Write(buffer, 0, buffer.Length);
outputStream.Close();
Alternatively, when the caller provides an Accept-Encoding request header, I try and write the response with compression as follows;
byte[] buffer = Encoding.UTF8.GetBytes(responseContent);
byte[] compressedBuffer;
using (var memoryStream = new MemoryStream())
{
using (Stream compressionStream = new DeflateStream(memoryStream, CompressionMode.Compress, false))
{
compressionStream.Write(buffer, 0, buffer.Length);
compressedBuffer = memoryStream.ToArray();
compressionStream.Close();
}
memoryStream.Close();
}
context.Response.ContentLength64 = compressedBuffer.Length;
context.Response.ContentType = ContentTypeJson;
Stream outputStream = context.Response.OutputStream;
outputStream.Write(compressedBuffer, 0, compressedBuffer.Length);
outputStream.Close();
If it helps, I am using an System.Net.HttpListener which is why I have to do this myself. Does anyone have any idea why this truncation may be occuring?

DeflateStream doesn't write to its output stream everything immediately after you write into it, but you can be sure it has done so after you close it. So the following will work:
compressionStream.Write(buffer, 0, buffer.Length);
compressionStream.Close();
compressedBuffer = memoryStream.ToArray();

How to compress a HttpWebRequest POST

I am trying to post data to server that accepts compressed data. The code below works just fine, but it is uncompressed. I have not worked with compression or Gzip beofre, so any help is appriciated.
HttpWebRequest request = WebRequest.Create(uri) as HttpWebRequest;
request.Timeout = 600000;
request.Method = verb; // POST
request.Accept = "text/xml";
if (!string.IsNullOrEmpty(data))
{
request.ContentType = "text/xml";
byte[] byteData = UTF8Encoding.UTF8.GetBytes(data);
request.ContentLength = byteData.Length;
// Here is where I need to compress the above byte array using GZipStream
using (Stream postStream = request.GetRequestStream())
{
postStream.Write(byteData, 0, byteData.Length);
}
}
XmlDocument xmlDoc = new XmlDocument();
HttpWebResponse response = null;
StreamReader reader = null;
try
{
response = request.GetResponse() as HttpWebResponse;
reader = new StreamReader(response.GetResponseStream());
xmlDoc.LoadXml(reader.ReadToEnd());
}
Do I gzip the entire byte array? Do I need to add other headers or remove the one that is already there?
Thanks!
-Scott

To answer the question you asked, to POST compressed data, all you need to do is wrap the request stream with a gzip stream
using (Stream postStream = request.GetRequestStream())
{
using(var zipStream = new GZipStream(postStream, CompressionMode.Compress))
{
zipStream.Write(byteData, 0, byteData.Length);
}
}
This is completely different than requesting a gzip response, which is a much more common thing to do.

I also received the "Cannot close stream until all bytes are written" error using code similar to tnyfst's. The problem was that I had:
request.ContentLength = binData.Length;
where binData is my raw data before the compression. Obviously the length of the compressed content would be different, so I just removed this line and ended up with this code:
using (GZipStream zipStream = new GZipStream(request.GetRequestStream(), CompressionMode.Compress))
{
zipStream.Write(binData, 0, binData.Length);
}

In Page_Load event:
Response.AddHeader("Content-Encoding", "gzip");
And for making compressed requests:
HttpWebRequest and GZip Http Responses by Rick Strahl

Try this extension method.
The stream will be left open (see the GZipStream constructor).
The stream position is set to 0 after the compression is done.
public static void GZip(this Stream stream, byte[] data)
{
using (var zipStream = new GZipStream(stream, CompressionMode.Compress, true))
{
zipStream.Write(data, 0, data.Length);
}
stream.Position = 0;
}
You can use the following test:
[Test]
public void Test_gzip_data_is_restored_to_the_original_value()
{
var stream = new MemoryStream();
var data = new byte[]{1,2,3,4,5,6,7,8,9,10};
stream.GZip(data);
var decompressed = new GZipStream(stream, CompressionMode.Decompress);
var data2 = new byte[10];
decompressed.Read(data2,0,10);
Assert.That(data, Is.EqualTo(data2));
}
For more information see: http://msdn.microsoft.com/en-us/library/hh158301(v=vs.110).aspx

How to stream video content in asp.net?

I have the following code which downloads video content:
WebRequest wreq = (HttpWebRequest)WebRequest.Create(url);
using (HttpWebResponse wresp = (HttpWebResponse)wreq.GetResponse())
using (Stream mystream = wresp.GetResponseStream())
{
using (BinaryReader reader = new BinaryReader(mystream))
{
int length = Convert.ToInt32(wresp.ContentLength);
byte[] buffer = new byte[length];
buffer = reader.ReadBytes(length);
Response.Clear();
Response.Buffer = false;
Response.ContentType = "video/mp4";
//Response.BinaryWrite(buffer);
Response.OutputStream.Write(buffer, 0, buffer.Length);
Response.End();
}
}
But the problem is that the whole file downloads before being played. How can I make it stream and play as it's still downloading? Or is this up to the client/receiver application to manage?

You're reading the entire file into a single buffer, then sending the entire byte array at once.
You should read into a smaller buffer in a while loop.
For example:
byte[] buffer = new byte[4096];
while(true) {
int bytesRead = myStream.Read(buffer, 0, buffer.Length);
if (bytesRead == 0) break;
Response.OutputStream.Write(buffer, 0, bytesRead);
}

This is more efficient for you especially if you need to stream a video from a file on your server or even this file is hosted at another server
File On your server:
context.Response.BinaryWrite(File.ReadAllBytes(HTTPContext.Current.Server.MapPath(_video.Location)));
File on external server:
var wc = new WebClient();
context.Response.BinaryWrite(wc.DownloadData(new Uri("http://mysite/video.mp4")));

Have you looked at Smooth Streaming?
Look at sample code here

How to convert Video to byte Array in C#?

I am using c# .net compact framework 3.5 and I want to convert a video file to byte array so that I may upload it on the server.
In the similar manner I am doing the image uploading which is getting the success result.
HttpWebRequest request;
request.ContentType = "image/jpeg";
request.ContentLength = byteArray.Length;
request.Method = "PUT";
imageToByteArray(img).CopyTo(byteArray, 0);
using (Stream requestStream = request.GetRequestStream())
{
requestStream.Write(byteArray, 0, (int)Fs.Length);
requestStream.Flush();
requestStream.Close();
}
public byte[] imageToByteArray(Image imageIn)
{
MemoryStream ms = new MemoryStream();
imageIn.Save(ms,System.Drawing.Imaging.ImageFormat.Jpeg);
return ms.ToArray();
}
How to do this for the video files?

You should copy the stream one block at a time instead of reading the entire file into an array. Otherwise, you'll use a potentially very large amount of memory as video files can grow quite big.
For example:
HttpWebRequest request;
request.Method = "PUT";
using(Stream requestStream = request.GetRequestStream())
using(Stream video = File.OpenRead("Path")) {
byte[] buffer = new byte[4096];
while(true) {
int bytesRead = video.Read(buffer, 0, buffer.Length);
if (bytesRead == 0) break;
requestStream.Write(buffer, 0, bytesRead);
}
}