I am trying to post data to server that accepts compressed data. The code below works just fine, but it is uncompressed. I have not worked with compression or Gzip beofre, so any help is appriciated.
HttpWebRequest request = WebRequest.Create(uri) as HttpWebRequest;
request.Timeout = 600000;
request.Method = verb; // POST
request.Accept = "text/xml";
if (!string.IsNullOrEmpty(data))
{
request.ContentType = "text/xml";
byte[] byteData = UTF8Encoding.UTF8.GetBytes(data);
request.ContentLength = byteData.Length;
// Here is where I need to compress the above byte array using GZipStream
using (Stream postStream = request.GetRequestStream())
{
postStream.Write(byteData, 0, byteData.Length);
}
}
XmlDocument xmlDoc = new XmlDocument();
HttpWebResponse response = null;
StreamReader reader = null;
try
{
response = request.GetResponse() as HttpWebResponse;
reader = new StreamReader(response.GetResponseStream());
xmlDoc.LoadXml(reader.ReadToEnd());
}
Do I gzip the entire byte array? Do I need to add other headers or remove the one that is already there?
Thanks!
-Scott
To answer the question you asked, to POST compressed data, all you need to do is wrap the request stream with a gzip stream
using (Stream postStream = request.GetRequestStream())
{
using(var zipStream = new GZipStream(postStream, CompressionMode.Compress))
{
zipStream.Write(byteData, 0, byteData.Length);
}
}
This is completely different than requesting a gzip response, which is a much more common thing to do.
I also received the "Cannot close stream until all bytes are written" error using code similar to tnyfst's. The problem was that I had:
request.ContentLength = binData.Length;
where binData is my raw data before the compression. Obviously the length of the compressed content would be different, so I just removed this line and ended up with this code:
using (GZipStream zipStream = new GZipStream(request.GetRequestStream(), CompressionMode.Compress))
{
zipStream.Write(binData, 0, binData.Length);
}
In Page_Load event:
Response.AddHeader("Content-Encoding", "gzip");
And for making compressed requests:
HttpWebRequest and GZip Http Responses by Rick Strahl
Try this extension method.
The stream will be left open (see the GZipStream constructor).
The stream position is set to 0 after the compression is done.
public static void GZip(this Stream stream, byte[] data)
{
using (var zipStream = new GZipStream(stream, CompressionMode.Compress, true))
{
zipStream.Write(data, 0, data.Length);
}
stream.Position = 0;
}
You can use the following test:
[Test]
public void Test_gzip_data_is_restored_to_the_original_value()
{
var stream = new MemoryStream();
var data = new byte[]{1,2,3,4,5,6,7,8,9,10};
stream.GZip(data);
var decompressed = new GZipStream(stream, CompressionMode.Decompress);
var data2 = new byte[10];
decompressed.Read(data2,0,10);
Assert.That(data, Is.EqualTo(data2));
}
For more information see: http://msdn.microsoft.com/en-us/library/hh158301(v=vs.110).aspx
Related
I sended a WebRequest, and got as Response a String with XML Code within my content.
WebRequest request = WebRequest.Create("...");
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream dataStream = response.GetResponseStream();
StreamReader reader = new StreamReader(dataStream);
string responseFromServer = reader.ReadToEnd();
byte[] bytes = Convert.FromBase64String(responseFromServer);
Image image;
using (MemoryStream ms = new MemoryStream(bytes))
{
image = Image.FromStream(ms);
}
image.Save("File", System.Drawing.Imaging.ImageFormat.Gif);
The Problem is that "Convert.FromBase64String(responseFromServer)" cant convert to base 64, because the response is:
"<string xmlns="*LINK*"> 'content' </string>"
Can I remove these Tags, or copy out the content?
I hope you got my problem:) Thanks for help and sorry for the bad english x)
you can use XmlDocument..
WebRequest request = WebRequest.Create("...");
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream dataStream = response.GetResponseStream();
StreamReader reader = new StreamReader(dataStream);
XmlDocument doc = new XmlDocument();
doc.Load(reader);
byte[] bytes = Convert.FromBase64String(doc.GetElementsByTagName("string")[0].InnerText);
System.Drawing.Image image;
using (MemoryStream ms = new MemoryStream(bytes))
{
image = System.Drawing.Image.FromStream(ms);
}
I'd recommend using XmlReader for this.
WebRequest request = WebRequest.Create("...");
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream dataStream = response.GetResponseStream();
//StreamReader reader = new StreamReader(dataStream);
//string responseFromServer = reader.ReadToEnd();
//byte[] bytes = Convert.FromBase64String(responseFromServer);
using (var reader = XmlReader.Create(dataStream))
{
reader.Read();
reader.Read();
string base64 = reader.Value;
byte[] bytes = Convert.FromBase64String(base64);
Image image;
using (MemoryStream ms = new MemoryStream(bytes))
{
image = Image.FromStream(ms);
image.Save("File", System.Drawing.Imaging.ImageFormat.Gif);
}
}
There is a method for converting from Base64 in the element value directly, but you'd need to know things that more or less require you to read your whole stream anyway to do that (you can look at the edit history to see me working through that ;).
When I apply gzip or deflate compression to my HTTP responses, I seem to be losing the last bracket in my JSON structures. For example:
Result without compression:
{"alist":{"P_1":0,"P_2":0,"P_3":0}}
Result with compression as received by the browser:
{"alist":{"P_1":0,"P_2":0,"P_3":0}
When writing the response without compression I am doing the following:
byte[] buffer = Encoding.UTF8.GetBytes(responseContent);
context.Response.ContentLength64 = buffer.Length;
context.Response.ContentType = ContentTypeJson;
Stream outputStream = context.Response.OutputStream;
outputStream.Write(buffer, 0, buffer.Length);
outputStream.Close();
Alternatively, when the caller provides an Accept-Encoding request header, I try and write the response with compression as follows;
byte[] buffer = Encoding.UTF8.GetBytes(responseContent);
byte[] compressedBuffer;
using (var memoryStream = new MemoryStream())
{
using (Stream compressionStream = new DeflateStream(memoryStream, CompressionMode.Compress, false))
{
compressionStream.Write(buffer, 0, buffer.Length);
compressedBuffer = memoryStream.ToArray();
compressionStream.Close();
}
memoryStream.Close();
}
context.Response.ContentLength64 = compressedBuffer.Length;
context.Response.ContentType = ContentTypeJson;
Stream outputStream = context.Response.OutputStream;
outputStream.Write(compressedBuffer, 0, compressedBuffer.Length);
outputStream.Close();
If it helps, I am using an System.Net.HttpListener which is why I have to do this myself. Does anyone have any idea why this truncation may be occuring?
DeflateStream doesn't write to its output stream everything immediately after you write into it, but you can be sure it has done so after you close it. So the following will work:
compressionStream.Write(buffer, 0, buffer.Length);
compressionStream.Close();
compressedBuffer = memoryStream.ToArray();
I was trying to convert an Url to Stream but I am not sure whether I am right or wrong.
protected Stream GetStream(String gazouUrl)
{
Stream rtn = null;
HttpWebRequest aRequest = (HttpWebRequest)WebRequest.Create(gazouUrl);
HttpWebResponse aResponse = (HttpWebResponse)aRequest.GetResponse();
using (StreamReader sReader = new StreamReader(aResponse.GetResponseStream(), System.Text.Encoding.Default))
{
rtn = sReader.BaseStream;
}
return rtn;
}
Am I on the right track?
I ended up doing a smaller version and using WebClient instead the old Http Request code:
private static Stream GetStreamFromUrl(string url)
{
byte[] imageData = null;
using (var wc = new System.Net.WebClient())
imageData = wc.DownloadData(url);
return new MemoryStream(imageData);
}
You don't need to create a StreamReader there. Just return aResponse.GetResponseStream();. The caller of that method will also need to call Dispose on the stream when it's done.
The current answer is missing an example in how to use GetResponseStream()
Here is an example
// Creates an HttpWebRequest with the specified URL.
HttpWebRequest myHttpWebRequest = (HttpWebRequest)WebRequest.Create(url);
// Sends the HttpWebRequest and waits for the response.
HttpWebResponse myHttpWebResponse = (HttpWebResponse)myHttpWebRequest.GetResponse();
// Gets the stream associated with the response.
Stream receiveStream = myHttpWebResponse.GetResponseStream();
Encoding encode = System.Text.Encoding.GetEncoding("utf-8");
// Pipes the stream to a higher level stream reader with the required encoding format.
StreamReader readStream = new StreamReader( receiveStream, encode );
Console.WriteLine("\r\nResponse stream received.");
Char[] read = new Char[256];
// Reads 256 characters at a time.
int count = readStream.Read( read, 0, 256 );
Console.WriteLine("HTML...\r\n");
while (count > 0)
{
// Dumps the 256 characters on a string and displays the string to the console.
String str = new String(read, 0, count);
Console.Write(str);
count = readStream.Read(read, 0, 256);
}
Console.WriteLine("");
// Releases the resources of the response.
myHttpWebResponse.Close();
// Releases the resources of the Stream.
readStream.Close();
For more details see - https://learn.microsoft.com/en-us/dotnet/api/system.net.httpwebresponse.getresponsestream?view=net-5.0
I’m using the following code to download a file from a remote ftp server:
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(serverPath);
request.KeepAlive = true;
request.UsePassive = true;
request.UseBinary = true;
request.Method = WebRequestMethods.Ftp.DownloadFile;
request.Credentials = new NetworkCredential(userName, password);
using (FtpWebResponse response = (FtpWebResponse)request.GetResponse())
using (Stream responseStream = response.GetResponseStream())
using (StreamReader reader = new StreamReader(responseStream))
using (StreamWriter destination = new StreamWriter(destinationFile))
{
destination.Write(reader.ReadToEnd());
destination.Flush();
}
The file that I’m downloading is a dll and my problem is that it is being altered by this process in some way. I know this because the file size is increasing. I have a suspicion that this section of code is at fault:
destination.Write(reader.ReadToEnd());
destination.Flush();
Can anyone offer any ideas as to what may be wrong?
StreamReader and StreamWriter work with character data, so you are decoding the stream from bytes to characters and then encoding it back to bytes again. A dll file contains binary data, so this round-trip conversion will introduce errors. You want to read bytes directly from the responseStream object and write to a FileStream that isn't wrapped in a StreamWriter.
If you are using .NET 4.0 you can use Stream.CopyTo, but otherwise you will have to copy the stream manually. This StackOverflow question has a good method for copying streams:
public static void CopyStream(Stream input, Stream output)
{
byte[] buffer = new byte[32768];
while (true)
{
int read = input.Read(buffer, 0, buffer.Length);
if (read <= 0)
return;
output.Write(buffer, 0, read);
}
}
So, your code will look like this:
using (FtpWebResponse response = (FtpWebResponse)request.GetResponse())
using (Stream responseStream = response.GetResponseStream())
using (FileStream destination = File.Create(destinationFile))
{
CopyStream(responseStream, destination);
}
I am using c# .net compact framework 3.5 and I want to convert a video file to byte array so that I may upload it on the server.
In the similar manner I am doing the image uploading which is getting the success result.
HttpWebRequest request;
request.ContentType = "image/jpeg";
request.ContentLength = byteArray.Length;
request.Method = "PUT";
imageToByteArray(img).CopyTo(byteArray, 0);
using (Stream requestStream = request.GetRequestStream())
{
requestStream.Write(byteArray, 0, (int)Fs.Length);
requestStream.Flush();
requestStream.Close();
}
public byte[] imageToByteArray(Image imageIn)
{
MemoryStream ms = new MemoryStream();
imageIn.Save(ms,System.Drawing.Imaging.ImageFormat.Jpeg);
return ms.ToArray();
}
How to do this for the video files?
You should copy the stream one block at a time instead of reading the entire file into an array. Otherwise, you'll use a potentially very large amount of memory as video files can grow quite big.
For example:
HttpWebRequest request;
request.Method = "PUT";
using(Stream requestStream = request.GetRequestStream())
using(Stream video = File.OpenRead("Path")) {
byte[] buffer = new byte[4096];
while(true) {
int bytesRead = video.Read(buffer, 0, buffer.Length);
if (bytesRead == 0) break;
requestStream.Write(buffer, 0, bytesRead);
}
}