We Keep Coding

File corrupted on download from Controller Action

I am generating some JSON content, and then GZipping that content, before returning the gzipped content to the user, from an MVC Controller Action.
The generation of the content, and gzipping, is working correctly as I can output the generated file to disk, and then I can open that file using GZip. However, when the content is returned to the browser, the content has been corrupted.
I have tried several different approaches to returning the content to the browser, such as
return File(byte[], "application/gzip");
return new FileStreamResult(stream, "application/gzip")
And also writing directly to the Response using BinaryWrite() and WriteFile() methods
No matter what I do, the file I receive in the browser is corrupt.
This code shows the manner in which I am currently trying to return the file content.
// This line writes my content byte[] array to disk. This file when opened with gzip works fine.
System.IO.File.WriteAllBytes(#"C:\temp\test.vcp", result.FileBytes);
// Writing out the byte array to the Response results in a corrupt file. I have also attempted to Response.WriteFile(#"C:\temp\test.vcp") which also results in a corrupt file.
Response.Clear();
Response.ContentType = "application/gzip";
Response.AppendHeader("Content-Disposition", cd.ToString());
Response.AddHeader("Content-Length", result.FileBytes.Length.ToString());
Response.BinaryWrite(result.FileBytes);
Response.Flush();
Response.Close();
Response.End();
As the file I am creating can be written to disk, and can be read using Gzip, but the file received by the browser is corrupt, I am confident that my file creation is OK. But somehow after writing the file to the Response, it is being corrupted.
I did wonder if maybe some sort of HTTPHandler is manipulating the result, but I haven't added any Handlers (that I can see).
I am running the application locally currently through IISExpress. How can I check what HttpHandlers/HttpModules are being applied to the pipeline?
Ultimately I expect to receive the exact same file in my browser as is written to disk.
For reference, my generated content is 132 bytes in length, but the browser receives 216 bytes. I have noticed when looking at the byte structure of the received data, there is a repeating pattern of 3 bytes in the content, with the values 239, 191, 189. It almost looks like the resultant byte array has been stuffed or padded with these 3 bytes.
EDIT
Here is a standalone Action method which demonstrated the issue.
[HttpGet]
public void GetFile()
{
byte[] text = Encoding.ASCII.GetBytes(#"{""PetName"":""Doggy McDocFace"",""OwnerName"":""Kurt""}");
byte[] compressed = Compress(text);
var cd = new System.Net.Mime.ContentDisposition
{
// for example foo.bak
FileName = "ExampleFile.vcp",
// always prompt the user for downloading, set to true if you want
// the browser to try to show the file inline
Inline = true,
};
System.IO.File.WriteAllBytes(#"C:\temp\ExampleFile.vcp", compressed);
Response.Clear();
Response.ContentType = "application/gzip";
Response.AppendHeader("Content-Disposition", cd.ToString());
Response.AddHeader("Content-Length", compressed.Length.ToString());
Response.BinaryWrite(compressed);
Response.Flush();
Response.Close();
Response.End();
}
public byte[] Compress(byte[] raw)
{
using (var memory = new MemoryStream())
{
using (var gzip = new GZipStream(memory, CompressionMode.Compress, true))
{
gzip.Write(raw, 0, raw.Length);
}
return memory.ToArray();
}
}
Here I am spoofing my JSON content, and then compressing it. The file written to disk works fine, and can be opened with my GZip application (I use 7-zip). However, the file received by the browser is corrupt. 7-zip cannot recognise it as a gzip file.
EDIT 2
So it looks like (Thanks to #Will) that the content when written to Response is falling foul of UTF-8 encoding. I cannot work out how though, as in my example above I am using Encoding.ASCII.GetBytes() to convert my string to a byte[] array.
I've tried setting the
Response.Charset = Encoding.ASCII.EncodingName;
Response.ContentEncoding = Encoding.ASCII;
But this still doesn't result in a valid file downloaded.
Edit 3
I've narrowed down the issue to the GZip encryption of the data. If I do not encrypt the data, then the plain text file downloads fine. However, encrypting the byte[] array and then writing that byte[] array to the Repsonse is resulting in what seems like UTF-8 encoding issues. Any bytes with a value over 127 are corrupted with the 3 bytes I mention further up. I cannot work out why the Response is treating this encrypted data in this way. My assumption is that when the Byte[] array is just plain text as a byte[] array, then this is handled fine. As soon as it is a proper byte[] array, i.e not just a string as a byte[] array, then some other conversion of encoding is going on in the Response.

You can try ActionFilterAttribute
Basically response filters look at the Response output stream as it's written and convert the data flowing through it.
GZip/Deflate Compression in ASP.NET MVC
public class CompressAttribute : ActionFilterAttribute
{
public override void OnActionExecuting(ActionExecutingContext filterContext)
{
var encodingsAccepted = filterContext.HttpContext.Request.Headers["Accept-Encoding"];
if (string.IsNullOrEmpty(encodingsAccepted)) return;
encodingsAccepted = encodingsAccepted.ToLowerInvariant();
var response = filterContext.HttpContext.Response;
if (encodingsAccepted.Contains("deflate"))
{
response.AppendHeader("Content-encoding", "deflate");
response.Filter = new DeflateStream(response.Filter, CompressionMode.Compress);
}
else if (encodingsAccepted.Contains("gzip"))
{
response.AppendHeader("Content-encoding", "gzip");
response.Filter = new GZipStream(response.Filter, CompressionMode.Compress);
}
}
}
[Compress]
[HttpGet]
public ActionResult GetFile()
{...}

Converting a byte[] string back to byte[] array

I have one scenario with class like this.
Class Document
{
public string Name {get;set;}
public byte[] Contents {get;set;}
}
Now I am trying to implement the import export functionality where I keep the document in binary so the document will be in json file with other fields and the document will be something in this format.
UEsDBBQABgAIAAAAIQCitGbRsgEAALEHAAATAAgCW0NvbnRlbnRfVHlwZXNdLnhtbCCiBAIooAACAAAAAAA==
Now when I upload this file back, I get this file as a string and I get the same data but when I try to convert this in binary bytes[] the file become corrupt.
How can I achieve this ?
I use something like this to convert
var ss = sr.ReadToEnd();
MemoryStream stream = new MemoryStream();
StreamWriter writer = new StreamWriter(stream);
writer.Write(ss);
writer.Flush();
stream.Position = 0;
var bytes = default(byte[]);
bytes = stream.ToArray();

This looks like base 64. Use:
System.Convert.ToBase64String(b)
https://msdn.microsoft.com/en-us/library/dhx0d524%28v=vs.110%29.aspx
And
System.Convert.FromBase64String(s)
https://msdn.microsoft.com/en-us/library/system.convert.frombase64string%28v=vs.110%29.aspx

You need to de-code it from base64, like this:
Assuming you've read the file into ss as a string.
var bytes = Convert.FromBase64String(ss);

There are several things going on here. You need to know the encoding for the default StreamWriter, if it is not specified it defaults to UTF-8 encoding. However, .NET strings are always either UNICODE or UTF-16.
MemoryStream from string - confusion about Encoding to use
I would suggest using System.Convert.ToBase64String(someByteArray) and its counterpart System.Convert.FromBase64String(someString) to handle this for you.

Send XML file on request

I'm trying to send a XML file on request, but I'm getting an error when I'm trying to copy the stream, which I'm loading the file into, to the output stream.
Right now it's working fine if I'm making the request (I use HttpListener btw) from a browser; it shows me my .xml just fine. But I'd also like to be able to download the .xml when I make the request.
Any suggestions?
string xString = #"C:\Src\Capabilities.xml";
XDocument capabilities = XDocument.Load(xString);
Stream stream = response.OutputStream;
response.ContentType = "text/xml";
capabilities.Save(stream);
CopyStream(stream, response.OutputStream);
stream.Close();
public static void CopyStream(Stream input, Stream output)
{
input.CopyTo(output);
}
The error I'm getting is at input.CopyTo(output); : "Stream does not support reading."

You probably get the error because the stream input actually is the response.OutputStream, which is an output stream and also makes the source and target of the copy operation the same stream - huh?
Essentially what your code does now (and this is wrong): You save the XML content to the response's output stream (which essentially already sends it to the browser). Then you try to copy the output stream into the output stream. This doesn't work and even if it did - why? You already wrote to the output stream.
You can simplify all this greatly in my opinion as follows:
// Read the XML text into a variable - why use XDocument at all?
string xString = #"C:\Src\Capabilities.xml";
string xmlText = File.ReadAllText(xString);
// Create an UTF8 byte buffer from it (assuming UTF8 is the desired encoding)
byte[] xmlBuffer = Encoding.UTF8.GetBytes(xmlText);
// Write the UTF8 byte buffer to the response stream
Stream stream = response.OutputStream;
response.ContentType = "text/xml";
response.ContentEncoding = Encoding.UTF8;
stream.Write(xmlBuffer, 0, xmlBuffer.Length);
// Done
stream.Close();

Download a PDF from a third party using ASP.NET HttpWebRequest/HttpWebResponse

I want to send a url as query string e.g.
localhost/abc.aspx?url=http:/ /www.site.com/report.pdf
and detect if the above URL returns the PDF file. If it will return PDF then it gets saved automatically otherwise it gives error.
There are some pages that uses Handler to fetch the files so in that case also I want to detect and download the same.
localhost/abc.aspx?url=http:/ /www.site.com/page.aspx?fileId=223344
The above may return a pdf file.
What is best way to capture this?
Thanks

You can download a PDF like this
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(uri);
HttpWebResponse response = req.GetResponse();
//check the filetype returned
string contentType = response.ContentType;
if(contentType!=null)
{
splitString = contentType.Split(';');
fileType = splitString[0];
}
//see if its PDF
if(fileType!=null && fileType=="application/pdf"){
Stream stream = response.GetResponseStream();
//save it
using(FileStream fileStream = File.Create(fileFullPath)){
// Initialize the bytes array with the stream length and then fill it with data
byte[] bytesInStream = new byte[stream.Length];
stream.Read(bytesInStream, 0, bytesInStream.Length);
// Use write method to write to the file specified above
fileStream.Write(bytesInStream, 0, bytesInStream.Length);
}
}
response.Close();
The fact that it may come from an .aspx handler doesn't actually matter, it's the mime returned in the server response that is used.
If you are getting a generic mime type, like application/octet-stream then you must use a more heuristical approach.
Assuming you cannot simply use the file extension (eg for .aspx), then you can copy the file to a MemoryStream first (see How to get a MemoryStream from a Stream in .NET?). Once you have a memory stream of the file, you can take a 'cheeky' peek at it (I say cheeky because it's not the correct way to parse a PDF file)
I'm not an expert on PDF format, but I believe reading the first 5 chars with an ASCII reader will yield "%PDF-", so you can identify that with
bool isPDF;
using( StreamReader srAsciiFromStream = new StreamReader(memoryStream,
System.Text.Encoding.ASCII)){
isPDF = srAsciiFromStream.ReadLine().StartsWith("%PDF-");
}
//set the memory stream back to the start so you can save the file
memoryStream.Position = 0;

Decoding base64 file contents between PHP and C#

I need to serve an AES encrypted, base64 encoded file from PHP to a C# client (Mono, on various platforms). I've successfully got the AES encryption/decryption working fine but as soon as I attempt the base64 encoding/decoding I run into trouble. Both the examples below have the AES disabled, so that shouldn't be a factor.
My simplest test case, a Hello World string, works fine:
PHP serving output-
// Save encoded data to file
$data = base64_encode("Hello encryption world!!");
$file = fopen($targetPath, 'w');
fwrite($file, $data);
fclose($file);
// Later on, serve the file
header("Pragma: public");
header("Expires: 0");
header("Cache-Control: must-revalidate, post-check=0, pre-check=0");
header("Cache-Control: private",false);
header("Content-Type: application/octet-stream");
header("Content-Disposition: attachment; filename=".basename($product->PackageFilename($packageId)));
header("Content-Transfer-Encoding: binary");
header("Content-Length: ".filesize($targetPath));
ob_clean();
flush();
$handle = fopen($targetPath, "r");
fpassthru($handle);
fclose($handle);
C# decoding and using-
StreamReader reader = new StreamReader(stream);
char[] buffer = DecodeBuffer;
string decoded = "";
int read = 0;
while (0 < (read = reader.Read(buffer, 0, DecodeBufferSize)))
{
byte[] decodedBytes = Convert.FromBase64CharArray(buffer, 0, read);
decoded += System.Text.Encoding.UTF8.GetString(decodedBytes);
}
Log(decoded); // Correctly logs "Hello encryption world!!"
However once I start trying to do the same thing with the contents of a file, a FormatException: Invalid character found is thrown by Convert.FromBase64CharArray:
PHP serving output-
// Save encoded data to file
$data = base64_encode(file_get_contents($targetPath));
$file = fopen($targetPath, 'w');
fwrite($file, $data);
fclose($file);
// Later on, serve the file
// Same as above
C# decoding and using-
using (Stream file = File.Open(zipPath, FileMode.Create))
{
using (StreamReader reader = new StreamReader(stream))
{
char[] buffer = DecodeBuffer;
byte[] decodedBytes;
int read = 0;
while (0 < (read = reader.Read(buffer, 0, DecodeBufferSize)))
{
// Throws FormatException: Invalid character found
decodedBytes = Convert.FromBase64CharArray(buffer, 0, read);
file.Write(decodedBytes, 0, decodedBytes.Length);
}
}
}
Is there some kind of additional processing that should be done on larger data for base64 to be valid? Is it perhaps just not appropriate to be doing this with large binary data-and if so how else would you prevent potential problems with characters unsafe for transmission?

Your reading Base64 text code is not correct.
Base64 is text, so consider using text reader instead
Base64 may contain new lines/white spaces. I.e. it is custom to have split whole Base64 encoded value into 70-80 character long strings.
To verify if data in the file is correct read whole file as string (StreamReader.ReadToEnd) and convert to byte array (Convert.FromBase64String).
If file contains valid Base64 data and you can't read it as single string you should implement your own Base64 decoding or manually read correct number of non white space characters (multiple of 4) and decode such chunks.

Base64 encoding converts 3 octets into 4 encoded characters. Thus, the length of the data you provide for decoding needs to be a multiple of 4.
First, ensure that DecodeBufferSize is such a multiple of 4. Next, since StreamReader.Read does not guarantee that all the requested bytes will be read, you should continue reading into buffer until either it has been filled, or the end of the stream been reached.