We Keep Coding

Streamed File Contains Strange Characters - An Encoding Issue

I have a WCF service end-point which generates an excel file and returns this file as a MemoryStream in the end in order to make client download the relevant file.
The file generated on the respective directory has no issues. I don't see any strange characters when I open and check it.
But, the file I returned with MemoryStream is full of strange unreadable characters.
My end-point is like that,
public Stream GetEngagementFeedFinalizeData(int workspaceId, string startDate, string endDate, Stream data)
{
try
{
string contentType = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet;";
string extension = "xls";
string fileName = "report-" + DateTime.Now.Ticks.ToString();
string contentDisposition = string.Format(CultureInfo.InvariantCulture, "attachment; filename={0}.{1}", fileName, extension);
WebOperationContext.Current.OutgoingResponse.ContentType = contentType;
WebOperationContext.Current.OutgoingResponse.Headers.Set("Content-Disposition", contentDisposition);
//Here is some business logic and fetching data from db. Not any encoding
//related issue. The data set is assigned to a variable
//named "feedFinalizeDataTable" in the end
feedFinalizeDataTable.TableName = "Summary";
DataSet dataSet = new DataSet();
dataSet.Tables.Add(feedFinalizeDataTable);
using (ExcelPackage excelPackage = new ExcelPackage())
{
foreach (DataTable dt in dataSet.Tables)
{
ExcelWorksheet sheet = excelPackage.Workbook.Worksheets.Add(dt.TableName);
sheet.Cells["A1"].LoadFromDataTable(dt, true);
}
var path = System.IO.Path.Combine(System.AppDomain.CurrentDomain.BaseDirectory);
var filePath = path + "\\" + "New.xls";
excelPackage.SaveAs(new System.IO.FileInfo(filePath)); //This file is flawless
FileStream fs = new FileStream(filePath, FileMode.Open);
int length = (int)fs.Length;
WebOperationContext.Current.OutgoingResponse.ContentLength = length;
byte[] buffer = new byte[length];
int sum = 0;
int count;
while ((count = fs.Read(buffer, sum, length - sum)) > 0)
{
sum += count;
}
fs.Close();
return new MemoryStream(buffer); //This file is full of unreadable chars as per above shared screenshot
}
I'm using OfficeOpenXml to generate excel files.
Then, I checked both files encoding by open them with notepad. I saw that the file on the directory (the flawless one) has ANSI encoding. And, the one which is returned by the end-point (the broken one) has UTF-8 encoding.
After that, I try to change the encoding type of the stream like this,
var byteArray = System.IO.File.ReadAllBytes(filePath);
string fileStr = new StreamReader(new MemoryStream(byteArray), true).ReadToEnd();
var encd = Encoding.GetEncoding(1252); //On the other topics I saw that ANSI represented with 1252
var end = encd.GetBytes(fileStr);
return new MemoryStream(end);
But, this doesn't help too. Though some of the strange characters are replaced with some other strange characters, but as I said, streamed file is still unreadable. And, when I open it with notepad to see its encoding, I saw that its still UTF-8.
Thus, I'm kind of stuck. I have also try directly to stream the generated excel file (without writing it to a directory and then reading it) with OfficeOpenXml's built in function called .GetAsByteArray(), but the downloaded file looks exactly the same as per above screenshot.
Thanks in advance.

C# Binary string to Bytearray without conversion

I call an API to get a PDF file. The API returns it as a string with binary data.
Now I need to save it to a file without any conversion of the data.
How can I do this in C#?
I have been trying
string file = await service.GetDocumentsAsync(document.FileId); // Gets the filedata
byte[] byteArray = file.Select (c => (byte)c).ToArray ();
using (var stream = new FileStream($"c:\\temp\\{document.Id}.pdf", FileMode.Create))
{
stream.Write (byteArray,0,file.Length);
stream.Close ();
}
I do get the PDF, but it only has blank pages.
The beginning of the string when i look at it in the Debugger:

As suggested we had to change what the API returned. We use RestSharp and had to use Response.RawByte iso. Response.

Converting a byte[] string back to byte[] array

I have one scenario with class like this.
Class Document
{
public string Name {get;set;}
public byte[] Contents {get;set;}
}
Now I am trying to implement the import export functionality where I keep the document in binary so the document will be in json file with other fields and the document will be something in this format.
UEsDBBQABgAIAAAAIQCitGbRsgEAALEHAAATAAgCW0NvbnRlbnRfVHlwZXNdLnhtbCCiBAIooAACAAAAAAA==
Now when I upload this file back, I get this file as a string and I get the same data but when I try to convert this in binary bytes[] the file become corrupt.
How can I achieve this ?
I use something like this to convert
var ss = sr.ReadToEnd();
MemoryStream stream = new MemoryStream();
StreamWriter writer = new StreamWriter(stream);
writer.Write(ss);
writer.Flush();
stream.Position = 0;
var bytes = default(byte[]);
bytes = stream.ToArray();

This looks like base 64. Use:
System.Convert.ToBase64String(b)
https://msdn.microsoft.com/en-us/library/dhx0d524%28v=vs.110%29.aspx
And
System.Convert.FromBase64String(s)
https://msdn.microsoft.com/en-us/library/system.convert.frombase64string%28v=vs.110%29.aspx

You need to de-code it from base64, like this:
Assuming you've read the file into ss as a string.
var bytes = Convert.FromBase64String(ss);

There are several things going on here. You need to know the encoding for the default StreamWriter, if it is not specified it defaults to UTF-8 encoding. However, .NET strings are always either UNICODE or UTF-16.
MemoryStream from string - confusion about Encoding to use
I would suggest using System.Convert.ToBase64String(someByteArray) and its counterpart System.Convert.FromBase64String(someString) to handle this for you.

Send file back in a readable format?

I was hoping someone could shed some light on this. Much to my chagrin, I realized that browsers such as IE8 and IE9 do not support any type of file reader.
So after a bit of research, I'm trying to have the server read the contents of the file, convert it to base64, and then send it back down to the client where the javascript takes it from there.
Is this possible? How would you recommend doing this?
For example right now I have set up a RESTful service that gets the file once the form is submitted...
public string Post()
{
string readableFile="";
HttpResponseMessage result = null;
var httprequest = HttpContext.Current.Request;
if (httprequest.Files.Count > 0)
{
foreach (string file in httprequest.Files)
{
var postedFile = httprequest.Files[file];
//convert to base64? somehow?
}
}
return readableFile;
}
the postedFile variable contains the file information, I would just need to make it into a readable format...though, I am having difficulty.
Any thoughts or help is greatly appreciated!

The HttpPostedFile class has an InputStream property which you can use to read the data. To covert that data to a base64 string you could do the following:
public string Post(HttpPostedFile file)
{
if (file.InputStream.Length > Int32.MaxValue) // Or some other file length limitation
throw new HttpResponseException(HttpStatusCode.BadRequest);
int length = (int)file.InputStream.Length;
byte[] buffer = new byte[length];
file.InputStream.Read(buffer, 0, length);
string encodedString = Convert.ToBase64String(buffer);
return encodedString;
}
You have to code it significantly differently if you want to support larger files over 4GB, but I'm assuming you want to set a reasonable limit on the file size to avoid getting an out of memory exception or having to cache the file to a hard drive.

You're looking for C#, right?
string sixtyfour = System.Convert.ToBase64String(
myFileUploadControl.FileBytes,
0,
myFileUploadControl.FileBytes.Length);
That should take the contents of a fileupload control and put it into base64.

Base64 decode in C# or Java

I have a Base64-encoded object with the following header:
application/x-xfdl;content-encoding="asc-gzip"
What is the best way to proceed in decoding the object? Do I need to strip the first line? Also, if I turn it into a byte array (byte[]), how do I un-gzip it?
Thanks!
I think I misspoke initially. By saying the header was
application/x-xfdl;content-encoding="asc-gzip"
I meant this was the first line of the file. So, in order to use the Java or C# libraries to decode the file, does this line need to be stripped?
If so, what would be the simplest way to strip the first line?

To decode the Base64 content in C# you can use the Convert Class static methods.
byte[] bytes = Convert.FromBase64String(base64Data);
You can also use the GZipStream Class to help deal with the GZipped stream.
Another option is SharpZipLib. This will allow you to extract the original data from the compressed data.

I was able to use the following code to convert an .xfdl document into a Java DOM Document.
I used iHarder's Base64 utility to do the Base64 Decode.
private static final String FILE_HEADER_BLOCK =
"application/vnd.xfdl;content-encoding=\"base64-gzip\"";
public static Document OpenXFDL(String inputFile)
throws IOException,
ParserConfigurationException,
SAXException
{
try{
//create file object
File f = new File(inputFile);
if(!f.exists()) {
throw new IOException("Specified File could not be found!");
}
//open file stream from file
FileInputStream fis = new FileInputStream(inputFile);
//Skip past the MIME header
fis.skip(FILE_HEADER_BLOCK.length());
//Decompress from base 64
Base64.InputStream bis = new Base64.InputStream(fis,
Base64.DECODE);
//UnZIP the resulting stream
GZIPInputStream gis = new GZIPInputStream(bis);
DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
DocumentBuilder db = dbf.newDocumentBuilder();
Document doc = db.parse(gis);
gis.close();
bis.close();
fis.close();
return doc;
}
catch (ParserConfigurationException pce) {
throw new ParserConfigurationException("Error parsing XFDL from file.");
}
catch (SAXException saxe) {
throw new SAXException("Error parsing XFDL into XML Document.");
}
}
Still working on successfully modifying and re-encoding the document.
Hope this helps.

In Java, you can use the Apache Commons Base64 class
String decodedString = new String(Base64.decodeBase64(encodedBytes));

It sounds like you're dealing with data that is both gzipped and Base 64 encoded. Once you strip off any mime headers, you should convert the Base64 data to a byte array using something like Apache commons codec. You can then wrap the byte[] in a ByteArrayInputStream object and pass that to a GZipInputStream which will let you read the uncompressed data.

For java, have you tried java's built in java.util.zip package? Alternately, Apache Commons has the Commons Compress library to work with zip, tar and other compressed file types. As to decoding Base 64, there are several open source libraries, or you can use Sun's sun.misc.BASE64Decoder class.

Copied from elsewhere, for Base64 I link to commons-codec-1.6.jar:
public static String decode(String input) throws Exception {
byte[] bytes = Base64.decodeBase64(input);
BufferedReader in = new BufferedReader(new InputStreamReader(
new GZIPInputStream(new ByteArrayInputStream(bytes))));
StringBuffer buffer = new StringBuffer();
char[] charBuffer = new char[1024];
while(in.read(charBuffer) != -1) {
buffer.append(charBuffer);
}
return buffer.toString();
}