We Keep Coding

How to send binary data over http, the right way?

I have the following code in C# that converts a string from one encoding to the other and send the response over http.
Everything in the code works fine. I'm just a bit confused 👇
When I return this over http it gets converted into a text representation. I thought this would just send the raw byte array?
Why is it converted? Should I never try and send a raw byte array or is there a case for that when communicating over http? Or is a text representation and a application/octet-stream the way to send binary data over http?
byte[] fromBytes = fromEncoding.GetBytes(body);
byte[] toBytes = Encoding.Convert(fromEncoding, toEncoding, fromBytes);
var response = req.CreateResponse(HttpStatusCode.OK);
MemoryStream ms = new MemoryStream(toBytes);
response.Content = new StreamContent(ms);
response.Content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
return response;

Forwarding a stream in c# api

In my service we need to get a zip file created by another service and return it.
This is my code (code has been simplified for the question):
[HttpGet("mediafiles/{id}")]
public async Task<IActionResult> DownloadMediaFiles(int id)
{
var fileIds = _myProvider.GetFileIdsForEntityId(id); // result be like "1,2,3,4"
using var httpClient = new HttpClient();
var response = await httpClient.GetAsync($"http://file-service/bulk/{fileIds}");
var stream = await response.Content.ReadAsStreamAsync();
return File(stream, "application/octet-stream", "media_files.zip");
}
With the id I can gather the info I need to create the fileIds string and call the other service.
Here's the api on the other service (code has been simplified for the question):
[HttpGet("bulk/{idList}")]
public async Task<IActionResult> DownloadBulk(string idList)
{
var ids = string.IsNullOrEmpty(idList) ? new List<int>() : idList.Split(',').Select(x => Convert.ToInt32(x));
using var memoryStream = new MemoryStream();
using (var archive = new ZipArchive(memoryStream, ZipArchiveMode.Create, true))
{
var index = archive.CreateEntry("hello.txt");
using (var entryStream = index.Open())
using (var streamWriter = new StreamWriter(entryStream))
{
streamWriter.Write("hello");
}
}
var byteArray = memoryStream.ToArray();
return File(byteArray, "application/octet-stream", "media_files.zip");
}
but when the client tries to open the zip we get
Exception has occurred. ArchiveException (FormatException: Could not
find End of Central Directory Record)
I'm absolutely not confident about these two lines of the /mediafiles/{id}
var stream = await response.Content.ReadAsStreamAsync();
return File(stream, "application/octet-stream", "media_files.zip");
And probably the issue might be there.
I just need to forward back the file-service response, but I don't know why

I believe the problem you're experiencing is that in DownloadMediaFiles(int id) you are using an HttpClient that gets disposed when leaving the function scope. The stream you created from the response therefore is closed and disposed of as well, before the response payload has finished writing its contents to the client. The client therefore receives an incomplete zip-file that you can't open. See here for reference.
In this answer there's a simple solution you could use, which is simply to read the response stream (the response stream from $"http://file-service/bulk/{fileIds}") into a byte array and then pass it to the response to the client:
using var httpClient = new HttpClient();
var response = await httpClient.GetAsync($"http://file-service/bulk/{fileIds}");
var byteArr = await response.Content.ReadAsByteArrayAsync();
return File(byteArr, "application/octet-stream", "media_files.zip");
You might realize that this means loading the whole file into memory, which can quickly become an issue if you plan on working with large files or even with medium sized files if the API is supposed to be used by a lot of clients simultaneously. Your web application would most likely run out of memory at some point.
Instead, I came upon this article which shows how you can return the contents of the stream from a request using an HttpClient. You should be able to stick with the first section of that article (all the ZIP-file and callback-based response stuff is unrelated).
To recap on that article all you need is something like this:
// Your ControllerClass.cs
private static HttpClient Client { get; } = new HttpClient();
[HttpGet("mediafiles/{id}")]
public async Task<IActionResult> DownloadMediaFiles(int id)
{
var fileIds = _myProvider.GetFileIdsForEntityId(id); // result be like "1,2,3,4"
var stream = await Client.GetStreamAsync($"http://file-service/bulk/{fileIds}");
return File(stream, "application/octet-stream", "media_files.zip");
}
You'll notice, that the stream object is not disposed of here but ASP.Net Core does this for you as part of writing the response payload to the client. The Client which is stored in a static global variable is not disposed of either, which means you can reuse it between requests (it's usually recommended not to instantiate a new HttpClient everytime you need it). ASP.Net Core 2.1 and up has special support for dependency injecting the client for you through the IHttpClientFactory interface. I would suggest you do that instead of a static variable. Read here for the most basic usage of injecting the client factory.
Now you should be able to enjoy streaming the file contents directly from your "other service" without loading it into memory in your API web application.

.NET Core 2.0 Web API for Video Streaming from FileStream

I have found a bunch of examples that use objects not available to me within my application and don't seem to match up to my version of .NET Core web API. In essence I am working on a project that will have <video> tags on a web page and want to load the videos using a stream from the server rather than directly serving the files via a path. One reason is the source of the files may change and serving them via path isn't what my customer wants. So I need to be able to open a stream and async write the video file.
This for some reason produces JSON data so that's wrong. But I just don't understand what I need to do to send a streamed video file to a <video> tag in HTML.
Current Code:
[HttpGet]
public HttpResponseMessage GetVideoContent()
{
if (Program.TryOpenFile("BigBuckBunny.mp4", FileMode.Open, out FileStream fs))
{
using (var file = fs)
{
var range = Request.Headers.GetCommaSeparatedValues("Range").FirstOrDefault();
if (range != null)
{
var msg = new HttpResponseMessage(HttpStatusCode.PartialContent);
var body = GetRange(file, range);
msg.Content = new StreamContent(body);
msg.Content.Headers.Add("Content-Type", "video/mp4");
//msg.Content.Headers.Add("Content-Range", $"0-0/{fs.Length}");
return msg;
}
else
{
var msg = new HttpResponseMessage(HttpStatusCode.OK);
msg.Content = new StreamContent(file);
msg.Content.Headers.Add("Content-Type", "video/mp4");
return msg;
}
}
}
else
{
return new HttpResponseMessage(HttpStatusCode.BadRequest);
}
}

HttpResponseMessage is not used as a return type in asp.net-core it will read that as an object model and serialize it in the response by design, as you have already observed.
Luckily in ASP.NET Core 2.0, you have
Enhanced HTTP header support
If an application visitor requests content with a Range Request header, ASP.NET will recognize that and handle that header. If the requested content can be partially delivered, ASP.NET will appropriately skip and return just the requested set of bytes. You don't need to write any special handlers into your methods to adapt or handle this feature; it's automatically handled for you.
So now all you have to do is return the file stream
[HttpGet]
public IActionResult GetVideoContent() {
if (Program.TryOpenFile("BigBuckBunny.mp4", FileMode.Open, out FileStream fs)) {
FileStreamResult result = File(
fileStream: fs,
contentType: new MediaTypeHeaderValue("video/mp4").MediaType,
enableRangeProcessing: true //<-- enable range requests processing
);
return result;
}
return BadRequest();
}
Making sure to enable range requests processing. Though, as stated in the documentation, that should be handled based on the request headers and whether that data can be partially delivered.
From there it is now a simple matter of pointing to the endpoint from the video client and let it do its magic

How to additionally include File as response of WebAPI

My current web API already responding JSON data as below.
public HttpResponseMessage GetFieldInfo()
{
//....
return Ok(GetFieldsInstance()); //GetFieldsInstance returning with DTO class instance.
}
Now, I need to include, file along with JSON response. I could not find any link which shows, how to include filestream and JSON in single response.
For file stream, it will work as below but, not able to find way, how to include JSON object property with filestream.
result = Request.CreateResponse(HttpStatusCode.OK);
result.Content = new StreamContent(new FileStream(localFilePath, FileMode.Open, FileAccess.Read));
result.Content.Headers.ContentDisposition = new System.Net.Http.Headers.ContentDispositionHeaderValue("attachment");
result.Content.Headers.ContentDisposition.FileName = "FieldFile";

You can covert (serialize) the file to a base64 string and include it as a property in the JSON response.
public IHttpActionResult GetFieldInfo() {
//...
var model = new {
//assuming: byte[] GetBinaryFile(...)
data = Convert.ToBase64String(GetBinaryFile(localFilePath)),
result = "final",
//...other properties...
};
return Ok(model);
}
The client would then need to convert (desrialize) the base64 string back to your desired file to be used as desired.
Take note that depending on the size of the file it can drastically increase the size of the response and the client should take that into consideration.

C# File Download is Corrupt

I've got some C# in a utility for a Web API project. The upload portion of the code works fine; I've verified the file that gets to the server matches the file that was uploaded. However, something is happening in the download that causes the client to view the file as corrupted, and when I do a diff I can see that something went wrong.
Unfortunately, I can't figure out what I'm doing wrong. The relevant parts of the utility are as follows:
public static HttpResponseMessage StreamResponse(this HttpRequestMessage request, Stream stream)
{
if (stream.CanSeek) stream.Position = 0;// Reset stream if possible
HttpResponseMessage response = request.CreateResponse(HttpStatusCode.OK);
response.Content = new StreamContent(stream);
if (stream is FileStream)
{// If this is a FileStream, might as well figure out the content type
string mimeType = MimeMapping.GetMimeMapping(((FileStream)stream).Name);
response.Content.Headers.ContentType = MediaTypeHeaderValue.Parse(mimeType);
}
return response;
}
public static HttpResponseMessage DownloadAs(this HttpResponseMessage response, string fileName)
{
response.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment");
response.Content.Headers.ContentDisposition.FileName = fileName;
response.Content.Headers.ContentType = MediaTypeHeaderValue.Parse(MimeMapping.GetMimeMapping(fileName));
return response;// For chaining or whatnot
}
My usage in the API controllers is return ResponseMessage(Request.StreamResponse(stream).DownloadAs("Filename.ext"));
I've double checked code for downloading, and this seems to match up with what I've found. What am I doing wrong or what am I missing? It looks like something's wrong with the encoding or charset, but I can't tell what the solution is.

Finally figured out the issue thanks to this Q&A. I was missing the responseType option/parameter in my $http call in the client-side code.