We Keep Coding

ASP.NET attempting to load streamed video source into HTML5

Over the last few weeks I have been working on an ASP.NET WebAPI that was designed to stream video from one of the company servers and play it on an HTML5 <video> element. Following a guide on C# Corner, we got the API published and now when the link for one of our videos is pasted into a browser, it starts to download (which, by the way, I'm not sure if it's supposed to do that when all we're trying to do is stream).
The files we need to stream are mp4 and are going to be used largely on iOS devices through Safari. And before anyone asks: srcVid is programmed and confirmed to be able to encode .mp4 files successfully, as we have managed to hard-code videos into this element with no issue. With that said, this is how the page handles its HTML5 elements:
<video autoplay muted id="trainVid" style="width: 75%; height: auto;" controls>
<source id="srcVid" runat="server"
type='video/mp4; codecs*="avc1.424085, mp4a.40.2"' />
</video>
On the API side, here are how the videos are processed, largely following the example set by the C# article, as well as some help from a Stephen Cleary article:
public class VidService
{
public async void WriteVidBytes(Stream outputStream,
HttpContent content, TransportContext tc)
{
try
{
var filePath = "\\\\server\\link\\to\\file.mp4";
int bufferSize = 1000;
byte[] buffer = new byte[bufferSize];
using (var fileStream = new FileStream(
filePath, FileMode.Open, FileAccess.Read, FileShare.Read))
{
int fileSize = (int)fileStream.Length;
while (fileSize > 0)
{
int cnt = fileSize > bufferSize ? bufferSize : fileSize,
readBufferSize = fileStream.Read(buffer, 0, cnt);
await outputStream.WriteAsync(buffer, 0, readBufferSize);
fileSize -= readBufferSize;
}
}
}
catch (HttpException ex) { return; }
finally { outputStream.Close(); }
}
public HttpResponseMessage GetVidContent()
{
// NOTE: please see the Edit on 6/10
var httpResponse = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new PushStreamContent(
(Action<Stream, HttpContent, TransportContext>)WriteVidBytes
)
};
return httpResponse;
}
}
public class VidController : ApiController
{
private static readonly VidService vs = new VidService();
[HttpGet]
public HttpResponseMessage GetVid(int id)
{
return vs.GetVidContent(id);
}
}
*Note that the actual program dynamically fetches video links through a Video.cs object
And finally on the C# side:
protected void LoadVideo(int vidId)
{
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(
string.Format("http://mobileAPI.website.com/Vid/GetVid/" + vidId.ToString()));
req.Method = "GET";
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
string jsonString;
using (Stream stream = resp.GetResponseStream())
{
StreamReader r = new StreamReader(stream, System.Text.Encoding.UTF8);
jsonString = r.ReadToEnd();
}
srcVid.Src = jsonString;
}
When opening the page, LoadVideo() seems to execute with no errors -- but after this, the page goes blank and hangs forever. I'm thinking this may be because I'm putting the wrong value into srcVid.Src, but if I don't put in the jsonString, then what do I put in for the source?
As always, any help would be greatly appreciated! If I missed anything obvious, please let me know, as this is the first time I have worked with WebAPI.
UPDATE 1 (6/10)
I made a secondary method that took WriteVidBytes and turned it into a Task -- and other than turning it into a Task, the code inside is exactly the same. Another difference, also, is how GetVidContent fetches the data:
public HttpResponseMessage GetVidContent(int vId)
{
var httpResponse = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new PushStreamContent(async
(outputStream, httpContext, transportContext) =>
{
await WriteVidTask(outputStream, httpContext, transportContext);
}),
};
return httpResponse;
}
However, the page still hangs even though there is no problem getting the file through Postman or Fiddler.

Following an example from Robert Huang on CodeProject, I was able to get the video to stream through API without any issue.
The first thing I changed was the way that the <video> reads the source. Rather than a JSON string, the video loads the API link.
srcVid.Src = "http://api.website.com/Vid/GetVid?id=" + vidId.ToString();
Following the CodeProject link, I created an HttpResponseMessage, very similar to the one provided -- only this one supports asynchronous loading:
public HttpResponseMessage GetVidContent(RangeHeaderValue rh, FileInfo fi)
{
HttpResponseMessage response = new HttpResponseMessage();
response.Headers.AcceptRanges.Add("bytes");
long totalLength = fi.Length;
if (rh == null || !rh.Ranges.Any())
{ // treat request normally if there is no range header
response.StatusCode = HttpStatusCode.OK;
response.Content = new PushStreamContent(async (outputStream,
httpContent, transpContext) =>
{
using (outputStream) // copy file to output stream straightforward
using (Stream inputStream = fi.OpenRead())
{
try
{
await inputStream.CopyToAsync(outputStream, ReadStreamBufferSize);
}
catch (Exception ex)
{
Debug.WriteLine(ex);
}
}
}, GetMimeNameFromExt(fi.Extension));
return response;
}
long start = 0, end = 0;
if (rh.Unit != "bytes" || rh.Ranges.Count > 1 || TryReadRangeItem(rh.Ranges.First(),
totalLength, out start, out end))
{
response.StatusCode = HttpStatusCode.RequestedRangeNotSatisfiable;
response.Content = new StreamContent(Stream.Null);
response.Content.Headers.ContentRange = new ContentRangeHeaderValue(totalLength);
response.Content.Headers.ContentType = GetMimeNameFromExt(fi.Extension);
return response;
}
var contentRange = new ContentRangeHeaderValue(start, end, totalLength);
response.StatusCode = HttpStatusCode.PartialContent;
response.Content = new PushStreamContent(async (outputStream, httpContent, transpContext) =>
{
using (outputStream)
using (Stream inputStream = fi.OpenRead())
await CreatePartialContent(inputStream, outputStream, start, end);
}, GetMimeNameFromExt(fi.Extension));
response.Content.Headers.ContentLength = end - start + 1;
response.Content.Headers.ContentRange = contentRange;
return response;
}
Any methods from the CodeProject article used in await areas were marked with the async expression.

Send large file from WebAPI.Content Length is 0

I am trying to send large file (GB) from one WebAPI (.NET Core) to another WebApi (.Net Core).
I already managed to send smaller file as part of Multipart Request like in last post here: link
To send bigger file I need (I think) send this file as StreamContent, however i am getting Content length = 0 in API which receives request.
Problem occurs even when I am sending (for test) smaller files (10 Mb).
Clientside code:
[HttpPost("UploadFiles")]
public async Task<IActionResult> Post(IFormFile file)
{
var filePath = Path.GetTempFileName();
using (var stream = new FileStream(filePath, FileMode.Create, FileAccess.ReadWrite))
{
await file.CopyToAsync(stream);
using (var formDataContent = new MultipartFormDataContent())
{
using (var httpClient = new HttpClient())
{
formDataContent.Add(CreateFileContent(stream, "myfile.test", "application/octet-stream"));
var response = await httpClient.PostAsync(
"http://localhost:56595/home/upload",
formDataContent);
return Json(response);
}
}
}
}
internal static StreamContent CreateFileContent(Stream stream, string fileName, string contentType)
{
var fileContent = new StreamContent(stream);
fileContent.Headers.ContentDisposition = new System.Net.Http.Headers.ContentDispositionHeaderValue("form-data")
{
Name = "\"file\"",
FileName = "\"" + fileName + "\"",
};
fileContent.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue(contentType);
return fileContent;
}
Serverside code:
[HttpPost]
public ActionResult Upload()
{
IFormFile fileFromRequest = Request.Form.Files.First();
string myFileName = fileFromRequest.Name;
// some code
return Ok();
}
Where is the problem?
To create Multipart request I used advices from:
HttpClient StreamContent append filename twice
POST StreamContent with Multiple Files

Finally I figured it out:
There were 2 problems:
1. Stream pointer position
In client side code, change this:
await file.CopyToAsync(stream);
to that:
await file.CopyToAsync(stream);
stream.Position = 0;
Problem was that file from request was copied to stream and left position of pointer in the end of the stream. That is why request send from client had stream with proper length, but actually when it started to read it, it could not (read 0 bytes).
2. Wrong way of handling request on server.
I used code from dotnetcoretutorials.com
Working code below:
Client side:
[HttpPost("UploadFiles")]
public async Task<IActionResult> Post(IFormFile file)
{
var filePath = Path.GetTempFileName();
using (var stream = new FileStream(filePath, FileMode.Create, FileAccess.ReadWrite))
{
await file.CopyToAsync(stream);
stream.Position = 0;
using (var formDataContent = new MultipartFormDataContent())
{
using (var httpClient = new HttpClient())
{
formDataContent.Add(CreateFileContent(stream, "myfile.test", "application/octet-stream"));
var response = await httpClient.PostAsync(
"http://localhost:56595/home/upload",
formDataContent);
return Json(response);
}
}
}
}
internal static StreamContent CreateFileContent(Stream stream, string fileName, string contentType)
{
var fileContent = new StreamContent(stream);
fileContent.Headers.ContentDisposition = new System.Net.Http.Headers.ContentDispositionHeaderValue("form-data")
{
Name = "\"file\"",
FileName = "\"" + fileName + "\"",
};
fileContent.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue(contentType);
return fileContent;
}
Server side:
Controller:
[HttpPost]
[DisableFormValueModelBinding]
public async Task<IActionResult> Upload()
{
var viewModel = new MyViewModel();
try
{
FormValueProvider formModel;
using (var stream = System.IO.File.Create("c:\\temp\\myfile.temp"))
{
formModel = await Request.StreamFile(stream);
}
var bindingSuccessful = await TryUpdateModelAsync(viewModel, prefix: "",
valueProvider: formModel);
if (!bindingSuccessful)
{
if (!ModelState.IsValid)
{
return BadRequest(ModelState);
}
}
}
catch(Exception exception)
{
throw;
}
return Ok(viewModel);
}
Helper classes for methods from controller:
public static class MultipartRequestHelper
{
// Content-Type: multipart/form-data; boundary="----WebKitFormBoundarymx2fSWqWSd0OxQqq"
// The spec says 70 characters is a reasonable limit.
public static string GetBoundary(MediaTypeHeaderValue contentType, int lengthLimit)
{
var boundary = HeaderUtilities.RemoveQuotes(contentType.Boundary);
if (string.IsNullOrWhiteSpace(boundary.ToString()))
{
throw new InvalidDataException("Missing content-type boundary.");
}
if (boundary.Length > lengthLimit)
{
throw new InvalidDataException(
$"Multipart boundary length limit {lengthLimit} exceeded.");
}
return boundary.ToString();
}
public static bool IsMultipartContentType(string contentType)
{
return !string.IsNullOrEmpty(contentType)
&& contentType.IndexOf("multipart/", StringComparison.OrdinalIgnoreCase) >= 0;
}
public static bool HasFormDataContentDisposition(ContentDispositionHeaderValue contentDisposition)
{
// Content-Disposition: form-data; name="key";
return contentDisposition != null
&& contentDisposition.DispositionType.Equals("form-data")
&& string.IsNullOrEmpty(contentDisposition.FileName.ToString())
&& string.IsNullOrEmpty(contentDisposition.FileNameStar.ToString());
}
public static bool HasFileContentDisposition(ContentDispositionHeaderValue contentDisposition)
{
// Content-Disposition: form-data; name="myfile1"; filename="Misc 002.jpg"
return contentDisposition != null
&& contentDisposition.DispositionType.Equals("form-data")
&& (!string.IsNullOrEmpty(contentDisposition.FileName.ToString())
|| !string.IsNullOrEmpty(contentDisposition.FileNameStar.ToString()));
}
}
public static class FileStreamingHelper
{
private static readonly FormOptions _defaultFormOptions = new FormOptions();
public static async Task<FormValueProvider> StreamFile(this HttpRequest request, Stream targetStream)
{
if (!MultipartRequestHelper.IsMultipartContentType(request.ContentType))
{
throw new Exception($"Expected a multipart request, but got {request.ContentType}");
}
// Used to accumulate all the form url encoded key value pairs in the
// request.
var formAccumulator = new KeyValueAccumulator();
string targetFilePath = null;
var boundary = MultipartRequestHelper.GetBoundary(
MediaTypeHeaderValue.Parse(request.ContentType),
_defaultFormOptions.MultipartBoundaryLengthLimit);
var reader = new MultipartReader(boundary, request.Body);
var section = await reader.ReadNextSectionAsync();
while (section != null)
{
ContentDispositionHeaderValue contentDisposition;
var hasContentDispositionHeader = ContentDispositionHeaderValue.TryParse(section.ContentDisposition, out contentDisposition);
if (hasContentDispositionHeader)
{
if (MultipartRequestHelper.HasFileContentDisposition(contentDisposition))
{
await section.Body.CopyToAsync(targetStream);
}
else if (MultipartRequestHelper.HasFormDataContentDisposition(contentDisposition))
{
// Content-Disposition: form-data; name="key"
//
// value
// Do not limit the key name length here because the
// multipart headers length limit is already in effect.
var key = HeaderUtilities.RemoveQuotes(contentDisposition.Name);
var encoding = GetEncoding(section);
using (var streamReader = new StreamReader(
section.Body,
encoding,
detectEncodingFromByteOrderMarks: true,
bufferSize: 1024,
leaveOpen: true))
{
// The value length limit is enforced by MultipartBodyLengthLimit
var value = await streamReader.ReadToEndAsync();
if (String.Equals(value, "undefined", StringComparison.OrdinalIgnoreCase))
{
value = String.Empty;
}
formAccumulator.Append(key.ToString(), value);
if (formAccumulator.ValueCount > _defaultFormOptions.ValueCountLimit)
{
throw new InvalidDataException($"Form key count limit {_defaultFormOptions.ValueCountLimit} exceeded.");
}
}
}
}
// Drains any remaining section body that has not been consumed and
// reads the headers for the next section.
section = await reader.ReadNextSectionAsync();
}
// Bind form data to a model
var formValueProvider = new FormValueProvider(
BindingSource.Form,
new FormCollection(formAccumulator.GetResults()),
CultureInfo.CurrentCulture);
return formValueProvider;
}
private static Encoding GetEncoding(MultipartSection section)
{
MediaTypeHeaderValue mediaType;
var hasMediaTypeHeader = MediaTypeHeaderValue.TryParse(section.ContentType, out mediaType);
// UTF-7 is insecure and should not be honored. UTF-8 will succeed in
// most cases.
if (!hasMediaTypeHeader || Encoding.UTF7.Equals(mediaType.Encoding))
{
return Encoding.UTF8;
}
return mediaType.Encoding;
}
}
[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method)]
public class DisableFormValueModelBindingAttribute : Attribute, IResourceFilter
{
public void OnResourceExecuting(ResourceExecutingContext context)
{
var formValueProviderFactory = context.ValueProviderFactories
.OfType<FormValueProviderFactory>()
.FirstOrDefault();
if (formValueProviderFactory != null)
{
context.ValueProviderFactories.Remove(formValueProviderFactory);
}
var jqueryFormValueProviderFactory = context.ValueProviderFactories
.OfType<JQueryFormValueProviderFactory>()
.FirstOrDefault();
if (jqueryFormValueProviderFactory != null)
{
context.ValueProviderFactories.Remove(jqueryFormValueProviderFactory);
}
}
public void OnResourceExecuted(ResourceExecutedContext context)
{
}
}
Additional thoughts:
(on clientside) line:
fileContent.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue(contentType);
is not necessary to send the file.
(on clientside) file is send when MediaTypeHeaderValue is one of these:
application/x-msdownload
application/json
application/octet-stream
(on serverside) to use lines with contentDisposition.FileNameStar on serverside you need to change them to contentDisposition.FileNameStar.ToString()
(on serverside) code used in question for serverside will work with smaller files (Mb's) but to send GB file we need code which is pasted in the answer.
some parts of code are taken from aspnet core docs

Large File download from SQL via WebApi after custom MultipartFormDataStreamProvider upload

This is a follow up to a question I had asked previously that was closed for being too broad.Previous Question
In that question I explained that I needed upload a large file (1-3GB) to the database by storing chunks as individual rows. I did this by overriding the MultipartFormDataStreamProvider.GetStream method. That method returned a custom stream that wrote the buffered chunks to the database.
The problem is that the overriden GetStream method is writing the entire request to the database (including the headers). It is successfully writing that data while keeping the Memory levels flat but when I download the file, in addition to the file contents, it's returning all the header information in the downloaded file contents so the file can't be opened.
Is there a way to, in the overriden GetStream method, write just the contents of the file to the database without writing the headers?
API
[HttpPost]
[Route("file")]
[ValidateMimeMultipartContentFilter]
public Task<HttpResponseMessage> PostFormData()
{
var provider = new CustomMultipartFormDataStreamProvider();
// Read the form data and return an async task.
var task = Request.Content.ReadAsMultipartAsync(provider).ContinueWith<HttpResponseMessage>(t =>
{
if (t.IsFaulted || t.IsCanceled)
{
Request.CreateErrorResponse(HttpStatusCode.InternalServerError, t.Exception);
}
return Request.CreateResponse(HttpStatusCode.OK);
});
return task;
}
[HttpGet]
[Route("file/{id}")]
public async Task<HttpResponseMessage> GetFile(string id)
{
var result = new HttpResponseMessage()
{
Content = new PushStreamContent(async (outputStream, httpContent, transportContext) =>
{
await WriteDataChunksFromDBToStream(outputStream, httpContent, transportContext, id);
}),
StatusCode = HttpStatusCode.OK
};
result.Content.Headers.ContentType = new MediaTypeHeaderValue("application/zipx");
result.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment") { FileName = "test response.zipx" };
return result;
}
return new HttpResponseMessage(HttpStatusCode.BadRequest);
}
private async Task WriteDataChunksFromDBToStream(Stream responseStream, HttpContent httpContent, TransportContext transportContext, string fileIdentifier)
{
// PushStreamContent requires the responseStream to be closed
// for signaling it that you have finished writing the response.
using (responseStream)
{
using (var myConn = new SqlConnection(System.Configuration.ConfigurationManager.ConnectionStrings["TestDB"].ConnectionString))
{
await myConn.OpenAsync();
using (var myCmd = new SqlCommand("ReadAttachmentChunks", myConn))
{
myCmd.CommandType = System.Data.CommandType.StoredProcedure;
var fileName = new SqlParameter("#Identifier", fileIdentifier);
myCmd.Parameters.Add(fileName);
// Read data back from db in async call to avoid OutOfMemoryException when sending file back to user
using (var reader = await myCmd.ExecuteReaderAsync(CommandBehavior.SequentialAccess))
{
while (await reader.ReadAsync())
{
if (!(await reader.IsDBNullAsync(3)))
{
using (var data = reader.GetStream(3))
{
// Asynchronously copy the stream from the server to the response stream
await data.CopyToAsync(responseStream);
}
}
}
}
}
}
}// close response stream
}
Custom MultipartFormDataStreamProvider GetStream method implementation
public override Stream GetStream(HttpContent parent, HttpContentHeaders headers)
{
// For form data, Content-Disposition header is a requirement
ContentDispositionHeaderValue contentDisposition = headers.ContentDisposition;
if (contentDisposition != null)
{
// If we have a file name then write contents out to AWS stream. Otherwise just write to MemoryStream
if (!String.IsNullOrEmpty(contentDisposition.FileName))
{
var identifier = Guid.NewGuid().ToString();
var fileName = contentDisposition.FileName;// GetLocalFileName(headers);
if (fileName.Contains("\\"))
{
fileName = fileName.Substring(fileName.LastIndexOf("\\") + 1).Replace("\"", "");
}
// We won't post process files as form data
_isFormData.Add(false);
var stream = new CustomSqlStream();
stream.Filename = fileName;
stream.Identifier = identifier;
stream.ContentType = headers.ContentType.MediaType;
stream.Description = (_formData.AllKeys.Count() > 0 && _formData["description"] != null) ? _formData["description"] : "";
return stream;
//return new CustomSqlStream(contentDisposition.Name);
}
// We will post process this as form data
_isFormData.Add(true);
// If no filename parameter was found in the Content-Disposition header then return a memory stream.
return new MemoryStream();
}
throw new InvalidOperationException("Did not find required 'Content-Disposition' header field in MIME multipart body part..");
#endregion
}
Implemented Write method of Stream called by CustomSqlStream
public override void Write(byte[] buffer, int offset, int count)
{
//write buffer to database
using (var myConn = new SqlConnection(System.Configuration.ConfigurationManager.ConnectionStrings["TestDB"].ConnectionString)) {
using (var myCmd = new SqlCommand("WriteAttachmentChunk", myConn)) {
myCmd.CommandType = System.Data.CommandType.StoredProcedure;
var pContent = new SqlParameter("#Content", buffer);
myCmd.Parameters.Add(pContent);
myConn.Open();
myCmd.ExecuteNonQuery();
if (myConn.State == System.Data.ConnectionState.Open)
{
myConn.Close();
}
}
}
((ManualResetEvent)_dataAddedEvent).Set();
}
The "ReadAttachmentChunks" stored procedure gets the rows respective to the file from the db ordered by the time they are inserted into the database. So, the way the code works is it pulls those chunks back and then async writes it back to the PushStreamContent to go back to the user.
So my question is:
Is there a way to write ONLY the content of the file being uploaded as opposed to the headers in addition to the content?
Any help would be greatly appreciated. Thank you.

I finally figured it out. I over-complicated the write process which brought about most of the struggle. Here is my solution to my initial issue:
To keep .net from buffering the file in memory (so that you can handle large file uploads), you first need to override the WebHostBufferPolicySelector so that it doesnt buffer the input stream for your controller and then replace the BufferPolicy Selector.
public class NoBufferPolicySelector : WebHostBufferPolicySelector
{
public override bool UseBufferedInputStream(object hostContext)
{
var context = hostContext as HttpContextBase;
if (context != null)
{
if (context.Request.RequestContext.RouteData.Values["controller"] != null)
{
if (string.Equals(context.Request.RequestContext.RouteData.Values["controller"].ToString(), "upload", StringComparison.InvariantCultureIgnoreCase))
return false;
}
}
return true;
}
public override bool UseBufferedOutputStream(HttpResponseMessage response)
{
return base.UseBufferedOutputStream(response);
}
}
then for replacing the BufferPolicy Selector
GlobalConfiguration.Configuration.Services.Replace(typeof(IHostBufferPolicySelector), new NoBufferPolicySelector());
Then to avoid the default behavior of having the file stream written to disk, you need to provide a stream provider that will write to the database instead. To do this you inherit MultipartStreamProvider and override the GetStream method to return the stream that will write to your database.
public override Stream GetStream(HttpContent parent, HttpContentHeaders headers)
{
// For form data, Content-Disposition header is a requirement
ContentDispositionHeaderValue contentDisposition = headers.ContentDisposition;
if (contentDisposition != null && !String.IsNullOrEmpty(contentDisposition.FileName))
{
// We won't post process files as form data
_isFormData.Add(false);
//create unique identifier for this file upload
var identifier = Guid.NewGuid();
var fileName = contentDisposition.FileName;
var boundaryObj = parent.Headers.ContentType.Parameters.SingleOrDefault(a => a.Name == "boundary");
var boundary = (boundaryObj != null) ? boundaryObj.Value : "";
if (fileName.Contains("\\"))
{
fileName = fileName.Substring(fileName.LastIndexOf("\\") + 1).Replace("\"", "");
}
//write parent container for the file chunks that are being stored
WriteLargeFileContainer(fileName, identifier, headers.ContentType.MediaType, boundary);
//create an instance of the custom stream that will write the chunks to the database
var stream = new CustomSqlStream();
stream.Filename = fileName;
stream.FullFilename = contentDisposition.FileName.Replace("\"", "");
stream.Identifier = identifier.ToString();
stream.ContentType = headers.ContentType.MediaType;
stream.Boundary = (!string.IsNullOrEmpty(boundary)) ? boundary : "";
return stream;
}
else
{
// We will post process this as form data
_isFormData.Add(true);
// If no filename parameter was found in the Content-Disposition header then return a memory stream.
return new MemoryStream();
}
}
The custom stream you create needs to inherit Stream and override the Write method. This is where I overthought the problem and thought I needed to parse out the boundary headers that were passed via the buffer parameter. But this is actually done for you by leveraging the offset and count parameters.
public override void Write(byte[] buffer, int offset, int count)
{
//no boundary is inluded in buffer
byte[] fileData = new byte[count];
Buffer.BlockCopy(buffer, offset, fileData, 0, count);
WriteData(fileData);
}
From there, it's just plugging in the api methods for upload and download.
For upload:
public Task<HttpResponseMessage> PostFormData()
{
var provider = new CustomMultipartLargeFileStreamProvider();
// Read the form data and return an async task.
var task = Request.Content.ReadAsMultipartAsync(provider).ContinueWith<HttpResponseMessage>(t =>
{
if (t.IsFaulted || t.IsCanceled)
{
Request.CreateErrorResponse(HttpStatusCode.InternalServerError, t.Exception);
}
return Request.CreateResponse(HttpStatusCode.OK);
});
return task;
}
For download, and in order to keep the memory footprint low, I leveraged the PushStreamContent to push the chunks back to the user:
[HttpGet]
[Route("file/{id}")]
public async Task<HttpResponseMessage> GetFile(string id)
{
string mimeType = string.Empty;
string filename = string.Empty;
if (!string.IsNullOrEmpty(id))
{
//get the headers for the file being sent back to the user
using (var myConn = new SqlConnection(System.Configuration.ConfigurationManager.ConnectionStrings["PortalBetaConnectionString"].ConnectionString))
{
using (var myCmd = new SqlCommand("ReadLargeFileInfo", myConn))
{
myCmd.CommandType = System.Data.CommandType.StoredProcedure;
var pIdentifier = new SqlParameter("#Identifier", id);
myCmd.Parameters.Add(pIdentifier);
myConn.Open();
var dataReader = myCmd.ExecuteReader();
if (dataReader.HasRows)
{
while (dataReader.Read())
{
mimeType = dataReader.GetString(0);
filename = dataReader.GetString(1);
}
}
}
}
var result = new HttpResponseMessage()
{
Content = new PushStreamContent(async (outputStream, httpContent, transportContext) =>
{
//pull the data back from the db and stream the data back to the user
await WriteDataChunksFromDBToStream(outputStream, httpContent, transportContext, id);
}),
StatusCode = HttpStatusCode.OK
};
result.Content.Headers.ContentType = new MediaTypeHeaderValue(mimeType);// "application/octet-stream");
result.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment") { FileName = filename };
return result;
}
return new HttpResponseMessage(HttpStatusCode.BadRequest);
}
private async Task WriteDataChunksFromDBToStream(Stream responseStream, HttpContent httpContent, TransportContext transportContext, string fileIdentifier)
{
// PushStreamContent requires the responseStream to be closed
// for signaling it that you have finished writing the response.
using (responseStream)
{
using (var myConn = new SqlConnection(System.Configuration.ConfigurationManager.ConnectionStrings["PortalBetaConnectionString"].ConnectionString))
{
await myConn.OpenAsync();
//stored proc to pull the data back from the db
using (var myCmd = new SqlCommand("ReadAttachmentChunks", myConn))
{
myCmd.CommandType = System.Data.CommandType.StoredProcedure;
var fileName = new SqlParameter("#Identifier", fileIdentifier);
myCmd.Parameters.Add(fileName);
// The reader needs to be executed with the SequentialAccess behavior to enable network streaming
// Otherwise ReadAsync will buffer the entire BLOB into memory which can cause scalability issues or even OutOfMemoryExceptions
using (var reader = await myCmd.ExecuteReaderAsync(CommandBehavior.SequentialAccess))
{
while (await reader.ReadAsync())
{
//confirm the column that has the binary data of the file returned is not null
if (!(await reader.IsDBNullAsync(0)))
{
//read the binary data of the file into a stream
using (var data = reader.GetStream(0))
{
// Asynchronously copy the stream from the server to the response stream
await data.CopyToAsync(responseStream);
await data.FlushAsync();
}
}
}
}
}
}
}// close response stream
}

Ugh. This is nasty. With the upload, you have to make sure to
separate the headers from the content portion - you must follow the requirements RFC documents for HTTP.
Allow for chunked transfers
Of course, the content portion (unless you are transmitting text) will be binary encoded into strings.
Allow for transfers that are compressed, i.e. GZIP or DEFLATE.
Maybe - just maybe - take the encoding into account (ASCII, Unicode, UTF8, etc).
You can't really ensure that you're persisting the right info to the database without looking at all of these. For the latter items, all of your metadata as to what to do will be somewhere in the header, so it's not just a throwaway.

Async Download file

I plan to write a function DownloadData return a byte array, another client will call it to get byte array. My point is I don't want client app is waiting file is download, so I need it download in async mode. But I so confuse how to do that.
This is my function:
public byte[] DownloadData(string serverUrlAddress, string path)
{
if(string.IsNullOrWhiteSpace(serverUrlAddress) || string.IsNullOrWhiteSpace(path))
return null;
// Create a new WebClient instance
WebClient client = new WebClient();
// Concatenate the domain with the Web resource filename.
string url = string.Concat(serverUrlAddress, "/", path);
if (url.StartsWith("http://") == false)
url = "http://" + url;
byte[] data = null;
client.DownloadDataCompleted += delegate(object sender, DownloadDataCompletedEventArgs e)
{
data = e.Result;
};
while (client.IsBusy) { }
return data;
}

I wrote a method that does just that.
public async Task<byte[]> DownloadData(string url)
{
TaskCompletionSource<byte[]> tcs = new TaskCompletionSource<byte[]>();
HttpWebRequest request = WebRequest.CreateHttp(url);
using (HttpWebResponse response = (HttpWebResponse)(await request.GetResponseAsync()))
using (Stream stream = response.GetResponseStream())
using (MemoryStream ms = new MemoryStream())
{
await stream.CopyToAsync(ms);
tcs.SetResult(ms.ToArray());
return await tcs.Task;
}
}

I know why I lost byte. On API, I return byte array, but I use HttpClient to get data. I change to HttpResponseMessage as return and accept type on both.

Silverlight Http Post to upload images

I have been trying to perform an HTTP Post request to upload an image in a silverlight application for a windows phone 7 application. The sample codes online do not get me the desired response from the API. Could anyone please provide a working code which does this?
By desired response I mean that the API responds saying that the uploaded file is in a format which cannot be read.
Thanks in advance!
Here is my code:
private void post_image(version, username,password,job-id, serviceUri)
{
if (session_free.bLoggedIn)
{
bool submit_success = false;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(new Uri(serviceUri));
IsolatedStorageFileStream stream = IsolatedStorageFile.GetUserStoreForApplication().OpenFile("file.jpg", FileMode.Open);
request.PostMultiPartAsync(new Dictionary<string, object> { { "version", version }, { "username", user }, { "password", pass }, { filename, stream } }, new AsyncCallback(asyncResult =>
{
Thread.Sleep(1000);
HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(asyncResult);
Stream responseStream = response.GetResponseStream();
StreamReader reader = new StreamReader(responseStream);
Post_Result = reader.ReadToEnd();
this.Dispatcher.BeginInvoke(delegate
{
MessageBox.Show(Post_Result);
response.Close();
});
}), filename);
Thread.Sleep(1000);
}
else
{
MessageBox.Show("User not signed in! Please login to continue...", "Invalid Authentication", MessageBoxButton.OK);
}
}
public class DataContractMultiPartSerializer
{
private string boundary;
public DataContractMultiPartSerializer(string boundary)
{
this.boundary = boundary;
}
private void WriteEntry(StreamWriter writer, string key, object value, string filename)
{
if (value != null)
{
writer.Write("--");
writer.WriteLine(boundary);
if (value is IsolatedStorageFileStream)
{
IsolatedStorageFileStream f = value as IsolatedStorageFileStream;
writer.WriteLine(#"Content-Disposition: form-data; name=""{0}""; filename=""{1}""", key, filename);
writer.WriteLine("Content-Type: image/jpeg");
writer.WriteLine("Content-Length: " + f.Length);
writer.WriteLine();
writer.Flush();
Stream output = writer.BaseStream;
Stream input = f;
byte[] buffer = new byte[4096];
for (int size = input.Read(buffer, 0, buffer.Length); size > 0; size = input.Read(buffer, 0, buffer.Length))
{
output.Write(buffer, 0, size);
}
output.Flush();
writer.WriteLine();
}
else
{
writer.WriteLine(#"Content-Disposition: form-data; name=""{0}""", key);
writer.WriteLine();
writer.WriteLine(value.ToString());
}
}
}
public void WriteObject(Stream stream, object data, string filename)
{
StreamWriter writer = new StreamWriter(stream);
if (data != null)
{
if (data is Dictionary<string, object>)
{
foreach (var entry in data as Dictionary<string, object>)
{
WriteEntry(writer, entry.Key, entry.Value, filename);
}
}
else
{
foreach (var prop in data.GetType().GetFields())
{
foreach (var attribute in prop.GetCustomAttributes(true))
{
if (attribute is System.Runtime.Serialization.DataMemberAttribute)
{
DataMemberAttribute member = attribute as DataMemberAttribute;
writer.Write("{0}={1}&", member.Name ?? prop.Name, prop.GetValue(data));
}
}
}
}
writer.Flush();
}
}
}

Is there a reason you're using a multipart message?
What is the PostMultiPartAsync method you're using? Presumably it's an extension method from somewhere?
In future, try and provide the smallest, complete, piece of code which demonstrates the issue.
Anyway, sorry it's not a full working example, but here are the steps for one way to do this.
Create request and set a calback for BeginGetRequestStream
var request = (HttpWebRequest)WebRequest.Create(App.Config.ServerUris.Login);
request.Method = "POST";
request.BeginGetRequestStream(ReadCallback, request);
In that callback, get the request stream and write your data to it.
using (var postStream = request.EndGetRequestStream(asynchronousResult))
{
// Serialize image to byte array, or similar (that's what imageBuffer is)
postStream.Write(imageBuffer, 0, imageBuffer.Length);
}
Set a callback for the response from the server.
request.BeginGetResponse(ResponseCallback, request);
Check that everything worked OK on the server
private void ResponseCallback(IAsyncResult asynchronousResult)
{
var request = (HttpWebRequest)asynchronousResult.AsyncState;
using (var resp = (HttpWebResponse)request.EndGetResponse(asynchronousResult))
{
using (var streamResponse = resp.GetResponseStream())
{
using (var streamRead = new StreamReader(streamResponse))
{
string responseString = streamRead.ReadToEnd(); // Assuming that the server will send a text based indication of upload success
// act on the response as appropriate
}
}
}
}
On the server you will need to deserialize the data and turn it back into an image as appropriate.
HTH.