We Keep Coding

Trying to Write Web Server Side Implementation of File Download via C# where body of message is the file contents

My question/challenge is very much akin to a couple of different Stack Overflow postings, but I've read them in detail and am still struggling to get even the most basic application to work. Specifically, I believe "Returning binary file from controller in ASP.NET Web API" and "How to return a file (FileContentResult) in ASP.NET WebAPI" are both very close to what I'm trying to achieve.
To give you context, I'm trying to write/stub a simple Firmware OTA (over the air) server. My initial goal is seemingly very simple, but I've spent a lot of time with almost zero progress. I want to provide a URL that can be hit from a browser (and ultimately some firmware running on an IoT device) that will take in some parameters (either via URL parameters or via the header). The call should return a file as the body of the response if a firmware update is available, or a suitable HTTP Reponse code (or just an empty message body) if no updated is currently available.
I'm not very experienced in Visual Studio for Website applications, but plenty comfortable in C# and it seemed like a good environment to get this running quickly. I am open to other implementations, but I thought writing a controller would be the simplest way. I am learning that it seems this Web API is primarily intended for .Net to .Net communication, so if there's a better platform, I'm happy to be pointed in a better direction. Here's what I put together (heavily leveraging links above, so no credit due to me!):
namespace OverTheAirApi.Controllers
{
[Route("api/[controller]")]
public class OTAController : ApiController
{
[HttpGet]
public HttpResponseMessage Download(string name)
{
string fileName = "hello-world.bin";
string filePath = "C:\\Devel\\Code\\...\\hello_world\\build\\" + fileName;
HttpResponseMessage result = new HttpResponseMessage(HttpStatusCode.OK);
var stream = new FileStream(filePath, FileMode.Open);
result.Content = new StreamContent(stream);
result.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment");
result.Content.Headers.ContentDisposition.FileName = Path.GetFileName(filePath);
result.Content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
result.Content.Headers.ContentLength = stream.Length;
return result;
}
}
}
When I type in the URL http://localhost:2265/api/ota, I see the following raw output on screen:
{"version":{"major":1,"minor":1,"build":-1,"revision":-1,"majorRevision":-1,"minorRevision":-1},"content":{"headers":[{"key":"Content-Disposition","value":["attachment; filename=hello-world.bin"]},{"key":"Content-Type","value":["application/octet-stream"]},{"key":"Content-Length","value":["407904"]}]},"statusCode":200,"reasonPhrase":"OK","headers":[],"requestMessage":null,"isSuccessStatusCode":true}
The entire response, captured in Fiddler2 looks like this:
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Content-Type: application/json; charset=utf-8
Server: Kestrel
X-SourceFiles: =?UTF-8?B? YzpcdXNlcnNcZG91Z2NcZG9jdW1lbnRzXHZpc3VhbCBzdHVkaW8gMjAxNVxQcm9qZWN0c1xPdmVyVGhl QWlyQXBpXHNyY1xPdmVyVGhlQWlyQXBpXGFwaVxvdGE=?=
X-Powered-By: ASP.NET
Date: Tue, 28 Feb 2017 16:43:01 GMT
192
{"version": {"major":1,"minor":1,"build":-1,"revision":-1,"majorRevision":-1,"minorRevision":-1},"content":{"headers":[{"key":"Content-Disposition","value":["attachment; filename=hello-world.bin"]},{"key":"Content-Type","value":["application/octet-stream"]},{"key":"Content-Length","value":["407904"]}]},"statusCode":200,"reasonPhrase":"OK","headers":[],"requestMessage":null,"isSuccessStatusCode":true}
0
(Sorry if that formatting is a bit ugly, I guess I need some tutoring on using MD for HTTP Responses as well!)
Once again, my goal is to write C# code that will return back an HTTP response where the header indicates that the file is an attachment and where the body of the message is just the file itself.
Thank you again for any assistance/insight you might be able to provide!
Thank You - doug.

Here's my standard method for sending a file to the browser. The method takes two parameters, one the contents of the file and one the default filename that the client will receive. Enjoy.
/// <summary>
/// Sends the fileData to the browser and prompts the visitor to save it with the
/// name specified in fileName.
/// </summary>
/// <param name="defaultFilename">File name to use when browser prompts visitor (spaces will be replaced with dashes)</param>
/// <param name="data">Data to be sent to visitor's browser</param>
/// <param name="errorMsg"></param>
// Mod 08/04/09 Ron C - Reworked code to work right in modern browsers.
public static bool DownloadToBrowser(string data, string defaultFilename, out string errorMsg){
errorMsg = "";
System.Web.HttpResponse response = HttpContext.Current.Response;
try {
defaultFilename = defaultFilename.Replace(' ', '-'); //firefox will cut the name off at the space if there is one, so get rid of 'em
response.Clear();
response.ContentType = "application/octet-stream";
response.AddHeader("Content-Disposition", "attachment; filename=" + defaultFilename);
//8/5/09 Adding a "Content-Length" header was cutting off part of my file. Apparently
// it would need to factor in the length of the headers as well as the conent.
// since I have no easy way to figure out the length of the headers, initially I was gonna
// eliminate "Content-Length" header. Doing so works great in IE 7 and FireFox 3, but not
// in Safari 3 or 4(which doesn't download the file without the "Content-Length" header. So I
// resorted to a cludge of using the header tag and setting it to content length + 1000 for headers.
// I'd love to have a better solution, but his one works and I've already wasted 6+ hours to get to this
// solution. Cross broswer downloading of a file shouldn't have to be so hard. -Ron
int len = data.Length + 1000;
response.AddHeader("Content-Length", len.ToString()); //file size for progress dialog
response.Write(data);
response.Flush();
response.Close(); //Close() needed to prevent html from page being streamed back after the file data we sent.
//Don't use response.End() cause it throws a thread abort exception that can't be caught! Actually you can
//catch it but then it rethrows after the catch block! (What bozo thought that up?). I found lots of threads on this.
} catch (Exception ex) {
errorMsg = "Unable to download file to browser.";
//Add code here to log the error in your environment
return false;
}
return true;
}

Dropbox file uploading not showing 409 error

I'm uploading file using Dropbox core API. I have written the upload code like-
RequestResult strReq = OAuthUtility.Put
(
"https://api-content.dropbox.com/1/files_put/auto/",
new HttpParameterCollection
{
{"access_token", "Token"},
{"path","/file.txt"},
{"overwrite", "false"},
{"autorename","false"},
{stream}
}
);
Suppose there is a existing file in root folder named file.txt and I'm again trying to upload the same name file to same folder.I have written
overwrite= false and autorename=false but surprisingly there is no error status code returning in the response.Always returning the success code 200 in the response.I need to show the proper error code.

Two things stand out:
Your URL is https://api-content.dropbox.com/1/files_put/auto/, but it should be (for this example) https://api-content.dropbox.com/1/files_put/auto/file.txt. The path parameter should be removed from the HttpParameterCollection.
I'm unfamiliar with the library you're using, but are you sure that those parameters are turned into query parameters and that stream becomes the HTTP body? I.e. the resulting URL should be https://api-content.dropbox.com/1/files_put/auto/file.txt?overwrite=false&autorename=false&access_token=<TOKEN>, and then the file content should go in the body of the request. Please make sure this is what's happening.
Please also share the body that comes back with the 200 response. It should tell you, for example, the path of the file that got written.
Note that if you upload the exact same file content to the same path, it doesn't count as a conflict, so when looking for a 409, make sure you're uploading different content to the file.

How to read only a small part of a .XML

I built an application in order to read a file, but even with the fact that my connection is fast, the page takes several seconds to load, I would like to know how to read only the first records of this .xml
string rssURL = "http://www.cnt.org.br/Paginas/feed.aspx?t=n";
System.Net.WebRequest myRequest = System.Net.WebRequest.Create(rssURL);
System.Net.WebResponse myResponse = myRequest.GetResponse();
System.IO.Stream rssStream = myResponse.GetResponseStream();
System.Xml.XmlDocument rssDoc = new System.Xml.XmlDocument();
rssDoc.Load(rssStream);
System.Xml.XmlNodeList rssItems = rssDoc.SelectNodes("rss/channel/item");
Tks..

As the fore posters mention you can’t download part of a web request. But you can start parsing Xml before the request finished. Using XmlDocument is the wrong approach for your use case, because it needs the complete request to create the object. Try using XmlTextReader.

There is no easy way to download part of a web request and ensure it is what you want. One workaround would be to use the Google Feed API.
You'd have to use the JSON interface since they don't provide a library for C#, but since it's going through Google's servers it will be much faster. You'd have to modify your code a little bit, since it returns JSON by default instead of XML, but that is a trivial change to make. You can also change the parameter output=xml to retrieve the XML representation of the data.
Try going to this page, that is your same feed, with fewer elements and loads much faster. That only returns a few elements, but if you want 10 elements, all you have to do is add num=10 to the URL. For example, this url has 10 elements. Read the API documentation a little more to see what variables you can add to cater the request to what you want to do.

Download content from the internet with code

I have to download some content from a website every day so I figure it will be nice to have a program that will do it... The problem is that the website requires authentication.
My current solution is by using System.Windows.Forms.WebBrowser control. I currently do something like:
/* Create browser */
System.Windows.Forms.WebBrowser browser = new System.Windows.Forms.WebBrowser();
/* navigate to desired site */
browser.Navigate("http://stackoverflow.com/");
// wait for browser to download dom
/* Get all tags of type input */
var elements = browser.Document.Body.GetElementsByTagName("input");
/* let's look for the one we are interested */
foreach (System.Windows.Forms.HtmlElement curInput in elements)
{
if (curInput.GetAttribute("name") == "q") //
{
curInput.SetAttribute("value", "I changed the value of this input");
break;
}
}
// etc
I think this approach works but is not the best solution. I have tried to use the webclient class and that seems to work but for some reason it does not work. I belive the reason why it does not work is because I have to save the cookies?
So my question is how will I be able to track all the bytes that get send to the server and all the bytes that get responded in order to download what I need. In other words I will like to have the webclient act as a webrowser and once I get to the part I need by just looking at the source I should be able to parser the data that I need.
I will appreciate if someone can show me an example of how to do so. Google chrome does a pretty good job displaying lots of information:
Thanks in advance,
Antonio

Answering your question:
The best utility i know to track traffic is Fiddler (its free).
For sending advanced HTTP requests, you should use class System.Net.HttpWebRequest, which also has property CookieContainer, and Headers, allowing you to do what ever you want.
Hope it helps.

C#: HttpListener Error Serving Content

I have implemented something similar to this
only real difference is
string filename = context.Request.RawUrl.Replace("/", "\\").Remove(0,1);
string path = Uri.UnescapeDataString(Path.Combine(_baseFolder, filename));
so that I can traverse to subdirectories. This works great for webpages and other text file types but when trying to serve up media content I get the exception
HttpListenerException: The I/O
operation has been aborted because of
either a thread exit or an application
request
Followed by
InvalidOperationException: Cannot close stream until all bytes are written.
In the using statement.
Any suggestions on how to handle this or stop these exceptions?
Thanks

I should mention that I am using Google Chrome for my browser (Google Chrome doesn't seem to care about the MIME types, when it sees audio it will try to use it like it's in a HTML5 player), but this is also applicable if you are trying to host media content in a page.
Anyways, I was inspecting my headers with fiddler and noticed that Chrome passes 3 requests to the server. I started playing with other browsers and noticed they did not do this, but depending on the browser and what I had hard coded as the MIME type I would either get a page of crazy text, or a download of the file.
On further inspection I noticed that chrome would first request the file. Then request the file again with a few different headers most notably the range header. The first one with byte=0- then the next with a different size depending on how large the file was (more than 3 requests can be made depending how large the file is).
So there was the problem. Chrome will first ask for the file. Once seeing the type it would send another request which seems to me looking for how large the file is (byte=0-) then another one asking for the second half of the file or something similar to allow for a sort of streaming experienced when using HTML5. I coded something quickly up to handle MIME types and threw a HTML5 page together with the audio component and found that other browsers also do this (except IE)
So here is a quick solution and I no longer get these errors
string range = context.Request.Headers["Range"];
int rangeBegin = 0;
int rangeEnd = msg.Length;
if (range != null)
{
string[] byteRange = range.Replace("bytes=", "").Split('-');
Int32.TryParse(byteRange[0], out rangeBegin);
if (byteRange.Length > 1 && !string.IsNullOrEmpty(byteRange[1]))
{
Int32.TryParse(byteRange[1], out rangeEnd);
}
}
context.Response.ContentLength64 = rangeEnd - rangeBegin;
using (Stream s = context.Response.OutputStream)
{
s.Write(msg, rangeBegin, rangeEnd - rangeBegin);
}

Try:
using (Stream s = context.Response.OutputStream)
{
s.Write(msg, 0, msg.Length);
s.Flush()
}