I am trying to implement file download feature in asp.net application. The application would be used by say around 200 users concurrently to download various files.
It would be hosted on IIS 7. I do not want the application server to crash because of multiple requests coming concurrently.
I am assuming that by calling Context.Response.Flush() in a loop, I am flushing out all the file data that I would have read till then, so application memory usage would be kept uniform. What other optimizations can I make to the current code or what other approach should be used in a scenario like this?
The requests would be for various files and the file sizes can be anywhere between 100 KB to 10 MB.
My current code is like this:
FileStream inStr = null;
byte[] buffer = new byte[1024];
String fileName = #"C:\DwnldTest\test.doc";
long byteCount; inStr = File.OpenRead(fileName);
Response.AddHeader("content-disposition", "attachment;filename=test.doc");
while ((byteCount = inStr.Read(buffer, 0, buffer.Length)) > 0)
{
if (Context.Response.IsClientConnected)
{
Context.Response.ContentType = "application/msword";
//Context.Response.BufferOutput = true;
Context.Response.OutputStream.Write(buffer, 0, buffer.Length);
Context.Response.Flush();
}
}
You can use Response.TransmitFile to save server memory when sending files.
Response.ContentType = "application/pdf";
Response.AddHeader("content-disposition", "attachment; filename=testdoc.pdf");
Response.TransmitFile(#"e:\inet\www\docs\testdoc.pdf");
Response.End();
In your code example, you're not closing / disposing inStr. That could affect performance.
Another more simple way to do this would be to use the built in method:
WriteFile
It should already be optimized and will take care of opening / closing files for you.
Maybe you want to use FileSystemWatcher class to check if the file was modified, and read it into memory only while such change was detected. For rest of the time just return the byte array that is already stored in memory. I don't know if HttpResponse.WriteFile method is sensitive for such file modification changes, or if always reads a file from given path, but this also seems to be a good option to use, as it is served by framework out of the box.
Since you are sending an existing file to the client, consider using HttpResponse.TransmitFile (http://msdn.microsoft.com/en-us/library/12s31dhy.aspx).
Looking at the .NET code it seems that this will forward the file writing to IIS instead of reading/writing it in ASP.NET process. HttpResponse.WriteFile(string, false) and HttpResponse.Write(string) seems to do the same thing.
In order to verify that the file sending is relayed to IIS, at HttpResponse.Output property - it should be of type HttpWriter. The HttpWriter._buffers array should now contain a new element HttpFileResponseElement).
Of course, you should always investigate if caching is appropriate in your scenario and test if it is being used.
Related
I'm using Response.Filter in order to implement stream compression in accordance with HTTP Request Header Accept-Encoding
Here's the important stuff:
if (AcceptEncoding.Contains("deflate") || AcceptEncoding == "*")
{
HttpApp.Response.Filter = new DeflateStream(PreviousOutputStream, CompressionMode.Compress);
HttpApp.Response.AppendHeader("Content-Encoding", "deflate");
}
By and large this works as intended. However, I'm in a situation where I'm using an ActionResult on an MVC Controller to serve up files to the user agent:
Response.Clear();
Response.Headers["Content-Type"] = contentType;
Response.Headers["Content-Length"] = contentLength;
if (Request.QueryString["dl"] == "1")
{
Response.Headers["Content-Disposition"] = "attachment; filename=" + fileInfo.Name;
}
Response.Flush();
Response.TransmitFile(fileInfo.FullName);
To be more exact, the action method returns new EmptyResult() after the Response.TransmitFile() call. This works exactly as intended without the Response.Filter modification.
In this situation, the response entity reaches the user agent garbled and unintelligible. FireFox's Poster addon shows empty entities or jumbled entities coming back.
If you can help it, definitely look for alternatives, because manually doing compression in ASP.NET is NOT fun. But, if you are as hard-headed as I am, I submit to you the following.
First of all: do NOT use .NET's built-in compression stream classes. They are buggy and can truncate bytes off the end of streams at random. I've been using DotNetZip with good results: http://dotnetzip.codeplex.com/
Now, some additional notes:
Response.TransmitFile() doesn't work with response filtering.
Response.BinaryWrite() doesn't work with response filtering, so you can't loop over the contents of the file and write it out that way.
Response.OutputStream doesn't work with response filtering, so you can't loop over the contents of the file and write it out THAT way, either.
Response.WriteFile() DOES work with response filtering, but it loads the entire file into memory and keeps it there until the client closes the connection, which doesn't work well for large files.
And to make things just that little bit more fun: response filtering stops working if you set Response.BufferOutput to false. (I just spent literally hours figuring that out)
Obviously, there's a LOT of different issues surrounding response filtering and writing to the output stream. Using Reflector and lots of experimentation, this is the best ("best" being in terms of working correctly in a variety of scenarios) solution I've found thus far:
Write a class that extends Encoding and call it BinaryEncoding. Implement all the methods so that they copy characters and bytes correctly, but of course doing the necessary type casts.
Set Response.ContentEncoding to an instance of BinaryEncoding (you can use the singleton pattern quite successfully for this).
Open your file with FileStream.
Create a new StreamReader(fileStream, new BinaryEncoding(), false). That "false" parameter is very important, it stops the StreamReader from eating byte order marks and overriding your BinaryEncoding.
Allocate a buffer of char[] (I've found that 32KB is a good size).
Then, in a loop:
int n = StreamReader.Read(buffer, 0, buffer.Length);
Response.Write(buffer, 0, n);
Response.Flush();
Until n is 0.
Be warned: this method results in fairly high CPU usage. On a 100 megabit LAN, CPU usage on one core goes to around 40-50% for a single client downloading at 10MB/sec. I wish I could find a better way... If I had Reflector Pro, I might be able to find one.
If you are using IIS7 or IIS7.5 I would suggest using the HTTP Compression module rather than rolling your own. It might help solve the problem.
http://technet.microsoft.com/en-us/library/cc771003(WS.10).aspx
I'm making tests with ASP.NET HttpHandler for download a file writting directly on the response stream, and I'm not pretty sure about the way I'm doing it. This is a example method, in the future the file could be stored in a BLOB in the database:
public void GetFile(HttpResponse response)
{
String fileName = "example.iso";
response.ClearHeaders();
response.ClearContent();
response.ContentType = "application/octet-stream";
response.AppendHeader("Content-Disposition", "attachment; filename=" + fileName);
using (FileStream fs = new FileStream(Path.Combine(HttpContext.Current.Server.MapPath("~/App_Data"), fileName), FileMode.Open))
{
Byte[] buffer = new Byte[4096];
Int32 readed = 0;
while ((readed = fs.Read(buffer, 0, buffer.Length)) > 0)
{
response.OutputStream.Write(buffer, 0, readed);
response.Flush();
}
}
}
But, I'm not sure if this is correct or there is a better way to do it.
My questions are:
When I open the url with the browser, appears the "Save File" dialog... but it seems like the server has started already to push data into the stream before I click "Save", is that normal?
If I remove the line"response.Flush()", when I open the url with the browser, ... I see how the web server is pushing data but the "Save File" dialog doesn't come up, (or at least not in a reasonable time fashion) why?
When I open the url with a WebRequest object, I see that the HttpResponse.ContentLength is "-1", although I can read the stream and get the file. What is the meaning of -1? When is HttpResponse.ContentLength going to show the length of the response? For example, I have a method that retrieves a big xml compresed with deflate as a binary stream, but in that case... when I access it with a WebRequest, in the HttpResponse I can actually see the ContentLength with the length of the stream, why?
What is the optimal length for the Byte[] array that I use as buffer for optimal performance in a web server? I've read that is between 4K and 8K... but which factors should I consider to make the correct decision.
Does this method bloat the IIS or client memory usage? or is it actually buffering the transference correctly?
Sorry for so many questions, I'm pretty new in web development :P
Cheers.
Yes; this is normal.
If you never flush, the browser doesn't get any response until the server finishes (Not even the Content-Disposition header). Therefore, it doesn't know to show a file dialog.
The Content-Length header only gets set if the entire response is buffered (If you never flush) or if you set it yourself. In this case, you can and should set it yourself; write
response.AppendHeader("Content-Length", new FileInfo(path).Length.ToString());
I recommend 4K; I don't have any hard basis for the recommendation.
This method is the best way to do it. By calling Flush inside the loop, you are sending the response down the wire immediately, without any buffering. However, for added performance, you can use GZIP compression.
Yes, it is buffering.
Flush pushes the cached content to the browser. If it is never pushed, you won't get a save dialog box.
Hard to tell without seeing the exact files/URLs/Streams you are using.
I think the factors depends on how sluggish your page is, really. You will have better performance toward 4k. And perhaps, the lower value will be better to accommodate slower connections.
See #1 & 2.
For #3 you need to set the content-length header in your http-response. Many of those values come from http headers.
I believe you can change the bufferring by changing a buffering property on the response object to false. Haven't done it in a while so I don't remember what it might be.
ok i am downloading a file from a server and i plan to delete the file that i have downloaded on the server after it gets downloaded on the client side..
My download code is working fine but i dont know when to put the command to delete the file.
string filepath = restoredFilename.ToString();
// Create New instance of FileInfo class to get the properties of the file being downloaded
FileInfo myfile = new FileInfo(filepath);
// Checking if file exists
if (myfile.Exists)
{
// Clear the content of the response
Response.ClearContent();
// Add the file name and attachment, which will force the open/cancel/save dialog box to show, to the header
Response.AddHeader("Content-Disposition", "attachment; filename=" + myfile.Name);
//Response.AddHeader("Content-Disposition", "inline; filename=" + myfile.Name);
// Add the file size into the response header
Response.AddHeader("Content-Length", myfile.Length.ToString());
// Set the ContentType
Response.ContentType = ReturnExtension(myfile.Extension.ToLower());
//// Write the file into the response (TransmitFile is for ASP.NET 2.0. In ASP.NET 1.1 you have to use WriteFile instead)
Response.TransmitFile(myfile.FullName);
// End the response
Response.End();
}
Now i know the response.End() will stop every thing and return the value, so is there another way too do so..
I need to call a function
DeleteRestoredFileForGUI(restoredFilename);
to delete the file but dont know where to put it.. i tried putting before and after Response.End() but it does not work..
any help is appreciated... thanks
Add
Response.Flush();
DeleteRestoredFileForGUI(restoredFilename);
after the call to TransmitFile() and ditch the call to Response.End() (you don't need it).
If that does not work, then ditch TransmitFile() and go with:
Stream s = myFile.OpenRead();
int bytesRead = 0;
byte[] buffer = new byte[32 * 1024] //32k buffer
while((bytesRead = s.Read(buffer, 0, buffer.Length)) > 0 &&
Response.IsClientConnected)
{
Response.OutputStream.Write(buffer, 0, bytesRead);
Response.Flush();
}
you can't delete the file straight away as it may not have been downloaded yet. from the server side there is no easy way of telling that the file was successfully downloaded. what if an open/save dialog is opened by the browser? download won't begin until the dialog is acknowledged. (this may not be immediately and/or the dialog may be cancelled)
or, what if it is a large file and the connection is dropped before it is fully downloaded? should it be possible to attempt the download again?
the normally recommended way of dealing with your situation is to do the deletion as a separate process, after a time period which allows you to be (fairly) sure the file is no longer required and/or it can be recreated/restored if need be.
depending on your situation you could have a separate process which periodically removes/processes old files. or, if you have a low volume of traffic, you could check for and delete old files each time a new one is requested.
the identification of old files will likely be based on a file time or associated value in a darabase. either way, if there are potentially lots of files to process you are unlikely to want the overhead of checking very frequently if it is unlikely to identify a lot of files to remove.
also, be sure to way up the consequences of lots of files not being removed ASAP (is disk space really an issue?) against the side effects of possibly deleting them while still needed or creating a performance side effect by checking to zealously.
The general pattern you are following makes me wonder, are you doing this?
Create Data for Client and Save to
Disk Transmit File to Client Delete
File
If you are, you might change your system to work in memory. Since memory is managed in .Net you wouldn't have to do this manual cleanup, and depending on the size of the file this could be a good bit faster too:
Create Data for Client and Save to MemoryStream
Transmit Stream to Client
Since you set the file name in the header, you have two options:
Read the file contents into a string, delete the file, echo/print the string as the body of the message.
Rename the file something like delete-filename.xxx and then have some external process (maybe a cron job?) that goes behind and deletes any files beginning with that prefix.
I have a database column that contains the contents of a file. I'm converting this into a byte[] on the server (I don't want to save the file to the disk) and then want to send this to the client to download. The file can be any thing (pdfs, pics, word, excel, etc).
I have the file name so I know the extension but I'm not sure how the best way to send it to the client is. Here's where I'm currently at:
string fileName = ds.Tables[0].Rows[0]["form_file_name"].ToString();
byte[] fileContents = (byte[])ds.Tables[0].Rows[0]["form_file_contents"];
Where do I go from here?
You should be able to write it out to the client via something like this...
Response.Clear();
Response.AddHeader("Content-Length", fileContents.Length.ToString());
Response.AddHeader("Content-Disposition", "attachment; filename=FILENAME");
Response.OutputStream.Write(fileContents, 0, fileContents.Length);
Response.Flush();
Response.End();
I'd a similar situation here; if you're dealing with files, you should consider what happens if you have a big one in database.
You could use DataReader.GetBytes() as in Memory effective way to read BLOB data in C#/SQL 2005 and write that data in chunks. This is a better approach since you don't need have entire file in memory, but just a small piece everytime.
Using this idea, you could to write code to read 64k data chunks and write them like Quintin Robinson said.
I have a custom HttpHandler that invokes a webservice to get a file. In test, I invoke the production webservice and the HttpHandler returns the file correctly. When I test it in the production environment on the server, it works as well. However, if I invoke the HttpHandler from a remote client (not on the server) the filename and size are set correctly, but the file bytes that are downloaded are zero. Any ideas?
So here's the deal. I created a multipart range handler (you need to implement the RFC in order to stream content to, say, an iPhone or Adobe Reader). The spec is suppose to enable handling a file when the client requests a range of bytes instead of the whole array. The issue with my handler came when the client wanted the whole BLOB:
if (context.Request.Headers[HEADER_RANGE] != null)
{
...
}
else
{
context.Response.ContentType = contentItem.MimeType;
addHeader(context.Response, HEADER_CONTENT_DISPOSITION, "attachment; filename=\"" + contentItem.Filename + "\"");
addHeader(context.Response, HEADER_CONTENT_LENGTH, contentItem.FileBytes.Length.ToString());
context.Response.OutputStream.Write(contentItem.FileBytes, 0, contentItem.FileBytes.Length);
}
Notice anything missing???
I forgot to include:
context.Response.Flush();
After adding that line of code, it started working in the production environment. I find it very odd, however, that this was working on the server and not on any clients. Anyone able to shed any light on why that would be?