I want to download a single file in a remote Zip file that is in the cloud. The zip file is too large for me to download as a whole therefore I have decided to look for a way to download only a single file(XML) that I need within the archive. I have tried and Tested a webclient and web request but it downloads the whole zip file(also too large file for these technuques usually fails). I'm eyeing the SharpZipLib but I dont know how to use it. Is it the right library I should use or there are other available ones I can get and test. Thank you so much.
Related
I can't seem to find any examples of downloading multiple files from a URL.
I have a URL lets say www.example.com/Document and I want to download all the files in that URL is that possible? or do I need to provide the name of each file in that URL? All the files would download to a folder specified here in the code.
Any examples of this would be great, this would be done by clicking a Download button in WPF.
The way I did this was to create a ZIP file and have all the files in there. You can either pre create a ZIP file and download or you can create the ZIP file as it downloads. I had to write my own software to do that but it does work.
I have around 1GB of data which my client want to download from my portal as a zip file, but in the backend the files are served from AWS S3.
Currently i am downloading all files to memory stream and zipping it, which is taking a lot of time and at times it timeout and client is not sure if request is processing as I dont have download progress in the browser.
So is there a best solution to download huge data zipped from S3
Thanks
You could run some code on an Amazon EC2 instance that downloads the data from Amazon S3 (very quick if in the same region), zips it, then puts the zip back into S3.
The user can then download it directly from S3.
If you want to get fancy, they could download via a Pre-Signed URL and you could have a lifecycle rule that deletes it after a day or two.
OK, so I have little experience with S3 myself, but in general cases like this call for asynchronous processing. As in - when your user clicks "download", you initiate a background process that downloads and zips the files in some temporary location. During this, your client sees something like "preparing download, please wait". Ideally with a progress bar so that he can see that the process isn't stalled. When it's done, the download starts for real and without any timeouts since you already have the full ZIP file in a temp location.
Alternatively, see if you can streamline the whole process. Right now it sounds like you're downloading all the files to memory, creating the ZIP file in memory, and only then you're starting to output the first bytes to the client. You can do better. There are libraries out there that allow zipping "on-the-fly". While you're still downloading the files from S3 on one end, the other end is already spitting out ZIP file to the client. This way you don't need to keep everything in memory either.
my users upload their files in my Web Application and the application saves their files in the server that has the most free space. The result is that one folder of a user may contain multiple files that are saved in multiple servers.
Now i want to give my users the option to download their whole folder as a ZIP file.
Can somebody provide an appropriate namespace or guidance of what will be the best practice to achieve this functionality?
The application is written in C# .NET. So far i saw
1) http://msdn.microsoft.com/en-us/library/system.io.packaging.aspx
together with
2) http://msdn.microsoft.com/en-us/library/system.io.packaging.zippackage.aspx
Am i looking at the right direction?
Let say a user has his files over 3 servers named server1,server3 and server6
point 1. You must have those locations somewhere? right?
now suppose server7 needs to pack those files into 1 zip and stream to the client.
You will need to download all those files to server7 and then pack those files to zip archive.
To download the files you can use webrequest method. Hope this is clear.
YOU WILL NEED TO DOWNLOAD THE FILES TO A SINGLE LOCATION BEFORE ZIPPING THEM, there is no shortcut over this.
Suppose, I have a list of MP3 files on my server. And I want a user to download multiple files (which he wants through any means). For, this what i want is to create the zip file dynamically and while saving it into the Output Stream using the dotnetzip or ioniczip libraries.
Well, that's not the perfect solution if the zip file got heavy in size. As, in that scenario the server doesn't support resumable downloads. So, to overcome this approach, I need to handle the zip file structure internally and provide the resume support.
So, is there any library (open source) which i can use to provide resumable dyanamic zip files stream directly to the Output Stream. Or, if possible I will be happy if someone let me know the structure of zip file specially the header content + data content.
Once a download has started, you should not alter the ZIP file anymore. Because then a resume will just result in a broken ZIP file. So make sure your dynamically created ZIP file stays available!
The issue of providing resume-functionality was solved in this article for .NET 1.1, and it is still valid and functional.
I've got a project which requires a fairly complicated process and I want to make sure I know the best way to do this. I'm using ASP.net C# with Adobe Flex 3. The app server is Mosso (cloud server) and the file storage server is Amazon S3. The existing site can be viewed at NoiseTrade.com
I need to do this:
Allow users to upload MP3 files to
an album "widget"
After the user has uploaded their
album/widget, I need to
automatically zip the mp3 (for other
users to download) and upload the
zip along with the mp3 tracks to
Amazon S3
I actually have this working already (using client side processing in Flex) but this no longer works because of Adobe's flash 10 "security" update. So now I need to implement this server-side.
The way I am thinking of doing this is:
Store the mp3 in a temporary folder
on the app server
When the artist "publishes" create a
zip of the files in that folder
using a c# library
Start the amazon S3 upload process (zip and mp3s)
and email the user when it is
finished (as well as deleting the
temporary folder)
The major problem I see with this approach is that if a user deletes or adds a track later on I'll have to update the zip file but the temporary files will not longer exist.
I'm at a loss at the best way to do this and would appreciate any advice you might have.
Thanks!
The bit about updating the zip but not having the temporary files if the user adds or removes a track leads me to suspect that you want to build zips containing multiple tracks, possibly complete albums. If this is incorrect and you're just putting a single mp3 into each zip, then StingyJack is right and you'll probably end up making the file (slightly) larger rather than smaller by zipping it.
If my interpretation is correct, then you're in luck. Command-line zip tools frequently have flags which can be used to add files to or delete files from an existing zip archive. You have not stated which library or other method you're using to do the zipping, but I expect that it probably has this capability as well.
MP3's are compressed. Why bother zipping them?
I would say it is not necessary to zip a compressed file format, you are only gong to get a five percent reduction in filesize, give or take a little. Mp3's dont really zip up by their nature the have compressed most of the possible data already.
DotNetZip can zip up files from C#/ASP.NET. I concur with the prior posters regarding compressibility of MP3s. DotNetZip will automatically skip compression on MP3, and just store the file, just for this reason. It still may be interesting to use a zip as a packaging/archive container, aside from the compression.
If you change the zip file later (user adds a track), you could grab the .zip file from S3, and just update it. DotNetZip can update zip files, too. But in this case you would have to pay for the transfer cost into and out of S3.
DotNetZip can do all of this with in-memory handling of the zips - though that may not be feasible for large archives with lots of MP3s and lots of concurrent users.