I need basic file downloading capabilities in my app and I cannot use WebClient.DownloadFile [1]. Is this (naïve?) implementation of a DownloadFile method enough? Are there any pitfalls that I don't address with this implementation?
public static void DownloadFile(String url, String destination)
{
using (var request = (HttpWebRequest)WebRequest.Create(url))
{
request.Method = "GET";
request.Timeout = 100000; // 100 seconds
using (var response = request.GetResponse())
{
using (var responseStream = response.GetResponseStream())
{
using (var fileStream = File.Open(destination,
FileMode.Create,
FileAccess.Write,
FileShare.None))
{
var MaxBytesToRead = 10 * 1024;
var buffer = new Byte[MaxBytesToRead];
var totalBytesRead = 0;
var bytesRead = responseStream.Read(buffer,
0,
MaxBytesToRead);
while (bytesRead > 0)
{
totalBytesRead += bytesRead;
fileStream.Write(buffer, 0, bytesRead);
bytesRead = responseStream.Read(buffer,
0,
MaxBytesToRead);
}
}
}
}
}
}
Thanks!
[1] .Net Compact Framework...
You're keeping track of totalBytesRead, but I can't see it used anywhere.
Since Method = "GET" is the default, I don't see anything that's specific to HTTP. If you remove the (HttpWebRequest) cast and the Method = line then you'll gain the ability to download over other protocols, such as FTP. Currently the code will throw an exception if somebody provides a URL other than http://.
Response should have a Content-Length header (unless content-encoding = chunked) which you can use to validate that the download was not interrupted.
Other than that, your implementation looks fine by me.
Related
I created a simple application that collects form data, generates an XML in memory as a MemoryStream object, and and delivers the XML to a server using SMB. The delivery method is simple for SMB:
var outputFile = new FileStream($#"{serverPath}\{filename}.xml", FileMode.Create);
int Length = 256;
Byte[] buffer = new Byte[Length];
int bytesRead = stream.Read(buffer, 0, Length);
while (bytesRead > 0)
{
outputFile.Write(buffer, 0, bytesRead);
bytesRead = stream.Read(buffer, 0, Length);
}
However, I need to create an alternative delivery method using FTP (with credentials). I don't want to rewrite my XML method, as creating it in memory saves writing to disk which has been a problem in our environment in the past.
I have not been able to find any examples that explain (for a person of very limited coding ability) how such a thing may be accomplished.
Generally when I have need to upload a file to an FTP server, I use something like this:
using (var client = new WebClient())
{
client.Credentials = new NetworkCredential("user", "pass");
client.UploadFile(uri, WebRequestMethods.Ftp.UploadFile, filename.xml);
}
Can this be adapted to upload from a MemoryStream instead of a file on disk?
If not, what other way could I upload a MemoryStream to an FTP server?
Either use FtpWebRequest, as you can see in Upload a streamable in-memory document (.docx) to FTP with C#?:
WebRequest request =
WebRequest.Create("ftp://ftp.example.com/remote/path/filename.xml");
request.Method = WebRequestMethods.Ftp.UploadFile;
request.Credentials = new NetworkCredential(username, password);
using (Stream ftpStream = request.GetRequestStream())
{
memoryStream.CopyTo(ftpStream);
}
or use WebClient.OpenWrite (as you can also see in the answer by #Neptune):
using (var webClient = new WebClient())
{
const string url = "ftp://ftp.example.com/remote/path/filename.xml";
using (Stream uploadStream = client.OpenWrite(url))
{
memoryStream.CopyTo(uploadStream);
}
}
Equivalently, your existing FileStream code can be simplified to:
using (var outputFile = File.Create($#"{serverPath}\{filename}.xml"))
{
stream.CopyTo(outputFile);
}
Though obviously, even better would be to avoid the intermediate MemoryStream and write the XML directly to FileStream and WebRequest.GetRequestStream (using their common Stream interface).
You can use the methods OpenWrite/OpenWriteAsync to get a stream that you can write to from any source (stream/array/...etc.)
Here is an example using OpenWrite to write from a MemoryStream:
var sourceStream = new MemoryStream();
// Populate your stream with data ...
using (var webClient = new WebClient())
{
using (Stream uploadStream = client.OpenWrite(uploadUrl))
{
sourceStream.CopyTo(uploadStream);
}
}
Problem still there while i tried below three methods.
Using Window API "URLDownloadToFile"
WebClient Method
webclient.DownloadFile(url,dest) ''With/Without credientials
HTTP WebRequest Method:
public static void Download(String strURLFileandPath, String strFileSaveFileandPath)
{
HttpWebRequest wr = (HttpWebRequest)WebRequest.Create(strURLFileandPath);
HttpWebResponse ws = (HttpWebResponse)wr.GetResponse();
Stream str = ws.GetResponseStream();
byte[] inBuf = new byte[100000];
int bytesToRead = (int) inBuf.Length;
int bytesRead = 0;
while (bytesToRead > 0)
{
int n = str.Read(inBuf, bytesRead,bytesToRead);
if (n==0)
break;
bytesRead += n;
bytesToRead -= n;
}
FileStream fstr = new FileStream(strFileSaveFileandPath, FileMode.OpenOrCreate, FileAccess.Write);
fstr.Write(inBuf, 0, bytesRead);
str.Close();
fstr.Close();
}
Still i m facing the problem, file i am able to download at my local system, but when i open that it show Corrupt pdf.
!!!!I just want to download the pdf from URL and thats my query in VB.net/C# not using response method of ASP.net.
Please help if someone face this real problem.
Thanks in Advance!!!
Your code only writes 100000 bytes of the downloaded PDF and hence every PDF that is bigger than 100000 bytes gets corrupted.
To read more bytes you have to write the contents of every buffer to the FileStream.
The following should do it:
HttpWebRequest wr = (HttpWebRequest)WebRequest.Create(strURLFileandPath);
using (HttpWebResponse ws = (HttpWebResponse)wr.GetResponse())
using (Stream str = ws.GetResponseStream())
using (FileStream fstr = new FileStream(strFileSaveFileandPath, FileMode.OpenOrCreate, FileAccess.Write))
{
byte[] inBuf = new byte[100000];
int bytesRead = 0;
while ((bytesRead = str.Read(inBuf, 0, inBuf.Length)) > 0)
fstr.Write(inBuf, 0, bytesRead);
}
(It's good coding practice to use a using on every IDisposable instead of manually closing the streams.)
I have a method for FTP download file, but I do not save file locally rather I parse the file in memory through ftp response. My question is, is returning stream reader after getting ftp response stream a good practice? Because do not want to do parsing and other stuff in the same method.
var uri = new Uri(string.Format("ftp://{0}/{1}/{2}", "somevalue", remotefolderpath, remotefilename));
var request = (FtpWebRequest)FtpWebRequest.Create(uri);
request.Credentials = new NetworkCredential(userName, password);
request.Method = WebRequestMethods.Ftp.DownloadFile;
var ftpResponse = (FtpWebResponse)request.GetResponse();
/* Get the FTP Server's Response Stream */
ftpStream = ftpResponse.GetResponseStream();
return responseStream = new StreamReader(ftpStream);
For me there are 2 disadvantages of using the stream directly, if you can live with them, you shouldn't waste memory or disk space.
In this stream you can not seek to a specific position, you can only read the contents as it comes in;
Your internet connection could suddenly drop and you will get an exception while parsing and processing your file, either split the parsing and processing or make sure your processing routine can handle the case that a file is processed for a second time (after a failure halfway through the first attempt).
To work around these issues, you could copy the stream to a MemoryStream:
using (var ftpStream = ftpResponse.GetResponseStream())
{
var memoryStream = new MemoryStream()
while ((bytesRead = ftpStream.Read(buffer, 0, buffer.Length)) > 0)
{
memoryStream.Write(buffer, 0, bytesRead);
}
memoryStream.Flush();
memoryStream.Position = 0;
return memoryStream;
}
If you are working with larger files I prefer writing it to a file, this way you minimize the memory footprint of your application:
using (var ftpStream = ftpResponse.GetResponseStream())
{
var fileStream = new FileStream(Path.GetTempFileName(), FileMode.CreateNew)
while ((bytesRead = ftpStream.Read(buffer, 0, buffer.Length)) > 0)
{
fileStream.Write(buffer, 0, bytesRead);
}
fileStream.Flush();
fileStream.Position = 0;
return fileStream;
}
I see more practical returning a responseStream when you are performing an HttpWebRequest. If you are using FtpWebRequest it means you are working with files. I would read the responseStream to byte[] and return the byte file content of the downloaded file, so you can easily work with the System.IO.Fileclasses to handle the file.
Thanks Carlos it was really helpful . I just return the byte[]
byte[] buffer = new byte[16 * 1024];
using (MemoryStream ms = new MemoryStream())
{
int read;
while ((read = ftpStream.Read(buffer, 0, buffer.Length)) > 0)
{
ms.Write(buffer, 0, read);
}
memoryStream=ms;
}
return memoryStream.ToArray();
and used byte[] in the method like this
public async Task ParseReport(byte[] bytesRead)
{
Stream stream = new MemoryStream(bytesRead);
using (StreamReader reader = new StreamReader(stream))
{
string line = null;
while (null != (line = reader.ReadLine()))
{
string[] values = line.Split(';');
}
}
stream.Close();
}
I need to download the file as http response for the current http request.
Until now I used code as
System.Uri uri = System.Web.HttpContext.Current.Request.Url;
HttpWebRequest httpRequest = (HttpWebRequest)WebRequest.Create(
Path.Combine(uri.ToString(), filename));
httpRequest.Method = "GET";
using (HttpWebResponse httpResponse = (HttpWebResponse)httpRequest.GetResponse())
{
using (Stream responseStream = httpResponse.GetResponseStream())
{
using (FileStream localFileStream = new FileStream(
Path.Combine(localFolder, filename), FileMode.Open))
{
int bytesRead;
while ((bytesRead = responseStream.Read(buffer, 0, buffer.Length)) > 0)
{
totalBytesRead += bytesRead;
localFileStream.Write(buffer, 0, bytesRead);
}
}
}
}
But this code the request is only sending but not getting any responses...
Is this possible?
You should get the file off disk then use the Response.OutputStream to write the file directly to the response. Make sure to set the correct content headers so the browser will know what is coming.
FileInfo file = new FileInfo(Path.Combine(localFolder, filename));
int len = (int)file.Length, bytes;
Response.ContentType = "text/plain"; //Set the file type here
Response.AddHeader "Content-Disposition", "attachment;filename=" + filename;
context.Response.AppendHeader("content-length", len.ToString());
byte[] buffer = new byte[1024];
using(Stream stream = File.OpenRead(path)) {
while (len > 0 && (bytes =
stream.Read(buffer, 0, buffer.Length)) > 0)
{
Response.OutputStream.Write(buffer, 0, bytes);
len -= bytes;
}
}
Not sure, but it looks like your making a web request, getting the response stream, then attempting to buffer it out to localFolder. If so, FileMode.Open looks suspect ("should open an existing file..."?). Maybe use FileMode.Create.
MSDN ref
Also, does your web app needs to have write permissions to localFolder.
Whenever I try to download a file from a server (the server is on a device, it's not on the interwebs) > 4Gb, the transfer only actually transfers what appears to be (FileSize) % 4Gb. In other words, for a file just over 4.5Gb, I end up only transferring around 600mb of data.
It's something to do with content-length headers and so on I think, but I'm not sure what the exact mechanism is. I've tried using WebClient and WebRequest but both exhibit the same behaviour.
Does anyone have any idea how I can get past this limit? Here's my current loop:
byte[] buffer = new byte[4096];
WebRequest request = WebRequest.Create(new Uri(transferDetails.URL));
using (WebResponse response = request.GetResponse())
{
using (Stream responseStream = response.GetResponseStream())
{
using (FileStream fileStream = new FileStream(actualPath, FileMode.Create, FileAccess.Write))
{
int count = 0;
do
{
// Read a block.
count = responseStream.Read(buffer, 0, buffer.Length);
// Write out to the local file.
if(count > 0)
{
fileStream.Write(buffer, 0, count);
}
} while (count != 0);
}
}
}