I'm saving an image from a web request and something really weird is happening. On roughly half of the 8,000 images I'm downloading I get IOEXCEPTION errors:
ERROR_ACCESS_DENIED (5)
INVALID_PARAMETER (87)
Before I save the file using file.open, I check to make sure the file does not exist. The exception is thrown at this line of code:
fileStream = File.Open(destination, FileMode.Create, FileAccess.Write, FileShare.None);
Below is the code:
public static bool DownloadFile(string url, string destination)
{
bool success = false;
System.Net.HttpWebRequest request = null;
System.Net.WebResponse response = null;
Stream responseStream = null;
FileStream fileStream = null;
try
{
request = (System.Net.HttpWebRequest)System.Net.WebRequest.Create(url);
request.Method = "GET";
request.Timeout = 100000; // 100 seconds
request.Proxy = System.Net.GlobalProxySelection.GetEmptyWebProxy();
response = request.GetResponse();
responseStream = response.GetResponseStream();
fileStream = File.Open(destination, FileMode.Create, FileAccess.Write, FileShare.None);
//fileStream = File.Create(destination);
// read up to ten kilobytes at a time
int maxRead = 10240;
byte[] buffer = new byte[maxRead];
int bytesRead = 0;
int totalBytesRead = 0;
// loop until no data is returned
while ((bytesRead = responseStream.Read(buffer, 0, maxRead)) > 0)
{
totalBytesRead += bytesRead;
fileStream.Write(buffer, 0, bytesRead);
}
// we got to this point with no exception. Ok.
success = true;
}
catch (System.Net.WebException we)
{
// something went terribly wrong.
success = false;
//MessageBox.Show(exp.ToString());
writeErrFile(we.ToString(), url);
//Debug.WriteLine(exp);
}
catch (System.IO.IOException ie)
{
// something went terribly wrong.
success = false;
//MessageBox.Show(ie.InnerException.ToString());
writeErrFile(ie.ToString(), destination + " -- " + url);
//Debug.WriteLine(exp);
}
catch (Exception exp)
{
// something went terribly wrong.
success = false;
//MessageBox.Show(exp.ToString());
writeErrFile(exp.ToString(), destination + " -- " + url);
//Debug.WriteLine(exp);
}
finally
{
// cleanup all potentially open streams.
if (null != responseStream)
responseStream.Close();
if (null != response)
response.Close();
if (null != fileStream)
fileStream.Close();
}
// if part of the file was written and the transfer failed, delete the partial file
if (!success && File.Exists(destination))
File.Delete(destination);
return success;
}
I've been stuck on this for a couple of days. Any help would be appreciated in unimaginable orders of magnitude.
Use file.exists() to check if the file exists and file.create or file.openwrite to write the file.
From your code I can't see how you are checking the file exists.
Related
I'm working on a project where I need to send large audio files via streams from a client to a server. I'm using the ASP.NET Web Api to communicate between client and server. My client has a "SendFile" method which I believe works fine, but I don't know how to make my server receive the data I'm sending via a stream. My client code looks like this so far:
private const int MAX_CHUNK_SIZE = (1024 * 5000);
private HttpWebRequest webRequest = null;
private FileStream fileReader = null;
private Stream requestStream = null;
public bool SendAudio(string uri, string file)
{
byte[] fileData;
fileReader = new FileStream(file, FileMode.Open, FileAccess.Read);
webRequest = (HttpWebRequest)WebRequest.Create(uri);
webRequest.Method = "POST";
webRequest.ContentLength = fileReader.Length;
webRequest.Timeout = 600000;
webRequest.Credentials = CredentialCache.DefaultCredentials;
webRequest.AllowWriteStreamBuffering = false;
requestStream = webRequest.GetRequestStream();
long fileSize = fileReader.Length;
long remainingBytes = fileSize;
int numberOfBytesRead = 0, done = 0;
while (numberOfBytesRead < fileSize)
{
SetByteArray(out fileData, remainingBytes);
done = WriteFileToStream(fileData, requestStream);
numberOfBytesRead += done;
remainingBytes -= done;
}
fileReader.Close();
return true;
}
public int WriteFileToStream(byte[] fileData, Stream requestStream)
{
int done = fileReader.Read(fileData, 0, fileData.Length);
requestStream.Write(fileData, 0, fileData.Length);
return done;
}
private void SetByteArray(out byte[] fileData, long bytesLeft)
{
fileData = bytesLeft < MAX_CHUNK_SIZE ? new byte[bytesLeft] : new byte[MAX_CHUNK_SIZE];
}
My server looks like this:
[HttpPost]
[ActionName("AddAudio")]
public async Task<IHttpActionResult> AddAudio([FromUri]string name)
{
try
{
isReceivingFile = true;
byte[] receivedBytes = await Request.Content.ReadAsByteArrayAsync();
if (WriteAudio(receivedBytes, name) == true)
{
isReceivingFile = false;
return Ok();
}
else
{
isReceivingFile = false;
return BadRequest("ERROR: Audio could not be saved on server.");
}
}
catch (Exception ex)
{
isReceivingFile = false;
return BadRequest("ERROR: Audio could not be saved on server.");
}
}
public bool WriteAudio(byte[] receivedBytes, string fileName)
{
string file = Path.Combine(#"C:\Users\username\Desktop\UploadedFiles", fileName);
using (FileStream fs = File.Create(file))
{
fs.Write(receivedBytes, 0, receivedBytes.Length);
}
return true;
}
The server code has the original code I wrote for it, before deciding to try and make it work with streams. The server code still works if I send a small file (under 30 MB), but if I send a large file my server gets a "outofmemoryexception". I can't figure out how to make the server take in the data via a stream. In my search for solutions I've come across a lot of examples with sockets and TCPClient, but that's not how we want to do it on this project. Can anybody help, please?
if I send a large file my server gets a "outofmemoryexception"
Well, it's reading the entire stream into memory right here:
byte[] receivedBytes = await Request.Content.ReadAsByteArrayAsync();
What you want to do is copy the stream from one location to another, without loading it all into memory at once. Something like this should work:
[HttpPost]
[ActionName("AddAudio")]
public async Task<IHttpActionResult> AddAudio([FromUri]string name)
{
try
{
string file = Path.Combine(#"C:\Users\username\Desktop\UploadedFiles", fileName);
using (FileStream fs = new FileStream(file, FileMode.Create, FileAccess.Write,
FileShare.None, 4096, useAsync: true))
{
await Request.Content.CopyToAsync(fs);
}
return Ok();
}
catch (Exception ex)
{
return BadRequest("ERROR: Audio could not be saved on server.");
}
}
In my code I want to upload and download a specific JSON-File to a FTP-Server.
The serializing works great and also the upload. When i look up the file via - for example - FileZilla, the content of the file is correct. (on the server)
But when i download this file with my application (and with my code - NOT with FileZilla), I don't get any exceptions, but the file is nearly empty. This is the only content:
{}
And here is my code for downloading:
string ResponseDescription = "";
FtpWebRequest req = (FtpWebRequest)FtpWebRequest.Create("ftp://" + "ftp.strato.com" + "/" + verzeichnis + "/" + file.Name);
req.Method = WebRequestMethods.Ftp.DownloadFile;
req.Credentials = new NetworkCredential(this.benutzer, this.passwort);
req.UseBinary = true;
req.UsePassive = false;
req.Proxy = null;
try
{
FtpWebResponse response = (FtpWebResponse)req.GetResponse();
Stream stream = response.GetResponseStream();
byte[] buffer = new byte[2048];
FileStream fs = new FileStream(destinationFolder + #"/" + destinationFile.Name, FileMode.Create);
int ReadCount = stream.Read(buffer, 0, buffer.Length);
while (ReadCount > 0)
{
fs.Write(buffer, 0, ReadCount);
ReadCount = stream.Read(buffer, 0, buffer.Length);
}
ResponseDescription = response.StatusDescription;
fs.Close();
stream.Close();
return true;
}
catch (Exception e)
{
MessageBox.Show(e.Message); // TODO - better Errorhandling
return false;
}
I found the solution. It was my mistake. The problem wasnt the download - the code was correct (as you said).
After downloading the file the JSON-Deserialization runs and here was the cause. I had a little mistake in my deserialization-logic.
I was trying to test a resumable download using c#. I found that add range will help by looking at some blogs. But in the following code add range have no meaning.
Guys Suggest me how to solve the issue.
What method will be effective to make a Resumable download??
HttpWebRequest myrequest = null;
HttpWebResponse myresponse = null;
private int interval=2048;
public bool set_url(string todonwloads,string tosaves)
{
this.todownload = todonwloads;
this.tosave = tosaves;
return true;
}
public bool start_download()
{
myrequest = (HttpWebRequest)WebRequest.Create(this.todownload);
// it the following code.
//If i dont write addrange. It will download same portion of the file.
myrequest.AddRange(4000,8000);
try
{
myresponse = (HttpWebResponse)myrequest.GetResponse();
if (myresponse.StatusCode == HttpStatusCode.OK)
{
Stream ReceiveSteam = myresponse.GetResponseStream();
FileStream fs = new FileStream(
this.tosave,
FileMode.Create,
FileAccess.Write,
FileShare.None);
int reads;
byte[] buffer = new byte[this.interval];
while ((reads = ReceiveSteam.Read(
buffer,
0,
this.interval)) > 0)
{
fs.Write(buffer, 0, reads);
}
return true;
}
}
catch (WebException ex)
{
throw ex;
}
finally
{
if (myresponse != null)
{
myresponse.Close();
}
}
return false;
}
You currently create and overwrite the file every time you download a section of the file:
// Overwrites the file each time -\/
... = new FileStream(this.tosave, FileMode.Create, ...
You instead need to open or create the file using FileMode.OpenOrCreate, then seek to the last section of the file you wrote to:
// seek to the last end offset, you'll need to save this somehow
fs.Seek(lastOffset, SeekOrigin.Begin);
int reads;
byte[] buffer = new byte[this.interval];
while ((reads = ReceiveSteam.Read(buffer, 0, this.interval)) > 0)
{
fs.Write(buffer, 0, reads);
lastOffset += reads;
}
First time poster, long-time reader. I have a really annoying problem thats been getting on my nerves. Ive got a program set up so I listen for new files on an FTP server, if theres a new file I download it. From there I work on some of the information in the file, etc. My problem comes when I run through my sequence the second time. That is, on the first file I download everything is totally fine, but as soon as a new file gets detected and my program tries downloading it, my program just hangs.
private static void DownloadFile(string s)
{
try
{
FtpWebRequest request = (FtpWebRequest)WebRequest.Create("ftp://blabla.com/"+s);
request.Method = WebRequestMethods.Ftp.DownloadFile;
request.Credentials = new NetworkCredential("xxx" ,"zzz");
using (FtpWebResponse partResponse = (FtpWebResponse)request.GetResponse())
{
Stream partReader = partResponse.GetResponseStream();
byte[] buffer = new byte[1024];
FileInfo fi = new FileInfo(path);
FileStream memStream = fi.Create();
while (true)
{
int bytesRead = partReader.Read(buffer, 0, buffer.Length - 1);
if (bytesRead == 0)
break;
memStream.Write(buffer, 0, bytesRead);
}
partResponse.Close();
memStream.Close();
}
Console.WriteLine(DateTime.Now + " file downloaded");
MoveFileToInProgress(s);
}
catch (Exception e)
{
Console.WriteLine(e.Message);
}
}
The line it hangs on is this one:
using (FtpWebResponse partResponse = (FtpWebResponse)request.GetResponse())
The reason my method here is static is because Im just running it in a different project to test it.. My question here is, how come it only ever dies on the second file? Ive been staring myself blind for hours now!
I ran into this problem as well... try finishing your request first and then closing it before trying to retrieve the response. That worked for me (actually tried it after reading comment by MartinNielsen). Here is what I did.
// connect to the ftp site
FtpWebRequest ftpRequest = (FtpWebRequest)WebRequest.Create(ftpUri);
ftpRequest.Method = WebRequestMethods.Ftp.UploadFile;
ftpRequest.Credentials = new NetworkCredential(ftpUser, ftpPassword);
// setting proxy to null so that it does not go through the proxy
ftpRequest.Proxy = null;
// get file information
StreamReader fileStream = new StreamReader(filePath);
byte[] fileBytes = Encoding.UTF8.GetBytes(fileStream.ReadToEnd());
ftpRequest.ContentLength = fileBytes.Length;
fileStream.Close();
// open connection to ftp site
Stream ftpRequestStream = ftpRequest.GetRequestStream();
// write the file to the stream
ftpRequestStream.Write(fileBytes, 0, fileBytes.Length);
// close the stream
ftpRequestStream.Close();
// get the response from the server
FtpWebResponse ftpUploadResponse = (FtpWebResponse)ftpRequest.GetResponse();
string result = ftpUploadResponse.StatusDescription;
// close response
ftpUploadResponse.Close();
// return response to calling code
return result;
Here are a couple of the resources that I used when writing this code (won't let me post more than 2, there were more)
How to: Upload Files with FTP
Uploading a file -- "The requested URI is invalid for this FTP command"
I'm not expert on C# but I use this code to download files from my ftp:
public void Download(string filename)
{
// I try to download five times before crash
for (int i = 1; i < 5; i++)
{
try
{
FtpWebRequest ftp = (FtpWebRequest)FtpWebRequest.Create(Global.Path + "/" + filename);
ftp.Credentials = new NetworkCredential(User, Pass);
ftp.KeepAlive = false;
ftp.Method = WebRequestMethods.Ftp.DownloadFile;
ftp.UseBinary = true;
ftp.Proxy = null;
int buffLength = 2048;
byte[] buff = new byte[buffLength];
int contentLen;
string LocalDirectory = Application.StartupPath.ToString() + "/downloads/" + filename;
using (FileStream fs = new FileStream(LocalDirectory, FileMode.Create, FileAccess.Write, FileShare.None))
using (Stream strm = ftp.GetResponse().GetResponseStream())
{
contentLen = strm.Read(buff, 0, buffLength);
while (contentLen != 0)
{
fs.Write(buff, 0, contentLen);
contentLen = strm.Read(buff, 0, buffLength);
}
}
Process.Start(LocalDirectory);
break;
}
catch (Exception exc)
{
if (i == 5)
{
MessageBox.Show("Can't download, try number: " + i + "/5 \n\n Error: " + exc.Message,
"Problem downloading the file");
}
}
}
}
Tell me if it works for you :)
one my application features is tod download files from our ftp server. And of course this feature reqiures to cancel this operation (Cancel Downloading).
Now, my Download Function is as follows:
try
{
reqFTP = (FtpWebRequest)FtpWebRequest.Create(new Uri("ftp://" + uri + "/" + fileName));
reqFTP.Method = WebRequestMethods.Ftp.DownloadFile;
reqFTP.UseBinary = true;
reqFTP.Credentials = new NetworkCredential(ftpUserID, ftpPassword);
reqFTP.UsePassive = true;
response = (FtpWebResponse)reqFTP.GetResponse();
ftpStream = response.GetResponseStream();
_isItOutputStream = true;
string dataLengthString = response.Headers["Content-Length"];
int dataLength = 0;
if (dataLengthString != null)
{
dataLength = Convert.ToInt32(dataLengthString);
}
long cl = response.ContentLength;
int bufferSize = 4048;
int readCount;
byte[] buffer = new byte[bufferSize];
readCount = ftpStream.Read(buffer, 0, bufferSize);
outputStream = new FileStream(filePath + "\\" + fileName, FileMode.Create);
bool first = true;
while (readCount > 0)
{
outputStream.Write(buffer, 0, readCount);
_actualDownloaded += readCount;
if (this.InvokeRequired)
{
ProgressBarDel _progressDel = new ProgressBarDel(ProgressBar);
this.Invoke(_progressDel, new object[] { _actualDownloaded, first });
}
first = false;
readCount = ftpStream.Read(buffer, 0, bufferSize);
}
ftpStream.Close();
outputStream.Close();
response.Close();
_isItOutputStream = false;
return true;
}
catch (Exception ee)
{
_downloadException = ee.Message;
if (ftpStream != null && outputStream!=null )
if (ftpStream.CanRead && outputStream.CanWrite)
{
ftpStream.Close();
outputStream.Close();
}
if (response != null)
response.Close();
return false;
}
Now as you can see in Catch Block you can see that i'm trying to interrupt this connection when the user clicks on Cancel button.
Canceling Operation Scenario:
1) Cancel button was clicked.
2) Call Function "DoSomeWorx()"
3) in "DoSomeWorx()" do:
if (_isItOutputStream)// where here i'm trying to check if it's downloading
{
ftpStream.Close();
outputStream.Close();
response.Close();
}
if (_startCopy)// check if copying to phone
{
IsCancelled();
}
_btnDownload2PhoneThread.Abort(); // actually this operation does what i did before but for some resoans it does this but it takes time...
_btnDownload2PhoneThread.Join();
The Problem is when I reach any of the following statments (ftpStream.Close();outputStream.Close();response.Close();)
it throws an exception "File unavailable(e.g file is busy)"
and this exception affects on re-downloading operation where it sees the file busy.
so how to avoid this exception?
I'm assuming that you have a form of some sort, so your performing the download on a thread.
What your probably better off doing is checking a "cancel" flag inside your while loop.
eg.
while(readcount > 0 && !cancel)
{
...
}
Then let your method gracefully cancel out.
Secondly you should use a using statement on your streams. That means that if you throw an exception, the finally block will guarantee that your stream gets disposed (which incidentally is why you would be receiving file busy, as the stream hasn't yet had it's destructor run, even though your method has completed)