We Keep Coding

C# How to download csv file from url [duplicate]

This question already has an answer here:
HttpWebRequest Unable to download data from nasdaq.com but able from browsers
(1 answer)
Closed 2 years ago.
I know it's kinda a silly question but i've read a lot of forums and nothing didn't work for me.
("https://www.nasdaq.com/api/v1/historical/HPQ/stocks/2010-11-14/2020-11-14")
I got url where i need to download csv file. When i paste this url into browsers it works fine but
when i paste it in my app it doesnt' work at all. My app just stop responding and create empty file.
WebClient webClient = new WebClient();
webClient.DownloadFile(
"https://www.nasdaq.com/api/v1/historical/HPQ/stocks/2010-11-14/2020-11-14",
#"HistoryDataStocks.csv");

You need send the proper web request. Try this code and it will work:
var request = (HttpWebRequest)WebRequest.Create(url);
request.Timeout = 30000;
request.AllowWriteStreamBuffering = false;
using (var response = (HttpWebResponse)request.GetResponse())
using (var s = response.GetResponseStream())
using (var fs = new FileStream("test.csv", FileMode.Create))
{
byte[] buffer = new byte[4096];
int bytesRead;
while ((bytesRead = s.Read(buffer, 0, buffer.Length)) > 0)
{
fs.Write(buffer, 0, bytesRead);
bytesRead = s.Read(buffer, 0, buffer.Length);
}
}
The file stream will contain your file.

Write to web text file

I am programming in Microsoft Visual C# 2010 Express.
I have a text file in a folder on my web server containing one character: '0'.
When I start my C# Application I want to read the number from my text file, increase it by 1, and then save the new number instead.
I've browsed the web but I can't find a good answer. All I get is questions and answers about writing/reading from local text files.
So basically, I want to write some text to a text file which is not on my computer but here:
http://mywebsite.xxx/something/something/myfile.txt
Is this possible?

You may have to adjust the path directory, but this works:
string path = Path.GetDirectoryName(AppDomain.CurrentDomain.SetupInformation.ConfigurationFile) + "\\something\\myfile.txt";
string previousNumber = System.IO.File.ReadAllText(path);
int newNumber;
if (int.TryParse(previousNumber, out newNumber))
{
newNumber++;
using (FileStream fs = File.Create(path, 1024))
{
Byte[] info = new UTF8Encoding(true).GetBytes(newNumber.ToString());
fs.Write(info, 0, info.Length);
}
}

I found a working solution, using File Transport Protocol as Barta Tamás mentioned.
However, I learned from Michael Todd that this is not safe, so I will not use it in my own application, but maybe it can be helpful to someone else.
I found information about uploading files using FTP here: http://msdn.microsoft.com/en-us/library/ms229715.aspx
void CheckNumberOfUses()
{
// Get the objects used to communicate with the server.
FtpWebRequest ftpRequest = (FtpWebRequest)WebRequest.Create("ftp://mywebsite.xx/public_html/something1/something2/myfile.txt");
HttpWebRequest httpRequest = (HttpWebRequest)WebRequest.Create("http://mywebsite.xx/something1/something2/myfile.txt");
StringBuilder sb = new StringBuilder();
byte[] buf = new byte[8192];
HttpWebResponse response = (HttpWebResponse)httpRequest.GetResponse();
Stream resStream = response.GetResponseStream();
string tempString = null;
int count = resStream.Read(buf, 0, buf.Length);
if (count != 0)
{
tempString = Encoding.ASCII.GetString(buf, 0, count);
int numberOfUses = int.Parse(tempString) + 1;
sb.Append(numberOfUses);
}
ftpRequest.Method = WebRequestMethods.Ftp.UploadFile;
// This example assumes the FTP site uses anonymous logon.
ftpRequest.Credentials = new NetworkCredential("login", "password");
// Copy the contents of the file to the request stream.
byte[] fileContents = Encoding.UTF8.GetBytes(sb.ToString());
ftpRequest.ContentLength = fileContents.Length;
Stream requestStream = ftpRequest.GetRequestStream();
requestStream.Write(fileContents, 0, fileContents.Length);
requestStream.Close();
FtpWebResponse ftpResponse = (FtpWebResponse)ftpRequest.GetResponse();
ftpResponse.Close();
}
Read in the comments of the question how this can be done better, not using FTP. My solution is not suggested if you have important files on your server.

FtpWebRequest download text file: CR/LF removed

I am using FtpWebRequest to download files, but in all text files all \r\n are removed when downloaded.
What am I doing wrong?
Uri u = new Uri(msg);
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(u);
request.Method = WebRequestMethods.Ftp.DownloadFile;
request.Credentials = credential;
request.UsePassive = true;
request.UseBinary = true;
request.KeepAlive = false;
//Get a reponse
WebResponse response = request.GetResponse();
Stream responseStream = response.GetResponseStream();
FileStream localfileStream = new FileStream(destination,
FileMode.Create, FileAccess.Write);
//create the file
byte[] buffer = new byte[1024];
int bytesRead = responseStream.Read(buffer, 0, 1024);
while (bytesRead != 0)
{
localfileStream.Write(buffer, 0, bytesRead);
bytesRead = responseStream.Read(buffer, 0, 1024);
}
localfileStream.Close();
response.Close();
responseStream.Close();.

Your code is correct. There is something else you are not telling that causing the problem. My guess would be
a) File on the server does not have \r\n
b) The way how you check that it does not have them on the client is wrong. Either you are checking a wrong file, or it had gone additional transformation or something else.

I've had a similar problem to this. The code I used is almost identical to yours. I found that the code for me was actually working correctly, but the file on the server only contained the the "LF" character at the end of the line, not the "CR/LF" combo. Most text editors ignore this, and display the text as one continuous line.

How to Download Zip file from Ftp in C#

I have to download zip file from ftp using c# code.
i have used the following code.
Uri url = new Uri("ftp://ftpurl");
if (url.Scheme == Uri.UriSchemeFtp)
{
FtpWebRequest objRequest = (FtpWebRequest)FtpWebRequest.Create(url);
//Set credentials if required else comment this Credential code
NetworkCredential objCredential = new NetworkCredential(userid, Pwd);
objRequest.Credentials = objCredential;
objRequest.Method = WebRequestMethods.Ftp.DownloadFile;
FtpWebResponse objResponse = (FtpWebResponse)objRequest.GetResponse();
StreamReader objReader = new StreamReader(objResponse.GetResponseStream());
byte[] buffer = new byte[16 * 1024];
int len = 0;
FileStream objFS = new FileStream(#"E:\ftpwrite", FileMode.Create, FileAccess.Write, FileShare.Read);
while ((len = objReader.BaseStream.Read(buffer, 0, buffer.Length)) != 0)
{
objFS.Write(buffer, 0, len);
}
objFS.Close();
objResponse.Close();
}
but this code is not giving me the correct response as i want to save file from ftp and this code is writing the data from file in bytes to my file.
my file is a zip file not a text file.
please help me what should i have to do or i am mistunderstanding.

My guess is that it's something to do with the fact that you're using StreamReader. It's possible that on construction it's reading from the stream to try to determine the encoding. As you're not really using it - just the BaseStream - it's pointless and leads to unclear code.
Just use:
Stream inputStream = objResponse.GetResponseStream();
Additionally, you should use using statements for all the streams, and the response.
Oh, and if you're using .NET 4 or higher, use Stream.CopyTo to save yourself some time.

If you're not opposed to using free 3rd party libraries, you can use http://www.enterprisedt.com/products/edtftpnet/download.html
It makes accessing the FTP a lot simpler IMO. Sample code (taken and slightly modified from their documentation):
using (FTPConnection ftp = new FTPConnection())
{
ftpConnection.ServerAddress = "myserver";
ftpConnection.UserName = userName;
ftpConnection.Password = password;
ftpConnection.Connect();
ftpConnection.DownloadFile(localFilePath, remoteFileName);
}

Simple file download via HTTP - is this sufficient?

I need basic file downloading capabilities in my app and I cannot use WebClient.DownloadFile [1]. Is this (naïve?) implementation of a DownloadFile method enough? Are there any pitfalls that I don't address with this implementation?
public static void DownloadFile(String url, String destination)
{
using (var request = (HttpWebRequest)WebRequest.Create(url))
{
request.Method = "GET";
request.Timeout = 100000; // 100 seconds
using (var response = request.GetResponse())
{
using (var responseStream = response.GetResponseStream())
{
using (var fileStream = File.Open(destination,
FileMode.Create,
FileAccess.Write,
FileShare.None))
{
var MaxBytesToRead = 10 * 1024;
var buffer = new Byte[MaxBytesToRead];
var totalBytesRead = 0;
var bytesRead = responseStream.Read(buffer,
0,
MaxBytesToRead);
while (bytesRead > 0)
{
totalBytesRead += bytesRead;
fileStream.Write(buffer, 0, bytesRead);
bytesRead = responseStream.Read(buffer,
0,
MaxBytesToRead);
}
}
}
}
}
}
Thanks!
[1] .Net Compact Framework...

You're keeping track of totalBytesRead, but I can't see it used anywhere.
Since Method = "GET" is the default, I don't see anything that's specific to HTTP. If you remove the (HttpWebRequest) cast and the Method = line then you'll gain the ability to download over other protocols, such as FTP. Currently the code will throw an exception if somebody provides a URL other than http://.

Response should have a Content-Length header (unless content-encoding = chunked) which you can use to validate that the download was not interrupted.
Other than that, your implementation looks fine by me.