How can I easily download a file from internet in C#?
Thank you very much.
WebClient.DownloadFIle is one simple way.
Have a look at the BCL class System.Net.WebRequest. Here's a brief example:
using System.Net;
...
var request = WebRequest.Create(new Uri("http://..."));
var response = request.GetResponse();
var responseStream = response.GetResponseStream();
// ^^^^^^^^^^^^^^
// read web resource content through this stream object!
...
Take note:
Some of these objects are IDisposable, so in real code you might want to wrap them in using blocks.
The above code example does not do any error checking. You might want to add appropriate code for this, also.
Related
I am trying to read a response from a server that I receive when I send a POST request. Viewing fiddler, it says it is a JSON response. How do I decode it to a normal string using C# Winforms with preferably no outside APIs. I can provide additional code/fiddler results if you need them.
The fiddler and gibberish images:
The gibberish came from my attempts to read the stream in the code below:
Stream sw = requirejs.GetRequestStream();
sw.Write(logBytes, 0, logBytes.Length);
sw.Close();
response = (HttpWebResponse)requirejs.GetResponse();
Stream stream = response.GetResponseStream();
StreamReader sr = new StreamReader(stream);
MessageBox.Show(sr.ReadToEnd());
As mentioned in the comments, Newtonsoft.Json is really a good library and worth using -- very lightweight.
If you really want to only use Microsoft's .NET libraries, also consider System.Web.Script.Serialization.JavaScriptSerializer.
var serializer = new System.Web.Script.Serialization.JavaScriptSerializer();
var jsonObject = serializer.DeserializeObject(sr.ReadToEnd());
Going to assume (you haven't clarified yet) that you need to actually decode the stream, since A) retrieving a remote stream of text is well documented, and B) you can't do anything much with a non-decoded JSON stream.
Your best course of action is to implement System.Web.Helpers.Json:
using System.Web.Helpers.Json
...
var jsonObj = Json.Decode(jsonStream);
I would like to load an excel file directly from an ftp site into a memory stream. Then I want to open the file in the FarPoint Spread control using the OpenExcel(Stream) method. My issue is I'm not sure if it's possible to download a file directly into memory. Anyone know if this is possible?
Yes, you can download a file from FTP to memory.
I think you can even pass the Stream from the FTP server to be processed by FarPoint.
WebRequest request = FtpWebRequest.Create("ftp://asd.com/file");
using (WebResponse response = request.GetResponse())
{
Stream responseStream = response.GetResponseStream();
OpenExcel(responseStream);
}
Using WebClient you can do nearly the same. Generally using WebClient is easier but gives you less configuration options and control (eg.: No timeout setting).
WebClient wc = new WebClient();
using (MemoryStream stream = new MemoryStream(wc.DownloadData("ftp://asd.com/file")))
{
OpenExcel(stream);
}
Take a look at WebClient.DownloadData. You should be able to download the file directory to memory and not write it to a file first.
This is untested, but something like:
var spreadSheetStream
= new MemoryStream(new WebClient().DownloadData(yourFilePath));
I'm not familiar with FarPoint though, to say whether or not the stream can be used directly with the OpenExcel method. Online examples show the method being used with a FileStream, but I'd assume any kind of Stream would be accepted.
Download file from URL to memory.
My answer does not exactly show, how to download a file for use in Excel, but shows how to create a generic-purpose in-memory byte array.
private static byte[] DownloadFile(string url)
{
byte[] result = null;
using (WebClient webClient = new WebClient())
{
result = webClient.DownloadData(url);
}
return result;
}
In the project I have in mind I want to be able to look at a website, retrieve text from that website, and do something with that information later.
My question is what is the best way to retrieve the data(text) from the website. I am unsure about how to do this when dealing with a static page vs dealing with a dynamic page.
From some searching I found this:
WebRequest request = WebRequest.Create("anysite.com");
// If required by the server, set the credentials.
request.Credentials = CredentialCache.DefaultCredentials;
// Get the response.
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
// Display the status.
Console.WriteLine(response.StatusDescription);
Console.WriteLine();
// Get the stream containing content returned by the server.
using (Stream dataStream = response.GetResponseStream())
{
// Open the stream using a StreamReader for easy access.
StreamReader reader = new StreamReader(dataStream, Encoding.UTF8);
// Read the content.
string responseString = reader.ReadToEnd();
// Display the content.
Console.WriteLine(responseString);
reader.Close();
}
response.Close();
So from running this on my own I can see it returns the html code from a website, not exactly what I'm looking for. I eventually want to be able to type in a site (such as a news article), and return the contents of the article. Is this possible in c# or Java?
Thanks
I hate to brake this to you but that's how webpages looks, it's a long stream of html markup/content. This gets rendered by the browser as what you see on your screen. The only way I can think of is to parse to html by yourself.
After a quick search on google I found this stack overflow article.
What is the best way to parse html in C#?
but I'm betting you figured this would be a bit easier than you expected, but that's the fun in programming always challenging problems
You can just use a WebClient:
using(var webClient = new WebClient())
{
string htmlFromPage = webClient.DownloadString("http://myurl.com");
}
In the above example htmlFromPage will contain the HTML which you can then parse to find the data you're looking for.
What you are describing is called web scraping, and there are plenty of libraries that do just that for both Java and C#. It doesn't really matter if the target site is static or dynamic since both output HTML in the end. JavaScript or Flash heavy sites on the other hand tend to be problematic.
Please try this,
System.Net.WebClient wc = new System.Net.WebClient();
string webData = wc.DownloadString("anysite.com");
I'm creating a general-purpose FTP class for .NET. I seem to have things working nicely but there are a few details I'm a little unsure about. And documentation such as that on MSDN provides absolutely no information on these details.
For example, when performing request methods such as WebRequestMethods.Ftp.UploadFile and WebRequestMethods.Ftp.RemoveDirectory, can I safely ignore the contents of the response? Or could there ever be information in the response that I need to check for?
I'm pretty sure GetResponse() throws an exception for pretty much all error conditions, but how can I be sure? Again, the docs just don't seem to cover this type of information.
Can anyone address this question or recommend the documentation I should be using?
Yes you can ignore these; however I would suggest ensuring that these are accessed in a controlled manner and within try...finally tags, ensuring the response is closed in the finally tag:
using (FtpWebResponse response = (FtpWebResponse)ftpWebRequest.GetResponse())
{
try
{
using (Stream dataStream = response.GetResponseStream())
{
using (StreamReader reader = new StreamReader(dataStream))
{
return reader.ReadToEnd();
}
}
}
finally
{
response.Close();
}
}
I'm trying to download a file from the web and save it locally, but I get an exception:
C# The process cannot access the file
'blah' because it is being used by
another process.
This is my code:
File.Create("data.csv"); // create the file
request = (HttpWebRequest)WebRequest.CreateDefault(new Uri(url));
request.Timeout = 30000;
response = (HttpWebResponse)request.GetResponse();
using (Stream file = File.OpenWrite("data.csv"), // <-- Exception here
input = response.GetResponseStream())
{
// Save the file using Jon Skeet's CopyStream method
CopyStream(input, file);
}
I've seen numerous other questions with the same exception, but none of them seem to apply here. Any help?
Update:
Thanks for the answers! Removing the File.Create(...) fixed it!
One comment on the documentation of OpenWrite: it is a little misleading, the brief description says:
Opens an existing file for writing.
The detailed description says:
If the file exists, it is opened for
writing at the beginning. The existing
file is not truncated.
Update 2.0:
It looks like the discrepancy is between IntelliSense/F1 and the online documentation. I thought it should be the same since I allow F1 to connect to the web when it's displaying documentation.
File.Create returns a FileStream - which you're not closing. That means you won't be able to open another stream writing to the same file until the finalizer has closed the existing stream.
Just get rid of the call to File.Create - File.OpenWrite will create it anyway. Alternatively, keep the FileStream around to write to:
using (Stream file = File.Create("data.csv"))
{
request = (HttpWebRequest)WebRequest.CreateDefault(new Uri(url));
request.Timeout = 30000;
using (var response = (HttpWebResponse)request.GetResponse())
using (Stream input = response.GetResponseStream())
{
// Save the file using Jon Skeet's CopyStream method
CopyStream(input, file);
}
}
Note that I'm also disposing of the WebResponse here, which you should do to make sure the connection is freed to the connection pool.
looks like File.Create returns an open FileStream object
http://msdn.microsoft.com/en-us/library/aa328775(v=VS.71).aspx
try
using (FileStream fs = File.Create("data.csv"))
and leave off the first File.Create
The problem is that the File.Create method actually opens the file and returns a FileStream object. It won't close the file until the object is garbage collected (which will happen at an indeterminate time). The FileStream object still gets created, regardless of whether or not you use it. Try this:
using (Stream file = File.Create("data.csv"),
input = response.GetResponseStream())
{
// Save the file using Jon Skeet's CopyStream method
CopyStream(input, file);
}
First. File.Create will return a stream that you should use for accessing the file.
Second, if that doesn't resolve the issue, if you google who lock you will find a program that let's you determine what process is accessing a file at the same time.