I'm trying to get a link from a pastebin. Where the link is the only text in the raw paste. Then I want to download a file from the link in pastebin.
WebRequest request = WebRequest.Create("http://pastebin.com/raw/Dtdf2qMp");
WebResponse response = request.GetResponse();
System.IO.StreamReader reader = new
System.IO.StreamReader(response.GetResponseStream());
Console.WriteLine(reader.ReadToEnd());
WebClient client = new WebClient();
client.DownloadFile (Link gotten from pastebin here, "c:\\File");
System.Threading.Thread.Sleep(5000);
Instead of dumping the text read to console output, you should assign it to a variable.
var pastebinOutput = reader.ReadToEnd();
Then just pass that as the link for the DownloadFile method. If you want to do verification that it's actually a URL you got from the original pastebin, you can look into System.Uri's TryCreate method.
I've got a solution - assuming you have your link in the raw pastebin link (mine is a .txt file saying 'it worked') I suggest you copy and paste the code below exactly - if you get an file saying 'it worked' then you can change the pastebin link & file names. If you don't want to open the file then remove Process.Start - if you want to change the delay just change the number (it's in milliseconds) Also, you can change the format from .txt to .exe or whatever your file is (or you can remove it so its the defualt name in the download link):
WebRequest request = WebRequest.Create("https://pastebin.com/raw/QAWufg1z");
WebResponse response = request.GetResponse();
System.IO.StreamReader reader = new
System.IO.StreamReader(response.GetResponseStream());
var pastebinOutput = reader.ReadToEnd();
WebClient client = new WebClient();
client.DownloadFile(pastebinOutput, #".\downloaded.txt");
MessageBox.Show("File should open automatically in the next minute. Please wait...");
await Task.Delay(3000); //3000 = 3 seconds
Process.Start(#".\downloaded.txt");
Related
I'm quite new to using RestSharp and I've got a question that I can't find an answer to here on SO.
I've have this situation where I must download a csv-file and output the file directly in the browser. The following code illustrates how to download a file and save it to a certain path on disc.
string tempFile = Path.GetTempFileName();
using (var writer = File.OpenWrite(tempFile))
{
var client = new RestClient(baseUrl);
var request = new RestRequest("Assets/LargeFile.7z");
request.ResponseWriter = (responseStream) => responseStream.CopyTo(writer);
var response = client.DownloadData(request);
}
I want to download the csv-file and directly output the result as a download file in the browser. You know, like in Chrome the file you download will be displayed in the left bottom corner of your browser.
Can this be done using RestSharp? And if so, how? Got an example? Please share it. ;-)
Thanx!
I am developing an application which is showing web pages through a web browser control.
When I click the save button, the web page with images should be stored in local storage. It should be save in .html format.
I have the following code:
WebRequest request = WebRequest.Create(txtURL.Text);
WebResponse response = request.GetResponse();
Stream data = response.GetResponseStream();
string html = String.Empty;
using (StreamReader sr = new StreamReader(data))
{
html = sr.ReadToEnd();
}
Now string html contains the webpage content. I need to save this into D:\Cache\
How do i save the html contents to disk?
You can use this code to write your HTML string to a file:
var path= #"D:\Cache\myfile.html";
File.WriteAllText(path, html);
Further refinement: Extract the filename from your (textual) URL.
Update:
See Get file name from URI string in C# for details. The idea is:
var uri = new Uri(txtUrl.Text);
var filename = uri.IsFile
? System.IO.Path.GetFileName(uri.LocalPath)
: "unknown-file.html";
you have to write below code on save button
File.WriteAllText(path, browser.Document.Body.Parent.OuterHtml, Encoding.GetEncoding(browser.Document.Encoding));
Now the 'Body.parent' must save whole the page instead of just saving only part.
check it.
There is nothing built-in to the .NET Framework as far I know.
So my approach would be like below:
Use System.NET.HttpWebRequest to get the main HTML document as a
string or stream (easy). (Which you have done already)
Load this into a HTMLAgilityPack document where you can now easily
query the document to get lists of all image elements, stylesheet
links, etc.
Then make a separate web request for each of these files and save
them to a subdirectory.
Finally update all relevent links in the main page to point to the
items in the subdirectory.
In the project I have in mind I want to be able to look at a website, retrieve text from that website, and do something with that information later.
My question is what is the best way to retrieve the data(text) from the website. I am unsure about how to do this when dealing with a static page vs dealing with a dynamic page.
From some searching I found this:
WebRequest request = WebRequest.Create("anysite.com");
// If required by the server, set the credentials.
request.Credentials = CredentialCache.DefaultCredentials;
// Get the response.
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
// Display the status.
Console.WriteLine(response.StatusDescription);
Console.WriteLine();
// Get the stream containing content returned by the server.
using (Stream dataStream = response.GetResponseStream())
{
// Open the stream using a StreamReader for easy access.
StreamReader reader = new StreamReader(dataStream, Encoding.UTF8);
// Read the content.
string responseString = reader.ReadToEnd();
// Display the content.
Console.WriteLine(responseString);
reader.Close();
}
response.Close();
So from running this on my own I can see it returns the html code from a website, not exactly what I'm looking for. I eventually want to be able to type in a site (such as a news article), and return the contents of the article. Is this possible in c# or Java?
Thanks
I hate to brake this to you but that's how webpages looks, it's a long stream of html markup/content. This gets rendered by the browser as what you see on your screen. The only way I can think of is to parse to html by yourself.
After a quick search on google I found this stack overflow article.
What is the best way to parse html in C#?
but I'm betting you figured this would be a bit easier than you expected, but that's the fun in programming always challenging problems
You can just use a WebClient:
using(var webClient = new WebClient())
{
string htmlFromPage = webClient.DownloadString("http://myurl.com");
}
In the above example htmlFromPage will contain the HTML which you can then parse to find the data you're looking for.
What you are describing is called web scraping, and there are plenty of libraries that do just that for both Java and C#. It doesn't really matter if the target site is static or dynamic since both output HTML in the end. JavaScript or Flash heavy sites on the other hand tend to be problematic.
Please try this,
System.Net.WebClient wc = new System.Net.WebClient();
string webData = wc.DownloadString("anysite.com");
I am downloading image files from web using the following code in my Console Application.
WebClient client = new WebClient();
client.DownloadFile(string address_of_image_file,string filename);
The code is running absolutely fine.
I want to know if there is a way i can get the size of this image file before I download it.
PS- Actually I have written code to make a crawler which moves around the site downloading image files. So I doesn't know its size beforehand. All I have is the complete path of file which has been extracted from the source of webpage.
Here is a simple example you can try
if you have files of different extensions like .GIF, .JPG, etc
you can create a variable or wrap the code within a Switch Case Statement
System.Net.WebClient client = new System.Net.WebClient();
client.OpenRead("http://someURL.com/Images/MyImage.jpg");
Int64 bytes_total= Convert.ToInt64(client.ResponseHeaders["Content-Length"])
MessageBox.Show(bytes_total.ToString() + " Bytes");
If the web-service gives you a Content-Length HTTP header then it will be the image file size. However, if the web-service wants to "stream" data to you (using Chunk encoding), then you won't know until the whole file is downloaded.
You can use this code:
using System.Net;
public long GetFileSize(string url)
{
long result = 0;
WebRequest req = WebRequest.Create(url);
req.Method = "HEAD";
using (WebResponse resp = req.GetResponse())
{
if (long.TryParse(resp.Headers.Get("Content-Length"), out long contentLength))
{
result = contentLength;
}
}
return result;
}
You can use an HttpWebRequest to query the HEAD Method of the file and check the Content-Length in the response
You should look at this answer: C# Get http:/…/File Size where your question is fully explained. It's using HEAD HTTP request to retrieve the file size, but you can also read "Content-Length" header during GET request before reading response stream.
I am using WebClient.DownloadFile to download a small executable file from the internet. This method is working very well. However, I would now like to download this executable file into a byte array rather than onto my hard drive. I did some reading and came across the WebClient.DownloadData method. The problem that I am having with the downloadData method is that rather than downloading my file, my code is downloading the HTML data behind my file's download page.
I have tried using dozens of sites - each brings me the same issue. Below is the code I am using.
// Create a new instance of the System.Net 'WebClient'
System.Net.WebClient client = new System.Net.WebClient();
// Download URL
Uri uri = new Uri("http://www35.multiupload.com:81/files/4D7B4D2BFC3F1A9F765A433BA32ED2C5883D0CE133154A0FDB7E7786547A3165DA62393141C4AF8FF36C75222566CF3EB64AF6FBCFC02099BB209C891529CF7B90C83D9C63D39D989CBB8ECE6DE2B83B/Project1.exe");
byte[] dbytes = client.DownloadData(uri);
MessageBox.Show(dbytes.Length.ToString()); // Not the size of my file
Keep in mind that I am attempting to download the data of an executable file into a byte array.
Thank you for any help,
Evan
You are attempting to download a file using an expired token url. See below:
URL: http://www35.multiupload.com:81/files/4D7B4D2BFC3F1A9F765A433BA32ED2C5883D0CE133154A0FDB7E7786547A3165DA62393141C4AF8FF36C75222566CF3EB64AF6FBCFC02099BB209C891529CF7B90C83D9C63D39D989CBB8ECE6DE2B83B/Project1.exe`
Server: www35
Token:
4D7B4D2BFC3F1A9F765A433BA32ED2C5883D0CE133154A0FDB7E7786547A3165DA62393141C4AF8FF36C75222566CF3EB64AF6FBCFC02099BB209C891529CF7B90C83D9C63D39D989CBB8ECE6DE2B83B
You can't just download a file by waiting for the timer to end, and copy the direct link, it's a "token" link. It will only work for a specified period of time before redirecting you back to the download page (which is why you are getting HTML instead of binary data).
Workaround
You will have to download the multiupload's HTML and parse the direct download link from the HTML source code. Only this way provides a sure-fire way of getting an up-to-date token url.
How #Dark Slipstream said, you're attempting to download a file using an expired token url
look how get the new url:
System.Net.WebClient client = new System.Net.WebClient();
// Download URL
Uri uri = new Uri("http://www.multiupload.com/39QMACX7XS");
byte[] dbytes = client.DownloadData(uri);
string responseStr = System.Text.Encoding.ASCII.GetString(dbytes);
HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(responseStr);
string urlToDownload = doc.DocumentNode.SelectNodes("//a[contains(#href,'files/')]")[0].Attributes["href"].Value;
byte[] data = client.DownloadData(uri);
length = data.Length;
I dont parsing the exceptions