C# downloading a csv file from https site - c#

I am trying to download a file into my app. From what I can see if I put this link into a browser I get a good file downloaded so I know the file is available for me to get. I have other files to download from different sites and they all work well but this one will not download for me in a usable manner. I guess I do not understand something, do you know the key fact I am missing?
After many attempts I have now coded the following
private string url =
"https://coronavirus.data.gov.uk/api/v1/data?filters=areaType=nation&structure={\"Name\":\"areaName\",\"date\":\"date\",\"FirstDose\":\"cumPeopleVaccinatedFirstDoseByPublishDate\",\"SecondDose\":\"cumPeopleVaccinatedSecondDoseByPublishDate\"}&format=csv";
private void btn_wc1_Click(object sender, EventArgs e)
{
WebClient wc = new WebClient();
wc.Encoding = Encoding.UTF8;
wc.DownloadFile(url, "wc1_uk_data.csv");
}
private void btn_wc2_Click(object sender, EventArgs e)
{
using (var webClient = new WebClient())
{
webClient.Encoding = Encoding.UTF8;
string s = webClient.DownloadString(url);
File.WriteAllText(#"wc2_uk_data.csv", s);
}
}
private async void btn_https_Click(object sender, EventArgs e)
{
HttpClient _client = new HttpClient();
byte[] buffer = null;
try
{
HttpResponseMessage task = await _client.GetAsync(url);
Stream task2 = await task.Content.ReadAsStreamAsync();
using (MemoryStream ms = new MemoryStream())
{
await task2.CopyToAsync(ms);
buffer = ms.ToArray();
}
File.WriteAllBytes("https_uk_data.csv", buffer);
}
catch
{
}
}
private void btn_wc3_Click(object sender, EventArgs e)
{
using (var webClient = new WebClient())
{
webClient.Encoding = Encoding.UTF8;
byte[] myDataBuffer = webClient.DownloadData(url);
MemoryStream ms = new MemoryStream(myDataBuffer);
FileStream f = new FileStream(Path.GetFullPath(Path.Combine(Application.StartupPath, "wc3_uk_data.csv")),
FileMode.OpenOrCreate);
ms.WriteTo(f);
f.Close();
ms.Close();
}
}
Using the following UI
All the different functions above will download a file but none of the downloaded files is usable. It seems like it is not the file but maybe something to do with information regarding the file. As my app does not know what to do with this I never reply with what ever the other end wants. I guess if I replied the next set of data I got would be the file.
If I put the URL into a browser then I get a file that is good. This link is good at the moment.
https://coronavirus.data.gov.uk/api/v1/data?filters=areaType=nation&structure={"Name":"areaName","date":"date","FirstDose":"cumPeopleVaccinatedFirstDoseByPublishDate","SecondDose":"cumPeopleVaccinatedSecondDoseByPublishDate"}&format=csv
Anyone got any idea on what I need to do in my app to get the file like the browser does?

You need to set the WebClient.Encoding before calling DownloadString
string url = "https://coronavirus.data.gov.uk/api/v1/data?filters=areaType=nation&structure={\"Name\":\"areaName\",\"date\":\"date\",\"FirstDose\":\"newPeopleReceivingFirstDose\",\"SecondDose\":\"newPeopleReceivingSecondDose\"}&format=csv";
using (var webClient = new WebClient())
{
webClient.Encoding = Encoding.UTF8;
string s = webClient.DownloadString(url);
}
Here is a related question:
WebClient.DownloadString results in mangled characters due to encoding issues, but the browser is OK

Related

There is one Url for DownLoad Excel.When I paste that url in browser it downloads.But in C# it is not Working

There is one Url for DownLoad Excel.When I paste that url in browser it downloads.But when I use C# WebClient.DownLoadFile(source,Destination),It does not download.)
Well, showing more lines of codes will make solving your problem super fast.
However, you can try this
Using the WebClient class
using System.Net;
//...
WebClient Client = new WebClient();
var address = "http://i.stackoverflow.com/Content/Img/stackoverflow-logo-250.png";
var destination = #"C:\folder\stackoverflowlogo.png";
Client.DownloadFile(address, destination);
Edited: I noticed you're using credentials. Try this,
Uri ur = new Uri("http://remotehost.do/images/img.jpg");
using (WebClient client = new WebClient()) {
//client.Credentials = new NetworkCredential("username", "password");
String credentials = Convert.ToBase64String(Encoding.ASCII.GetBytes("Username" + ":" + "MyNewPassword"));
client.Headers[HttpRequestHeader.Authorization] = $"Basic {credentials}";
client.DownloadProgress += WebClientDownloadProgress;
client.DownloadDataDone += WebClientDownloadDone;
client.DownloadFileAsync(ur, #"C:\path\newImage.jpg");
}
Implement the functions:
void WebClientDownloadProgress(object sender, DownloadProgressEventArgs e)
{
Console.WriteLine("Download status: {0}%.", e.ProgressPercentage);
// updating the UI or do something else
}
void WebClientDownloadDone(object sender, DownloadDataDoneEventArgs e)
{
Console.WriteLine("Download finished!");
}
In summary, this might not be exactly what you want to do, but it gives you an idea of to use the authorizations. Happy coding!

Async Download file

I plan to write a function DownloadData return a byte array, another client will call it to get byte array. My point is I don't want client app is waiting file is download, so I need it download in async mode. But I so confuse how to do that.
This is my function:
public byte[] DownloadData(string serverUrlAddress, string path)
{
if(string.IsNullOrWhiteSpace(serverUrlAddress) || string.IsNullOrWhiteSpace(path))
return null;
// Create a new WebClient instance
WebClient client = new WebClient();
// Concatenate the domain with the Web resource filename.
string url = string.Concat(serverUrlAddress, "/", path);
if (url.StartsWith("http://") == false)
url = "http://" + url;
byte[] data = null;
client.DownloadDataCompleted += delegate(object sender, DownloadDataCompletedEventArgs e)
{
data = e.Result;
};
while (client.IsBusy) { }
return data;
}
I wrote a method that does just that.
public async Task<byte[]> DownloadData(string url)
{
TaskCompletionSource<byte[]> tcs = new TaskCompletionSource<byte[]>();
HttpWebRequest request = WebRequest.CreateHttp(url);
using (HttpWebResponse response = (HttpWebResponse)(await request.GetResponseAsync()))
using (Stream stream = response.GetResponseStream())
using (MemoryStream ms = new MemoryStream())
{
await stream.CopyToAsync(ms);
tcs.SetResult(ms.ToArray());
return await tcs.Task;
}
}
I know why I lost byte. On API, I return byte array, but I use HttpClient to get data. I change to HttpResponseMessage as return and accept type on both.

Upload image file in Windows Phone 7 Application to PHP

I am trying to upload a picture from the Pictures Library (on WP7) and save it in a folder on the server.
On the server, I'm using PHP to receive the file using POST Method.
The PHP Code is:
<?php
$uploads_dir = 'files/'; //Directory to save the file that comes from client application.
if ($_FILES["file"]["error"] == UPLOAD_ERR_OK) {
$tmp_name = $_FILES["file"]["tmp_name"];
$name = $_FILES["file"]["name"];
move_uploaded_file($tmp_name, "$uploads_dir/$name");
}
?>
I've already tried some approaches, but they all just seem to fail.
I've already done this job in a Windows Forms application using the Client.UploadFile method but it seems that it cannot be used on a Windows Phone Application.
I think httpwebrequest can help, right?
This is my C# code so far:
public partial class SamplePage : PhoneApplicationPage
{
public SamplePage()
{
InitializeComponent();
}
PhotoChooserTask selectphoto = null;
private void SampleBtn_Click(object sender, RoutedEventArgs e)
{
selectphoto = new PhotoChooserTask();
selectphoto.Completed += new EventHandler<PhotoResult>(selectphoto_Completed);
selectphoto.Show();
}
void selectphoto_Completed(object sender, PhotoResult e)
{
if (e.TaskResult == TaskResult.OK)
{
BinaryReader reader = new BinaryReader(e.ChosenPhoto);
image1.Source = new BitmapImage(new Uri(e.OriginalFileName));
txtBX.Text = e.OriginalFileName;
}
}
}
I read it somewhere that the image is required to be converted to a string of bytes, I don't know for sure.
But, please help me.
Thanks a lot in advance.
I would convert the image to base64 (see System.Convert) then transfer via POST:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://mydomain.cc/saveimage.php");
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
string postData = String.Format("image={0}", myBase64EncodedImage);
// Getting the request stream.
request.BeginGetRequestStream
(result =>
{
// Sending the request.
using (var requestStream = request.EndGetRequestStream(result))
{
using (StreamWriter writer = new StreamWriter(requestStream))
{
writer.Write(postData);
writer.Flush();
}
}
// Getting the response.
request.BeginGetResponse(responseResult =>
{
var webResponse = request.EndGetResponse(responseResult);
using (var responseStream = webResponse.GetResponseStream())
{
using (var streamReader = new StreamReader(responseStream))
{
string srresult = streamReader.ReadToEnd();
}
}
}, null);
}, null);
}
saveimage.php should look like this:
<?
function base64_to_image( $imageData, $outputfile ) {
/* encode & write data (binary) */
$ifp = fopen( $outputfile, "wb" );
fwrite( $ifp, base64_decode( $imageData ) );
fclose( $ifp );
/* return output filename */
return( $outputfile );
}
if (isset($_POST['image'])) {
base64_to_jpeg($_POST['image'], "my_path_to_store_images.jpg");
}
else
die("no image data found");
?>
Note: I have not tested the code. There might be typos or other errors. It's just to illustrate how I would transfer an image using POST.
Edit as a repy to your comment: I don't have code to encode to base64 at hand but here is how you would decode a base64 encoded image in C#:
byte[] image = Convert.FromBase64String(str);

How can i download a website content to a string?

I tried this and i want that the source content of the website will be download to a string:
public partial class Form1 : Form
{
WebClient client;
string url;
string[] Search(string SearchParameter);
public Form1()
{
InitializeComponent();
url = "http://chatroll.com/rotternet";
client = new WebClient();
webBrowser1.Navigate("http://chatroll.com/rotternet");
}
private void Form1_Load(object sender, EventArgs e)
{
}
static void DownloadDataCompleted(object sender,
DownloadDataCompletedEventArgs e)
{
}
public string SearchForText(string SearchParameter)
{
client.DownloadDataCompleted += DownloadDataCompleted;
client.DownloadDataAsync(new Uri(url));
return SearchParameter;
}
I want to use WebClient and downloaddataasync and in the end to have the website source content in a string.
No need for async, really:
var result = new System.Net.WebClient().DownloadString(url)
If you don't want to block your UI, you can put the above in a BackgroundWorker. The reason I suggest this rather than the Async methods is because it is dramatically simpler to use, and because I suspect you are just going to stick this string into the UI somewhere anyway (where BackgroundWorker will make your life easier).
If you are using .Net 4.5,
public async void Downloader()
{
using (WebClient wc = new WebClient())
{
string page = await wc.DownloadStringTaskAsync("http://chatroll.com/rotternet");
}
}
For 3.5 or 4.0
public void Downloader()
{
using (WebClient wc = new WebClient())
{
wc.DownloadStringCompleted += (s, e) =>
{
string page = e.Result;
};
wc.DownloadStringAsync(new Uri("http://chatroll.com/rotternet"));
}
}
Using WebRequest:
WebRequest request = WebRequest.Create(url);
request.Method = "GET";
WebResponse response = request.GetResponse();
Stream stream = response.GetResponseStream();
StreamReader reader = new StreamReader(stream);
string content = reader.ReadToEnd();
reader.Close();
response.Close();
You can easily call the code from within another thread, or use background worer - that will make your UI responsive while retrieving data.

How to retrieve a webpage with C#?

How to retrieve a webpage and diplay the html to the console with C# ?
Use the System.Net.WebClient class.
System.Console.WriteLine(new System.Net.WebClient().DownloadString(url));
I have knocked up an example:
WebRequest r = WebRequest.Create("http://www.msn.com");
WebResponse resp = r.GetResponse();
using (StreamReader sr = new StreamReader(resp.GetResponseStream()))
{
Console.WriteLine(sr.ReadToEnd());
}
Console.ReadKey();
Here is another option, using the WebClient this time and do it asynchronously:
static void Main(string[] args)
{
System.Net.WebClient c = new WebClient();
c.DownloadDataCompleted +=
new DownloadDataCompletedEventHandler(c_DownloadDataCompleted);
c.DownloadDataAsync(new Uri("http://www.msn.com"));
Console.ReadKey();
}
static void c_DownloadDataCompleted(object sender,
DownloadDataCompletedEventArgs e)
{
Console.WriteLine(Encoding.ASCII.GetString(e.Result));
}
The second option is handy as it will not block the UI Thread, giving a better experience.
// Save HTML code to a local file.
WebClient client = new WebClient ();
client.DownloadFile("http://yoursite.com/page.html", #"C:\htmlFile.html");
// Without saving it.
string htmlCode = client.DownloadString("http://yoursite.com/page.html");

Categories