I'm trying to automate a task. I have to repeat the task 14 times and want my application to do it for me. my application runs once without a problem. Can anyone explain how I do this without duplicating code?
My application scrapes html from a url and then saves it to a location on my computer. (the html is an email signature). I need the application to do this for 14 different people.
Example.
//person1
string urlAddress1 = "http://www.url.com/person1";
filestream 1 = #"C:\Users\ellio\Desktop\test\person1.htm
//person2
string urlAddress2 = "http://www.url.com/person2";
filestream 2 = #"C:\Users\ellio\Desktop\test\person2.htm
etc
using System;
using System.Net;
using System.Text;
using System.IO;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
namespace GetSignatureHtml
{
class Program
{
static void Main(string[] args)
{
string urlAddress = "http://www.url.com/person1";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(urlAddress);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
if (response.StatusCode == HttpStatusCode.OK)
{
Stream receiveStream = response.GetResponseStream();
StreamReader readStream = null;
if (response.CharacterSet == null)
{
readStream = new StreamReader(receiveStream);
}
else
{
readStream = new StreamReader(receiveStream, Encoding.GetEncoding(response.CharacterSet));
}
string data = readStream.ReadToEnd();
using (FileStream fs = new FileStream(#"C:\Users\ellio\Desktop\test\person1.htm", FileMode.Create))
{
using (StreamWriter w = new StreamWriter(fs, Encoding.UTF8))
{
w.WriteLine(data);
}
}
Console.Write(data);
Console.ReadKey(true);
response.Close();
readStream.Close();
}
}
}
}
I'd suggest passing an argument into the Main method which is a file containing all the URLs to fetch. You can then read all those URLs into a list or array (e.g. with File.ReadAllLines, and infer the output file from the URL. Move most of your current code into a method that just accepts the URL, so the code will look something like this:
class Program
{
static void Main(string[] args)
{
if (args.Length != 1)
{
// Display some error message here
return;
}
string[] urls = File.ReadAllLines(args[0]);
foreach (var url in urls)
{
DownloadUrl(url);
}
}
private static void DownloadUrl(string url)
{
// Put most of your current code in here.
// You need to infer the name of the file to save -
// consider using new Uri(url), then Uri.LocalPath
}
}
Then you just need to put the URLs into a text file, and specify that when you run the code.
Note that this could all be done with just one URL per command line argument, but by the time you've got 14 URLs on a command line, it's going to be a bit painful to check. I tend to find that using a file for the data makes life easier.
As asides on the rest of the code, you should use a using statement for the response, and I'd encourage you to use File.WriteAllText as a simpler way of creating the file. You might want to look at using HttpClient as an alternative approach to the whole thing, although I'm not sure whether it applies the response encoding in the same way that you do.
Move your work into a method, then iterate over that method.
using System;
using System.Net;
using System.Text;
using System.IO;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
namespace GetSignatureHtml
{
class Program
{
static void Main(string[] args)
{
foreach(arg in args)
{
FindData(arg.urlAddress);
}
}
public void FindData(arg)
{
string urlAddress = "http://www.url.com/person1";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(urlAddress);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
if (response.StatusCode == HttpStatusCode.OK)
{
Stream receiveStream = response.GetResponseStream();
StreamReader readStream = null;
if (response.CharacterSet == null)
{
readStream = new StreamReader(receiveStream);
}
else
{
readStream = new StreamReader(receiveStream, Encoding.GetEncoding(response.CharacterSet));
}
string data = readStream.ReadToEnd();
using (FileStream fs = new FileStream(#"C:\Users\ellio\Desktop\test\person1.htm", FileMode.Create))
{
using (StreamWriter w = new StreamWriter(fs, Encoding.UTF8))
{
w.WriteLine(data);
}
}
Console.Write(data);
Console.ReadKey(true);
response.Close();
readStream.Close();
}
}
}
}
Related
when I try to upload any kidn of file through my SlackApp(via c# using HttpClient),
I allways get the following response:
{"ok":false,"error":"no_file_data"}
I checked my ByteArray (I stream the file to an array and then try to upload) and wrote my data back into a .txt and .jpg - I tried both types of data. When i write them back they are exact copies from the original, so I guess my streaming and writing to an ByteArrayworks fine. But something is off with my upload.
I'll show you my code:
The Client and the method to upload:
using System;
using System.Net.Http;
using System.Threading.Tasks;
using System.Net.Http.Headers;
namespace SlackApp
{
public class SlackClient
{
private readonly Uri _webhookUrl;
private readonly HttpClient _httpClient = new HttpClient {};
public SlackClient(Uri webhookUrl)
{
_webhookUrl = webhookUrl;
}
public async Task<HttpResponseMessage> UploadFile(byte[] file)
{
var requestContent = new MultipartFormDataContent();
var fileContent = new ByteArrayContent(file);
fileContent.Headers.ContentType = MediaTypeHeaderValue.Parse("multipart/form-data");
requestContent.Add(fileContent, "slack", "slack.txt");
var response = await _httpClient.PostAsync(_webhookUrl, requestContent);
return response;
}
}
}
the creation of the bytearray:
public class PostFile
{
String path = #"C:\Users\f.held\Desktop\Held-Docs\dagged.jpg";
public byte[] ReadImageFile()
{
FileInfo fileInfo = new FileInfo(path);
long imageFileLength = fileInfo.Length;
FileStream fs = new FileStream(path, FileMode.Open, FileAccess.ReadWrite);
BinaryReader br = new BinaryReader(fs);
byte[] imageData = br.ReadBytes((int)imageFileLength);
return imageData;
}
}
the Main:
using System;
using System.Net.Http;
using System.Threading.Tasks;
namespace SlackApp
{
class TestArea
{
public static void Main(string[] args)
{
Task.WaitAll(IntegrateWithSlackAsync());
}
private static async Task IntegrateWithSlackAsync()
{
var webhookUrl = new Uri("https://slack.com/api/files.upload?token=xoxp-hereStandsMyToken&channel=MyChannel");
var slackClient = new SlackClient(webhookUrl);
PostMessage PM = new PostMessage();
PostFile PF = new PostFile();
var testFile = PF.ReadImageFile();
while (true)
{
var message = Console.ReadLine();
FormUrlEncodedContent payload = PM.Content(message, "");
var response = await slackClient.SendMessageAsync(payload);
string content = await response.Content.ReadAsStringAsync();
Console.WriteLine(content); //I build these two lines in here so I got the response from the method, and this is where it says "no_file_data"
var isValid = response.IsSuccessStatusCode ? "valid" : "invalid";
Console.WriteLine($"Received {isValid} response.");
Console.WriteLine(response); //this puts out a "valid" response - oddly enough
}
}
}
}
Does anybody have an idea what is wrong here? Why isn't it taking the data?
You have two bugs in your code:
main(): The parameter to specify the channels is called
channels, not channel
UploadFile(): When you add your file content to the multipart you
need to include the correct API parameter for the file which is file,
not slack. And also want to include a reasonable filename (instead of slack.txt).
Additional comments
UploadFile(): Its wrong to set the content type to multipart/form-data. The
correct type for that content would be image/jpeg. However, the
correct type seams to be detected automatically, so just remove the
line.
main(): The Slack API will always return OK (http 200, unless there is a network problem), so you want to also look on the ok and error properties of the JSON response instead.
Here is an update version of your code. I changed your main() method to include a call to `UploadFile()?.
using System;
using System.IO;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Threading.Tasks;
namespace SlackApp
{
public class PostFile
{
string path = #"C:\Users\Stratios_down.jpg";
public byte[] ReadImageFile()
{
FileInfo fileInfo = new FileInfo(path);
long imageFileLength = fileInfo.Length;
FileStream fs = new FileStream(path, FileMode.Open, FileAccess.ReadWrite);
BinaryReader br = new BinaryReader(fs);
byte[] imageData = br.ReadBytes((int)imageFileLength);
return imageData;
}
}
public class SlackClient
{
private readonly Uri _webhookUrl;
private readonly HttpClient _httpClient = new HttpClient { };
public SlackClient(Uri webhookUrl)
{
_webhookUrl = webhookUrl;
}
public async Task<HttpResponseMessage> UploadFile(byte[] file)
{
var requestContent = new MultipartFormDataContent();
var fileContent = new ByteArrayContent(file);
requestContent.Add(fileContent, "file", "stratios.jpg");
var response = await _httpClient.PostAsync(_webhookUrl, requestContent);
return response;
}
}
class TestArea
{
public static void Main(string[] args)
{
Task.WaitAll(IntegrateWithSlackAsync());
}
private static async Task IntegrateWithSlackAsync()
{
var webhookUrl = new Uri(
"https://slack.com/api/files.upload?token=xoxp-MY-TOKEN&channels=test"
);
var slackClient = new SlackClient(webhookUrl);
PostFile PF = new PostFile();
var testFile = PF.ReadImageFile();
var response = await slackClient.UploadFile(testFile);
string content = await response.Content.ReadAsStringAsync();
Console.WriteLine(content);
Console.ReadKey();
}
}
}
In addition I would have a couple of suggestions to improve your code.
Instead of including the additional API parameters in the URL, I
would send them in the POST request as recommended by the API
documentation.
Including the file as FileStream instead of loading it yourself into
a ByteArray is the better approach and recommended for larger files.
Not sure why you need an infinite loop in your main. Those are really
bad and should be avoided.
Please also take also a look at my new async example for uploading a file to Slack where I applied those two ideas.
I was running into the no_file_data error as well. I found out you the file needs to exist AND it needs actual content inside. Make sure to do a size check or content length check in addition to the file exists check before uploading
I am trying to download file from a URL and I have to choose between WebClient and HttpClient. I have referenced this article and several other articles on the internet. Everywhere, it is suggested to go for HttpClient due to its great async support and other .Net 4.5 privileges. But I am still not totally convinced and need more inputs.
I am using below code to download file from internet:
WebClient:
WebClient client = new WebClient();
client.DownloadFile(downloadUrl, filePath);
HttpClient:
using (HttpClient client = new HttpClient())
{
using (HttpResponseMessage response = await client.GetAsync(url))
using (Stream streamToReadFrom = await response.Content.ReadAsStreamAsync())
{
}
}
From my perspective, I can see only one disadvantage in using WebClient, that would be the non async call, blocking the calling thread. But what if I am not worried about the blocking of thread or use client.DownloadFileAsync() to leverage the async support?
On the other hand, if I use HttpClient, ain't I loading every single byte of a file into memory and then writing it to a local file? If the file size is too large, won't memory overhead be expensive? Which could be avoided if we use WebClient, since it will directly write to local file and not consume system memory.
So, if performance is my utter priority, which approach should I use for download? I would like to be clarified if my above assumption is wrong, and I am open to alternate approach as well.
You can do it natively with .Net 4.5+. I tried doing it your way and then I just found a method in Intellisense that seemed to make sense.
https://learn.microsoft.com/en-us/dotnet/api/system.io.stream.copytoasync?view=netframework-4.7.2
uri = new Uri(generatePdfsRetrieveUrl + pdfGuid + ".pdf");
HttpClient client = new HttpClient();
var response = await client.GetAsync(uri);
using (var fs = new FileStream(
HostingEnvironment.MapPath(string.Format("~/Downloads/{0}.pdf", pdfGuid)),
FileMode.CreateNew))
{
await response.Content.CopyToAsync(fs);
}
Here is my approach.
If you are calling a WebApi to get a file, then from a controller method you can use HttpClient GET request and return file stream using FileStreamResult return type.
public async Task<ActionResult> GetAttachment(int FileID)
{
UriBuilder uriBuilder = new UriBuilder();
uriBuilder.Scheme = "https";
uriBuilder.Host = "api.example.com";
var Path = "/files/download";
uriBuilder.Path = Path;
using (HttpClient client = new HttpClient())
{
client.BaseAddress = new Uri(uriBuilder.ToString());
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Add("authorization", access_token); //if any
client.DefaultRequestHeaders.Accept.Add(new System.Net.Http.Headers.MediaTypeWithQualityHeaderValue("application/json"));
HttpResponseMessage response = await client.GetAsync(uriBuilder.ToString());
if (response.IsSuccessStatusCode)
{
System.Net.Http.HttpContent content = response.Content;
var contentStream = await content.ReadAsStreamAsync(); // get the actual content stream
return File(contentStream, content_type, filename);
}
else
{
throw new FileNotFoundException();
}
}
}
To use HttpClient on my existing code that used WebClient, I wrote a small extension method to use it on the same way I used DownloadFileTaskAsync on my code.
using (var client = new System.Net.Http.HttpClient()) // WebClient
{
var fileName = #"C:\temp\imgd.jpg";
var uri = new Uri("https://yourwebsite.com/assets/banners/Default.jpg");
await client.DownloadFileTaskAsync(uri, fileName);
}
To use it we can have this extension method:
public static class HttpClientUtils
{
public static async Task DownloadFileTaskAsync(this HttpClient client, Uri uri, string FileName)
{
using (var s = await client.GetStreamAsync(uri))
{
using (var fs = new FileStream(FileName, FileMode.CreateNew))
{
await s.CopyToAsync(fs);
}
}
}
}
For code being called repeatedly, you do not want to put HttpClient in a using block (it will leave hanging ports open)
For downloading a file with HttpClient, I found this extension method which seemed like a good and reliable solution to me:
public static class HttpContentExtensions
{
public static Task ReadAsFileAsync(this HttpContent content, string filename, bool overwrite)
{
string pathname = Path.GetFullPath(filename);
if (!overwrite && File.Exists(filename))
{
throw new InvalidOperationException(string.Format("File {0} already exists.", pathname));
}
FileStream fileStream = null;
try
{
fileStream = new FileStream(pathname, FileMode.Create, FileAccess.Write, FileShare.None);
return content.CopyToAsync(fileStream).ContinueWith(
(copyTask) =>
{
fileStream.Close();
});
}
catch
{
if (fileStream != null)
{
fileStream.Close();
}
throw;
}
}
}
Here’s one way to use it to download a URL and save it to a file: (I am using windows 7, therefore no WindowsRT available to me, so I’m also using System.IO.)
public static class WebUtils
{
private static Lazy<IWebProxy> proxy = new Lazy<IWebProxy>(() => string.IsNullOrEmpty(Settings.Default.WebProxyAddress) ? null : new WebProxy { Address = new Uri(Settings.Default.WebProxyAddress), UseDefaultCredentials = true });
public static IWebProxy Proxy
{
get { return WebUtils.proxy.Value; }
}
public static Task DownloadAsync(string requestUri, string filename)
{
if (requestUri == null)
throw new ArgumentNullException(“requestUri”);
return DownloadAsync(new Uri(requestUri), filename);
}
public static async Task DownloadAsync(Uri requestUri, string filename)
{
if (filename == null)
throw new ArgumentNullException("filename");
if (Proxy != null)
WebRequest.DefaultWebProxy = Proxy;
using (var httpClient = new HttpClient())
{
using (var request = new HttpRequestMessage(HttpMethod.Get, requestUri))
{
using (Stream contentStream = await (await httpClient.SendAsync(request)).Content.ReadAsStreamAsync(), stream = new FileStream(filename, FileMode.Create, FileAccess.Write, FileShare.None, Constants.LargeBufferSize, true))
{
await contentStream.CopyToAsync(stream);
}
}
}
}
}
Note that code is saving the address of the proxy server I use (at work) in a setting, and using that if such setting is specified. Otherwise, it should tell you all you need to know regarding using the HttpClient beta to download and save a file.
If you want (or have) to do this synchronously, but using the nice HttpClient class, then there's this simple approach:
string requestString = #"https://example.com/path/file.pdf";
var GetTask = httpClient.GetAsync(requestString);
GetTask.Wait(WebCommsTimeout); // WebCommsTimeout is in milliseconds
if (!GetTask.Result.IsSuccessStatusCode)
{
// write an error
return;
}
using (var fs = new FileStream(#"c:\path\file.pdf", FileMode.CreateNew))
{
var ResponseTask = GetTask.Result.Content.CopyToAsync(fs);
ResponseTask.Wait(WebCommsTimeout);
}
My approach is very simple. Using FileStream you can store it in the local folder, or return it from API using FileStreamResult.
Example for store into local folder:
private async Task SaveDataIntoLocalFolder(string url,string fileName)
{
using (var client = new HttpClient())
{
var response = await client.GetAsync(url);
if (response.IsSuccessStatusCode)
{
var stream = await response.Content.ReadAsStreamAsync();
var fileInfo = new FileInfo(fileName);
using (var fileStream = fileInfo.OpenWrite())
{
await stream.CopyToAsync(fileStream);
}
}
else
{
throw new Exception("File not found");
}
}
}
This is a simple demo UWP application for downloading an image file.
Just paste the image URL link and press the download button. You can identify the file type and change the fileName to download the desired file.
MainPage.xaml
<Page
x:Class="HttpDownloader.MainPage"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:local="using:HttpDownloader"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
mc:Ignorable="d"
Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
<Grid>
<StackPanel>
<TextBox x:Name="uriInput"
Header="URI:" PlaceholderText="Please provide an uri"
Width="300"
HorizontalAlignment="Center"/>
<Button Content="Dowload"
HorizontalAlignment="Center"
Click="Button_Click"/>
</StackPanel>
</Grid>
</Page>
MainPage.xaml.xs
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net.Http;
using System.Runtime.InteropServices.WindowsRuntime;
using System.Threading.Tasks;
using Windows.Foundation;
using Windows.Foundation.Collections;
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Controls.Primitives;
using Windows.UI.Xaml.Data;
using Windows.UI.Xaml.Input;
using Windows.UI.Xaml.Media;
using Windows.UI.Xaml.Navigation;
using System.Net.Http;
using System.Net;
using Windows.Storage.Streams;
using Windows.Storage.Pickers;
using Windows.Storage;
using Windows.Graphics.Imaging;
using System.Threading;
// The Blank Page item template is documented at https://go.microsoft.com/fwlink/?LinkId=402352&clcid=0x409
namespace HttpDownloader
{
/// <summary>
/// An empty page that can be used on its own or navigated to within a Frame.
/// </summary>
public sealed partial class MainPage : Page
{
public MainPage()
{
this.InitializeComponent();
}
private async void Button_Click(object sender, RoutedEventArgs e)
{
HttpClient client = new HttpClient();
string imageUrl = uriInput.Text;
try
{
using (var cancellationTokenSource = new CancellationTokenSource(50000))
{
var uri = new Uri(WebUtility.HtmlDecode(imageUrl));
using (var response = await client.GetAsync(uri, cancellationTokenSource.Token))
{
response.EnsureSuccessStatusCode();
var mediaType = response.Content.Headers.ContentType.MediaType;
string fileName = DateTime.Now.ToString("yyyyMMddhhmmss");
if (mediaType.IndexOf("jpg", StringComparison.OrdinalIgnoreCase) >= 0
|| mediaType.IndexOf("jpeg", StringComparison.OrdinalIgnoreCase) >= 0)
{
fileName += ".jpg";
}
else if (mediaType.IndexOf("png", StringComparison.OrdinalIgnoreCase) >= 0)
{
fileName += ".png";
}
else if (mediaType.IndexOf("gif", StringComparison.OrdinalIgnoreCase) >= 0)
{
fileName += ".gif";
}
else if (mediaType.IndexOf("bmp", StringComparison.OrdinalIgnoreCase) >= 0)
{
fileName += ".bmp";
}
else
{
fileName += ".png";
}
// Get the app's local folder.
StorageFolder localFolder = Windows.Storage.ApplicationData.Current.LocalFolder;
// Create a new subfolder in the current folder.
// Replace the folder if already exists.
string desiredName = "Images";
StorageFolder newFolder = await localFolder.CreateFolderAsync(desiredName, CreationCollisionOption.ReplaceExisting);
StorageFile newFile = await newFolder.CreateFileAsync(fileName, CreationCollisionOption.ReplaceExisting);
using (Stream streamStream = await response.Content.ReadAsStreamAsync())
{
using (Stream streamToWriteTo = File.Open(newFile.Path, FileMode.Create))
{
await streamStream.CopyToAsync(streamToWriteTo);
}
}
}
}
}
catch (Exception ex)
{
Console.WriteLine("Exception occur");
Console.WriteLine(ex.ToString());
}
}
}
}
You will find the image in this folder.
Users/[current user name]/AppData/Local/Packages/[Application package name]/LocalState/Images
HttpClient _client=new HttpClient();
byte[] buffer = null;
try
{
HttpResponseMessage task = await _client.GetAsync("https://**FILE_URL**");
Stream task2 = await task.Content.ReadAsStreamAsync();
using (MemoryStream ms = new MemoryStream())
{
await task2.CopyToAsync(ms);
buffer = ms.ToArray();
}
File.WriteAllBytes("C:/**PATH_TO_SAVE**", buffer);
}
catch
{
}
I get continous [ non-stop ] mesages from web url:
string webUrl = "xxxxx/status.cgi";
WebClient client = new WebClient();
client.Credentials = new NetworkCredential("UUU", "PPP");
StreamReader reader = new StreamReader(client.OpenRead(webUrl), Encoding.UTF8,true);
string line;
int counter = 0;
while ((line = reader.ReadLine()) != null)
{
if( line == "XXXXXX")
{
break;
}
}
Console.WriteLine("Try to Close Stream Reader...");
reader.Close();
Console.WriteLine("Stream Reader is closed...");
The problem is that when I break from while loop, i want to close the stream reader...But stream reader does not close...."reader.Close();" hangs/block the code...
Why this happen? How to fix it?
UPDATE:
"using" DOES NOT WORK IN MY CASE: Exit the loop but stream reader is not disposed...Hang/Block
using System;
using System.Collections.Generic;
using System.IO;
using System.Net;
using System.Linq;
using System.Text;
namespace TestStreamReader
{
class Program
{
static void Main(string[] args)
{
string webUrl = "http://X.Y.Z:7000/status.cgi";
int counter = 0;
using (WebClient client = new WebClient())
{
client.Credentials = new NetworkCredential("admin", "000000");
using (StreamReader reader = new StreamReader(client.OpenRead(webUrl), Encoding.UTF8, true))
{
string line;
while ((line = reader.ReadLine()) != null)
{
counter++;
Console.WriteLine("Input"+ line);
if (counter == 10)
{
Console.WriteLine("I am exiting the loop");
break;
}
}
Console.WriteLine("Exit the while loop");
}
Console.WriteLine("Reader should be desposed");
}
Console.WriteLine("Web Client should be disposed!");
}
}
}
I had the same problem, but none of the suggestions made in this thread were helpful for me. But I WAS able to fix it a different way, so I thought I'd share...
I used the asynchronous version of client.OpenRead(uri) to open the stream, and then I was able to call client.CancelAsync() when I need to shut it down. The implementation in there must be disposing the StreamReader differently, because it works that way.
Instead of using WebClient, have you thought about using HttpWebRequest and getting the response stream?
//using System.Net;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(ApiProcedure.FunctionUri);
request.Credentials = (CredentialCache)Credentials;
request.PreAuthenticate = true;
//define the type of request
request.Method = HttpMethod;
request.ContentType = "application/json";
//execute
StreamReader sr = new StreamReader(response.GetResponseStream());
return sr.ReadToEnd();
I know you're doing a continuous stream, but you should be able to treat the stream reader normally like you're doing in your example. I think this method will give you a little more control.
I normally only use WebClient if I need to perform a simple task where all the nitty gritty setup/streams are done for me. Find I get better results when managing the request and response separately.
Good luck!
EDIT: Also, this is just a snippet from my code. You might want to take a look here: C# - How to read a continuous stream of XML over HTTP for a good example of reading continuous chunks of data.
A Quick idea frm my side, try put the StreamReader in a Using.
string webUrl = "http://www.google.com";
using (WebClient client = new WebClient())
{
client.Credentials = new NetworkCredential("UUU", "PPP");
int counter = 0;
using (StreamReader reader = new StreamReader(client.OpenRead(webUrl), Encoding.UTF8, true))
{
string line;
while ((line = reader.ReadLine()) != null)
{
if (line == "XXXXXX")
{
break;
}
counter++;
}
}
}
Console.WriteLine("Try to Close Stream Reader...");
Console.WriteLine("Stream Reader İSclosed...");
This will keep Connection open until it's no longer used. The Dispose() will be called automatically after then. No need of manual detach or close. [Update] I also move the WebClient into a using and the counter variable outside the StreamReaders using.
[Edit]
Due to comments the problem looks like a (untested speculation) side-effect of the WebClient can't quit, because it can't see the end of the stream. Therefore, the StreamReader wait. I attach Another approach to read from a webrequest. However this will far from sure work better.
string webUrl = "http://www.google.com";
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(webUrl);
req.Credentials = new NetworkCredential("UUU", "PPP");
using (HttpWebResponse resp = (HttpWebResponse)req.GetResponse())
{
using (Stream stream = resp.GetResponseStream())
{
int read;
string line;
byte[] data = new byte[4096];
while ((read = stream.Read(data, 0, data.Length)) > 0)
{
line = Encoding.GetEncoding("ASCII").GetString(data, 0, data.Length);
if (line.Contains("(function(){try{var a=window.gbar;"))
{
Console.WriteLine("End Bit founded..");
// Some more logic?
break;
}
data = new byte[4096];
Console.WriteLine(line);
}
}
}
Console.WriteLine("End of Stream");
Please note I used Stream here and manually split to 4096 datablocks, just for the sake of the problem. The usual would be using (StreamReader stream2 = new StreamReader(resp.GetResponseStream())) { string test = stream2.ReadToEnd(); }.
Hope it will give any step further.
Also mind that Encoding is set to "ASCII" above.
I've seen something similar. The problem is that it's waiting for a ReadTimeOut to occur. You can try setting the Stream.ReadTimeOut property to something sufficiently small.
I am currently trying to login and get cookies to a remote server using windows phone 7 silverlight in visual studio. I managed to login and get a successful result for login, but when I try to put in the codes to get cookies, it just failed.
It produced error "Cannot set CookieContainer due to the state of the HttpWebRequest object." on my code "request.CookieContainer = new CookieContainer();"
Can anyone helps me? I can't seem to find the error and I try to look at the documentation and examples but with no luck. Below is my full codes on windows phone 7.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Documents;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Animation;
using System.Windows.Shapes;
using Microsoft.Phone.Controls;
using System.IO;
using System.IO.IsolatedStorage;
namespace Testing_Login_
{
public partial class MainPage : PhoneApplicationPage
{
// Constructor
public MainPage()
{
InitializeComponent();
}
private void buttonLogin_Click(object sender, RoutedEventArgs e)
{
//HttpWebRequest req = WebRequest.Create();
string POST_ADDRESS = "http://mywebsite.com";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(new Uri(POST_ADDRESS, UriKind.Absolute));
request.Method = "POST";
// don't miss out this
request.ContentType = "application/x-www-form-urlencoded";
request.BeginGetRequestStream(new AsyncCallback(RequestReady), request);
}
// Sumbit the Post Data
void RequestReady(IAsyncResult asyncResult)
{
HttpWebRequest request = asyncResult.AsyncState as HttpWebRequest;
Stream stream = request.EndGetRequestStream(asyncResult);
// Hack for solving multi-threading problem
// I think this is a bug
this.Dispatcher.BeginInvoke(delegate()
{
// Send the post variables
StreamWriter writer = new StreamWriter(stream);
writer.Write("username=" + textBoxUsername.Text + "&password=" + passwordBoxSTAMP.Password);
writer.Flush();
writer.Close();
request.CookieContainer = new CookieContainer();
request.BeginGetResponse(new AsyncCallback(ResponseReady), request);
});
}
// Get the Result
void ResponseReady(IAsyncResult asyncResult)
{
HttpWebRequest request = asyncResult.AsyncState as HttpWebRequest;
HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(asyncResult);
this.Dispatcher.BeginInvoke(delegate()
{
Stream responseStream = response.GetResponseStream();
StreamReader reader = new StreamReader(responseStream);
// get the result text
string result = reader.ReadToEnd();
if (result == "TRUE")
{
MessageBox.Show("Login Successful!");
//CookieCollection cookieValue = response.Cookies;
using (IsolatedStorageFile isf = IsolatedStorageFile.GetUserStoreForApplication())
{
using (IsolatedStorageFileStream isfs = isf.OpenFile("CookieExCookies", FileMode.OpenOrCreate, FileAccess.Write))
{
using (StreamWriter sw = new StreamWriter(isfs))
{
foreach (Cookie cookieValue in response.Cookies)
{
sw.WriteLine("Cookie: " + cookieValue.ToString());
}
sw.Close();
}
}
}
//MessageBox.Show(cookieValue.ToString());
}
else if (result == "ICRED")
{
MessageBox.Show("Username or Password incorrect!");
}
else
{
MessageBox.Show("Unknown Error!"+result);
}
});
}
private void ReadFromIsolatedStorage()
{
using (IsolatedStorageFile isf = IsolatedStorageFile.GetUserStoreForApplication())
{
using (IsolatedStorageFileStream isfs =
isf.OpenFile("CookieExCookies", FileMode.Open))
{
using (StreamReader sr = new StreamReader(isfs))
{
textBoxCookies.Text = sr.ReadToEnd();
sr.Close();
}
}
}
}
}
}
Move the line:-
request.CookieContainer = new CookieContainer();
out of RequestReady and put it in buttonLogin_Click just after you have constructed it.
OR since you aren't re-using it anywhere ditch line altogether.
OR if in fact for things to really work you need to re-use it then construct and hold it somewhere else (such as a field in the class) and assign it to each Request object created before you invoke any BeginGetRequestStream or BeginGetResponse.
I just recieve my unique developer API key from Imgur and I'm aching to start cracking on this baby.
First a simple test to kick things off. How can I upload an image using C#? I found this using Python:
#!/usr/bin/python
import pycurl
c = pycurl.Curl()
values = [
("key", "YOUR_API_KEY"),
("image", (c.FORM_FILE, "file.png"))]
# OR: ("image", "http://example.com/example.jpg"))]
# OR: ("image", "BASE64_ENCODED_STRING"))]
c.setopt(c.URL, "http://imgur.com/api/upload.xml")
c.setopt(c.HTTPPOST, values)
c.perform()
c.close()
looks like the site uses HTTP Post to upload images. Take a look at the HTTPWebRequest class and using it to POST to a URL: Posting data with HTTPRequest.
The Imgur API now provide a complete c# example :
using System;
using System.IO;
using System.Net;
using System.Text;
namespace ImgurExample
{
class Program
{
static void Main(string[] args)
{
PostToImgur(#"C:\Users\ashwin\Desktop\image.jpg", IMGUR_ANONYMOUS_API_KEY);
}
public static void PostToImgur(string imagFilePath, string apiKey)
{
byte[] imageData;
FileStream fileStream = File.OpenRead(imagFilePath);
imageData = new byte[fileStream.Length];
fileStream.Read(imageData, 0, imageData.Length);
fileStream.Close();
string uploadRequestString = "image=" + Uri.EscapeDataString(System.Convert.ToBase64String(imageData)) + "&key=" + apiKey;
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create("http://api.imgur.com/2/upload");
webRequest.Method = "POST";
webRequest.ContentType = "application/x-www-form-urlencoded";
webRequest.ServicePoint.Expect100Continue = false;
StreamWriter streamWriter = new StreamWriter(webRequest.GetRequestStream());
streamWriter.Write(uploadRequestString);
streamWriter.Close();
WebResponse response = webRequest.GetResponse();
Stream responseStream = response.GetResponseStream();
StreamReader responseReader = new StreamReader(responseStream);
string responseString = responseReader.ReadToEnd();
}
}
}
Why don't you use the NuGet for this: called Imgur.API and for upload
you would have a method like this:
/*
The refresh token and all the values represented by constans are given when you allow the application in your imgur panel on the response url
*/
public OAuth2Token CreateToken()
{
var token = new OAuth2Token(TOKEN_ACCESS, REFRESH_TOKEN, TOKEN_TYPE, ID_ACCOUNT, IMGUR_USER_ACCOUNT, int.Parse(EXPIRES_IN));
return token;
}
//Use it only if your token is expired
public Task<IOAuth2Token> RefreshToken()
{
var client = new ImgurClient(CLIENT_ID, CLIENT_SECRET);
var endpoint= new OAuth2Endpoint(client);
var token = endpoint.GetTokenByRefreshTokenAsync(REFRESH_TOKEN);
return token;
}
public async Task UploadImage()
{
try
{
var client = new ImgurClient(CLIENT_ID, CLIENT_SECRET, CreateToken());
var endpoint = new ImageEndpoint(client);
IImage image;
//Here you have to link your image location
using (var fs = new FileStream(#"IMAGE_LOCATION", FileMode.Open))
{
image = await endpoint.UploadImageStreamAsync(fs);
}
Debug.Write("Image uploaded. Image Url: " + image.Link);
}
catch (ImgurException imgurEx)
{
Debug.Write("Error uploading the image to Imgur");
Debug.Write(imgurEx.Message);
}
}
Also you can find all the reference here: Imgur.API NuGet