Okay, so I've googled on several occasions regarding this, but each one suggests respectfully decent choices like : "Selenium" which works fine, but isn't use-able without firefox (or even within an API to my knowledge?).
I have this code :
public byte[] GetFileViaHttp(string url)
{
using (WebClient client = new WebClient())
{
return client.DownloadData(url);
}
}
Then I also have this code :
byte[] result = GetFileViaHttp(#"http://ip-lookup.net/");
string str = Encoding.UTF8.GetString(result);
richTextBox1.Text = str;
Works fine, returns my IP's information, but I want to automate this with other IP addresses, rather than return my own.
How would this be done ?
By this I mean, I want the API to take txtBox1.Text (IP) & print the details into richTextBox1.Text (Host/Country) ..
How could this be done ?
I looked around the site and found a help document that details exactly what you want.
Simply pass the IP value as an unnamed query string parameter:
http://ip-lookup.net/?127.0.0.1
In your code:
byte[] result = GetFileViaHttp(string.Format("http://ip-lookup.net?{0}", ipAddress));
where you are injecting a string ip address as ipAddress.
You can find their help page here. I looked for a legal agreement but I wasn't able to find one, so please use at your own risk and discretion.
UPDATE:
If you are getting 403s, you need to pass along a user agent header. Your WebClient instance can be modified to include a header in the request.
public byte[] GetFileViaHttp(string url)
{
using (WebClient client = new WebClient())
{
client.Headers.Add("User-Agent: Other");
return client.DownloadData(url);
}
}
Related
There are several similar questions but not one quite the same:
We have an S3 bucket with CloudFront on it configured to use signedURLs. I can use the command line to generate a signed URL and access it. When I try to do this via C# the signed URL generated is not valid.
This is the command line which returns a valid url:
aws cloudfront sign --url https://MyPath.com/testfile.txt
--key-pair-id "keypairIDvalue" --private-key file://bas_key.pem --date-less-than 2022-01-01
The code is this:
static void Main(string[] args)
{
// technique 1
string policyDoc = Amazon.CloudFront.AmazonCloudFrontUrlSigner.BuildPolicyForSignedUrl(
#"https://MyPath/testfile.txt",
DateTime.Today.AddDays(+4),
"myIPAddress");
string signurl = Amazon.CloudFront.AmazonCloudFrontUrlSigner.SignUrl(
#"https://MyPath/testfile.txt",
"KeypairID",
new StreamReader(#"c:\keys\Bas_key.pem"),
policyDoc);
// the signurl does not work
// technique 2
string newurl = Amazon.CloudFront.AmazonCloudFrontUrlSigner.GetCustomSignedURL(
#"https://MyPath/testfile.txt",
new StreamReader(#"c:\keys\Bas_key.pem"),
"KeypairID",
DateTime.Today.AddDays(4),
DateTime.Today.AddDays(-1),
"MyIPAddress");
// newurl is different to signurl but also does not work
Console.WriteLine(newurl);
}
Most examples I have seem to have a lot of extra code to do various bits and bobs but here everything is setup and configured. I just need the API command to get the URL, the documentation seems to imply this is an expected way of doing this but I cannot get it to work. Neither of the above techniques work.
Can anyone see what I have done wrong?
Fixed it!
The working code is this!
static void Main(string[] args)
{
// technique 1
string policyDoc = Amazon.CloudFront.AmazonCloudFrontUrlSigner.BuildPolicyForSignedUrl(
null,
DateTime.Today.AddDays(+1),
null);
string signurl = Amazon.CloudFront.AmazonCloudFrontUrlSigner.SignUrl(
#"https://MyPath/testfile.txt",
"KeypairID",
new StreamReader(#"c:\keys\Bas_key.pem"),
policyDoc);
Console.WriteLine(signurl);
}
so after searching both on here and Google I have been unable to find a solution. Basically I want to allow the user to add a list of proxies from a text file, but I want to check that what gets passed in is a valid Proxy format before I make the WebProxy. I know using try catch like try {var proxy = new WebProxy(host + ":" + port;} catch{} will work, but as you may already know using try catch is slow, even more so when doing it in bulk.
So what would be a good and fast way to test a string to see if it's in a valid WebProxy format?
Thanks
This is how I do it:
var ValidIpAddressPortRegex = #"^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5]):[\d]+$";
if (System.Text.RegularExpressions.Regex.IsMatch(proxy_string_to_validate, ValidIpAddressPortRegex ))
{
do_stuff_with_proxy();
}
else
{
//MessageBox.Show("IP:PORT url not valid!","Information");
}
Please bear with me as I am new to Silverlight. I need to write a web api wrapper (I've named it WebClientWrapper) which is supposed to consume rest services. The project uses Silverlight 5. I am facing many problems while writing such a wrapper. There are a lot of examples demonstrating rest service consumption in C#. But unfortunately none of them worked for me. It's been a challenge for me to get done with what I need. Below are my requirements:
1) UI should not freeze while I make any GET request,
2) Calling methods from WebClientWrapper should be as easier as possible.
3) .NET framework version of project should not be changed.
I tried following things so far:
1) Use of HttpClient. I referred this link: http://www.asp.net/web-api/overview/web-api-clients/calling-a-web-api-from-a-net-client. The problem with this approach is I cannot call ReadAsAsync method. Calling this method requires change in framework (i.e. replacing dlls or changing framework version) which is not feasible.
2) Use of WebClient. http://www.kastory.net/index.php?option=com_content&view=article&id=25:rest-web-service-in-c-with-silverlight-and-asp-net-web-api&catid=32:web-services&Itemid=130.
The problem with this is I will probably (not sure!) have to make changes in the way I call methods from WebClientWrapper. I am trying to accomplish calling method from WebClientWrapper something like this:
var appointments = WebClientWrapper.Get<Appointments>(new Dictionary<string, string>()
{
{"hospitalId", "19465654546"}
}, "appointment");
Below is the code snippet I tried in WebClientWrapper.
private const string BaseUrl = "http://localhost:63455/api";
private static WebClient GetClient()
{
int leadingInt = new Random().Next(10000, 99999);
int trailingInt = new Random().Next(1000, 9999);
string date = DateTime.Now.ToString("ddHHmmMMssMMyyyyyss");
string ticketString = string.Format("{0}{1}{2}", leadingInt, date, trailingInt);
var client = new WebClient();
client.Headers["Accept"] = ticketString;
client.Headers["UserAgent"] = "ReceptionistApp";
return client;
}
private static void DownloadCompletionHandler<T>(object sender, DownloadStringCompletedEventArgs e)
{
Encoding messageEncoding = Encoding.UTF8;
var serializer = new DataContractJsonSerializer(typeof (T));
var memoryStream = new MemoryStream(messageEncoding.GetBytes(e.Result));
var objectToReturn = (T) serializer.ReadObject(memoryStream);
}
public static T Get<T>(Dictionary<string, string> paramDictionary, string controller)
{
string absoluteUrl = BaseUrl + controller + "?";
absoluteUrl = paramDictionary.Aggregate(absoluteUrl,
(current, keyValuePair) => current + (keyValuePair.Key + "=" + keyValuePair.Value + "&"));
absoluteUrl = absoluteUrl.TrimEnd('&');
WebClient client = GetClient();
client.DownloadStringCompleted += DownloadCompletionHandler<T>;
client.DownloadStringAsync(new Uri(absoluteUrl));
}
Below are the things I want to mention regarding the code above:
1) As obvious, compiler throws error for the method Get<T> since I've not returned object of type T. How shall I get that object from DownloadStringAsync? I know I can use DownloadStringTaskAsync. But it is not available with the current framework. What changes do I have to make in my code so that I get appointments as shown in the Get<Appointments> method call?
2) DownloadCompletionHandler<T> is bound to return void but actually I want to return objectToReturn as shown in code.
Help will be greatly appreciated. Any new code snippets that will fulfill my requirements are welcome.
RestSharp helped me instead of WebClient.
I'm developing an android application in unity 3d that will have network communications for user accounts updating said accounts as well as controlling everything on the back end of the app. I use the WWW class in unity to send info to the server. The backend is php and all the data is stored in a mysql database. How can I make a secure connection between the app and the backend without someone just simply getting the servers address and blocking it in their hosts file and feeding the app false info and going online with it.(as an example) I'm no security expert but I'm not sure what I need to look in too in order to create secure connections between server and client. Any help would be greatly apericiated. Thank you.
you just need to implement the www class
void start()
{
StartCoroutine(retrieveHighscores()); //Start out by getting the current scores.
}
IEnumerator retrieveHighscores()
{
var form = new WWWForm(); // create a new form
form.AddField("Nipun",name); // add the data you want to retrieve in the form fields
var rawData = form.data;
var headers = form.headers; // here headers will be used to authenticate the credentials of the person trying to access
headers["Authorization"]="Basic " + System.Convert.ToBase64String(System.Text.Encoding.ASCII.GetBytes("username:password"));
WWW webRequest = new WWW("https://abc.com/test.php", rawData, headers); //
yield return webRequest;
if (webRequest != null) {
//here you have successfully got the response back from the server , here i am adding the whole response in a string and then splitting the string based on the format of the data i received.
string x = webRequest.text;
string[] lines = webRequest.text.Split(new string[] { System.Environment.NewLine }, System.StringSplitOptions.RemoveEmptyEntries); //Split the response by newlines.
Debug.Log(x); // to check what you received
scores = new Dictionary<string, int>(); //Always reset our scores, as we just got new ones.
foreach (string line in lines) //Parse every line
{
// code here how you want to use the split up data you received
}
}
else
Debug.Log("error");
}
}
How to screen scrape HTTPS using C#?
You can use System.Net.WebClient to start an HTTPS connection, and pull down the page to scrape with that.
Look into the Html Agility Pack.
You can use System.Net.WebClient to grab web pages. Here is an example: http://www.codersource.net/csharp_screen_scraping.html
If for some reason you're having trouble with accessing the page as a web-client or you want to make it seem like the request is from a browser, you could use the web-browser control in an app, load the page in it and use the source of the loaded content from the web-browser control.
Here's a concrete (albeit trivial) example. You can pass a ship name to VesselFinder in the querystring, but even if it only finds one ship with that name it still shows you the search results screen with one ship. This example detects that case and takes the user straight to the tracking map for the ship.
string strName = "SAFMARINE MAFADI";
string strURL = "https://www.vesselfinder.com/vessels?name=" + HttpUtility.UrlEncode(strName);
string strReturnURL = strURL;
string strToSearch = "/?imo=";
string strPage = string.Empty;
byte[] aReqtHTML;
WebClient objWebClient = new WebClient();
objWebClient.Headers.Add("User-Agent: Other"); //You must do this or HTTPS won't work
aReqtHTML = objWebClient.DownloadData(strURL); //Do the name search
UTF8Encoding utf8 = new UTF8Encoding();
strPage = utf8.GetString(aReqtHTML); // get the string from the bytes
if (strPage.IndexOf(strToSearch) != strPage.LastIndexOf(strToSearch))
{
//more than one instance found, so leave return URL as name search
}
else if (strPage.Contains(strToSearch) == true)
{
//find the ship's IMO
strPage = strPage.Substring(strPage.IndexOf(strToSearch)); //cut off the stuff before
strPage = strPage.Substring(0, strPage.IndexOf("\"")); //cut off the stuff after
}
strReturnURL = "https://www.vesselfinder.com" + strPage;