HTTPS availablity of http url - c#

I am having list of http urls, I need to find https url is available or not. Example : http://www.apra.gov.au/Insight/Pages/insight-issue2-2017.html, need to check whether https is available on the same domain via c# code. since I have a list of 5k http urls. I need to verify all these url available on HTTPS?

You can probably do a simle string replace (http: = https: ) and then loop through them all calling httpget to check:
Psuedo code:
var httpClient = new HttpClient()
foreach(var url in urls)
var httpUrl = url.Replace("http:","https:");
httpClient.Get(url);

Related

How to get request url who used my webapi in asp.net?

I already write a Web API method, now I want to get a client website URL who used my API method.
I tried this line of code but I get my API URL. But I want a client URL.
var requestUrl = Request.Url;
var clientRequest = HttpContext.Request.Headers["Referer"];

HttpClient GetAsync with a hash in URL

.NET Core 2.2 console application on Windows.
I'm exploring how to use HttpClient GetAsync on a Stackoverflow share style URL eg: https://stackoverflow.com/a/29809054/26086 which returns a 302 redirect URL with a hash in it
static async Task Main()
{
var client = new HttpClient();
// 1. Doesn't work - has a hash in URL
var url = "https://stackoverflow.com/questions/29808915/why-use-async-await-all-the-way-down/29809054#29809054";
HttpResponseMessage rm = await client.GetAsync(url);
Console.WriteLine($"Status code: {(int)rm.StatusCode}"); // 400 Bad Request
// 2. Does work - no hash
url = "https://stackoverflow.com/questions/29808915/why-use-async-await-all-the-way-down/29809054";
rm = await client.GetAsync(url);
Console.WriteLine($"Status code: {(int)rm.StatusCode}"); // 200 Okay
// 3. Doesn't work as the 302 redirect goes to the first URL above with a hash
url = "https://stackoverflow.com/a/29809054/26086";
rm = await client.GetAsync(url);
Console.WriteLine($"Status code: {(int)rm.StatusCode}"); // 400 Bad Request
}
I'm crawling my blog which has many SO short codes in it.
Update/Workaround
With thanks to #rohancragg I found that turning off AutoRedirect then getting the URI from the returned header worked
// as some autoredirects fail due to #fragments in url, handle redirects manually
var handler = new HttpClientHandler { AllowAutoRedirect = false };
var client = new HttpClient(handler);
var url = "https://stackoverflow.com/a/29809054/26086";
HttpResponseMessage rm = await client.GetAsync(url);
// gives the desired new URL which can then GetAsync
Uri u = rm.Headers.Location;
As #Damien_The_Unbeliever implies in a comment, you'll just need to strip off the hash and everything after it - all that does is tell the browser to jump to that anchor tag in the HTML page (see: https://w3schools.com/jsref/prop_anchor_hash.asp).
You could also use the Uri class to parse the Uri and ignore any 'fragments': https://learn.microsoft.com/en-us/dotnet/api/system.uri.fragment
Because the share-style Urls are only ever going to return a 302 then I'd suggest capturing the Uri to which the 302 is referring and do as I suggest above and just get the path and ignore the fragment.
So you need to use some mechanism (which I'm just looking up!) to handle a 302 gracefully followed by option 2
Update: this looks relevant! How can I get System.Net.Http.HttpClient to not follow 302 redirects?
Update 2 Steve Guidi has a very important bit of advice in a comment here: https://stackoverflow.com/a/17758758/5351
In response to the advice that you need to use HttpResponseMessage.RequestMessage.RequestUri:
it is very important to add HttpCompletionOption.ResponseHeadersRead
as the second parameter of the GetAsync() call
Disclaimer - I've not tried the above, this is just based on reading ;-)
Maybe you need to encode your URL before send the request using HttpUtility class, this way any special character will be escaped.
using System.Web;
var url = $"htpps://myurl.com/{HttpUtility.UrlEncode("#1234567")}";

Get instance of ApiController class from a URL directly

I am looking for a way to call the appropriate method (get, post etc.) on an ApiController class based on the URL and request type etc. without making a http request.
Background: We have an API application with numerous controllers that needs to also accept requests from a remote server. Due to restrictions I cannot control there is no way to open ports between the two servers to allow the remote server to make the request directly so we decided to forward the data using websockets (SignalR). I can send (within reason) whatever information is required.
I have tried the below:
HttpRequestMessage request = new HttpRequestMessage();
var bld = new UriBuilder
{
Port = 123,
Path = "api/v1/search",
Query = "query=search_string"
};
request.RequestUri = bld.Uri;
var httpCfg = AppConfiguration.Get().HttpConfig; //this is the same config that UseWebApi was called with and contains the routes.
var route = httpCfg.Routes.GetRouteData(request);
var controllerSelector = new DefaultHttpControllerSelector(httpCfg);
var descriptor = controllerSelector.SelectController(request);
route contains the controller name (search) but the call to SelectController throws an exception with a 404 response in it (I presume this indicates I am missing something from the fake request). The same URI works when sent as a direct http request so the routes do work as best I can tell.
Is there a better way to do this, or if not what am I missing from the request that is causing the 404?

How to get raw url from Owin?

How can I get raw url from Owin (the url what was passed to HTTP request), independently on Owin hosting?
For instance, both http://localhost/myapp and http://localhost/myapp/ contains in IOwinRequest.Path the /. The PathBase contains always /myapp and Uri.OriginalString contains always http://localhost/myapp/.
(In ASP.NET I would call HttpContext.Current.Request.RawUrl which returns either /myapp or /myapp/.)
Reason: Currently, I need it to do the server side redirect to add the trailing / if it is missing (independently on the hosting).
You can get the raw Url in Owin by accessing the HttpListenerContext that was used to receive the request.
public static string RealUrlFromOwin(HttpRequestMessage request)
{
var owincontext = ((OwinContext) request.Properties["MS_OwinContext"]);
var env = owincontext.Environment;
var listenerContext = (System.Net.HttpListenerContext) env["System.Net.HttpListenerContext"];
return listenerContext.Request.RawUrl;
}
This will not only recover things like a trailing / in the Url, but will also get the Url string before any decoding is applied, so you can distinguish between '!' and %21, for instance.

Get And Use Facebook Access Token Without HTTPS

I have the following url for the first stage. getting code http://www.facebook.com/dialog/oauth?client_id=myappid&redirect_uri=myurl;state=e879888c-7090-4c09-98a5-ff361b30d55;scope=email
and
url = String.Format(#"http://graph.facebook.com/oauth/access_token?client_id={0}&redirect_uri={1}&client_secret={2}&code={3}",FacebookAppId, redirectUrl, FacebookSecret, code);
WebClient webClient = new WebClient();
var tokenVars = webClient.DownloadString(url);
var responseVars = HttpUtility.ParseQueryString(tokenVars);
string access_token = responseVars["access_token"];
string data = webClient.DownloadString(String.Format("http://graph.facebook.com/me?access_token={0}", access_token));
var jobject = JObject.Parse(data);
400 BAD REQUEST is returned.
Question: Is it possible to get facebook access token without connecting by http s ?
Question: Is it possible to use access_token without HTTPS? NOT HTTP
for a secure you must take https,
maybe if you aren't, some bad thing will show, #long process..
i just browse for your issue, and same thing

Categories