Hi I'm trying to implement simple http proxy service using Owin infrastructure. This proxy must authenticate user with windows authentication, pull user's properties from the enterprise AD, add this info as a cookie value into original request and then redirect request to the application on the Internet (lets call it External Application).
I'm using HttpClient to send request to the External Application.
However HttpClient does not fit well for this scenario. It seems that the only allowed way to send cookie using it is to put them into CookieContainer and set this CookieContainer as a property of HttpClientHandler. It's ok when you have single user but in case of a proxy service cookie values from different users will mix and overwrite each other.
Is there any way to set cookie or CookieContainer per request? Or maybe there is a better way to redirect requests?
P.S. Here is some code:
Initialization of http handlers:
private void RegisterRoutes(HttpConfiguration config)
{
config.Routes.MapHttpRoute(
name: "Proxy",
routeTemplate: "{*path}",
handler: HttpClientFactory.CreatePipeline
(
innerHandler: new HttpClientHandler(),
handlers: new DelegatingHandler[]
{
new ProxyHandler()
}
),
defaults: new { path = RouteParameter.Optional },
constraints: null);
}
ProxyHandler
internal class ProxyHandler : DelegatingHandler
{
private readonly HttpClient _client;
public ProxyHandler()
{
var handler = new HttpClientHandler { ClientCertificateOptions = ClientCertificateOption.Automatic};
_client = new HttpClient(handler);
}
protected async override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
var forwardUri = new UriBuilder(request.RequestUri);
forwardUri.Host = "localhost";
forwardUri.Port = 23016;
forwardUri.Scheme = Uri.UriSchemeHttp;
request.RequestUri = forwardUri.Uri;
request.Headers.Host = forwardUri.Host;
//Explicitly null it to avoid protocol violation
if (request.Method == HttpMethod.Get || request.Method == HttpMethod.Trace)
request.Content = null;
try
{
var response = await _client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, cancellationToken);
//Explicitly null it to avoid protocol violation
if (request.Method == HttpMethod.Head)
response.Content = null;
return response;
}
catch (Exception ex)
{
var response = request.CreateErrorResponse(HttpStatusCode.InternalServerError, ex);
string message = ex.Message;
if (ex.InnerException != null)
message += ':' + ex.InnerException.Message;
response.Content = new StringContent(message);
Trace.TraceError("Error:{0}", message);
return response;
}
}
private void SetCookies(HttpRequestMessage request)
{
var container = new CookieContainer();
var authCookieValue = "2EF91D8FD9EDC594F2DB82";
var authCookie = new Cookie("cookieByProxy", authCookieValue);
var targetUri = new Uri("http://localhost:23016/");
container.Add(targetUri, authCookie);
var cookieHeader = container.GetCookieHeader(targetUri);
if (!string.IsNullOrEmpty(cookieHeader))
request.Headers.TryAddWithoutValidation("Cookie", cookieHeader);//Overwriting cookie header with custom values. However cookie values are ignored by HttpClient (both old and new)
}
}
Found solution here: enter link description here
The trick is in explicitly setting UseCookie flag of the HttpClientHandler to false.
var handler = new HttpClientHandler { ClientCertificateOptions = ClientCertificateOption.Automatic, UseCookie = false };
Related
I need to check if certain HTTPS proxies are working. To do so, I originally thought of using the following code:
var proxy = new WebProxy($"https://proxy-server:port");
var handler = new HttpClientHandler { Proxy = proxy };
using (var client = new HttpClient(handler))
{
client.BaseAddress = new Uri("https://base-address");
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", credentials);
try
{
var content = client.GetStringAsync(filepath).Result;
Console.WriteLine(content);
}
catch (Exception e)
{
Console.WriteLine("Failed. Reason: " + e);
}
}
If I try to access the base-address without the proxy, it works fine. Using the proxy I get an error saying I can only use HTTP or SOCKS proxies. The problem is, I wasn't able to find an alternate solution that supports HTTPS proxies.
You're getting the error because the WebProxy class in .NET does not support HTTPS proxies. However, you can use the HttpClientHandler's SslOptions property to configure the SSL/TLS settings.
var proxy = new WebProxy($"http://proxy-server:port");
var handler = new HttpClientHandler
{
Proxy = proxy,
SslOptions = new SslClientAuthenticationOptions
{
RemoteCertificateValidationCallback = (sender, certificate, chain, sslPolicyErrors) => true
}
};
using (var client = new HttpClient(handler))
{
client.BaseAddress = new Uri("https://base-address");
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", credentials);
try
{
var content = client.GetStringAsync(filepath).Result;
Console.WriteLine(content);
}
catch (Exception e)
{
Console.WriteLine("Failed. Reason: " + e);
}
}
Create a new instance of HttpClientHandler and set its Proxy property to a new WebProxy that represents the HTTP proxy server.
Set the SslOptions property to a new instance of SslClientAuthenticationOptions and configure the RemoteCertificateValidationCallback to return true, indicating that you want to accept any server certificate.
Create a new instance of HttpClient and pass the handler to its constructor.
Finally, use the client to send an HTTPS GET request to the base-address website.
ASP.Net Core Web API Call Thirds party API fails intermittently.
The following exception raises intermittently when load test with postman.
"Call failed with status code 500 (Internal Server Error): POST https://sample.com/apiendpoint."
I tried the Named/Typed with HttpClient/IHttpClientFactory approach and the problem continues.
How to make sure it uses connection pooling and not create new on one.
what is the right value for SetHandlerLifetime to keep the connection in the pool for future call to use.
services.AddHttpClient<IRestService, RestServiceOne>()
.SetHandlerLifetime(TimeSpan.FromMinutes(5)) //Set lifetime to five minutes
.AddPolicyHandler(GetRetryPolicy());
The following code is in RestServiceOne.cs
public class RestServiceOne : IRestService
{
private readonly IHttpClientFactory _httpClientFactory;
public RestServiceOne(IHttpClientFactory httpClientFactory)
{
_httpClientFactory = httpClientFactory;
}
public async Task<HttpResponseMessage> GetDataAsync(string destinationUrl, string user,
string password, string requestXml, string orderNumber, CancellationToken cancellationToken)
{
var endpoint = $"{destinationUrl}";
var authToken = Encoding.ASCII.GetBytes($"{user}:{password}");
var data = new StringContent(requestXml, Encoding.UTF8, "text/xml");
var httpRequestMessage = new HttpRequestMessage(
HttpMethod.Post,
endpoint)
{
Headers =
{
{ "Accept", "application/vnd.github.v3+json" },
{ "User-Agent", "HttpRequestsConsoleSample" }
}
};
httpRequestMessage.Content = data;
var httpClient = _httpClientFactory.CreateClient();
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic",
Convert.ToBase64String(authToken));
var httpResponseMessage = await httpClient.SendAsync(httpRequestMessage);
return httpResponseMessage;
}
}
I also tried HttpClient injection given in Microsoft type example.
public class RestService
{
private readonly HttpClient _httpClient;
public RestService(HttpClient httpClient)
{
_httpClient = httpClient;
try
{
_httpClient.BaseAddress = new Uri("https://testxx.com/test");
// GitHub API versioning
_httpClient.DefaultRequestHeaders.Add("Accept",
"application/vnd.github.v3+json");
// GitHub requires a user-agent
_httpClient.DefaultRequestHeaders.Add("User-Agent",
"HttpClientFactory-Sample");
}
catch(Exception ex)
{
}
}
public async Task<HttpResponseMessage> GetDataAsync(string destinationUrl, string user,
string password, string requestXml, string orderNumber, CancellationToken cancellationToken)
{
var endpoint = $"{destinationUrl}";
var authToken = Encoding.ASCII.GetBytes($"{user}:{password}");
_httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic",
Convert.ToBase64String(authToken));
var data = new StringContent(requestXml, Encoding.UTF8, "text/xml");
try
{
var response = await _httpClient.PostAsync(endpoint, data);
return response;
}
catch(Exception ex)
{
throw;
}
}
}
I tried with Flurl directly in service layer
var endpoint = $"{destinationUrl}";
var authToken = Encoding.ASCII.GetBytes($"{user}:{password}");
var data = new StringContent(requestXml, Encoding.UTF8, "text/xml");
try
{
var response = await endpoint
.WithHeader("Content-Type", "text/xml")
.WithHeader("app-bu-id", "SANIDERMMEDICAL")
.WithHeader("Authorization", new AuthenticationHeaderValue("Basic", Convert.ToBase64String(authToken)))
.PostAsync(data);
The above all 3 approach failed.
what is right value for .SetHandlerLifetime() to keep the connection in the pool and avoid creating new.
I saw some other example using the following approach but how to use this with IHttpClientFactory / Flurl.
var socketsHandler = new SocketsHttpHandler
{
PooledConnectionLifetime = TimeSpan.FromMinutes(10),
PooledConnectionIdleTimeout = TimeSpan.FromMinutes(5),
MaxConnectionsPerServer = 10
};
var client = new HttpClient(socketsHandler);
How can I ensure it use connection pooling and avoid the 500 error when calling 3rd party API from Azure.
I found solution when I use httpclient as given below.
private static readonly HttpClient httpClient = new HttpClient(new SocketsHttpHandler
{
PooledConnectionLifetime = TimeSpan.FromSeconds(60),
PooledConnectionIdleTimeout = TimeSpan.FromSeconds(20),
MaxConnectionsPerServer = 10
});
This article helped me
Tried couple of value for SocketsHttpHandler but finally choose this since no error.
I have this method running in a .NET Core 2.X app running in Azure app service. I have a remote server that we use this method to call from button presses in our Angular website. that calls a remote device.
Angular button --> .NET Core app service in Azure --> another app service --> internet\cell connected device. We wait for the response from the device to return a status code.
If I quickly send commands [2 or 3 in a second] to this method it causes the app service to stop responding until I restart it. I read this post and added the [, HttpCompletionOption.ResponseHeadersRead).ConfigureAwait(false)].
However I can still freeze the entire app and require a restart from quickly sending commands to this method.
private async void SetEndPointValueAsync(string stunnelUrl, string username, string password)
{
try
{
//set the user name and password
var httpClientHandler = new HttpClientHandler()
{
Credentials = new NetworkCredential(username, password)
};
using (var client = new HttpClient(httpClientHandler))
{
using (var response = await client.GetAsync(stunnelUrl**, HttpCompletionOption.ResponseHeadersRead).ConfigureAwait(false)**)
{
if (response.IsSuccessStatusCode)
{
LogInfo(typeof(IntegrationService), stunnelUrl, LogAction.EndpointUpdate);
}
else
{
//request failed.
LogWarning(typeof(IntegrationService), stunnelUrl, LogAction.DeviceRequest);
}
//using (var content = response.Content)
//{
// //do here your changes when required
//}
}
}
}
catch (Exception e)
{
LogErrorDetailed(e, typeof(IntegrationService), stunnelUrl, LogAction.DeviceRequest);
}
}
Generally, you don't want to create so many instances of HttpClient as you lose a lot of the benefits of the management it provides.
You could reduce some overhead by only having a single instance of it, like so...
private readonly HttpClient _client;
public ClassConstructor(HttpClient client)
{
_client = client ?? new HttpClient();
}
Then you could change your method to look something like this...
private async Task SetEndPointValueAsync(Uri stunnelUri, string username, string password)
{
if (stunnelUri == null) throw new ArgumentNullException(nameof(stunnelUri));
if (String.IsNullOrEmpty(username)) throw new ArgumentNullException(nameof(username));
if (String.IsNullOrEmpty(password)) throw new ArgumentNullException(nameof(password));
byte[] byteArray = Encoding.ASCII.GetBytes($"{username}:{password}");
string scheme = "Basic";
string parameter = Convert.ToBase64String(byteArray);
HttpRequestMessage request = new HttpRequestMessage();
request.Method = HttpMethod.Post;
request.RequestUri = stunnelUri;
request.Headers.Authorization = new AuthenticationHeaderValue(scheme, parameter);
try
{
HttpResponseMessage response = await _client.SendAsync(request);
// This will throw an HttpRequestException if there is a failure above
response.EnsureSuccessStatusCode();
// do your stuff here...
// { ... }
}
catch (HttpRequestException)
{
// { log your connection / bad request issue. }
}
catch (Exception)
{
// I don't recommend this, but since you had it...
// { handle all other exceptions }
}
}
I am trying to pull REST data from an API but I need to handle the calls to the API with some server side solution. I have tried using the following code
try
{
HttpClient client = new HttpClient();
client.Timeout = TimeSpan.FromSeconds(60);
var request = new HttpRequestMessage()
{
RequestUri = new Uri(string.Format("https://jsonodds.com/{0}{1}{2}", "api/odds/", "?source=", "3")),
Method = HttpMethod.Get,
};
request.Headers.Add("JsonOdds-API-Key", "your key");
HttpResponseMessage response = client.SendAsync(request).Result;
if (response.IsSuccessStatusCode)
{
String.Format("Success");
}
}
catch (Exception ex)
{ //log error }
I receive a 407() error. Any ideas or tips how to do this?
If you are going through a proxy server then you need to use a different constructor for HttpClient.
_httpClient = new HttpClient(new HttpClientHandler
{
UseProxy = true,
Proxy = new WebProxy
{
Address = new Uri(proxyUrl),
BypassProxyOnLocal = false,
UseDefaultCredentials = true
}
})
{
BaseAddress = url
};
Replace proxyUrl with your proxy address then replacing the credential with those that are valid for your proxy. This example uses the default credentials, but you can pass a NetworkCredential to the WebProxy.
I work on a Xamarin.Forms project where I call some WebServices that using cookies, as they was intially made for a website. There are some webservices that only return cookies, but ohers need to receive cookies for working well.
For example, the Login webservice gets a JSON and returns 2 cookies, while the Logout webservice gets an "empty" JSON, the 2 previous cookies and must return an updated value for one of these cookies.
I based on the following link to manage cookies with HttpClient:
Struggling trying to get cookie out of response with HttpClient in .net 4.5
My problem is that I can send a cookie to a webservice, I can receive the returned cookie by a webservice, but I can't receive any "updated" cookie if I've sent it before the call...
The code of the call to the Login webservice looks like this:
public async Task Login()
{
Uri uri = new Uri("http://www.website.com/Login");
CookieContainer cookies = new CookieContainer();
HttpClientHandler handler = new HttpClientHandler();
handler.CookieContainer = cookies;
var httpClient = new HttpClient(handler);
var jsonParam = "{\"data\":{\"device\":\"xxx\",\"login\":\"my#email.com\",\"password\":\"password\"}}";
HttpContent httpContent = new StringContent(jsonParam);
httpContent.Headers.ContentType = new MediaTypeHeaderValue("application/json");
httpClient.DefaultRequestHeaders.Add("X-HTTP-Method-Override", "PUT");
httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
try
{
HttpResponseMessage httpResponse = httpClient.PostAsync(uri), httpContent).Result;
//Treatment of the recovered cookies
IEnumerable<Cookie> responseCookies = cookies.GetCookies(uri).Cast<Cookie>();
foreach (Cookie cookie in responseCookies)
{
Debug.WriteLine(cookie.Name + " : " + cookie.Value);
wsCookies.Add(new KeyValuePair<string, string>(cookie.Name, cookie.Value));
}
if (httpResponse.IsSuccessStatusCode)
{
var responseText = await httpResponse.Content.ReadAsStringAsync();
}
}
catch (Exception e)
{
}
}
=> it works well: I get the expected cookies: SESSIONID=xxx and USERID=xxx
The code of my Logout method looks like this:
public async Task Logout(String test)
{
Uri uri = new Uri("http://www.website.com/Logout");
CookieContainer cookies = new CookieContainer();
HttpClientHandler handler = new HttpClientHandler();
handler.CookieContainer = cookies;
var httpClient = new HttpClient(handler);
var jsonParam = "{\"data\":{}}";
HttpContent httpContent = new StringContent(jsonParam);
httpContent.Headers.ContentType = new MediaTypeHeaderValue("application/json");
httpClient.DefaultRequestHeaders.Add("X-HTTP-Method-Override", "GET");
httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
//Retrieving cookies to send
foreach (KeyValuePair<string, string> kvpCookie in wsCookies)
{
cookies.Add(uri, new Cookie(kvpCookie.Key, kvpCookie.Value));
}
try
{
HttpResponseMessage httpResponse = httpClient.PostAsync(uri, httpContent).Result;
//Treatment of the recovered cookies
IEnumerable<Cookie> responseCookies = cookies.GetCookies(uri).Cast<Cookie>();
foreach (Cookie cookie in responseCookies)
{
Debug.WriteLine(cookie.Name + " : " + cookie.Value);
wsCookies.Add(new KeyValuePair<string, string>(cookie.Name, cookie.Value));
}
if (httpResponse.IsSuccessStatusCode)
{
var responseText = await httpResponse.Content.ReadAsStringAsync();
}
}
catch (Exception e)
{
}
}
=> this method works but I don't get the expected cookies. I send the cookies (names and values) that I've received earlier (SESSIONID=xxx and USERID=xxx), but I don't get a new value for the cookies, whereas I wait USERID=deleted as I can see when I sniff it in Fiddler: I only find the 2 cookies I've sent in "responseCookies"...
=> Do you have an explanation? Is there something that I'm doing wrong? Is there another way to manage webservices and cookies?
Not sure if that is the issue, but why create the cookie container every time? You can pass the cookies along every time.
So initialize once, and then assign the existing cookie container to the requests in Login and Logout.