I have 2 sites, the first site is basic forms authentication system and some method, that needs to authenticate user. In the second I use modified WebClient (it can use cookies) and send request for authenticate and after request for secure operation. At first step all are OK, first server return AUTH cookie in response and remember it, and on the second it send request for secure operation with AUTH cookie, but at the first site, our request is not authorized! Request.isauthenticated = false.
Why is it? The AUTH cookie at this request are valid.
It is code for site №1 WebClient requests.
//Create an instance of your new CookieAware Web Client
var client = new CookieAwareWebClient();
//Authenticate (username and password can be either hard-coded or pulled from a settings area)
var values = new NameValueCollection { { "Name", "name" }, { "Password", "1234" } };
//Perform authentication - after this has been performed the cookie will be stored within the Web Client
client.UploadValues(new Uri("http://localhost:15536/Plugins/ProductListGetter/login"), "POST", values);
var _cookies = client.ResponseHeaders["Set-Cookie"];
client.UploadString(new Uri("http://localhost:15536/Plugins/ProductListGetter/ChangeNameForCurrentUser"), "POST", "Example Message");
client.UploadString(new Uri("http://localhost:15536/Plugins/ProductListGetter/ChangeNameForCurrentUser"), "POST", "Example Message");
client.Dispose();
And code for modified WebClient
public class CookieAwareWebClient : WebClient
{
//Properties to handle implementing a timeout
private int? _timeout = null;
public int? Timeout
{
get
{
return _timeout;
}
set
{
_timeout = value;
}
}
//A CookieContainer class to house the Cookie once it is contained within one of the Requests
public CookieContainer CookieContainer { get; private set; }
//Constructor
public CookieAwareWebClient()
{
CookieContainer = new CookieContainer();
}
//Method to handle setting the optional timeout (in milliseconds)
public void SetTimeout(int timeout)
{
_timeout = timeout;
}
//This handles using and storing the Cookie information as well as managing the Request timeout
protected override WebRequest GetWebRequest(Uri address)
{
//Handles the CookieContainer
var request = (HttpWebRequest)base.GetWebRequest(address);
request.CookieContainer = CookieContainer;
//Sets the Timeout if it exists
if (_timeout.HasValue)
{
request.Timeout = _timeout.Value;
}
return request;
}
Related
I'm trying to access a resource on a Lighttpd server which enforces that the full request URI matches the URI in the Authorization request header. This is specified in RFC 7616
The authenticating server MUST assure that the resource designated
by the "uri" parameter is the same as the resource specified in the
Request-Line; if they are not, the server SHOULD return a 400 Bad
Request error. (Since this may be a symptom of an attack, server
implementers may want to consider logging such errors.) The purpose
of duplicating information from the request URL in this field is to
deal with the possibility that an intermediate proxy may alter the
client's Request-Line. This altered (but presumably semantically
equivalent) request would not result in the same digest as that
calculated by the client.
I'm using the Flurl library (v1.4), which is just a wrapper around HttpClient. However, HttpClientHandler is from .Net.
Why does it use the base URI and not the full URI? Is it a bug? How do I make it use the full URI?
I thought of adding another HttpMessageHandler into the pipeline and modifying the Authentication header with the full URI, but HttpClientHandler doesn't let you set an InnerHandler.
The full Uri should be:
http://base/resource.cgi?my=1¶ms=2
But this is what appears in the request header:
Authorization: Digest
username="user",realm="serve",nonce="5b911545:eaf4352d2113e5e4b1ca253bd70fd90a",
uri="/base/resource.cgi",cnonce="bf7cf40f1289bc10bd07e8bf4784159c",nc=00000001,qop="auth",response="cf3c731ec93f7e5928f19f880f8325ab"
Which results in a 400 Bad Request response.
My Flurl Code:
HttpClient Factory
/// <summary>
/// Custom factory to generate HttpClient and handler to use digest authentication and a continuous stream
/// </summary>
private class DigestHttpFactory : Flurl.Http.Configuration.DefaultHttpClientFactory
{
private CredentialCache CredCache { get; set; }
public DigestHttpFactory(string url, string username, string password) : base()
{
Url = url;
CredCache = new CredentialCache
{
{ new Uri(Url), "Digest", new NetworkCredential(username, password) }
};
}
private string Url { get; set; }
public override HttpClient CreateHttpClient(HttpMessageHandler handler)
{
var client = base.CreateHttpClient(handler);
client.Timeout = TimeSpan.FromMilliseconds(Timeout.Infinite); // To keep stream open indefinietly
return client;
}
public override HttpMessageHandler CreateMessageHandler()
{
var handler = new HttpClientHandler
{
Credentials = CredCache.GetCredential(new Uri(Url), "Digest")
};
return handler;
}
}
Request code
public class MyConnection
{
public string BaseUrl => "http://base/resource.cgi";
public async Task ConnectAsync(CancellationToken cancellationToken = default(CancellationToken))
{
ConnectionCancellation = new CancellationTokenSource();
var url = BaseUrl
.SetQueryParam("my", 1)
.SetQueryParam("params", 2)
FlurlHttp.ConfigureClient(url, client =>
{
client.Configure(settings =>
{
settings.HttpClientFactory = new DigestHttpFactory(url, Username, Password);
});
});
try
{
using (var getResponse = url.GetAsync(cancellationToken, HttpCompletionOption.ResponseHeadersRead))
{
var responseMessage = await getResponse;
using (var linkedCts = CancellationTokenSource.CreateLinkedTokenSource(ConnectionCancellation.Token, cancellationToken))
using (var stream = await responseMessage.Content.ReadAsStreamAsync())
await Process(stream, linkedCts.Token);
}
}
catch (OperationCanceledException ex)
{
throw ex;
}
catch (Exception ex)
{
throw ex;
}
}
}
It appears that this is indeed a bug with Microsoft's implementation of the RFC:
System.Net classes don't include the query string in the 'Uri'
attribute of the digest authentication header. This is a violation of
the RFC, and some web server implementations reject those requests.
That post details a work-around taken from this answer.
I login with HTTPWebRequest and transfer the CookieContainer to the new class and want my WebClient to log in with this cookie and download a site as string. But I only get the source code of the login page, so it seems like the webclient won't login.
This is my current code:
BasicAuthentication login = new BasicAuthentication();
CookieContainer cookieJar = login.getCookie();
WebClientEx client = new WebClientEx();
client.Cookies = cookieJar;
And the WebClientEx Class
class WebClientEx : WebClient
{
private CookieContainer _cookies;
private string _ref;
public WebClientEx()
{
_cookies = new CookieContainer();
}
public CookieContainer Cookies
{
get { return _cookies; }
set { _cookies = value; }
}
protected override WebRequest GetWebRequest(System.Uri address)
{
var webReq = base.GetWebRequest(address);
if (webReq is HttpWebRequest)
{
var req = (HttpWebRequest)webReq;
req.CookieContainer = _cookies;
if (_ref != null)
{
req.Referer = _ref;
}
}
_ref = address.ToString();
return webReq;
}
protected override void Dispose(bool disposing)
{
_cookies = null;
base.Dispose(disposing);
}
}
I just get an Answer to my question.
Here's a example of my code:
WebClient client = new WebClient();
client.Headers.Add(HttpRequestHeader.Cookie, "cookievalue");
string download = client.DownloadString("url");
output.Text = download;
In the variable cookievalue I only enter the cookiename=cookievalue.
I get the name and the value from Postman, a useful google chrome tool.
I just have to make sure that i can redirect to any login proteccted page with postman and when this is true, then I can use cookiename and cookievalue to login my webclient and see all pages and download sites with downloadString.
I've previously used a CookieContainer with HttpWebRequest and HttpWebResponse sessions, but now, I want to use it with a WebClient. As far as I understand, there is no built-in method like there is for HttpWebRequests (request.CookieContainer). How can I collect cookies from a WebClient in a CookieContainer?
I googled for this and found the following sample:
public class CookieAwareWebClient : WebClient
{
private readonly CookieContainer m_container = new CookieContainer();
protected override WebRequest GetWebRequest(Uri address)
{
WebRequest request = base.GetWebRequest(address);
HttpWebRequest webRequest = request as HttpWebRequest;
if (webRequest != null)
{
webRequest.CookieContainer = m_container;
}
return request;
}
}
Is this the best way to do it?
WebClient wb = new WebClient();
wb.Headers.Add(HttpRequestHeader.Cookie, "somecookie");
From Comments
How do you format the name and value of the cookie in place of "somecookie" ?
wb.Headers.Add(HttpRequestHeader.Cookie, "cookiename=cookievalue");
For multiple cookies:
wb.Headers.Add(HttpRequestHeader.Cookie,
"cookiename1=cookievalue1;" +
"cookiename2=cookievalue2");
Yes. IMHO, overriding GetWebRequest() is the best solution to WebClient's limited functionalty. Before I knew about this option, I wrote lots of really painful code at the HttpWebRequest layer because WebClient almost, but not quite, did what I needed. Derivation is much easier.
Another option is to use the regular WebClient class, but manually populate the Cookie header before making the request and then pull out the Set-Cookies header on the response. There are helper methods on the CookieContainer class which make creating and parsing these headers easier: CookieContainer.SetCookies() and CookieContainer.GetCookieHeader(), respectively.
I prefer the former approach since it's easier for the caller and requires less repetitive code than the second option. Also, the derivation approach works the same way for multiple extensibility scenarios (e.g. cookies, proxies, etc.).
This one is just extension of article you found.
public class WebClientEx : WebClient
{
public WebClientEx(CookieContainer container)
{
this.container = container;
}
public CookieContainer CookieContainer
{
get { return container; }
set { container= value; }
}
private CookieContainer container = new CookieContainer();
protected override WebRequest GetWebRequest(Uri address)
{
WebRequest r = base.GetWebRequest(address);
var request = r as HttpWebRequest;
if (request != null)
{
request.CookieContainer = container;
}
return r;
}
protected override WebResponse GetWebResponse(WebRequest request, IAsyncResult result)
{
WebResponse response = base.GetWebResponse(request, result);
ReadCookies(response);
return response;
}
protected override WebResponse GetWebResponse(WebRequest request)
{
WebResponse response = base.GetWebResponse(request);
ReadCookies(response);
return response;
}
private void ReadCookies(WebResponse r)
{
var response = r as HttpWebResponse;
if (response != null)
{
CookieCollection cookies = response.Cookies;
container.Add(cookies);
}
}
}
The HttpWebRequest modifies the CookieContainer assigned to it. There is no need to process returned cookies. Simply assign your cookie container to every web request.
public class CookieAwareWebClient : WebClient
{
public CookieContainer CookieContainer { get; set; } = new CookieContainer();
protected override WebRequest GetWebRequest(Uri uri)
{
WebRequest request = base.GetWebRequest(uri);
if (request is HttpWebRequest)
{
(request as HttpWebRequest).CookieContainer = CookieContainer;
}
return request;
}
}
I think there's cleaner way where you don't have to create a new webclient (and it'll work with 3rd party libraries as well)
internal static class MyWebRequestCreator
{
private static IWebRequestCreate myCreator;
public static IWebRequestCreate MyHttp
{
get
{
if (myCreator == null)
{
myCreator = new MyHttpRequestCreator();
}
return myCreator;
}
}
private class MyHttpRequestCreator : IWebRequestCreate
{
public WebRequest Create(Uri uri)
{
var req = System.Net.WebRequest.CreateHttp(uri);
req.CookieContainer = new CookieContainer();
return req;
}
}
}
Now all you have to do is opt in for which domains you want to use this:
WebRequest.RegisterPrefix("http://example.com/", MyWebRequestCreator.MyHttp);
That means ANY webrequest that goes to example.com will now use your custom webrequest creator, including the standard webclient. This approach means you don't have to touch all you code. You just call the register prefix once and be done with it.
You can also register for "http" prefix to opt in for everything everywhere.
I am using WebClient to try and access a web page. I have to login first and the site sets cookies.
I am using code from this question
The cookies are being set correctly, however when I try to access the second page I am just returned to the login page.
private WebClientEx client;
public string GetFile(string URL)
{
Login();
// Download desired page
return client.DownloadString(URL);
}
private void Login()
{
using (client)
{
var values = new NameValueCollection
{
{ "Login", "xxxx" },
{ "Password", "xxxx" },
};
// Authenticate
client.UploadValues("http://www.xxxxxx.com/backoffice/login.php");
}
return;
}
}
/// <summary>
/// A custom WebClient featuring a cookie container
/// </summary>
public class WebClientEx : WebClient
{
public CookieContainer CookieContainer { get; private set; }
public WebClientEx()
{
CookieContainer = new CookieContainer();
}
protected override WebRequest GetWebRequest(Uri address)
{
WebRequest request = base.GetWebRequest(address);
if (request.GetType() == typeof(HttpWebRequest))
((HttpWebRequest)request).CookieContainer = CookieContainer;
return request;
}
}
I've previously used a CookieContainer with HttpWebRequest and HttpWebResponse sessions, but now, I want to use it with a WebClient. As far as I understand, there is no built-in method like there is for HttpWebRequests (request.CookieContainer). How can I collect cookies from a WebClient in a CookieContainer?
I googled for this and found the following sample:
public class CookieAwareWebClient : WebClient
{
private readonly CookieContainer m_container = new CookieContainer();
protected override WebRequest GetWebRequest(Uri address)
{
WebRequest request = base.GetWebRequest(address);
HttpWebRequest webRequest = request as HttpWebRequest;
if (webRequest != null)
{
webRequest.CookieContainer = m_container;
}
return request;
}
}
Is this the best way to do it?
WebClient wb = new WebClient();
wb.Headers.Add(HttpRequestHeader.Cookie, "somecookie");
From Comments
How do you format the name and value of the cookie in place of "somecookie" ?
wb.Headers.Add(HttpRequestHeader.Cookie, "cookiename=cookievalue");
For multiple cookies:
wb.Headers.Add(HttpRequestHeader.Cookie,
"cookiename1=cookievalue1;" +
"cookiename2=cookievalue2");
Yes. IMHO, overriding GetWebRequest() is the best solution to WebClient's limited functionalty. Before I knew about this option, I wrote lots of really painful code at the HttpWebRequest layer because WebClient almost, but not quite, did what I needed. Derivation is much easier.
Another option is to use the regular WebClient class, but manually populate the Cookie header before making the request and then pull out the Set-Cookies header on the response. There are helper methods on the CookieContainer class which make creating and parsing these headers easier: CookieContainer.SetCookies() and CookieContainer.GetCookieHeader(), respectively.
I prefer the former approach since it's easier for the caller and requires less repetitive code than the second option. Also, the derivation approach works the same way for multiple extensibility scenarios (e.g. cookies, proxies, etc.).
This one is just extension of article you found.
public class WebClientEx : WebClient
{
public WebClientEx(CookieContainer container)
{
this.container = container;
}
public CookieContainer CookieContainer
{
get { return container; }
set { container= value; }
}
private CookieContainer container = new CookieContainer();
protected override WebRequest GetWebRequest(Uri address)
{
WebRequest r = base.GetWebRequest(address);
var request = r as HttpWebRequest;
if (request != null)
{
request.CookieContainer = container;
}
return r;
}
protected override WebResponse GetWebResponse(WebRequest request, IAsyncResult result)
{
WebResponse response = base.GetWebResponse(request, result);
ReadCookies(response);
return response;
}
protected override WebResponse GetWebResponse(WebRequest request)
{
WebResponse response = base.GetWebResponse(request);
ReadCookies(response);
return response;
}
private void ReadCookies(WebResponse r)
{
var response = r as HttpWebResponse;
if (response != null)
{
CookieCollection cookies = response.Cookies;
container.Add(cookies);
}
}
}
The HttpWebRequest modifies the CookieContainer assigned to it. There is no need to process returned cookies. Simply assign your cookie container to every web request.
public class CookieAwareWebClient : WebClient
{
public CookieContainer CookieContainer { get; set; } = new CookieContainer();
protected override WebRequest GetWebRequest(Uri uri)
{
WebRequest request = base.GetWebRequest(uri);
if (request is HttpWebRequest)
{
(request as HttpWebRequest).CookieContainer = CookieContainer;
}
return request;
}
}
I think there's cleaner way where you don't have to create a new webclient (and it'll work with 3rd party libraries as well)
internal static class MyWebRequestCreator
{
private static IWebRequestCreate myCreator;
public static IWebRequestCreate MyHttp
{
get
{
if (myCreator == null)
{
myCreator = new MyHttpRequestCreator();
}
return myCreator;
}
}
private class MyHttpRequestCreator : IWebRequestCreate
{
public WebRequest Create(Uri uri)
{
var req = System.Net.WebRequest.CreateHttp(uri);
req.CookieContainer = new CookieContainer();
return req;
}
}
}
Now all you have to do is opt in for which domains you want to use this:
WebRequest.RegisterPrefix("http://example.com/", MyWebRequestCreator.MyHttp);
That means ANY webrequest that goes to example.com will now use your custom webrequest creator, including the standard webclient. This approach means you don't have to touch all you code. You just call the register prefix once and be done with it.
You can also register for "http" prefix to opt in for everything everywhere.