Which is the preferred method to open a Url (and are there any differences behind the scenes between):
driver.Url = "http://example.com";
or
driver.Navigate().GoToUrl("http://example.com");
Also, if the driver is already pointing at the same page, will setting the Url a second time cause the page to refresh?
i.e.
...
driver.Url = "http://example.com";
driver.Url = "http://example.com"; //does this reload the page?
...
FWIW I'm using the Chrome driver chromedriver.exe, but it doesn't appear to be a managed assembly (I tried opening it with ILSpy but no luck).
Selenium is an open source framework, so please have a look at the source code here.
GoToUrl() is defined in RemoteNavigator.cs:
/// <summary>
/// Navigate to a url for your test
/// </summary>
/// <param name="url">String of where you want the browser to go to</param>
public void GoToUrl(string url)
{
this.driver.Url = url;
}
/// <summary>
/// Navigate to a url for your test
/// </summary>
/// <param name="url">Uri object of where you want the browser to go to</param>
public void GoToUrl(Uri url)
{
if (url == null)
{
throw new ArgumentNullException("url", "URL cannot be null.");
}
this.driver.Url = url.ToString();
}
So basically driver.Navigate().GoToUrl(); sets driver.Url under the hood and I don't see a difference there.
However, driver.Navigate().GoToUrl() is more flexible, which allows sending either string or Uri as parameter types, while only string is allowed when setting through driver.Url.
To your second question, the source code shows that driver.Navigate().Refresh() asks browsers to refresh, while driver.Url tells browsers to navigate. So these two are fundamentally different. For more details, please see Difference between Refresh and Navigate function in browser control?
If you want to refresh the page, please use driver.Navigate().Refresh();
Refresh() is defined in RemoteNavigator.cs:
/// <summary>
/// Refresh the browser
/// </summary>
public void Refresh()
{
// driver.SwitchTo().DefaultContent();
this.driver.InternalExecute(DriverCommand.Refresh, null);
}
driver.Url is defined in RemoteWebDriver.cs:
public string Url
{
...
set
{
...
try
{
this.Execute(DriverCommand.Get, parameters);
}
...
}
}
Adding to Yi Zeng's answer, the difference between reloading and refreshing has to do with the driver information about the page. If we re-assign driver.Url = url again, the memory will host this overridden string value one more time —regardless of being the same. While refreshing has to do with displaying updated data.
Related
In my application, a toast message(pop up) appears when i try to enter invalid cell phone number and i want to assert the text of that toast message.
Below is the image of my HTML.I am unable to copy paste here.
Below is the code tried but no such element error occurs.
String actual_error = driver.FindElement(By.XPath("//div[#class='toast-message']")).Text;
String expected_error = "CellphoneNumber:Please enter a valid cellphone number. :";
Assert.AreEqual(actual_error, expected_error);
Console.WriteLine("Phone Number error message validated successfully");
Kindly suggest the right way to assert the toast message.
I think you can do like this:
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Support.UI;
using Serilog;
using System;
using System.IO;
using System.Reflection;
namespace StackOverFlow.Answer.Selenium.AssertMessageText
{
class AssertMessagePhone
{
public static IWebDriver driver;
public static WebDriverWait wait;
public static int timeOut = 30;
[Test]
[Category("ClickPartilLink")]
public void AssertMessagePhoneTest()
{
string messageError = "CellphoneNumber:Please enter a valid cellphone number. :";
Log.Information("Get instance Chrome Browser");
driver = new ChromeDriver(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location), OptionsChrome());
Console.WriteLine("Acess your url site");
driver.Navigate().GoToUrl("http://YourUrlSite/index.aspx");
// Steps until to fill and submit phone
Log.Information("Wait notificantion message error");
WaiElement(By.CssSelector(".toast-message"));
Log.Information("Check message");
Assert.True(GetText(By.CssSelector(".toast-message")).ToUpper().Contains(messageError.ToUpper()));
Log.Information("Click on message to disaper");
Click(By.CssSelector("div[class='noty_bar noty_type_success'] > div > span"));
}
/// <summary>
/// Wait elements
/// </summary>
/// <param name="locator"></param>
private void WaiElement(By locator)
{
wait = new WebDriverWait(driver, TimeSpan.FromSeconds(timeOut));
wait.Until(condition: ExpectedConditions.PresenceOfAllElementsLocatedBy(locator));
}
/// <summary>
/// click on elements
/// </summary>
/// <param name="locator"></param>
private void Click(By locator)
{
WaiElement(locator);
driver.FindElement(locator).Click();
}
/// <summary>
/// return text from element
/// </summary>
/// <param name="locator"></param>
/// <returns></returns>
private string GetText(By locator)
{
WaiElement(locator);
return driver.FindElement(locator).Text;
}
}
}
Your verification seems fine. You need to make sure that:
- your tests are in proper order if page is opened in another one
- make sure you wait for all elements, i.e use method:
ExpectedConditions.PresenceOfAllElementsLocatedBy(/*element to test*/)
I agree with lukbl that I would create a bool to verify the element it present but I believe the issue is that your text is in a div tag and not in a span.
Try this:
String expected_error = driver.FindElement(By.XPath("//div[#class='toast-message']/text()[1]")).Text;
This is most likely a error of logic in my code.
In first place, what i'm trying to do is:
go to the respective page of my website which is in this case link and collect the data, with my public void GDataPicker.
now where i want you to help me is, i use the following code to see if the button next exists in the webpage, and collect it's respective data, but always give me the same error:
OpenQA.Selenium.StaleElementReferenceException: 'stale element reference: element is not attached to the page document
(Session info: chrome=58.0.3029.110)
(Driver info: chromedriver=2.30.477700 (0057494ad8732195794a7b32078424f92a5fce41),platform=Windows NT 10.0.15063 x86_64)' , i think it's probably because i don´t update my NextButtonElement.
Code:
Boolean ElementDisplayed;
try
{
Gdriver.Navigate().GoToUrl("http://www.codigo-postal.pt/");
IWebElement searchInput1 = Gdriver.FindElement(By.Id("cp4"));
searchInput1.SendKeys("4710");//4730
IWebElement searchInput2 = Gdriver.FindElement(By.ClassName("cp3"));
searchInput2.SendKeys("");//324
searchInput2.SendKeys(OpenQA.Selenium.Keys.Enter);
IWebElement NextButtonElement = Gdriver.FindElement(By.XPath("/html/body/div[4]/div/div/div[2]/ul/li[13]/a"));
GDataPicker();
while (ElementDisplayed = NextButtonElement.Displayed)
{
GDataPicker();
Gdriver.Manage().Timeouts().ImplicitlyWait(TimeSpan.FromSeconds(2000));
NextButtonElement.SendKeys(OpenQA.Selenium.Keys.Enter);
}
}
catch (NoSuchElementException i)
{
ElementDisplayed = false;
GDataPicker();
}
I cant help you with C#, however StaleElementReferenceException occurs when the element you act upon is still in the dom but has been replaced with an identical one. what i would do is catch that exception and find the element again
catch (StaleElementReferenceException i)
{
IWebElement NextButtonElement = Gdriver.FindElement(By.XPath("/html/body/div[4]/div/div/div[2]/ul/li[13]/a"));
}
http://www.seleniumhq.org/exceptions/stale_element_reference.jsp
I would use ExpectedConditions.ElementToBeClickable with the dynamic wait feature selenium has.
var wait = new WebDriverWait(GDriver, TimeSpan.FromSeconds(5));
IWebElement NextButtonElement = wait.Until(ExpectedConditions.ElementToBeClickable(By.XPath("/html/body/div[4]/div/div/div[2]/ul/li[13]/a")));
ExpectedConditions.ElementToBeClickable does exactly what you want it to do, wait a little bit until the element is displayed and not stale.
/// <summary>
/// An expectation for checking an element is visible and enabled such that you
/// can click it.
/// </summary>
/// <param name="locator">The locator used to find the element.</param>
/// <returns>The <see cref="IWebElement"/> once it is located and clickable (visible and enabled).</returns>
public static Func<IWebDriver, IWebElement> ElementToBeClickable(By locator)
{
return (driver) =>
{
var element = ElementIfVisible(driver.FindElement(locator));
try
{
if (element != null && element.Enabled)
{
return element;
}
else
{
return null;
}
}
catch (StaleElementReferenceException)
{
return null;
}
};
}
From https://github.com/SeleniumHQ/selenium/blob/master/dotnet/src/support/UI/ExpectedConditions.cs
I'm working on a video recording app that supports the VideoStabilization effect, but when I start recording, I receive the following through the MediaCapture.Failed event almost instantly:
The sample allocator is currently empty, due to outstanding requests.
(0xC00D4A3E)
It only happens when I use the recommended configuration from the effect, though. If I don't call SetUpVideoStabilizationRecommendationAsync, it works fine.
Here is how I'm setting it up:
private MediaEncodingProfile _encodingProfile = MediaEncodingProfile.CreateMp4(VideoEncodingQuality.Auto);
private async Task CreateVideoStabilizationEffectAsync()
{
var definition = new VideoStabilizationEffectDefinition();
_videoStabilizationEffect = (VideoStabilizationEffect)await _mediaCapture.AddVideoEffectAsync(definition, MediaStreamType.VideoRecord);
_videoStabilizationEffect.Enabled = true;
await SetUpVideoStabilizationRecommendationAsync();
}
private async Task SetUpVideoStabilizationRecommendationAsync()
{
var properties = _mediaCapture.VideoDeviceController.GetMediaStreamProperties(MediaStreamType.VideoRecord) as VideoEncodingProperties;
var recommendation = _videoStabilizationEffect.GetRecommendedStreamConfiguration(_mediaCapture.VideoDeviceController, properties);
if (recommendation.InputProperties != null)
{
await _mediaCapture.VideoDeviceController.SetMediaStreamPropertiesAsync(MediaStreamType.VideoRecord, recommendation.InputProperties);
}
if (recommendation.OutputProperties != null)
{
_encodingProfile.Video = recommendation.OutputProperties;
}
}
private async Task StartRecordingAsync()
{
var videoFile = await KnownFolders.PicturesLibrary.CreateFileAsync("StableVideo.mp4", CreationCollisionOption.GenerateUniqueName);
await _mediaCapture.StartRecordToStorageFileAsync(_encodingProfile, videoFile);
}
The desiredProperties parameter of the GetRecommendedStreamConfiguration method needs to get MediaEncodingProfile that will be used when calling your choice of MediaCapture.StartRecordTo* (i.e. the "output properties") to see what your desired VideoEncodingProperties are.
The error is being triggered because the VideoEncodingProperties from the VideoDeviceController (i.e. the "input properties") are being passed instead. If you think about it, an instance of the the VideoDeviceController is already being passed in as a parameter to the method, so the effect can already access the information in that properties var; it wouldn't make much sense to have to pass those in separately at the same time. Instead, what it needs is information about the other endpoint. Does that make sense? At least that's how I try to rationalize it.
The official SDK sample for VideoStabilization on the Microsoft github repo shows how to do this correctly:
/// <summary>
/// Configures the pipeline to use the optimal resolutions for VS based on the settings currently in use
/// </summary>
/// <returns></returns>
private async Task SetUpVideoStabilizationRecommendationAsync()
{
Debug.WriteLine("Setting up VS recommendation...");
// Get the recommendation from the effect based on our current input and output configuration
var recommendation = _videoStabilizationEffect.GetRecommendedStreamConfiguration(_mediaCapture.VideoDeviceController, _encodingProfile.Video);
// Handle the recommendation for the input into the effect, which can contain a larger resolution than currently configured, so cropping is minimized
if (recommendation.InputProperties != null)
{
// Back up the current input properties from before VS was activated
_inputPropertiesBackup = _mediaCapture.VideoDeviceController.GetMediaStreamProperties(MediaStreamType.VideoRecord) as VideoEncodingProperties;
// Set the recommendation from the effect (a resolution higher than the current one to allow for cropping) on the input
await _mediaCapture.VideoDeviceController.SetMediaStreamPropertiesAsync(MediaStreamType.VideoRecord, recommendation.InputProperties);
Debug.WriteLine("VS recommendation for the MediaStreamProperties (input) has been applied");
}
// Handle the recommendations for the output from the effect
if (recommendation.OutputProperties != null)
{
// Back up the current output properties from before VS was activated
_outputPropertiesBackup = _encodingProfile.Video;
// Apply the recommended encoding profile for the output, which will result in a video with the same dimensions as configured
// before VideoStabilization was added if an appropriate padded capture resolution was available. Otherwise, it will be slightly
// smaller (due to cropping). This prevents upscaling back to the original size, which can result in a loss of quality
_encodingProfile.Video = recommendation.OutputProperties;
Debug.WriteLine("VS recommendation for the MediaEncodingProfile (output) has been applied");
}
}
Is there a way to write to this event log:
Or at least, some other Windows default log, where I don't have to register an event source?
Yes, there is a way to write to the event log you are looking for. You don't need to create a new source, just simply use the existent one, which often has the same name as the EventLog's name and also, in some cases like the event log Application, can be accessible without administrative privileges*.
*Other cases, where you cannot access it directly, are the Security EventLog, for example, which is only accessed by the operating system.
I used this code to write directly to the event log Application:
using (EventLog eventLog = new EventLog("Application"))
{
eventLog.Source = "Application";
eventLog.WriteEntry("Log message example", EventLogEntryType.Information, 101, 1);
}
As you can see, the EventLog source is the same as the EventLog's name. The reason of this can be found in Event Sources # Windows Dev Center (I bolded the part which refers to source name):
Each log in the Eventlog key contains subkeys called event sources. The event source is the name of the software that logs the event. It is often the name of the application or the name of a subcomponent of the application if the application is large. You can add a maximum of 16,384 event sources to the registry.
You can using the EventLog class, as explained on How to: Write to the Application Event Log (Visual C#):
var appLog = new EventLog("Application");
appLog.Source = "MySource";
appLog.WriteEntry("Test log message");
However, you'll need to configure this source "MySource" using administrative privileges:
Use WriteEvent and WriteEntry to write events to an event log. You must specify an event source to write events; you must create and configure the event source before writing the first entry with the source.
As stated in MSDN (eg. https://msdn.microsoft.com/en-us/library/system.diagnostics.eventlog(v=vs.110).aspx ), checking an non existing source and creating a source requires admin privilege.
It is however possible to use the source "Application" without.
In my test under Windows 2012 Server r2, I however get the following log entry using "Application" source:
The description for Event ID xxxx from source Application cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.
If the event originated on another computer, the display information had to be saved with the event.
The following information was included with the event:
{my event entry message}
the message resource is present but the message is not found in the string/message table
I defined the following method to create the source:
private string CreateEventSource(string currentAppName)
{
string eventSource = currentAppName;
bool sourceExists;
try
{
// searching the source throws a security exception ONLY if not exists!
sourceExists = EventLog.SourceExists(eventSource);
if (!sourceExists)
{ // no exception until yet means the user as admin privilege
EventLog.CreateEventSource(eventSource, "Application");
}
}
catch (SecurityException)
{
eventSource = "Application";
}
return eventSource;
}
I am calling it with currentAppName = AppDomain.CurrentDomain.FriendlyName
It might be possible to use the EventLogPermission class instead of this try/catch but not sure we can avoid the catch.
It is also possible to create the source externally, e.g in elevated Powershell:
New-EventLog -LogName Application -Source MyApp
Then, using 'MyApp' in the method above will NOT generate exception and the EventLog can be created with that source.
This is the logger class that I use. The private Log() method has EventLog.WriteEntry() in it, which is how you actually write to the event log. I'm including all of this code here because it's handy. In addition to logging, this class will also make sure the message isn't too long to write to the event log (it will truncate the message). If the message was too long, you'd get an exception. The caller can also specify the source. If the caller doesn't, this class will get the source. Hope it helps.
By the way, you can get an ObjectDumper from the web. I didn't want to post all that here. I got mine from here: C:\Program Files (x86)\Microsoft Visual Studio 10.0\Samples\1033\CSharpSamples.zip\LinqSamples\ObjectDumper
using System;
using System.Diagnostics;
using System.Diagnostics.CodeAnalysis;
using System.Globalization;
using System.Linq;
using System.Reflection;
using Xanico.Core.Utilities;
namespace Xanico.Core
{
/// <summary>
/// Logging operations
/// </summary>
public static class Logger
{
// Note: The actual limit is higher than this, but different Microsoft operating systems actually have
// different limits. So just use 30,000 to be safe.
private const int MaxEventLogEntryLength = 30000;
/// <summary>
/// Gets or sets the source/caller. When logging, this logger class will attempt to get the
/// name of the executing/entry assembly and use that as the source when writing to a log.
/// In some cases, this class can't get the name of the executing assembly. This only seems
/// to happen though when the caller is in a separate domain created by its caller. So,
/// unless you're in that situation, there is no reason to set this. However, if there is
/// any reason that the source isn't being correctly logged, just set it here when your
/// process starts.
/// </summary>
public static string Source { get; set; }
/// <summary>
/// Logs the message, but only if debug logging is true.
/// </summary>
/// <param name="message">The message.</param>
/// <param name="debugLoggingEnabled">if set to <c>true</c> [debug logging enabled].</param>
/// <param name="source">The name of the app/process calling the logging method. If not provided,
/// an attempt will be made to get the name of the calling process.</param>
public static void LogDebug(string message, bool debugLoggingEnabled, string source = "")
{
if (debugLoggingEnabled == false) { return; }
Log(message, EventLogEntryType.Information, source);
}
/// <summary>
/// Logs the information.
/// </summary>
/// <param name="message">The message.</param>
/// <param name="source">The name of the app/process calling the logging method. If not provided,
/// an attempt will be made to get the name of the calling process.</param>
public static void LogInformation(string message, string source = "")
{
Log(message, EventLogEntryType.Information, source);
}
/// <summary>
/// Logs the warning.
/// </summary>
/// <param name="message">The message.</param>
/// <param name="source">The name of the app/process calling the logging method. If not provided,
/// an attempt will be made to get the name of the calling process.</param>
public static void LogWarning(string message, string source = "")
{
Log(message, EventLogEntryType.Warning, source);
}
/// <summary>
/// Logs the exception.
/// </summary>
/// <param name="ex">The ex.</param>
/// <param name="source">The name of the app/process calling the logging method. If not provided,
/// an attempt will be made to get the name of the calling process.</param>
public static void LogException(Exception ex, string source = "")
{
if (ex == null) { throw new ArgumentNullException("ex"); }
if (Environment.UserInteractive)
{
Console.WriteLine(ex.ToString());
}
Log(ex.ToString(), EventLogEntryType.Error, source);
}
/// <summary>
/// Recursively gets the properties and values of an object and dumps that to the log.
/// </summary>
/// <param name="theObject">The object to log</param>
[SuppressMessage("Microsoft.Globalization", "CA1303:Do not pass literals as localized parameters", MessageId = "Xanico.Core.Logger.Log(System.String,System.Diagnostics.EventLogEntryType,System.String)")]
[SuppressMessage("Microsoft.Naming", "CA1720:IdentifiersShouldNotContainTypeNames", MessageId = "object")]
public static void LogObjectDump(object theObject, string objectName, string source = "")
{
const int objectDepth = 5;
string objectDump = ObjectDumper.GetObjectDump(theObject, objectDepth);
string prefix = string.Format(CultureInfo.CurrentCulture,
"{0} object dump:{1}",
objectName,
Environment.NewLine);
Log(prefix + objectDump, EventLogEntryType.Warning, source);
}
private static void Log(string message, EventLogEntryType entryType, string source)
{
// Note: I got an error that the security log was inaccessible. To get around it, I ran the app as administrator
// just once, then I could run it from within VS.
if (string.IsNullOrWhiteSpace(source))
{
source = GetSource();
}
string possiblyTruncatedMessage = EnsureLogMessageLimit(message);
EventLog.WriteEntry(source, possiblyTruncatedMessage, entryType);
// If we're running a console app, also write the message to the console window.
if (Environment.UserInteractive)
{
Console.WriteLine(message);
}
}
private static string GetSource()
{
// If the caller has explicitly set a source value, just use it.
if (!string.IsNullOrWhiteSpace(Source)) { return Source; }
try
{
var assembly = Assembly.GetEntryAssembly();
// GetEntryAssembly() can return null when called in the context of a unit test project.
// That can also happen when called from an app hosted in IIS, or even a windows service.
if (assembly == null)
{
assembly = Assembly.GetExecutingAssembly();
}
if (assembly == null)
{
// From http://stackoverflow.com/a/14165787/279516:
assembly = new StackTrace().GetFrames().Last().GetMethod().Module.Assembly;
}
if (assembly == null) { return "Unknown"; }
return assembly.GetName().Name;
}
catch
{
return "Unknown";
}
}
// Ensures that the log message entry text length does not exceed the event log viewer maximum length of 32766 characters.
private static string EnsureLogMessageLimit(string logMessage)
{
if (logMessage.Length > MaxEventLogEntryLength)
{
string truncateWarningText = string.Format(CultureInfo.CurrentCulture, "... | Log Message Truncated [ Limit: {0} ]", MaxEventLogEntryLength);
// Set the message to the max minus enough room to add the truncate warning.
logMessage = logMessage.Substring(0, MaxEventLogEntryLength - truncateWarningText.Length);
logMessage = string.Format(CultureInfo.CurrentCulture, "{0}{1}", logMessage, truncateWarningText);
}
return logMessage;
}
}
}
try
System.Diagnostics.EventLog appLog = new System.Diagnostics.EventLog();
appLog.Source = "This Application's Name";
appLog.WriteEntry("An entry to the Application event log.");
I'm trying to use Rohit Agarwal's BrowserSession class together with HtmlAgilityPack to login to and subsequently navigate around Facebook.
I've previously managed doing the same by writing my own HttpWebRequest's. However, it then only works when I manually fetch the cookie from my browser and insert a fresh cookie-string to the request each time I'm doing a new "session". Now I'm trying to use BrowserSession to get smarter navigation.
Here's the current code:
BrowserSession b = new BrowserSession();
b.Get(#"http://www.facebook.com/login.php");
b.FormElements["email"] = "some#email.com";
b.FormElements["pass"] = "xxxxxxxx";
b.FormElements["lsd"] = "qDhIH";
b.FormElements["trynum"] = "1";
b.FormElements["persistent_inputcheckbox"] = "1";
var response = b.Post(#"https://login.facebook.com/login.php?login_attempt=1");
The above works fine. Trouble comes when I try to use this BrowserSession again to fetch another page. I'm doing it this way since BrowserSession saves the cookies from the last response and inserts them into the next request, thus I should not have to manually inser cookiedata fetched from my browser anymore.
However, when I try to do something like this:
var profilePage = b.Get(#"https://m.facebook.com/profile.php?id=1111111111");
the doc I get back is empty. I would appreciate any input on what I'm doing wrong.
I fixed the root cause of this if anyone cares. It turns out the cookies were being saved in the CookieContainer of the REQUEST object and not the response object. I also added the ability to download a file (provided that file is string based). Code definitely is NOT thread-safe, but the object wasn't thread-safe to begin with:
public class BrowserSession
{
private bool _isPost;
private bool _isDownload;
private HtmlDocument _htmlDoc;
private string _download;
/// <summary>
/// System.Net.CookieCollection. Provides a collection container for instances of Cookie class
/// </summary>
public CookieCollection Cookies { get; set; }
/// <summary>
/// Provide a key-value-pair collection of form elements
/// </summary>
public FormElementCollection FormElements { get; set; }
/// <summary>
/// Makes a HTTP GET request to the given URL
/// </summary>
public string Get(string url)
{
_isPost = false;
CreateWebRequestObject().Load(url);
return _htmlDoc.DocumentNode.InnerHtml;
}
/// <summary>
/// Makes a HTTP POST request to the given URL
/// </summary>
public string Post(string url)
{
_isPost = true;
CreateWebRequestObject().Load(url, "POST");
return _htmlDoc.DocumentNode.InnerHtml;
}
public string GetDownload(string url)
{
_isPost = false;
_isDownload = true;
CreateWebRequestObject().Load(url);
return _download;
}
/// <summary>
/// Creates the HtmlWeb object and initializes all event handlers.
/// </summary>
private HtmlWeb CreateWebRequestObject()
{
HtmlWeb web = new HtmlWeb();
web.UseCookies = true;
web.PreRequest = new HtmlWeb.PreRequestHandler(OnPreRequest);
web.PostResponse = new HtmlWeb.PostResponseHandler(OnAfterResponse);
web.PreHandleDocument = new HtmlWeb.PreHandleDocumentHandler(OnPreHandleDocument);
return web;
}
/// <summary>
/// Event handler for HtmlWeb.PreRequestHandler. Occurs before an HTTP request is executed.
/// </summary>
protected bool OnPreRequest(HttpWebRequest request)
{
AddCookiesTo(request); // Add cookies that were saved from previous requests
if (_isPost) AddPostDataTo(request); // We only need to add post data on a POST request
return true;
}
/// <summary>
/// Event handler for HtmlWeb.PostResponseHandler. Occurs after a HTTP response is received
/// </summary>
protected void OnAfterResponse(HttpWebRequest request, HttpWebResponse response)
{
SaveCookiesFrom(request, response); // Save cookies for subsequent requests
if (response != null && _isDownload)
{
Stream remoteStream = response.GetResponseStream();
var sr = new StreamReader(remoteStream);
_download = sr.ReadToEnd();
}
}
/// <summary>
/// Event handler for HtmlWeb.PreHandleDocumentHandler. Occurs before a HTML document is handled
/// </summary>
protected void OnPreHandleDocument(HtmlDocument document)
{
SaveHtmlDocument(document);
}
/// <summary>
/// Assembles the Post data and attaches to the request object
/// </summary>
private void AddPostDataTo(HttpWebRequest request)
{
string payload = FormElements.AssemblePostPayload();
byte[] buff = Encoding.UTF8.GetBytes(payload.ToCharArray());
request.ContentLength = buff.Length;
request.ContentType = "application/x-www-form-urlencoded";
System.IO.Stream reqStream = request.GetRequestStream();
reqStream.Write(buff, 0, buff.Length);
}
/// <summary>
/// Add cookies to the request object
/// </summary>
private void AddCookiesTo(HttpWebRequest request)
{
if (Cookies != null && Cookies.Count > 0)
{
request.CookieContainer.Add(Cookies);
}
}
/// <summary>
/// Saves cookies from the response object to the local CookieCollection object
/// </summary>
private void SaveCookiesFrom(HttpWebRequest request, HttpWebResponse response)
{
//save the cookies ;)
if (request.CookieContainer.Count > 0 || response.Cookies.Count > 0)
{
if (Cookies == null)
{
Cookies = new CookieCollection();
}
Cookies.Add(request.CookieContainer.GetCookies(request.RequestUri));
Cookies.Add(response.Cookies);
}
}
/// <summary>
/// Saves the form elements collection by parsing the HTML document
/// </summary>
private void SaveHtmlDocument(HtmlDocument document)
{
_htmlDoc = document;
FormElements = new FormElementCollection(_htmlDoc);
}
}
/// <summary>
/// Represents a combined list and collection of Form Elements.
/// </summary>
public class FormElementCollection : Dictionary<string, string>
{
/// <summary>
/// Constructor. Parses the HtmlDocument to get all form input elements.
/// </summary>
public FormElementCollection(HtmlDocument htmlDoc)
{
var inputs = htmlDoc.DocumentNode.Descendants("input");
foreach (var element in inputs)
{
string name = element.GetAttributeValue("name", "undefined");
string value = element.GetAttributeValue("value", "");
if (!this.ContainsKey(name))
{
if (!name.Equals("undefined"))
{
Add(name, value);
}
}
}
}
/// <summary>
/// Assembles all form elements and values to POST. Also html encodes the values.
/// </summary>
public string AssemblePostPayload()
{
StringBuilder sb = new StringBuilder();
foreach (var element in this)
{
string value = System.Web.HttpUtility.UrlEncode(element.Value);
sb.Append("&" + element.Key + "=" + value);
}
return sb.ToString().Substring(1);
}
}
Sorry, I don't know much about the HTML agility pack or BrowserSession class you've mentioned. But I did try the same scenario with HtmlUnit and it working just fine. I'm using a .NET wrapper (the source code of which can be found here and is explained a bit more here), and here's the code I've used (some details removed to protect the innocent):
var driver = new HtmlUnitDriver(true);
driver.Url = #"http://www.facebook.com/login.php";
var email = driver.FindElement(By.Name("email"));
email.SendKeys("some#email.com");
var pass = driver.FindElement(By.Name("pass"));
pass.SendKeys("xxxxxxxx");
var inputs = driver.FindElements(By.TagName("input"));
var loginButton = (from input in inputs
where input.GetAttribute("value").ToLower() == "login"
&& input.GetAttribute("type").ToLower() == "submit"
select input).First();
loginButton.Click();
driver.Url = #"https://m.facebook.com/profile.php?id=1111111111";
Assert.That(driver.Title, Is.StringContaining("Title of page goes here"));
Hope this helps.
You might want to use WatiN (Web Application Testing In .Net) Or Selenium to drive your browser. This will help make sure you don't have to fiddle with the cookies and do any custom work to make subsequent requests work since you're simulating actual user.
I had similar symptoms - login worked but authentication cookie was not present in the cookie container and so it was not sent on subsequent requests. I found out this was because the web request was handling the Location: header internally, redirecting behind the scenes to a new page, losing the cookies in the process. I fixed this by adding:
request.AllowAutoRedirect = false; // Location header messing up cookie handling!
...to the OnPreRequest() function. It now looks like this:
protected bool OnPreRequest(HttpWebRequest request)
{
request.AllowAutoRedirect = false; // Location header messing up cookie handling!
AddCookiesTo(request); // Add cookies that were saved from previous requests
if (_isPost) AddPostDataTo(request); // We only need to add post data on a POST request
return true;
}
I hope this can help someone experiencing the same issue.
Today I was facing the same problem. I also worked with Rohit Agarwal's BrowserSession class together with HtmlAgilityPack.
After trial and error programming the whole day, I figured out that the problem is caused, because of not setting the correct cookies in the subsequent requests.
I couln't change the initial BrowserSession code to work correctly but I added the following functions and slightly modified the SameCookieFrom-function. In the end it worked nicely for me.
The added/modified functions are the following:
class BrowserSession{
private bool _isPost;
private HtmlDocument _htmlDoc;
public CookieContainer cookiePot; //<- This is the new CookieContainer
...
public string Get2(string url)
{
HtmlWeb web = new HtmlWeb();
web.UseCookies = true;
web.PreRequest = new HtmlWeb.PreRequestHandler(OnPreRequest2);
web.PostResponse = new HtmlWeb.PostResponseHandler(OnAfterResponse2);
HtmlDocument doc = web.Load(url);
return doc.DocumentNode.InnerHtml;
}
public bool OnPreRequest2(HttpWebRequest request)
{
request.CookieContainer = cookiePot;
return true;
}
protected void OnAfterResponse2(HttpWebRequest request, HttpWebResponse response)
{
//do nothing
}
private void SaveCookiesFrom(HttpWebResponse response)
{
if ((response.Cookies.Count > 0))
{
if (Cookies == null)
{
Cookies = new CookieCollection();
}
Cookies.Add(response.Cookies);
cookiePot.Add(Cookies); //-> add the Cookies to the cookiePot
}
}
What it does: It basically saves the cookies from the initial "Post-Response" and adds the same CookieContainer to the request called later. I do not fully understand why it was not working in the initial version because it somehow does the same in the AddCookiesTo-function. (if (Cookies != null && Cookies.Count > 0) request.CookieContainer.Add(Cookies);)
Anyhow, with these added functions it should work fine now.
It can be used like this:
//initial "Login-procedure"
BrowserSession b = new BrowserSession();
b.Get("http://www.blablubb/login.php");
b.FormElements["username"] = "yourusername";
b.FormElements["password"] = "yourpass";
string response = b.Post("http://www.blablubb/login.php");
all subsequent calls should use:
response = b.Get2("http://www.blablubb/secondpageyouwannabrowseto");
response = b.Get2("http://www.blablubb/thirdpageyouwannabrowseto");
...
I hope it helps many people facing the same problem!
Have you checked out their new API?
http://developers.facebook.com/docs/authentication/
You can call a straightforward URL to get an oauth2.0 access token and attach that on the rest of your requests...
https://graph.facebook.com/oauth/authorize?
client_id=...&
redirect_uri=http://www.example.com/oauth_redirect
Change redirect_uri to whatever URL you want, and it will get called back with a parameter called "access_token" on it. Get that and make whatever automated SDK calls you want.