I have attempted using a somewhat complex XPath syntax in the EventLogQuery to no avail. I have looked through all Microsoft docs I found by myself and through other posts here. I have primarily looked through these two posts:
EventLogQuery: How to form query string?
EventLogQuery time format expected?
Here are the queries, first one grabs all logon events with the LogonType attribute within a data node. This is redundant as all logon events will have this node, so I was going to further narrow the query to only type 11 for testing:
// This works fine
*[System/EventID=4624 and EventData/Data[#Name='LogonType']]
// This is considered invalid
*[System/EventID=4624 and EventData/Data[#Name='LogonType' and text()='11']]
What is wrong with this query and how could I fix it? I would prefer not to have to retrieve every single logon event and then filter again (i.e. loop through returns and only act on those with an 11) as this seems inefficient. I will be querying against domain controllers with millions of events currently on them.
For those still searching, I have likely found what the issue is. Microsoft did not fully implement XPath 1.0 for EventLogQuery. They half implemented it and twisted it into what they saw "useful" for querying events. text() is not a useable function. In order to achieve what I was trying, you have to use extremely long, ugly queries such as the following:
*[System/EventID=4624 and (EventData/Data[#Name='LogonType']='2' or EventData/Data[#Name='LogonType']='3' or EventData/Data[#Name='LogonType']='11')]
How to efficiently implement cursor based pagination with EF? Traditionally Take and Skip solve the common way of do it, but for scenarios where data is added and removed frequently traditional pagination is not the best way to go for.
To put things in context suppose you need to list a huge list of products, you can store last product id and go with a where clause asking for ids greater than or less than the value stored. Things get complicated when you need to provide the ability to sort based on criteria like price, date added etc where you can have equals values for many items, then greater than or less than is not enough.
LINQ has SkipWhile and TakeWile but this work over objects not over SQL, but I can go for it if a decent solution come to my mind or with a smart answer / comment. I am trying to implement graphql pagination as per Relay.js
Thanks in advance
I am working in C#, and am trying to use DirectorySearch to query the groups of an extremely large Microsoft ActiveDirectory LDAP server.
So, in my application, I'm going to have a paged list of groups, with searching capability. Naturally, I don't want to hammer my LDAP server with passing me the entire result set for these queries every time I hit "Next Page".
Is there a way, using DirectorySearch, to retrieve ONLY a single arbitrary page's results, rather than returning the entire result-set in one method call?
Similar questions:
DirectorySearch.PageSize = 2 doesn't work
c# Active Directory Services findAll() returns only 1000 entries
Many questions like these exist, where someone asks about paging (meaning from LDAP server to app server), and gets responses involving PageSize and SizeLimit. However, those properties only affect paging between the C# server and the LDAP server, and in the end, the only relevant methods that DirectorySearch has are FindOne() and FindAll().
What I'm looking for is basically "FindPaged(pageSize, pageNumber)" (the pageNumber being the really important bit. I don't just want the first 1000 results, I want (for example) the 100'th set of 1000 results. The app can't wait for 100,000 records to be passed from the LDAP server to the app server, even if they are broken up into 1,000-record chunks.
I understand that DirectoryServices.Protocols has SearchRequest, which (I think?) allows you to use a "PageResultRequestControl", which looks like it has what I'm looking for (although it looks like the paging information comes in "cookies", which I'm not sure how I'd be supposed to retrieve). But if there's a way to do this without rewriting the entire thing to use Protocols instead, I'd rather not have to do so.
I just can't imagine there's no way to do this... Even SQL has Row_Number.
UPDATE:
The PageResultRequestControl does not help - It's forward-only and sequential (You must call and get the first N results before you can get the "cookie" token necessary to make a call to get result N+1).
However, the cookie does appear to have some sort of reproducible ordering... On a result set I was working on, I iterated one by one through the results, and each time the cookie came out thusly:
1: {8, 0, 0, 0}
2: {11, 0, 0, 0}
3: {12, 0, 0, 0}
4: {16, 0, 0, 0}
When I iterated through two by two, I got the same numbers (11, 16).
This makes me think that if I could figure out the code of how those numbers are generated, I could create a cookie ad-hoc, which would give me exactly the paging I'm looking for.
The PageResultRequestControl is indeed the way to do this, it's part of the LDAP protocol. You'll just have to figure out what that implies for your code, sorry. There should be a way to use it from where you are, but, having said that, I'm working in Java and I've just had to write a dozen or so request controls and extended-operation classes for use with JNDI so you might be out of luck ... or you might have to do like I did. Warning, ASN.1 parsing follows not that far behind :-|
Sadly, it appears there may not be a way to do this given current C# libraries.
All of the standard C#4.0 LDAP libraries return Top-N results (As in, FindAll(), which returns every result, FindOne(), which returns the first result, or SearchResult with PageResultRequestControl, which returns results N through N+M but requires you to retrieve results 1 through N-1 before you'll have a cookie token that you can pass with the request in order to get the next set.
I haven't been able to find any third-party LDAP libraries that allow this, either.
Unless a better solution is found, my path forward will be to modify the interface to instead display the top X results, with no client paging capabilities (obviously still using server-side paging as appropriate).
I may pursue a forward-only paging system at a later date, by passing the updated cookie to the client with the response, and passing it back with a click of a "More Results" type of button.
It might be worth pursuing at a later date, whether or not these cookies can be hand-crafted.
UPDATE:
I spoke with Microsoft Support and confirmed this - There is no way to do dynamic paging with LDAP servers. This is a limitation of LDAP servers themselves.
You can use Protocols and the Paging control (if your LDAP server supports it) to step forward at will, but there is no cross-server (or even cross-version) standard for the cookie, so you can't reasonably craft your own, and there's no guarantee that the cookie can be reused for repeated queries.
A full solution involves using Protocols (with Paging as above) to pull your pageable result set into SQL, whether into a temp table or a permanent storage table, and allow your user to page and sort through THAT result set in the traditional manner. Bear in mind your results won't be precisely up to date, but with some smart cache updating you can minimize that risk.
Maybe you want to iterate through your "pages" using the range-attribute accordingly:
----copy & paste----
This sample retrieves entries 0-500, inclusively.
DirectoryEntry group = new DirectoryEntry("LDAP://CN=Sales,DC=Fabrikam,DC=COM");
DirectorySearcher groupMember = new DirectorySearcher
(group,"(objectClass=*)",new string[]{"member;Range=0-500"},SearchScope.Base);
SearchResult result = groupMember.FindOne();
// Each entry contains a property name and the path (ADsPath).
// The following code returns the property name from the PropertyCollection.
String propName=String.Empty;
foreach(string s in result.Properties.PropertyNames)
{
if ( s.ToLower() != "adspath")
{
propName = s;
break;
}
}
foreach(string member in result.Properties[propName])
{
Console.WriteLine(member);
}
----copy & paste----
for more Information see:
Enumerating Members in a Large Group
https://msdn.microsoft.com/en-us/library/ms180907.aspx
Range Retrieval of Attribute Values
https://msdn.microsoft.com/en-us/library/cc223242.aspx
Searching Using Range Retrieval
https://msdn.microsoft.com/en-us/library/aa367017.aspx
Please check http://www.alibaba.com/catalogs/cid/702/Laptops.html they have nice filter here with number of items for each. Note one detail - they have locations here. Same thing on olx.com - location and number of items for each category.
Now imagine I have tables:
[products] (Id, Name, CategoryId, LocationId)
[Categories] (Id,Name)
[Location] (Id, Name)
My question how can I do the same, cause count things even with caching looks expensive? And they give results pretty fast...
Please advice with possible ways to do that in ASP.NET, C#, MVC, MS SQL, but avoice simple answers like "count and change"
Thank you in advance.
I would go with ASP.NET MVC for server side and Jquery for client side , for your requirement.
This combination allows you to have full control over the UI , and manipulate them using Javascript.
Some of the complex controls rendered by ASP.NET has its 'id' value dynamically generated which might be an issue.
As for your requirement http://www.alibaba.com/catalogs/cid/702/Laptops.html
There are good number of Jquery plugins that you can take advantage of to render the visual effects.
Please check out the link below (Table Sorter plugin) which i found useful for adding display effects to tables in particular.
Table Sorter
Hope this helps.
Thanks ,
Vijay
They are showing the possible filters at level one. When you choose a filter, they are running the search with that filter and fetching narrowed filters. The key is running the filters each time. This is quite achievable. I have done this already in asp.net MVC. In your case, it would be even easier as you do not have many filters.
Decide the level-1 filters.
For each level-1 filters, decide which are the level-2 filters.
Decide what is the meanigful depth levels for each filter type.
As the user keeps adding filters, run the filters that user has selected or unselected so for.
you get the items after applying the filters. Show them according to the level of filters.
Is there a limit to the rows that IEnumerable.Count() (or IQueryable.Count()) using LINQ to SQL? For whatever reason, if my query returns more than 200 records, I only get 200 from IEnumerable.Count(). I've even tried using IEnumerable.LongCount() (even though the number of results shouldn't be high enough to need it). I've also verified that calling COUNT on the database returns more than 200 records.
I've checked MSDN and tried Googling it, but to no avail.
Am I going crazy, or is there something that I'm missing somewhere? I suppose this isn't really a big deal (as the program is inserting the right number of records), but it'd be nice to be able to log the number of records transferred.
Could we see some sample code?
public IEnumerable<Irms_tx_modify_profile_ban> ExtractNewAdmits()
{
var results = from a in _dc.Applications
select (Irms_tx_modify_profile_ban)new RMSProfile
{
//assign column names to property names
};
//Don't know why, but Bad Things happen if you don't put the
//OfType call here.
IEnumerable<Irms_tx_modify_profile_ban> narrowed_results = results.OfType<Irms_tx_modify_profile_ban>();
Program.log.InfoFormat("Read {0} records from Banner.", narrowed_results.Count());
return narrowed_results;
}
The reason for the comment about bad things happening is due to the issues brought up in this thread. What I did just find out is that if I call Count on narrowed_results (IEnumerable), it returns the right amount. If I call it on results (IQueryable), it returns just 200. I'll go ahead and solve Skeet's answer (since he mentioned the difference between IQueryable and IEnumerable), but if anyone is able to explain why this is happening, I'd like to hear it.
I've not heard of anything like that, and it does sound very odd.
The most obvious thing to check is what query is being sent to the database. Also, it matters a great deal whether you're calling Enumerable.Count() (i.e. on an IEnumerable<T>) or Queryable.Count() (i.e. on an IQueryable<T>). The former will be iterating through the actual rows in .NET code to retrieve the count; the latter will put the count into the query.
Could we see some sample code?
EDIT: Okay, so having seen the code:
When you didn't call OfType, it was executing the count at the SQL level. That should have been visible in the SQL logged, and should be reproducible with any other SQL tool.
I suspect you didn't really have to call OfType. You could have called AsEnumerable, or just declared results as IEnumerable<Irms_tx_modify_profile_ban>. The important thing is that the type of the variable decides the extension method to use - and thus where the count is executed.
It's worth noting that your current solution is really inefficient - it's fetching all the data, and counting it but ignoring everything but the count. It would be much better to get the count onto the server side - and while I know that doesn't work at the moment, I'm sure with a bit of analysis we can make it work :)