We Keep Coding

Memory exceptions when scraping Active Directory

I am trying to scrape all the groups a user is a member of in Active Directory (including groups that they are indirect members of), but I am running into memory exception errors.
Latest error message passed was:
Insufficient memory to continue the execution of the program
The code I run is started in a backgroundworker on a schedule.
I have looked through ALL the SO active directory group scraping posts, and tried as many of the iterations of this process as I could get to work. This one will (usually) work if i don't run it on a schedule. Sometimes it will crash the program though.
Our Active Directory database is quite considerably large.
Any help you could offer with this would be great. I don't mind it taking a bit longer, speed is not an overly great concern, as long as I can get it to perform well, and correctly.
Here's what I have:
DataTable resultsTable = new DataTable();
resultsTable.Columns.Add("EmailID");
resultsTable.Columns.Add("UserID");
resultsTable.Columns.Add("memberOf");
resultsTable.Columns.Add("groupType");
resultsTable.Columns.Add("record_date");
try
{
string RecordDate = DateTime.Now.ToString(format: "dd/MM/yyyy", provider: CultureInfo.CreateSpecificCulture("en-GB"));
string ou = "OU=Groups,OU=nonya,DC=my,DC=domain,DC=net";
using (PrincipalContext context = new PrincipalContext(ContextType.Domain, "my.domain.net", ou))
{
using (GroupPrincipal GroupPrin = new GroupPrincipal(context))
{
using (var searcher = new PrincipalSearcher(GroupPrin))
{
using (var results = searcher.FindAll())
{
foreach (var result in results)
{
DirectoryEntry de = result.GetUnderlyingObject() as DirectoryEntry;
string GUID = de.Guid.ToString();
string testname1 = de.Name;
string testParentName1 = de.Parent.Name;
string MasterGroupname = testname1.Substring(3, (testname1.Length - 3));
string type = testParentName1.Substring(3, (testParentName1.Length - 3));
using (var group = GroupPrincipal.FindByIdentity(context, IdentityType.Guid, GUID))
{
using (var users = group.GetMembers(true))
{
foreach (Principal user in users)
{
DataRow dr1 = resultsTable.NewRow();
dr1["EmailID"] = user.UserPrincipalName;
dr1["UserID"] = user.SamAccountName;
dr1["memberOf"] = MasterGroupname;
dr1["groupType"] = type;
dr1["record_date"] = RecordDate;
resultsTable.Rows.Add(dr1);
user.Dispose();
}
}
}
}
}
}
}
}
//Table is uploaded to SQL database here, unless it crashes before reaching this point
}
catch (Exception E)
{
logger.Error($"ADGroupScrape error. System says: {E.Message}");
return false;
}
finally
{
resultsTable.Dispose();
}

Disposing each object at the end of the loop will probably solve your problem (result.Dispose()). I've had memory problems with long loops when working with AD. Assuming you're working with thousands of results, and you're allocating memory for each, and there's no break between processing each result, the garbage collector doesn't have an opportunity to clean up for you.
But you're also unnecessarily going back out to AD to search for the group you already found (GroupPrincipal.FindByIdentity). That would also increase memory consumption, but also slow down the whole operation. Instead, just cast result to a GroupPrincipal so you can call .GetMembers() on it.
Unrelated, but helpful: When you have multiple using blocks nested like that, you can combine them all into one block. It saves you indenting so much.
This is what your code would look like with all those suggestions:
using (PrincipalContext context = new PrincipalContext(ContextType.Domain, "my.domain.net", ou))
using (GroupPrincipal GroupPrin = new GroupPrincipal(context))
using (var searcher = new PrincipalSearcher(GroupPrin))
using (var results = searcher.FindAll())
{
foreach (GroupPrincipal result in results)
{
DirectoryEntry de = result.GetUnderlyingObject() as DirectoryEntry;
string GUID = de.Guid.ToString();
string testname1 = de.Name;
string testParentName1 = de.Parent.Name;
string MasterGroupname = testname1.Substring(3, (testname1.Length - 3));
string type = testParentName1.Substring(3, (testParentName1.Length - 3));
using (var users = result.GetMembers(true))
{
foreach (Principal user in users)
{
DataRow dr1 = resultsTable.NewRow();
dr1["EmailID"] = user.UserPrincipalName;
dr1["UserID"] = user.SamAccountName;
dr1["memberOf"] = MasterGroupname;
dr1["groupType"] = type;
dr1["record_date"] = RecordDate;
user.Dispose();
}
}
result.Dispose();
}
}
This will probably work, but could probably still be much faster. Using Principal objects (and the whole AccountManagement namespace) is a wrapper around DirectoryEntry/DirectorySearcher which makes things somewhat easier for you, but at the cost of performance. Using DirectoryEntry/DirectorySearcher directly is always faster.
If you want to experiment with that, I wrote a couple articles that will help:
Active Directory: Better performance
Active Directory: Find all the members of a group

So the issue didn't appear to be with my code, but thank you all for the suggestions in cleaning it up.
I changed the Target Platform in Configuration Manager (Visual Studio) to X64, as it was originally set to "Any CPU", and my code now runs successfully every time.

Couple of months down the line and i started getting out of memory exceptions again.
added pagination (which i didnt actually know you could do) and it seems to have resolved the issue. The line i added was ((DirectorySearcher)searcher.GetUnderlyingSearcher()).PageSize = 500; in between the searcher declaration and the searcher.FindAll().
using (var searcher = new PrincipalSearcher(GroupPrin))
{
((DirectorySearcher)searcher.GetUnderlyingSearcher()).PageSize = 500;
using (var results = searcher.FindAll())
{

c# Directory Services Synchronization does not return changed relationships

I'm using C# to work with AD (Win 2012R2).
We are syncing AD users,groups and their relationship to SQL database.
Full sync works well.
But when using synchronization cookie, the relationship changes does not detected.
What may be the reason?
Thanks.
Here is my code:
public void DirSyncChanges(DirectoryEntry de, byte[] cookie)
{
DirectorySynchronization syncData = new DirectorySynchronization(cookie);
srch = new DirectorySearcher(de)
{
Filter = "(&(objectClass=user)(objectCategory=person))",
SizeLimit = Int32.MaxValue,
Tombstone = true
};
srch.DirectorySynchronization = syncData;
syncData.Option = DirectorySynchronizationOptions.None;
using(SearchResultCollection results = srch.FindAll())
foreach (SearchResult res in results)
{
//results is empty. no loop
}
}

Please specify the DirectorySearcher.PropertiesToLoad. Only if any of the attributes in PropertiesToLoad are updated, you will get them in delta sync.
As i remember the search root of DirSync must be naming context root object.
Better use paged search. No matter how large the value you set to SizeLimit. It will only return at most 1000 or 1500 (forgot exact number) results.
My answer is based on .NET 3.5.

Updating Name Field on UserPrincipal

When I try to update the Name field (corresponds to the CN) on UserPrincipal (Principal, really), I get an error "The server is unwilling to process the request" on the call to UserPrincipal.Save().
I've checked to make sure there isn't another object in the same OU with the same Name (CN).
The PrincipalContext I'm operating at is the domain root (not exactly at the OU level where the user account exists).
What reason might there be for this error? Is it something that might be security policy related (even though I'm able to update all the other fields)?
using (var context = new PrincipalContext(ContextType.Domain, ConfigurationManager.AppSettings["domain"], ConfigurationManager.AppSettings["rootDN"], ContextOptions.Negotiate, ConfigurationManager.AppSettings["username"], ConfigurationManager.AppSettings["password"])) {
var user = UserPrincipal.FindByIdentity(context, IdentityType.Sid, "..."); // SID abbreviated
user.Name = "Name, Test";
user.Save();
}
The user I am using to create the PrincipalContext has the security rights to modify AD objects. If I update any other of the other fields (e.g. Surname, GivenName), everything works fine.
EDIT:
I've been able to accomplish what I need to do (using ADSI), but I have to run the following code under impersonation. The impersonation code is ugly, and the code below breaks away from the other way I'm updating AD data (using DirectoryServices.AccountManagement), so I'd like to get a better solution.
using (var companyOU = new DirectoryEntry("LDAP://" + company.UserAccountOU)) {
companyOU.Invoke("MoveHere", "LDAP://" + user.DistinguishedName, "cn=Name\, Test");
}

This is a cleaner way
using (var context = new PrincipalContext(ContextType.Domain))
{
var group = GroupPrincipal.FindByIdentity(context, groupName);
group.SamAccountName = newGroupName;
group.DisplayName = newGroupName;
group.Save();
var dirEntry = (DirectoryEntry)group.GetUnderlyingObject();
dirEntry.Rename("CN=" + newGroupName);
dirEntry.CommitChanges();
}

The only way I've found to do this is in the EDIT section in my question. Basically, you cannot use the UserPrincipal class. There is something special about the CN attribute, and you need to drop down a level and use DirectoryEntry, an LDAP string, and invoke the "MoveHere" ADSI command to rename the user account.

How to get the groups of a user in Active Directory? (c#, asp.net)

I use this code to get the groups of the current user. But I want to manually give the user and then get his groups. How can I do this?
using System.Security.Principal;
public ArrayList Groups()
{
ArrayList groups = new ArrayList();
foreach (IdentityReference group in System.Web.HttpContext.Current.Request.LogonUserIdentity.Groups)
{
groups.Add(group.Translate(typeof(NTAccount)).ToString());
}
return groups;
}

If you're on .NET 3.5 or up, you can use the new System.DirectoryServices.AccountManagement (S.DS.AM) namespace which makes this a lot easier than it used to be.
Read all about it here: Managing Directory Security Principals in the .NET Framework 3.5
Update: older MSDN magazine articles aren't online anymore, unfortunately - you'll need to download the CHM for the January 2008 MSDN magazine from Microsoft and read the article in there.
Basically, you need to have a "principal context" (typically your domain), a user principal, and then you get its groups very easily:
public List<GroupPrincipal> GetGroups(string userName)
{
List<GroupPrincipal> result = new List<GroupPrincipal>();
// establish domain context
PrincipalContext yourDomain = new PrincipalContext(ContextType.Domain);
// find your user
UserPrincipal user = UserPrincipal.FindByIdentity(yourDomain, userName);
// if found - grab its groups
if(user != null)
{
PrincipalSearchResult<Principal> groups = user.GetAuthorizationGroups();
// iterate over all groups
foreach(Principal p in groups)
{
// make sure to add only group principals
if(p is GroupPrincipal)
{
result.Add((GroupPrincipal)p);
}
}
}
return result;
}
and that's all there is! You now have a result (a list) of authorization groups that user belongs to - iterate over them, print out their names or whatever you need to do.
Update: In order to access certain properties, which are not surfaced on the UserPrincipal object, you need to dig into the underlying DirectoryEntry:
public string GetDepartment(Principal principal)
{
string result = string.Empty;
DirectoryEntry de = (principal.GetUnderlyingObject() as DirectoryEntry);
if (de != null)
{
if (de.Properties.Contains("department"))
{
result = de.Properties["department"][0].ToString();
}
}
return result;
}
Update #2: seems shouldn't be too hard to put these two snippets of code together.... but ok - here it goes:
public string GetDepartment(string username)
{
string result = string.Empty;
// if you do repeated domain access, you might want to do this *once* outside this method,
// and pass it in as a second parameter!
PrincipalContext yourDomain = new PrincipalContext(ContextType.Domain);
// find the user
UserPrincipal user = UserPrincipal.FindByIdentity(yourDomain, username);
// if user is found
if(user != null)
{
// get DirectoryEntry underlying it
DirectoryEntry de = (user.GetUnderlyingObject() as DirectoryEntry);
if (de != null)
{
if (de.Properties.Contains("department"))
{
result = de.Properties["department"][0].ToString();
}
}
}
return result;
}

GetAuthorizationGroups() does not find nested groups. To really get all groups a given user is a member of (including nested groups), try this:
using System.Security.Principal
private List<string> GetGroups(string userName)
{
List<string> result = new List<string>();
WindowsIdentity wi = new WindowsIdentity(userName);
foreach (IdentityReference group in wi.Groups)
{
try
{
result.Add(group.Translate(typeof(NTAccount)).ToString());
}
catch (Exception ex) { }
}
result.Sort();
return result;
}
I use try/catch because I had some exceptions with 2 out of 200 groups in a very large AD because some SIDs were no longer available. (The Translate() call does a SID -> Name conversion.)

First of all, GetAuthorizationGroups() is a great function but unfortunately has 2 disadvantages:
Performance is poor, especially in big company's with many users and groups. It fetches a lot more data then you actually need and does a server call for each loop iteration in the result
It contains bugs which can cause your application to stop working 'some day' when groups and users are evolving. Microsoft recognized the issue and is related with some SID's. The error you'll get is "An error occurred while enumerating the groups"
Therefore, I've wrote a small function to replace GetAuthorizationGroups() with better performance and error-safe. It does only 1 LDAP call with a query using indexed fields. It can be easily extended if you need more properties than only the group names ("cn" property).
// Usage: GetAdGroupsForUser2("domain\user") or GetAdGroupsForUser2("user","domain")
public static List<string> GetAdGroupsForUser2(string userName, string domainName = null)
{
var result = new List<string>();
if (userName.Contains('\\') || userName.Contains('/'))
{
domainName = userName.Split(new char[] { '\\', '/' })[0];
userName = userName.Split(new char[] { '\\', '/' })[1];
}
using (PrincipalContext domainContext = new PrincipalContext(ContextType.Domain, domainName))
using (UserPrincipal user = UserPrincipal.FindByIdentity(domainContext, userName))
using (var searcher = new DirectorySearcher(new DirectoryEntry("LDAP://" + domainContext.Name)))
{
searcher.Filter = String.Format("(&(objectCategory=group)(member={0}))", user.DistinguishedName);
searcher.SearchScope = SearchScope.Subtree;
searcher.PropertiesToLoad.Add("cn");
foreach (SearchResult entry in searcher.FindAll())
if (entry.Properties.Contains("cn"))
result.Add(entry.Properties["cn"][0].ToString());
}
return result;
}

Within the AD every user has a property memberOf. This contains a list of all groups he belongs to.
Here is a little code example:
// (replace "part_of_user_name" with some partial user name existing in your AD)
var userNameContains = "part_of_user_name";
var identity = WindowsIdentity.GetCurrent().User;
var allDomains = Forest.GetCurrentForest().Domains.Cast<Domain>();
var allSearcher = allDomains.Select(domain =>
{
var searcher = new DirectorySearcher(new DirectoryEntry("LDAP://" + domain.Name));
// Apply some filter to focus on only some specfic objects
searcher.Filter = String.Format("(&(&(objectCategory=person)(objectClass=user)(name=*{0}*)))", userNameContains);
return searcher;
});
var directoryEntriesFound = allSearcher
.SelectMany(searcher => searcher.FindAll()
.Cast<SearchResult>()
.Select(result => result.GetDirectoryEntry()));
var memberOf = directoryEntriesFound.Select(entry =>
{
using (entry)
{
return new
{
Name = entry.Name,
GroupName = ((object[])entry.Properties["MemberOf"].Value).Select(obj => obj.ToString())
};
}
});
foreach (var item in memberOf)
{
Debug.Print("Name = " + item.Name);
Debug.Print("Member of:");
foreach (var groupName in item.GroupName)
{
Debug.Print(" " + groupName);
}
Debug.Print(String.Empty);
}
}

My solution:
UserPrincipal user = UserPrincipal.FindByIdentity(new PrincipalContext(ContextType.Domain, myDomain), IdentityType.SamAccountName, myUser);
List<string> UserADGroups = new List<string>();
foreach (GroupPrincipal group in user.GetGroups())
{
UserADGroups.Add(group.ToString());
}

In my case the only way I could keep using GetGroups() without any expcetion was adding the user (USER_WITH_PERMISSION) to the group which has permission to read the AD (Active Directory). It's extremely essential to construct the PrincipalContext passing this user and password.
var pc = new PrincipalContext(ContextType.Domain, domain, "USER_WITH_PERMISSION", "PASS");
var user = UserPrincipal.FindByIdentity(pc, IdentityType.SamAccountName, userName);
var groups = user.GetGroups();
Steps you may follow inside Active Directory to get it working:
Into Active Directory create a group (or take one) and under secutiry tab add "Windows Authorization Access Group"
Click on "Advanced" button
Select "Windows Authorization Access Group" and click on "View"
Check "Read tokenGroupsGlobalAndUniversal"
Locate the desired user and add to the group you created (taken) from the first step

The answer depends on what kind of groups you want to retrieve. The System.DirectoryServices.AccountManagement namespace provides two group retrieval methods:
GetGroups - Returns a collection of group objects that specify the groups of which the current principal is a member.
This overloaded method only returns the groups of which the principal is directly a member; no recursive searches are performed.
GetAuthorizationGroups - Returns a collection of principal objects that contains all the authorization groups of which this user is a member. This function only returns groups that are security groups; distribution groups are not returned.
This method searches all groups recursively and returns the groups in which the user is a member. The returned set may also include additional groups that system would consider the user a member of for authorization purposes.
So GetGroups gets all groups of which the user is a direct member, and GetAuthorizationGroups gets all authorization groups of which the user is a direct or indirect member.
Despite the way they are named, one is not a subset of the other. There may be groups returned by GetGroups not returned by GetAuthorizationGroups, and vice versa.
Here's a usage example:
PrincipalContext domainContext = new PrincipalContext(ContextType.Domain, "MyDomain", "OU=AllUsers,DC=MyDomain,DC=Local");
UserPrincipal inputUser = new UserPrincipal(domainContext);
inputUser.SamAccountName = "bsmith";
PrincipalSearcher adSearcher = new PrincipalSearcher(inputUser);
inputUser = (UserPrincipal)adSearcher.FindAll().ElementAt(0);
var userGroups = inputUser.GetGroups();

This works for me
public string[] GetGroupNames(string domainName, string userName)
{
List<string> result = new List<string>();
using (PrincipalContext principalContext = new PrincipalContext(ContextType.Domain, domainName))
{
using (PrincipalSearchResult<Principal> src = UserPrincipal.FindByIdentity(principalContext, userName).GetGroups())
{
src.ToList().ForEach(sr => result.Add(sr.SamAccountName));
}
}
return result.ToArray();
}

In case Translate works locally but not remotly e.i group.Translate(typeof(NTAccount)
If you want to have the application code executes using the LOGGED IN USER identity, then enable impersonation. Impersonation can be enabled thru IIS or by adding the following element in the web.config.
<system.web>
<identity impersonate="true"/>
If impersonation is enabled, the application executes using the permissions found in your user account. So if the logged in user has access, to a specific network resource, only then will he be able to access that resource thru the application.
Thank PRAGIM tech for this information from his diligent video
Windows authentication in asp.net Part 87:
https://www.youtube.com/watch?v=zftmaZ3ySMc
But impersonation creates a lot of overhead on the server
The best solution to allow users of certain network groups is to deny anonymous in the web config
<authorization><deny users="?"/><authentication mode="Windows"/>
and in your code behind, preferably in the global.asax, use the HttpContext.Current.User.IsInRole :
Sub Session_Start(ByVal sender As Object, ByVal e As EventArgs)
If HttpContext.Current.User.IsInRole("TheDomain\TheGroup") Then
//code to do when user is in group
End If
NOTE: The Group must be written with a backslash \ i.e. "TheDomain\TheGroup"

This is quick and dirty but someone may find it helpful. You will need to add the reference to System.DirectoryServices.AccountManagement for this to work. This is just for getting user roles but can be expanded to include other things if needed.
using System.DirectoryServices.AccountManagement;
PrincipalContext ctx = new PrincipalContext(ContextType.Domain, "DaomainName");
UserPrincipal u = UserPrincipal.FindByIdentity(ctx, "Username");
List<UserRole> UserRoles = u.GetGroups().Select(x => new UserRole { Role = x.Name }).ToList();
public partial class UserRole
{
public string Role { get; set; }
}

How can I retrieve Active Directory users by Common Name more quickly?

I am querying information from Active Directory. I have code that works, but it's really slow.
This is the code I currently use:
static void Main(string[] args)
{
SearchResultCollection sResults = null;
try
{
//modify this line to include your domain name
string path = "LDAP://EXTECH";
//init a directory entry
DirectoryEntry dEntry = new DirectoryEntry(path);
//init a directory searcher
DirectorySearcher dSearcher = new DirectorySearcher(dEntry);
//This line applies a filter to the search specifying a username to search for
//modify this line to specify a user name. if you want to search for all
//users who start with k - set SearchString to "k"
dSearcher.Filter = "(&(objectClass=user))";
//perform search on active directory
sResults = dSearcher.FindAll();
//loop through results of search
foreach (SearchResult searchResult in sResults)
{
if (searchResult.Properties["CN"][0].ToString() == "Adit")
{
////loop through the ad properties
//foreach (string propertyKey in
//searchResult.Properties["st"])
//{
//pull the collection of objects with this key name
ResultPropertyValueCollection valueCollection =
searchResult.Properties["manager"];
foreach (Object propertyValue in valueCollection)
{
//loop through the values that have a specific name
//an example of a property that would have multiple
//collections for the same name would be memberof
//Console.WriteLine("Property Name: " + valueCollection..ToString());
Console.WriteLine("Property Value: " + (string)propertyValue.ToString());
//["sAMAccountName"][0].ToString();
}
//}
Console.WriteLine(" ");
}
}
}
catch (InvalidOperationException iOe)
{
//
}
catch (NotSupportedException nSe)
{
//
}
finally
{
// dispose of objects used
if (sResults != null)
sResults.Dispose();
}
Console.ReadLine();
}
What would faster code look like to get user information from AD?

You can call UserPrincipal.FindByIdentity inside System.DirectoryServices.AccountManagement:
using System.DirectoryServices.AccountManagement;
using (var pc = new PrincipalContext(ContextType.Domain, "MyDomainName"))
{
var user = UserPrincipal.FindByIdentity(pc, IdentityType.SamAccountName, "MyDomainName\\" + userName);
}

The reason why your code is slow is that your LDAP query retrieves every single user object in your domain even though you're only interested in one user with a common name of "Adit":
dSearcher.Filter = "(&(objectClass=user))";
So to optimize, you need to narrow your LDAP query to just the user you are interested in. Try something like:
dSearcher.Filter = "(&(objectClass=user)(cn=Adit))";
In addition, don't forget to dispose these objects when done:
DirectoryEntry dEntry
DirectorySearcher dSearcher

Well, if you know where your user lives in the AD hierarchy (e.g. quite possibly in the "Users" container, if it's a small network), you could also bind to the user account directly, instead of searching for it.
DirectoryEntry deUser = new DirectoryEntry("LDAP://cn=John Doe,cn=Users,dc=yourdomain,dc=com");
if (deUser != null)
{
... do something with your user
}
And if you're on .NET 3.5 already, you could even use the vastly expanded System.DirectorySrevices.AccountManagement namespace with strongly typed classes for each of the most common AD objects:
// bind to your domain
PrincipalContext pc = new PrincipalContext(ContextType.Domain, "LDAP://dc=yourdomain,dc=com");
// find the user by identity (or many other ways)
UserPrincipal user = UserPrincipal.FindByIdentity(pc, "cn=John Doe");
There's loads of information out there on System.DirectoryServices.AccountManagement - check out this excellent article on MSDN by Joe Kaplan and Ethan Wilansky on the topic.

You can simplify this code to:
DirectorySearcher searcher = new DirectorySearcher();
searcher.Filter = "(&(objectCategory=user)(cn=steve.evans))";
SearchResultCollection results = searcher.FindAll();
if (results.Count == 1)
{
//do what you want to do
}
else if (results.Count == 0)
{
//user does not exist
}
else
{
//found more than one user
//something is wrong
}
If you can narrow down where the user is you can set searcher.SearchRoot to a specific OU that you know the user is under.
You should also use objectCategory instead of objectClass since objectCategory is indexed by default.
You should also consider searching on an attribute other than CN. For example it might make more sense to search on the username (sAMAccountName) since it's guaranteed to be unique.

I'm not sure how much of your "slowness" will be due to the loop you're doing to find entries with particular attribute values, but you can remove this loop by being more specific with your filter. Try this page for some guidance ... Search Filter Syntax