We Keep Coding

Fetching more than 1000 rows from Domino LDAP server using .NET Core 5 and Novell.Directory.Ldap.NETStandard

I want to fetch all the users from a large location of our Domino LDAP, around ~2000 users altogether. Since .NET Core sadly doesn't have a platform independent LDAP library, I'm using Novell.Directory.Ldap.NETStandard with this POC:
var cn = new Novell.Directory.Ldap.LdapConnection();
cn.Connect("dc.internal", 389);
cn.Bind("user", "pw");
string filter = "location=MyLoc";
var result = cn.Search("", Novell.Directory.Ldap.LdapConnection.ScopeOne, filter, new string[] { Novell.Directory.Ldap.LdapConnection.AllUserAttrs }, typesOnly: false);
int count = 0;
while (result.HasMore()) {
var entry = result.Next();
count++;
Console.WriteLine(entry.Dn);
}
It prints me a lot of entries, but not all. When count = 1000 I got an Size Limit Exceeded exception. I guess this is because I need to use some kind of pagination, so not all entries woult be returned in a single request. There are different questions like this or this one. Both in Java, the .NET Core API seems somehow different.
Approach 1: Try to find out how LdapSearchRequest works in .NET Core
byte[] resumeCookie = null;
LdapMessageQueue queue = null;
var searchReq = new LdapSearchRequest("", LdapConnection.ScopeOne, filter, new string[] { LdapConnection.AllUserAttrs },
LdapSearchConstraints.DerefNever, maxResults: 3000, serverTimeLimit: 0, typesOnly: false, new LdapControl[] { new SimplePagedResultsControl(size: 100, resumeCookie) });
var searchRequest = cn.SendRequest(searchReq, queue);
I'm trying to figure out how the Java examples can be used in .NET Core. This looks good, however I can't figure out how to fetch the LDAP entries. I only get an message id. By looking into the source it seems that I'm on the right way, but they're using MessageAgent which cannot be used outside since it's internal sealed. This is propably the reason why searching for LdapRearchRequest in the source code doesn't give many results.
Approach 2: Using SimplePagedResultsControlHandler
var opts = new SearchOptions("", LdapConnection.ScopeOne, filter, new string[] { LdapConnection.AllUserAttrs });
// For testing purpose: https://github.com/dsbenghe/Novell.Directory.Ldap.NETStandard/issues/163
cn.SearchConstraints.ReferralFollowing = false;
var pageControlHandler = new SimplePagedResultsControlHandler(cn);
var rows = pageControlHandler.SearchWithSimplePaging(opts, pageSize: 100);
This throws a Unavaliable Cricital Extension exception. First I thought that this is an issue of the .NET port, which may doesn't support all the features of the original Java library yet. It seems complete and according to further researches, it looks like to be an LDAP error code. So this must be something which has to be supported by the server, but is not supported by Domino.

I couldn't make at least one of those approachs work, but found another way: Cross platform support for the System.DirectoryServices.Protocols namespace was was added in .NET 5. This was missing for a long time in .NET Core and I guess this is the main reason why libraries like Novell.Directory.Ldap.NETStandard were ported to .NET Core - in times of .NET Core 1.x this was the only way I found to authenticate against LDAP wich works on Linux too.
After having a deeper look into System.DirectoryServices.Protocols, it works well out of the box, even for ~2k users. My basic POC class looks like this:
public class DominoLdapManager {
LdapConnection cn = null;
public DominoLdapManager(string ldapHost, int ldapPort, string ldapBindUser, string ldapBindPassword) {
var server = new LdapDirectoryIdentifier(ldapHost, ldapPort);
var credentials = new NetworkCredential(ldapBindUser, ldapBindPassword);
cn = new LdapConnection(server);
cn.AuthType = AuthType.Basic;
cn.Bind(credentials);
}
public IEnumerable<DominoUser> Search(string filter, string searchBase = "") {
string[] attributes = { "cn", "mail", "companyname", "location" };
var req = new SearchRequest(searchBase, filter, SearchScope.Subtree, attributes);
var resp = (SearchResponse)cn.SendRequest(req);
foreach (SearchResultEntry entry in resp.Entries) {
var user = new DominoUser() {
Name = GetStringAttribute(entry, "cn"),
Mail = GetStringAttribute(entry, "mail"),
Company = GetStringAttribute(entry, "companyname"),
Location = GetStringAttribute(entry, "location")
};
yield return user;
}
yield break;
}
string GetStringAttribute(SearchResultEntry entry, string key) {
if (!entry.Attributes.Contains(key)) {
return string.Empty;
}
string[] rawVal = (string[])entry.Attributes[key].GetValues(typeof(string));
return rawVal[0];
}
}
Example usage:
var ldapManager = new DominoLdapManager("ldap.host", 389, "binduser", "pw");
var users = ldapManager.Search("objectClass=person");
But it's not solved with Novell.Directory.Ldap.NETStandard as the title said
This doesn't solve my problem with the Novell.Directory.Ldap.NETStandard library as the title suggested, yes. But since System.DirectoryServices.Protocols is a official .NET package maintained by Microsoft and the .NET foundation, this seems the better aproach for me. The foundation will take care to keep it maintained and compatible with further .NET releases. When I wrote the question, I was not aware of the fact that Linux support is added now.
Don't get me wrong, I don't want to say that third packages are bad by design - that would be completely wrong. However, when I have the choice between a official package and a third party one, I think it makes sense to prefer the official one. Except there would be a good reason against that - which is not the case here: The official package (which doesn't exist in the past) works better to solve this issue than the third party one.

How to load an extension in Edge using Selenium?

I want to load an extension in Edge while using Selenium in C#.
I have figured I have to use EdgeOptions.AddExtensionPath() but when passing the path no extensions are loaded.
The extension was extracted from %localappdata%\Microsoft\Edge\User Data\Default\Extensions
Here is the part of the code used to initialize the EdgeDriver:
using OpenQA.Selenium.Edge;
public void InitializeNewDriver() {
string microsoftWebDriverPath = #"some\path";
EdgeDriverService service = EdgeDriverService.CreateDefaultService(microsoftWebDriverPath);
EdgeOptions options = new EdgeOptions();
options.AddExtensionPath(#"path\to\extension\folder");
Driver = new EdgeDriver(service, options);
}
Edit: I'm using Selenium.Webdriver v4.0.0

I try to test the issue and found that if you pass the location of the extension to the options.AddExtensionPath() then the extension is not getting a load.
I suggest trying to use the options.AddExtensions() method and pass the .CRX file of the extension as a parameter.
It can help you to load the extension successfully.
C# code example:
var options = new EdgeOptions();
options.UseChromium = true;
options.BinaryLocation = #"C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe";
options.AddExtensions(#"Path_of_the_extension_here\extension.crx");
var driver = new Microsoft.Edge.SeleniumTools.EdgeDriver(#"Web_driver_path_here......", options);
driver.Navigate().GoToUrl("https://Your_URL_will_be_here...");
Output:
Further, you can modify the code sample as per your own requirements.

How to create new AutoML DataSet for simple classification (C#)

As part of ML automation process I want to dynamically create new AutoML model. I'm using C# (.net framework) and Google.Cloud.AutoML.V1.
After trying to run CreateDataSet code:
var autoMlClient = AutoMlClient.Create();
var parent = LocationName.FromProjectLocation(_projectId, _locationId);
var dataset = new Google.Cloud.AutoML.V1.Dataset();
dataset.DisplayName = "NewDataSet";
var response = autoMlClient.CreateDataset(parent, dataset);
I get the following error:
Field: dataset.dataset_metadata; Message: Required field not set
According to this user manual I should set Dataset Metadata Type, but the list contains only specific types of classifications (Translation/ImageClassifications etc.), I can't find a simple classification type.
How do I create a simple classification data set with the API ? in the AutoML UI its just with a simple button click ("NEW DATASET") - and have to provide only name & region - no classification type.
I also tried to set:
dataset.TextClassificationDatasetMetadata =
new TextClassificationDatasetMetadata() { ClassificationType = ClassificationType.Multiclass };
But I was unable to import data to it (got too many errors of invalid inputs from the input CSV file), I guess its related to the reason that the input format is not suitable for Text Classification.
UPDATE
I've just notice that the Nuget works with AutoML v1 but v1 beta does contains TablesDatasetMetadata Dataset Metadata Type for normal classifications. I'm speechless.

I also experienced this scenario today while creating a dataset using the NodeJS client. Since the Google AutoML table service is in the beta level you need to use the beta version of the AutoML client. In the Google cloud documentation they have used the beta client to create a dataset.
In NodeJS importing the beta version require('#google-cloud/automl').v1beta1.AutoMlClient instead of importing the normal version (v1) require('#google-cloud/automl').v1 worked for me to successfully execute the create dataset functionality.
In C# you can achieve the same through a POST request. Hope this helps :)

After #RajithaWarusavitarana comment, and my last question update , below is the code that did the trick. The token is being generated by GoogleClientAPI nuget and AutoML is handled by REST.
string GcpGlobalEndPointUrl = "https://automl.googleapis.com";
string GcpGlobalLocation = "us-central1"; // api "parent" parameter
public string GetToken(string jsonFilePath)
{
var serviceAccountCredentialFileContents = System.IO.File.ReadAllText(jsonFilePath);
var credentialParameters = NewtonsoftJsonSerializer.Instance.Deserialize<JsonCredentialParameters>(serviceAccountCredentialFileContents);
var initializer = new ServiceAccountCredential.Initializer(credentialParameters.ClientEmail)
{
Scopes = new List<string> { "https://www.googleapis.com/auth/cloud-platform" }
};
var cred = new ServiceAccountCredential(initializer.FromPrivateKey(credentialParameters.PrivateKey));
string accessToken = cred.GetAccessTokenForRequestAsync("https://oauth2.googleapis.com/token").Result;
return accessToken;
}
public void GetDataSetList(string projectId, string token)
{
var restClient = new RestClient(GcpGlobalEndPointUrl);
var createDataSetReqUrl = $"v1beta1/projects/{projectId}/locations/{GcpGlobalLocation}/datasets";
var createDataSetReq = new RestRequest(createDataSetReqUrl, Method.GET);
createDataSetReq.AddHeader("Authorization", $"Bearer {token}");
var createDatasetResponse = restClient.Execute(createDataSetReq);
createDatasetResponse.Dump();
}
I took the token generation code from google-api-dotnet-client Test File

How to create indexes in MongoDB via .NET

I've programmatically created a new document collection using the MongoDB C# driver.
At this point I want to create and build indexes programmatically. How can I do that?

Starting from v2.0 of the driver there's a new async-only API. The old API should no longer be used as it's a blocking facade over the new API and is deprecated.
The currently recommended way to create an index is by calling and awaiting CreateOneAsync with an IndexKeysDefinition you get by using Builders.IndexKeys:
static async Task CreateIndexAsync()
{
var client = new MongoClient();
var database = client.GetDatabase("HamsterSchool");
var collection = database.GetCollection<Hamster>("Hamsters");
var indexKeysDefinition = Builders<Hamster>.IndexKeys.Ascending(hamster => hamster.Name);
await collection.Indexes.CreateOneAsync(new CreateIndexModel<Hamster>(indexKeysDefinition));
}

you should use CreateIndex as EnsureIndex is marked obsolete for future compatibility with the next versions of MongoDB:
var client = new MongoClient("mongodb://localhost");
var db = client.GetServer().GetDatabase("db");
var collection = db.GetCollection<Hamster>("Hamsters");
collection.CreateIndex(IndexKeys<Hamster>.Ascending(_ => _.Name));

The overload of CreateOneAsync in the currently accepted answer is now marked as obsolete with the message "Use CreateOneAsync with a CreateIndexModel instead." Here's how you do it:
static async Task CreateIndex(string connectionString)
{
var client = new MongoClient(connectionString);
var database = client.GetDatabase("HamsterSchool");
var collection = database.GetCollection<Hamster>("Hamsters");
var indexOptions = new CreateIndexOptions();
var indexKeys = Builders<Hamster>.IndexKeys.Ascending(hamster => hamster.Name);
var indexModel = new CreateIndexModel<Hamster>(indexKeys, indexOptions);
await collection.Indexes.CreateOneAsync(indexModel);
}

Something like this should do:
var server = MongoServer.Create("mongodb://localhost");
var db = server.GetDatabase("myapp");
var users = db.GetCollection<User>("users");
users.EnsureIndex(new IndexKeysBuilder().Ascending("EmailAddress"));
Please see the following bits in the documentation:
http://api.mongodb.org/csharp/current/html/06bcd201-8844-3df5-d170-15f2b423675c.htm

There is an entire area on Indexing under the Definitions and Builders documentation page:
http://mongodb.github.io/mongo-csharp-driver/2.4/reference/driver/definitions/#index-keys
Example:
IndexKeysDefinition<MyModel> keys = "{ Reference: 1 }";
var indexModel = new CreateIndexModel<MyModel>(keys);
await _context.Indexes.CreateOneAsync(indexModel);

the easiest way to create indexes in c# is by using the driver wrapper library MongoDB.Entities. here's an example of creating a text index:
DB.Index<Author>()
.Key(a => a.Name, Type.Text)
.Key(a => a.Surname, Type.Text)
.Create();
and to do a full-text search, you simply do:
DB.SearchText<Author>("search term");
haven't seen anything else that makes it simpler than that.

Delete Record from C# using ManagedInterop

I am working with a C# project that uses the dll, Microsoft.Dynamics.AX.ManagedInterop to work with an AX 2012 environment. From within the code, I need to find a SalesQuotationLine based on specific criteria and delete it. So far, I can get the record I need but I am unable to delete it because I am not using a TTSBEGIN/TTSCOMMIT statement and I am not using FORUPDATE. This is my code:
DictTable dictTable = new DictTable(Global.tableName2Id("SalesQuotationLine"));
int quotationIdFieldId = (int)dictTable.Call("fieldName2Id", "QuotationId");
int bdcParentRecIdFieldId = (int)dictTable.Call("fieldName2Id", "BDCParentRecId");
var query = new Query();
var datasource = query.addDataSource(Global.tableName2Id("SalesQuotationLine"));
var queryRange1 = datasource.addRange(quotationIdFieldId);
queryRange1.value = "=" + line.QuotationId;
QueryRun queryRun = new QueryRun(query as object);
while (queryRun.next())
{
var result = queryRun.get(Global.tableName2Id("SalesQuotationLine"));
result.Delete();
}
I also looked at the code here, http://msdn.microsoft.com/en-us/library/cc197113.aspx but I found that I cannot use it since the code I am working with does not use the .NET Business Connector (I am not sure when one dll should be used over the other).

Use TTSBegin() and TTSCommit() methods on Microsoft.Dynamics.AX.ManagedInterop.RuntimeContext.Current. The forUpdate flag can be set by QueryBuildDataSource's update().
It may be easier (and better for maintenance) to write it in an X++ method just call the method from C#.