We Keep Coding

Cannot read all users from Active Directory - [DirectoryServicesCOMException] MoveNext()

My team is using a program written in C# to read all users from a specific OU. The program behaves very strange. Sometimes it is working for a couple of weeks and then without any big changes on our AD or any other related component, it throws an exception. Then it is not working for a couple of weeks and after some time it start to run normally again.
Code
DirectoryEntry searchRoot = new DirectoryEntry("<LDAP string>")
searchRoot.AuthenticationType = AuthenticationTypes.None;
DirectorySearcher search = new DirectorySearcher(searchRoot);
search.Filter = <our filter>;
search.PropertiesToLoad.Add("<some property>");
search.PageSize = 1;
SearchResult result;
SearchResultCollection resultCol = null;
try
{
resultCol = search.FindAll();
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
if (resultCol != null)
{
Console.WriteLine("Result Count: " + resultCol.Count); //.Count throws the Exception
}
Exception
Unhandled Exception: System.DirectoryServices.DirectoryServicesCOMException: An operations error occurred.
at System.DirectoryServices.SearchResultCollection.ResultsEnumerator.MoveNext()
at System.DirectoryServices.SearchResultCollection.get_InnerList()
at System.DirectoryServices.SearchResultCollection.get_Count()
Data: System.Collections.ListDictionaryInternal
Error Code: -2147016672
Extended Error: 8431
Extended Error Message: 000020EF: SvcErr: DSID-020A07A7, problem 5012 (DIR_ERROR), data -1018
HResult: -2147016672
Message: An operations error occured.
Source: System.DirectoryServices
Stack Trace: at System.DirectoryServices.SearchResultCollection.ResultsEnumerator.MoveNext()
Target Site: Boolean MoveNext()
Additional Information
Target Framework: .Net Framework 4.6.1 (no additional libraries)
The program is executed on a domain controller
What I have tried
I have created a loop to use the MoveNext() function of the
enumerator and found out that it loads results up to a specific
element and then crashes
It is always the same element
After the first exception all retries fail as well
The user that starts it is a domain admin (but I
have also tried it with an enterprise admin account, so it is
probably not a permission issue)
I have deleted the user that should be read when the exception happens but dring the next run the exception was thrown for a previous user
I have come to a point, where I have no more ideas on solving this problem. I would appreciate all your support.

This answer just summarizes our conversation in comments.
This thread partially matches the error you are getting:
problem 5012 (DIR_ERROR) data -1018
And the answer from a Microsoft MVP is:
That is a checksum error in the database, you have corruption in your
database which is usually due to a failing disk or disk subsystem or
possibly a system crash and data not being written from a write cache.
So it sounds like you might have the same thing going on.
But it may only be one DC that has the problem. So to help you narrow down which one, you can specify the DC in the LDAP path like so:
LDAP://dc1.example.com/OU=Target,OU=My User Group,OU=My Users,DC=example,DC=com
This can help you in two ways:
It can identify the bad DC so you know which one you need to fix (and possibly take it offline until it is fixed), and
You can specifically target a good DC so your script will keep working.

An error occurred while closing the provider connection…OCI invalid handle

I searched a lot trying to find why am I getting this error but no help, can you please advise.
I am getting the following error when executing a linq query
var test = (Linq query).ToList<Testdata>();
System.Data.EntityException: An error occurred while closing the
provider connection. See the inner exception for details. --->
Devart.Data.Oracle.OracleException: OCI invalid handle.
at Devart.Data.Oracle.a1.c(Int32 A_0) at Devart.Data.Oracle.aq.d()
at Devart.Data.Oracle.OracleInternalConnection.a(Boolean A_0)
at Devart.Common.DbConnectionInternal.CloseInternalConnection()
at
Devart.Data.Oracle.OracleInternalConnection.CloseInternalConnection()
at Devart.Common.DbConnectionInternal.Close()
at Devart.Data.Oracle.OracleInternalConnection.Close()
at Devart.Common.DbConnectionBase.Close()
at Devart.Data.Oracle.OracleConnection.Close()
at System.Data.EntityClient.EntityConnection.StoreCloseHelper() ---
End of inner exception stack trace ---
at System.Data.EntityClient.EntityConnection.StoreCloseHelper()
at System.Data.EntityClient.EntityConnection.CloseHelper()
at System.Data.EntityClient.EntityConnection.Close()
at System.Data.Objects.ObjectContext.ReleaseConnection()
at System.Data.Common.Internal.Materialization.Shaper`1.Finally()
at
ystem.Data.Common.Internal.Materialization.Shaper`1.ObjectQueryNestedEnumerator.Dispose()
at System.Collections.Generic.List1..ctor(IEnumerable1 collection)
at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)

Please send us a small test project and DDL/DML script for reproducing the issue in our environment. Also specify the versions of your Oracle Server and Oracle Client.

Issue was due to pool reaching max capacity

"Fluent methods may not be invoked on a Query created via CloudTable.CreateQuery<T>()" exception

What does the following exception means?
System.NotSupportedException was unhandled
Message: An unhandled exception of type 'System.NotSupportedException' occurred in mscorlib.dll
Additional information: Fluent methods may not be invoked on a Query created via CloudTable.CreateQuery()
It does not show the code throwing the exception so I don't know how to start debugging it.
Result StackTrace:
at System.Web.Http.ApiController.d__1.MoveNext()
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task1.GetResultCore(Boolean waitCompletionNotification)
at System.Threading.Tasks.Task1.get_Result()
at TestFramework.ExecuteRequest(HttpRequestMessage request) in d:\
at TestFramework.Post(String uri, Object tniObject) in d:\
at TestFramework.PostCall(String uri, Object o) in d:\
at TestFramework.MyMethod(String one, String two, MyStruct three) in d:\
... (Removed for privacy)
I believe the problem is in the following instruction.
string queryString = TableQuery.CombineFilters(TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, myId),
TableOperators.And,
TableQuery.GenerateFilterCondition("RowKey", QueryComparisons.Equal, number));
var theQuery = MyTable.CreateQuery<MyEntity>().Where(queryString);
can I use theQuery then to perform a segmented async query?
var returnList = new List<T>();
TableQuerySegment<T> querySegment = null;
querySegment = await theQuery.AsTableQuery().ExecuteSegmentedAsync(null);
// The query could potentially return more than one object
returnList.AddRange(querySegment);
Well, changing the CreateQuery method call to the following code made the exception to go away.
var query = new TableQuery<TenantTNEntity>().Where(queryString);
The exception I get now says:
Result Message:
Test method MyMethod threw exception:
System.AggregateException: One or more errors occurred. ---> System.InvalidOperationException: Unknown Table. The TableQuery does not have an associated CloudTable Reference. Please execute the query via the CloudTable ExecuteQuery APIs.

I don't know if this is specifically your problem, but it certainly was mine, and it took me a while to figure out. For reasons that I'm not entirely clear on, the Azure team has provided two different and incompatible ways to execute queries - and although they're incompatible at runtime, the Azure team has helpfully (not!) made sure that they have compatible signatures at compile time.
See the difference between the "Fluent" and "IQueryable" modes described here:
http://blogs.msdn.com/b/windowsazurestorage/archive/2013/09/07/announcing-storage-client-library-2-1-rtm.aspx
In other words, both of these will compile and run and do more or less the same thing:
var fluentQuery = new TableQuery<T>().Where(filter);
var fluentResult = Table.ExecuteQuery(fluentQuery);
var iQueryableQuery= from ent in Table.CreateQuery<T>()
where ent.PartitionKey == "some value"
select ent;
var iQueryableResult = iQueryableQuery.AsTableQuery().Execute();
But while this one will compile just fine, it will blow up at runtime with the System.NotSupportedException that you (and I) ran into.
var fluentQuery2 = Table.CreateQuery<T>().Where(filter);
var fluentResult2 = fluentQuery2.Execute();
I'm sure that the folks at MS had some good reason for this particular violation of the Principle of Least Astonishment - they're not idiots - but it certainly remains the case that this behavior is, shall we say, unusual.

Adding ".AsQueryable()" before ".Take(x)" will make it run

hdinsight new hiveconnection not working

Im using the hdinsight hadoop locally and after successfully running mapreduce jobs on the hdfs i am trying with hive, unfortunately i am getting errors when running the hive query when creating a table.
I got the following code from another stack solution as i am only running it locally and not using azure:
var db = new HiveConnection(
webHCatUri: new Uri("http://localhost:50111"),
userName: (string)"hadoop", password: (string)null);
// also tried command with no ; inside quotes
string command = "CREATE TABLE log4jLogs(t1 string, t2 string, t3 string," +
"t4 string, t5 string, t6 string, t7 string)" +
"ROW FORMAT DELIMITED FIELDS TERMINATED BY ' ';";
db.ExecuteHiveQuery(command).Wait();
This always throws an exception after a while of processing. the exception is:
System.AggregateException: One or more errors occurred. --->
System.NullReferenc eException: Object reference not set to an
instance of an object. at
System.Threading.Tasks.TaskAwaiter1.GetResult() at
Microsoft.Hadoop.Hive.HiveConnection.<ExecuteHiveQuery>d__2.MoveNext()
--- End of inner exception stack trace --- at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout,
CancellationTo ken cancellationToken) at
System.Threading.Tasks.Task.Wait() at
ConsoleApplication1.Program.CreateActorsTable() in
c:\Users\Administrator\ Documents\Visual Studio
2012\Projects\ConsoleApplication1\ConsoleApplication1\Pr ogram.cs:line
90
---> (Inner Exception #0) System.NullReferenceException: Object reference not se t to an instance of an object. at
System.Threading.Tasks.TaskAwaiter1.GetResult() at
Microsoft.Hadoop.Hive.HiveConnection.d__2.MoveNext()<---
I have checked the logs from hadoop in which i get the following:
java.io.IOException: Invalid status dir URI at
org.apache.hcatalog.templeton.tool.TempletonControllerJob$LaunchMapper.run(TempletonControllerJob.java:155)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:762)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364) at
org.apache.hadoop.mapred.Child$4.run(Child.java:266) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:396) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136)
at org.apache.hadoop.mapred.Child.main(Child.java:260) Caused by:
java.net.URISyntaxException: Illegal character in authority at index
6: asv://{0}#{1}/6f8d3bc9-89a2-4e1d-9749-2805d31f05f0 at
java.net.URI$Parser.fail(URI.java:2810) at
java.net.URI$Parser.parseAuthority(URI.java:3148) at
java.net.URI$Parser.parseHierarchical(URI.java:3059) at
java.net.URI$Parser.parse(URI.java:3015) at
java.net.URI.(URI.java:577) at
org.apache.hcatalog.templeton.tool.TempletonUtils.addUserHomeDirectoryIfApplicable(TempletonUtils.java:227)
at
org.apache.hcatalog.templeton.tool.TempletonControllerJob$LaunchMapper.run(TempletonControllerJob.java:152)
... 7 more
I would love if anyone could point me in the right direction with this as i cannot work out why this is failing.

If you want to use hive connection first have to start the hive server. Hive server executables are residing in hdp/hive/bin. Using command prompt start the hiveserver2.exe . You can modify the Hive server configuration using hiveserver2.xml which is also residing in the same folder.

Entity Framework with SQLite exception: The underlying provider failed on Commit

I'm using the Entity Framework (.NET 4.0) with SQLite as the underlying database and I get an exception when I try to commit some changes to the database:
The underlying provider failed on Commit.
The stack trace is:
System.Data.EntityException: The underlying provider failed on Commit. ---> System.Data.SQLite.SQLiteException: The database file is locked
database is locked
at System.Data.SQLite.SQLite3.Step(SQLiteStatement stmt)
at System.Data.SQLite.SQLiteDataReader.NextResult()
at System.Data.SQLite.SQLiteDataReader..ctor(SQLiteCommand cmd, CommandBehavi
or behave)
at System.Data.SQLite.SQLiteCommand.ExecuteReader(CommandBehavior behavior)
at System.Data.SQLite.SQLiteCommand.ExecuteNonQuery()
at System.Data.SQLite.SQLiteTransaction.Commit()
at System.Data.EntityClient.EntityTransaction.Commit()
--- End of inner exception stack trace ---
at System.Data.EntityClient.EntityTransaction.Commit()
at System.Data.Objects.ObjectContext.SaveChanges(SaveOptions options)
at MySystem.MySystemService.UpdateFollowersAndUsers() in C:\Path\To\MySystem\MySystemService.cs:line 295
My code is pretty simple:
// Gets more followers for the competitor and updates the database
List<Follower> moreFollowers = GetMoreFollowers(competitor);
// Add the new followers to the database
using (MySystemEntities db = new MySystemEntities())
{
try
{
foreach (Follower f in moreFollowers)
{
db.AddToFollowers(f);
}
db.SaveChanges();
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
}
This snippet of code is inside MyService, it comes in with a batch of 5,000 followers and it's getting the above exception. However, when I pull the same snippet of code out into the Main function and just manually add a few followers, then it no longer throws the exception and the followers are successfully added to my database. Apparently the database file is locked, what may be causing this issue?

I just found out the answer in another SO question: 'The database file is locked' with System.Data.SQLite
Ironically, the OP answers his own question too :).