We Keep Coding

Reading a json file asynchronously, the object property results are always null

I have a Json file, it contains connectionstring. I want to asynchronously read the file and deserialize it to a ConnectionString object and I always get a null result. I'm using .NET Core 6 and System.Text.Json.
Here is contents of my Json file:
{
"ConnectionStrings": {
"ConnStr": "Data Source=(local);Initial Catalog=MyData;Integrated Security=False;TrustServerCertificate=True;Persist Security Info=False;Async=True;MultipleActiveResultSets=true;User ID=sa;Password=MySecret;",
"ProviderName": "SQLServer"
}
}
Here are the contents of my classes:
internal class DBConnectionString
{
[JsonPropertyName("ConnStr")]
public string ConnStr { get; set; }
[JsonPropertyName("ProviderName")]
public string ProviderName { get; set; }
public DBConnectionString()
{
}
}
public class DBConnStr {
private static string AppSettingFilePath => "appsettings.json";
public static async Task<string> GetConnectionStringAsync()
{
string connStr = "";
if (File.Exists((DBConnStr.AppSettingFilePath)))
{
using (FileStream sr = new FileStream(AppSettingFilePath, FileMode.Open, FileAccess.Read))
{
//string json = await sr.ReadToEndAsync();
System.Text.Json.JsonDocumentOptions docOpt = new System.Text.Json.JsonDocumentOptions() { AllowTrailingCommas = true };
using (var document = await System.Text.Json.JsonDocument.ParseAsync(sr, docOpt))
{
System.Text.Json.JsonSerializerOptions opt = new System.Text.Json.JsonSerializerOptions() { AllowTrailingCommas = true, PropertyNameCaseInsensitive = true };
System.Text.Json.JsonElement root = document.RootElement;
System.Text.Json.JsonElement element = root.GetProperty("ConnectionStrings");
sr.Position = 0;
var dbConStr = await System.Text.Json.JsonSerializer.DeserializeAsync<DBConnectionString>(sr, opt);
if (dbConStr != null)
{
connStr = dbConStr.ConnStr;
}
}
}
}
return connStr;
}
}
The following is the syntax that I use to call the GetConnectionStringAsync method:
string ConnectionString = DBConnStr.GetConnectionStringAsync().Result;
When the application is running in debug mode, I checked, on line
var dbConStr = await
System.Text.Json.JsonSerializer.DeserializeAsync(sr,
opt);
The DBConnectionString object property is always empty.
I also tried the reference on the Microsoft website, https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/how-to?pivots=dotnet-6-0 but it doesn't work succeed.
using System.Text.Json;
namespace DeserializeFromFileAsync
{
public class WeatherForecast
{
public DateTimeOffset Date { get; set; }
public int TemperatureCelsius { get; set; }
public string? Summary { get; set; }
}
public class Program
{
public static async Task Main()
{
string fileName = "WeatherForecast.json";
using FileStream openStream = File.OpenRead(fileName);
WeatherForecast? weatherForecast =
await JsonSerializer.DeserializeAsync<WeatherForecast>(openStream);
Console.WriteLine($"Date: {weatherForecast?.Date}");
Console.WriteLine($"TemperatureCelsius: {weatherForecast?.TemperatureCelsius}");
Console.WriteLine($"Summary: {weatherForecast?.Summary}");
}
}
}
Do you have a solution for my problem or a better solution? I appreciate all your help. Thanks
Sorry about my English if it's not good, because I'm not fluent in English and use google translate to translate it

To begin with, if you want to read information from appSettings.json, you should explore more into reading configurations. There are helper classes provided by .Net for the same.
Coming back to your code, if you want to use your own code for Json Deserialization, then you need to make the following change to it.
var dbConStr = System.Text.Json.JsonSerializer.Deserialize<DBConnectionString>(element.GetRawText(), opt);
where, element according to code shared in the question is defined as
System.Text.Json.JsonElement element = root.GetProperty("ConnectionStrings");
This ensures the Raw Json associated with the JsonElement ConnectStrings is de-serialized.
However, I recommend you to read more into Reading configurations using the IConfiguration and related .Net helpers.

ldap principalsearcher very slow

I want to get information from only 1 user out of 20,000 users. The response time of the method I used below is 40 seconds. What is the solution to this problem?
public AuthenticatedUserProperties Info(string Username)
{
try
{
var context = new PrincipalContext(ContextType.Domain, Settings.LDAPDomain, Settings.LDAPContainer, Settings.LDAPUsername, Settings.LDAPPassword);
UserPrincipal user = new UserPrincipal(context);
user.SamAccountName = Username;
var searcher = new PrincipalSearcher(user);
var searchResults = searcher.FindOne();
DirectoryEntry de = searchResults.GetUnderlyingObject() as DirectoryEntry;
ActiveDirectoryUserProperties prop = ConvertLdapUserPropertyToArray(de);
return new AuthenticatedUserProperties
{
Status = true,
Properties = prop
};
}
catch (Exception e)
{
return new AuthenticatedUserProperties
{
Status = false,
Properties = null
};
}
}

I use the System.DirectoryServices.Protocols library instead. It is always blazing fast. I can never get System.DirectoryServices.AccountManagement to have reliable performance and it is often agonizingly slow (10+ seconds) to get just one user. TBH - I think our Network setup is likely to blame causing the bind to be dysfunctional - but the Protocols library yields good results without much effort regardless of our network dysfunction.
You have to do slightly more work - but nothing particularly difficult. I'm not an expert with this library - but this sample code works reliably for me.
using System.DirectoryServices.Protocols;
public class UserInfo
{
public string SAMAccountName;
public string DomainHostName;
public string ADSDirectory;
public Dictionary<string, string> UserAttributes;
// Some attributes not really strings and require extra handling - but simplied for example
// This is really just for illustrative purposes
public UserInfo(string a_SAMAccountName, string a_DomainHostName = "ldap.mydomain:3268", string a_ADSDirectory = "ours.net")
{
UserAttributes = new Dictionary<string, string>();
SAMAccountName = a_SAMAccountName;
DomainHostName = a_DomainHostName;
ADSDirectory = a_ADSDirectory;
}
}
public static class GetUserAttributes
{
public static List<string> WantedAttributes;
static GetUserAttributes()
{
WantedAttributes = new List<string>();
WantedAttributes.Add("mail");
//... Add Properties Wanted
}
public static void GetUserAttributes(UserInfo a_user)
{
using (HostingEnvironment.Impersonate())
{
LdapDirectoryIdentifier z_entry = new LdapDirectoryIdentifier(a_user.DomainHostName, true, false);
using (LdapConnection z_remote = new LdapConnection(z_entry))
{
z_remote.SessionOptions.VerifyServerCertificate = delegate (LdapConnection l, X509Certificate c) { return true; };
z_remote.SessionOptions.ReferralChasing = ReferralChasingOptions.None;
z_remote.SessionOptions.ProtocolVersion = 3;
z_remote.Bind();
SearchRequest z_search = new SearchRequest();
z_search.Scope = System.DirectoryServices.Protocols.SearchScope.Subtree;
z_search.Filter = "(SAMAccountName=" + a_user.SAMAccountName + ")";
z_search.DistinguishedName = a_user.ADSdirectory;
foreach (List<string> z_item in WantedAttributes)
{
z_search.Attributes.Add(z_item);
}
SearchResponse z_response = (SearchResponse)z_remote.SendRequest(z_search);
if (z_response != null)
{
foreach (SearchResultEntry z_result in z_response.Entries)
{
foreach (string z_property in z_result.Attributes.AttributeNames)
{
if (WantedAttributes.ContainsKey(z_property))
{
DirectoryAttribute z_details = a_result.Attributes[z_property];
if (z_details.Count == 1)
{
// Special handling required for Attributes that aren't strings objectSid, objectGUID, etc
string z_value = z_details[0].ToString().Trim();
if (!string.IsNullOrWhiteSpace(z_value))
{
a_user.UserAttributes.Add(z_property, z_value);
}
}
}
}
}
}
}
}
}
}

LinqToTwitter - How to dispose of observable stream correctly taking concurrency into account

I created an observable collection IObservable<Tweet> with LinqToTwitter as shown below. The problem is that this implementation has a concurrency issue when I dispose of the first observable and subscribe to a new observable.
How can I dispose of the first observable correctly?
(The samples below should be complete and work as they are, just add referenced packages and Twitter credentials.)
Here is an example where this problem occurs:
using System;
using System.Reactive.Linq;
namespace Twitter.Cli
{
class Program
{
public static void Main(string[] args)
{
var twitter = new TwitterApi.Twitter();
var search1 = twitter.AllTweetsAbout("windows")
.Sample(TimeSpan.FromSeconds(1));
var search2 = twitter.AllTweetsAbout("android")
.Sample(TimeSpan.FromSeconds(1));
var sub = search1.Subscribe(
x =>
Console.WriteLine("TOPIC = {0} - CONTAINS STRING: {1}", x.Topic, x.Text.ToLower().Contains(x.Topic.ToLower()) ? "YES" : "NO"));
Console.ReadLine();
sub.Dispose();
/*
* If you stop the processing here for a while so that the StartAsync method can be executed
* within the closure everything works fine because disposed is set to true
* before the second observable is created
*/
//Console.ReadLine();
search2.Subscribe(
x =>
Console.WriteLine("TOPIC = {0} - CONTAINS STRING: {1}", x.Topic, x.Text.ToLower().Contains(x.Topic.ToLower()) ? "YES" : "NO"));
Console.ReadLine();
}
}
}
If the StartAsync method in the closure of the first observable creation is executed before the second observable is created then disposed will be set to true and everything is fine.
But if the second observable is created before the next execution of the first closure in StartAsync disposed is set to false again and s.CloseStream(); is never called.
Here is the creation of the observable:
using System;
using System.Configuration;
using System.Linq;
using System.Reactive.Disposables;
using System.Reactive.Linq;
using LinqToTwitter;
namespace TwitterApi
{
public class Twitter
{
private readonly SingleUserAuthorizer _auth = new SingleUserAuthorizer
{
CredentialStore = new InMemoryCredentialStore
{
ConsumerKey = ConfigurationManager.AppSettings["consumerKey"],
ConsumerSecret = ConfigurationManager.AppSettings["consumerSecret"],
OAuthToken = ConfigurationManager.AppSettings["authtoken"],
OAuthTokenSecret = ConfigurationManager.AppSettings["authtokensecret"],
}
};
private readonly TwitterContext _twitterCtx;
public Twitter()
{
if (String.IsNullOrWhiteSpace(_auth.CredentialStore.ConsumerKey)
|| String.IsNullOrWhiteSpace(_auth.CredentialStore.ConsumerSecret)
|| String.IsNullOrWhiteSpace(_auth.CredentialStore.OAuthToken)
|| String.IsNullOrWhiteSpace(_auth.CredentialStore.OAuthTokenSecret))
throw new Exception("User Credentials are not set. Please update your App.config file.");
_twitterCtx = new TwitterContext(_auth);
}
public IObservable<Tweet> AllTweetsAbout(string topic)
{
return Observable.Create<Tweet>(o =>
{
var query = from s in _twitterCtx.Streaming
where s.Type == StreamingType.Filter &&
s.Track == topic
select s;
var disposed = false;
query.StartAsync(async s =>
{
if (disposed)
s.CloseStream();
else
{
Tweet t;
if (Tweet.TryParse(s.Content, topic, out t))
{
o.OnNext(t);
}
}
});
return Disposable.Create(() => disposed = true);
});
}
}
}
And finally the Tweet class:
using System;
using Newtonsoft.Json.Linq;
namespace TwitterApi
{
public class Tweet
{
public string User { get; private set; }
public string Text { get; private set; }
public string Topic { get; private set; }
public static bool TryParse(string json, string topic, out Tweet tweet)
{
try
{
dynamic parsed = JObject.Parse(json);
tweet = new Tweet
{
User = parsed.user.screen_name,
Text = parsed.text,
Topic = topic,
};
return true;
}
catch (Exception)
{
tweet = null;
return false;
}
}
private Tweet()
{
}
}
}

Writing enumerable to csv file

I'm sure its very straightforward but I am struggling to figure out how to write an array to file using CSVHelper.
I have a class for example
public class Test
{
public Test()
{
data = new float[]{0,1,2,3,4};
}
public float[] data{get;set;}
}
i would like the data to be written with each array value in a separate cell. I have a custom converter below which is instead providing one cell with all the values in it.
What am I doing wrong?
public class DataArrayConverter<T> : ITypeConverter
{
public string ConvertToString(TypeConverterOptions options, object value)
{
var data = (T[])value;
var s = string.Join(",", data);
}
public object ConvertFromString(TypeConverterOptions options, string text)
{
throw new NotImplementedException();
}
public bool CanConvertFrom(Type type)
{
return type == typeof(string);
}
public bool CanConvertTo(Type type)
{
return type == typeof(string);
}
}

To further detail the answer from Josh Close, here what you need to do to write any IEnumerable (including arrays and generic lists) in a recent version (anything above 3.0) of CsvHelper!
Here the class under test:
public class Test
{
public int[] Data { get; set; }
public Test()
{
Data = new int[] { 0, 1, 2, 3, 4 };
}
}
And a method to show how this can be saved:
static void Main()
{
using (var writer = new StreamWriter("db.csv"))
using (var csv = new CsvWriter(writer))
{
var list = new List<Test>
{
new Test()
};
csv.Configuration.HasHeaderRecord = false;
csv.WriteRecords(list);
writer.Flush();
}
}
The important configuration here is csv.Configuration.HasHeaderRecord = false;. Only with this configuration you will be able to see the data in the csv file.
Further details can be found in the related unit test cases from CsvHelper.
In case you are looking for a solution to store properties of type IEnumerable with different amounts of elements, the following example might be of any help:
using CsvHelper;
using CsvHelper.Configuration;
using System.Collections.Generic;
using System.IO;
using System.Linq;
namespace CsvHelperSpike
{
class Program
{
static void Main(string[] args)
{
using (var writer = new StreamWriter("db.csv"))
using (var csv = new CsvWriter(writer))
{
csv.Configuration.Delimiter = ";";
var list = new List<AnotherTest>
{
new AnotherTest("Before String") { Tags = new List<string> { "One", "Two", "Three" }, After="After String" },
new AnotherTest("This is still before") {After="after again", Tags=new List<string>{ "Six", "seven","eight", "nine"} }
};
csv.Configuration.RegisterClassMap<TestIndexMap>();
csv.WriteRecords(list);
writer.Flush();
}
using(var reader = new StreamReader("db.csv"))
using(var csv = new CsvReader(reader))
{
csv.Configuration.IncludePrivateMembers = true;
csv.Configuration.RegisterClassMap<TestIndexMap>();
var result = csv.GetRecords<AnotherTest>().ToList();
}
}
private class AnotherTest
{
public string Before { get; private set; }
public string After { get; set; }
public List<string> Tags { get; set; }
public AnotherTest() { }
public AnotherTest(string before)
{
this.Before = before;
}
}
private sealed class TestIndexMap : ClassMap<AnotherTest>
{
public TestIndexMap()
{
Map(m => m.Before).Index(0);
Map(m => m.After).Index(1);
Map(m => m.Tags).Index(2);
}
}
}
}
By using the ClassMap it is possible to enable HasHeaderRecord (the default) again. It is important to note here, that this solution will only work, if the collection with different amounts of elements is the last property. Otherwise the collection needs to have a fixed amount of elements and the ClassMap needs to be adapted accordingly.
This example also shows how to handle properties with a private set. For this to work it is important to use the csv.Configuration.IncludePrivateMembers = true; configuration and have a default constructor on your class.

Unfortunately, it doesn't work like that. Since you are returning , in the converter, it will quote the field, as that is a part of a single field.
Currently the only way to accomplish what you want is to write manually, which isn't too horrible.
foreach( var test in list )
{
foreach( var item in test.Data )
{
csvWriter.WriteField( item );
}
csvWriter.NextRecord();
}
Update
Version 3 has support for reading and writing IEnumerable properties.

Parsing JSON page

Been trying to figure out how to parse out "in_reply_to_status_id_str -> id_str" form the twitter search page:
https://twitter.com/phoenix_search.phoenix?q=hello&headers%5BX-Twitter-Polling%5D=true&headers%5BX-PHX%5D=true&since_id=203194965877194752&include_entities=1&include_available_features=1&contributor_details=true&mode=relevance&query_source=unknown
Anyone that could write a small example to show how it can be done?

Using Json.Net
dynamic jObj = JsonConvert.DeserializeObject(new WebClient().DownloadString("your url"));
foreach (var item in jObj.statuses)
{
Console.WriteLine("{0} {1}", item.in_reply_to_status_id_str, item.id_str);
}

SO here is where I pull my Json, this is where my list gets made, which you all ready have:
public JsonResult AllStatuses() //from the json called in the _client view
{
var buildStatuses = new List<BuildStatus>();
var projects = Client.AllProjects();
foreach (var project in projects)
{
try
{
var buildConfigs = Client.BuildConfigsByProjectId(project.Id);
foreach (var buildConfig in buildConfigs)
{
var b = new BuildStatus();
var build = Client.LastBuildByBuildConfigId(buildConfig.Id);
var status = build.Status; // Used to loop through BuildConfigID's to find which is a FAILURE, SUCCESS, ERROR, or UNKNOWN
var change = Client.LastChangeDetailByBuildConfigId(buildConfig.Id); // Provides the changeID
var changeDetail = Client.ChangeDetailsByChangeId(change.Id); // Provides the username, this one populates the usernames
if (changeDetail != null)
b.user = changeDetail.Username;
b.id = buildConfig.Id.ToString();
// If the date isn't null place the start date in long format
if (build.StartDate != null)
b.date = build.StartDate.ToString();
// If block; set the status based on the BuildconfigID from the var status
if (status.Contains("FAILURE")){
b.status = "FAILURE";
}
else if (status.Contains("SUCCESS")){
b.status = "SUCCESS";
}
else if (status.Contains("ERROR")){
b.status = "ERROR";
}
else{
b.status = "UNKNOWN";
}
buildStatuses.Add(b);
}
} catch { }
}
var query = buildStatuses.OrderBy(x => x.status); // Create a sorted list from Error - Unknown
return Json(query, JsonRequestBehavior.AllowGet);
Then I copied the JsonConverter I linked you too.
On my Website I finally pulled apart the list of Json with.
public JsonResult AllStatuses() //from the json called in the _client view
{
List<Client> clients = storeDB.Clients.Include("Projects").Include("Projects.Builds").ToList();
var buildStatuses = new List<BuildStatus>();
foreach (var client in clients) {
// Network credentials
// Used to get the Json Service request // URL here: client.ClientURL
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://localhost:81/Status/AllStatuses");
var response = request.GetResponse();
var reader = new StreamReader(response.GetResponseStream());
var responseString = reader.ReadToEnd();
var serializer = new JavaScriptSerializer();
serializer.RegisterConverters((new[] { new DynamicJsonConverter() }));
dynamic obj = serializer.Deserialize(responseString, typeof(object)) as dynamic;
foreach (var objects in obj) // Pull apart the dynamic object
{
var id = objects.id;
var status = objects.status;
var date = objects.date;
var user = objects.user;
var bs = new BuildStatus();
try
{
bs.status = status;
bs.date = date;
bs.id = id;
bs.user = user;
}
catch { throw; }
buildStatuses.Add(bs);
}
}
return Json(buildStatuses, JsonRequestBehavior.AllowGet);
}

Go for a jQuery approach:
var obj = jQuery.parseJSON(jsonString);
alert(obj.in_reply_to_status_id_str.id_str);

You can use this json libraryfor accomplish this.

You could also use the DataContractJsonSerializer class available in .NET once you add a reference to System.Runtime.Serialization.
All you need to do is a create two DataContract classes. Something like:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using System.Text;
namespace MyNamespace
{
[DataContract]
public class TwitterObject
{
[DataMember(Name = "statuses")]
public TwitterStatus[] Statuses { get; set; }
}
[DataContract]
public class TwitterStatus
{
[DataMember(Name = "in_reply_to_status_id_str")]
public string InReplyToStatusIdStr { get; set; }
[DataMember(Name = "id_str")]
public string IdStr { get; set; }
}
}
Then from any other method you wish, you just have to use the DataContractJsonSerializer to build your JSON into a .NET object:
DataContractJsonSerializer jsonSerializer = new DataContractJsonSerializer(typeof(TwitterObject));
// assume the twitterResponse is the JSON you receive
MemoryStream memoryStream = new MemoryStream(Encoding.ASCII.GetBytes(twitterResponse));
var twitterJson = jsonSerializer.ReadObject(memoryStream) as TwitterObject;
There may be some typos, but this should give you the hint. I'm currently working on an extensive synchronization between a server app and a website and this is the method I currently use for JSON communication between the two. I've found the combination of DataContracts and DataContractJsonSerializer is easier to use than 3rd party libraries.