We Keep Coding

Reading a json file asynchronously, the object property results are always null

I have a Json file, it contains connectionstring. I want to asynchronously read the file and deserialize it to a ConnectionString object and I always get a null result. I'm using .NET Core 6 and System.Text.Json.
Here is contents of my Json file:
{
"ConnectionStrings": {
"ConnStr": "Data Source=(local);Initial Catalog=MyData;Integrated Security=False;TrustServerCertificate=True;Persist Security Info=False;Async=True;MultipleActiveResultSets=true;User ID=sa;Password=MySecret;",
"ProviderName": "SQLServer"
}
}
Here are the contents of my classes:
internal class DBConnectionString
{
[JsonPropertyName("ConnStr")]
public string ConnStr { get; set; }
[JsonPropertyName("ProviderName")]
public string ProviderName { get; set; }
public DBConnectionString()
{
}
}
public class DBConnStr {
private static string AppSettingFilePath => "appsettings.json";
public static async Task<string> GetConnectionStringAsync()
{
string connStr = "";
if (File.Exists((DBConnStr.AppSettingFilePath)))
{
using (FileStream sr = new FileStream(AppSettingFilePath, FileMode.Open, FileAccess.Read))
{
//string json = await sr.ReadToEndAsync();
System.Text.Json.JsonDocumentOptions docOpt = new System.Text.Json.JsonDocumentOptions() { AllowTrailingCommas = true };
using (var document = await System.Text.Json.JsonDocument.ParseAsync(sr, docOpt))
{
System.Text.Json.JsonSerializerOptions opt = new System.Text.Json.JsonSerializerOptions() { AllowTrailingCommas = true, PropertyNameCaseInsensitive = true };
System.Text.Json.JsonElement root = document.RootElement;
System.Text.Json.JsonElement element = root.GetProperty("ConnectionStrings");
sr.Position = 0;
var dbConStr = await System.Text.Json.JsonSerializer.DeserializeAsync<DBConnectionString>(sr, opt);
if (dbConStr != null)
{
connStr = dbConStr.ConnStr;
}
}
}
}
return connStr;
}
}
The following is the syntax that I use to call the GetConnectionStringAsync method:
string ConnectionString = DBConnStr.GetConnectionStringAsync().Result;
When the application is running in debug mode, I checked, on line
var dbConStr = await
System.Text.Json.JsonSerializer.DeserializeAsync(sr,
opt);
The DBConnectionString object property is always empty.
I also tried the reference on the Microsoft website, https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/how-to?pivots=dotnet-6-0 but it doesn't work succeed.
using System.Text.Json;
namespace DeserializeFromFileAsync
{
public class WeatherForecast
{
public DateTimeOffset Date { get; set; }
public int TemperatureCelsius { get; set; }
public string? Summary { get; set; }
}
public class Program
{
public static async Task Main()
{
string fileName = "WeatherForecast.json";
using FileStream openStream = File.OpenRead(fileName);
WeatherForecast? weatherForecast =
await JsonSerializer.DeserializeAsync<WeatherForecast>(openStream);
Console.WriteLine($"Date: {weatherForecast?.Date}");
Console.WriteLine($"TemperatureCelsius: {weatherForecast?.TemperatureCelsius}");
Console.WriteLine($"Summary: {weatherForecast?.Summary}");
}
}
}
Do you have a solution for my problem or a better solution? I appreciate all your help. Thanks
Sorry about my English if it's not good, because I'm not fluent in English and use google translate to translate it

To begin with, if you want to read information from appSettings.json, you should explore more into reading configurations. There are helper classes provided by .Net for the same.
Coming back to your code, if you want to use your own code for Json Deserialization, then you need to make the following change to it.
var dbConStr = System.Text.Json.JsonSerializer.Deserialize<DBConnectionString>(element.GetRawText(), opt);
where, element according to code shared in the question is defined as
System.Text.Json.JsonElement element = root.GetProperty("ConnectionStrings");
This ensures the Raw Json associated with the JsonElement ConnectStrings is de-serialized.
However, I recommend you to read more into Reading configurations using the IConfiguration and related .Net helpers.

How to map any csv file to object with 1 method

I'm trying to map CSV file into class object with C#. My problem is that i have 3 different files, but I want to fallow DRY principles. Can someone tell me how to change 'ParseLine' method to make it possible?
C# consol app.
This is how my FileReader looks like:
public class FileReader<T> : IFileReader<T> where T : Entity
{
private readonly ITransactionReader<T> _transactionReader;
public FileReader(ITransactionReader<T> transactionReader)
{
_transactionReader = transactionReader;
}
public List<T> GetInfoFromFile(string filePath)
{
var lines = File.ReadAllLines(filePath);
var genericLines = new List<T>();
foreach (var line in lines)
{
genericLines.Add(_transactionReader.ParseLine(line));
}
return genericLines;
}
}
public interface IFileReader<T> where T : Entity
{
List<T> GetInfoFromFile(string filePath);
}
This is how the object should look like.
public class TransactionReader : ITransactionReader<Transaction>
{
public Transaction ParseLine(string line)
{
var fields = line.Split(";");
var transaction = new Transaction()
{
Id = fields[0],
Month = int.Parse(fields[1]),
Day = int.Parse(fields[2]),
Year = int.Parse(fields[3]),
IncomeSpecification = fields[4],
TransactionAmount = int.Parse(fields[5])
};
return transaction;
}
}
public interface ITransactionReader<T>
{
T ParseLine(string line);
}
This is how I run it for test purposes.
class Program
{
private static readonly string filePath = "C:/Users/<my_name>/Desktop/C# Practice/ERP/ERP/CsvFiles/Transaction.csv";
static void Main(string[] args)
{
ITransactionReader<Transaction> transactionReader = new TransactionReader();
IFileReader<Transaction> fileReader = new FileReader<Transaction>(transactionReader);
List<Transaction> Test()
{
var obj = fileReader.GetInfoFromFile(filePath);
return obj;
}
var list = Test();
}
}
I'm looking to modify that line:
genericLines.Add(_transactionReader.ParseLine(line));
and method arguments to make it open for any CSV fil.
I don't mind to change that composition into something more effective.

Create Moq for Epplus Excel file

My first question here. I have looked my query but could not find a helpful answer.
My task is to write unit test case for my excel file. The issue I am facing is that we using Epplus for excel files and I am not sure how can we write unit test cases for this. I looked up and found that we can also use MOQ to mock up. But again I could not find any helpful links for mocking an excel file that uses Epplus. I found this link Unit testing classes that use EPPlus but I am not sure how I can implement this .
I would appreciate if someone can provide a sample of how to write a simple unit test for the excel file. The test can be to check if file uploaded is an excel file or not, checking if the excel is empty or not etc.
Sorry at this moment I dont have any sample. What I can share is the code where I am reading the excel file:
public class MyController : Controller
{
[HttpPost("upload")]
public async Task<IActionResult> UploadFile(IFormFile file)
{
JArray data = new JArray();
using (ExcelPackage package = new ExcelPackage(file.OpenReadStream()))
{
ExcelWorksheet worksheet = package.Workbook.Worksheets[1];
//Check if excel is empty.
if (worksheet.Dimension == null)
{
return BadRequest("File is blank.");
}
data = Helper.CreateJson(worksheet);
}
return Ok(data);
}
}
I had created a helper class as:
public static JArray CreateJson(ExcelWorksheet worksheet)
{
JArray data = new JArray();
JObject jobject = new JObject();
int rowCount = worksheet.Dimension.End.Row;
int colCount = worksheet.Dimension.End.Column;
for (int row = 1; row <= rowCount; row++)
{
for (int col = 1; col <= colCount; col++)
{
var value = worksheet.Cells[row, col].Value;
//Excel has 2 columns and I want to create a json from that.
if (col == 1)
{
jObject.Add("ID", rowValue.ToString());
}
else
{
jObject.Add("Name", rowValue.ToString());
}
}
data.Add(jObject);
jObject= new JObject();
}
return data;
}
This is the Test Class i have so far.
public class TestClass
{
private MyController _controller;
public TestClass()
{
_controller = new MyController ();
}
[Fact]
public void Upload_WhenCalled()
{
//var file = new FileInfo(#"C:\myfile.xlsx");
//...what next?
var file = new Mock<IFormFile>();
var content = File.OpenRead(#"C:\myfile.xlsx");
var result = _controller.UploadFile(file.Object);
//When I debug it throws error "Object reference not set to an instance of an object."
}
}

In this case mock IFormFile to return the file stream in your test and pass that to the action under test. Make sure all other necessary dependencies are satisfied.
public class TestClass {
private MyController _controller;
public TestClass() {
_controller = new MyController ();
}
[Fact]
public void Upload_WhenCalled() {
//Arrange
var content = File.OpenRead(#"C:\myfile.xlsx");
var file = new Mock<IFormFile>();
file.Setup(_ => _.OpenReadStream()).Returns(content);
//Act
var result = _controller.UploadFile(file.Object);
//Assert
//...
}
}
Now while this should help get you through the current problem, you should really take the advice suggested by other answers about abstracting that tight coupling of ExcelPackage out of the controller into its own concern. Would make unit testing the controller easier in isolation.
You could always do an integration test of the wrapper separately as needed.
A simplified example of an interface abstracted from what is currently in the controller
public interface IExcelService {
Task<JArray> GetDataAsync(Stream stream);
}
which would have an implementation that mirrors the code in the controller
public class ExcelService: IExcelService {
public async Task<JArray> GetDataAsync(Stream stream) {
JArray data = new JArray();
using (ExcelPackage package = new ExcelPackage(stream)) {
ExcelWorksheet worksheet = package.Workbook.Worksheets[1];
if (worksheet.Dimension != null) {
data = await Task.Run(() => createJson(worksheet));
}
}
return data;
}
private JArray createJson(ExcelWorksheet worksheet) {
JArray data = new JArray();
int colCount = worksheet.Dimension.End.Column; //get Column Count
int rowCount = worksheet.Dimension.End.Row; //get row count
for (int row = 1; row <= rowCount; row++) {
JObject jobject = new JObject();
for (int col = 1; col <= colCount; col++) {
var value = worksheet.Cells[row, col].Value;
//Excel has 2 columns and I want to create a json from that.
if (col == 1) {
jObject.Add("ID", rowValue.ToString());
} else {
jObject.Add("Name", rowValue.ToString());
}
data.Add(jObject);
}
}
return data;
}
}
The controller can now be simplified to follow the Explicit Dependencies Principle
public class MyController : Controller {
private readonly IExcelService excel;
public MyController(IExcelService excel) {
this.excel = excel;
}
[HttpPost("upload")]
public async Task<IActionResult> UploadFile(IFormFile file) {
JArray data = await excel.GetDataAsync(myFile.OpenReadStream());
if(data.Count == 0)
return BadRequest("File is blank.");
return Ok(data);
}
}
You would make sure that the interface and implementation are registered with the Dependency Inversion framework in Startup
services.AddScoped<IExcelService, ExcelService>();
So now the controller is only concerned with what it is suppose to do when called at run time. I has no reason to be dealing with implementation concerns
public class MyControllerTests {
[Fact]
public async Task Upload_WhenCalled() {
//Arrange
var content = new MemoryStream();
var file = new Mock<IFormFile>();
file.Setup(_ => _.OpenReadStream()).Returns(content);
var expected = new JArray();
var service = new Mock<IExcelService>();
service
.Setup(_ => _.GetDataAsync(It.IsAny<Stream>()))
.ReturnsAsync(expected);
var controller = new MyController(service.Object);
//Act
var result = await controller.UploadFile(file.Object);
//Assert
service.Verify(_ => _.GetDataAsync(content));
//...other assertions like if result is OkContentResult...etc
}
}
To do an integration test that involves an actual file you can test the service
public class ExcelServiceTests {
[Fact]
public async Task GetData_WhenCalled() {
//Arrange
var stream = File.OpenRead(#"C:\myfile.xlsx");
var service = new ExcelService();
//Act
var actual = await service.GetDataAsync(stream);
//Assert
//...assert the contents of actual data.
}
}
Each concern can now be tested on its own.

You don't need to mock EPPlus to test. Your focus should be on testing your code, not EPPlus itself. Just like you wouldn't test any other library you consume. So have your code generate an Excel file in memory using EPPlus and return it. Then in your test use EPPlus to verify your assertions about the file.
Here's an example of a pattern to use:
public class MyReportGenerator : IReportGenerator
{
/* implementation here */
}
public interface IReportGenerator
{
byte[] GenerateMyReport(ReportParameters parameters);
}
[TestMethod]
public void TestMyReportGenerate()
{
//arrange
var parameters = new ReportParameters(/* some values */);
var reportGenerator = new MyReportGenerator(/* some dependencies */);
//act
byte[] resultFile = reportGenerator.GenerateMyReport(parameters);
//assert
using(var stream = new MemoryStream(resultFile))
using(var package = new ExcelPackage(stream))
{
//now test that it generated properly, such as:
package.Workbook.Worksheets["Sheet1"].Cells["C6"].GetValue<decimal>().Should().Be(3.14m);
package.Workbook.Worksheets["Sheet1"].Column(5).Hidden.Should().BeTrue();
}
}
The example above is using the Fluent Assertions library, though obviously this isn't necessary.

Change files using Roslyn

I'm trying to write a command line tool that modifies some code using Roslyn. Everything seems to go well: the solution is opened, the solution is changed, the Workspace.TryApplyChanges method returns true. However no actual files are changed on disk. What's up? Below is the top level code I'm using.
static void Main(string[] args)
{
var solutionPath = args[0];
UpdateAnnotations(solutionPath).Wait();
}
static async Task<bool> UpdateAnnotations(string solutionPath)
{
using (var workspace = MSBuildWorkspace.Create())
{
var solution = await workspace.OpenSolutionAsync(solutionPath);
var newSolution = await SolutionAttributeUpdater.UpdateAttributes(solution);
var result = workspace.TryApplyChanges(newSolution);
Console.WriteLine(result);
return result;
}
}

I constructed a short program using your code and received the results I expected - the problem appears to reside within the SolutionAttributeUpdater.UpdateAttributes method. I received these results using the following implementation with your base main and UpdateAnnotations-methods:
public class SolutionAttributeUpdater
{
public static async Task<Solution> UpdateAttributes(Solution solution)
{
foreach (var project in solution.Projects)
{
foreach (var document in project.Documents)
{
var syntaxTree = await document.GetSyntaxTreeAsync();
var root = syntaxTree.GetRoot();
var descentants = root.DescendantNodes().Where(curr => curr is AttributeListSyntax).ToList();
if (descentants.Any())
{
var attributeList = SyntaxFactory.AttributeList(
SyntaxFactory.SingletonSeparatedList(
SyntaxFactory.Attribute(SyntaxFactory.IdentifierName("Cookies"), SyntaxFactory.AttributeArgumentList(SyntaxFactory.SeparatedList(new[] { SyntaxFactory.AttributeArgument(
SyntaxFactory.LiteralExpression(
SyntaxKind.StringLiteralExpression, SyntaxFactory.Literal(#"Sample"))
)})))));
root = root.ReplaceNodes(descentants, (node, n2) => attributeList);
solution = solution.WithDocumentSyntaxRoot(document.Id, root);
}
}
}
return solution;
}
}
It was tested using the following class in the sample solution:
public class SampleClass<T>
{
[DataMember("Id")]
public int Property { get; set; }
[DataMember("Id")]
public void DoStuff()
{
DoStuff();
}
}
And it resulted in the following Output:
public class SampleClass<T>
{
[Cookies("Sample")] public int Property { get; set; }
[Cookies("Sample")] public void DoStuff()
{
DoStuff();
}
}
If you take a look at the UpdateAttributes method I had to replace the nodes with ReplaceNodes and updated the solution by calling WithDocumentSyntaxRoot.
I would assume that either one of those two calls is missing or that nothing was changed at all - if you call workspace.TryApplyChanges(solution) you would still receive true as an Output.
Note that using multiple calls of root.ReplaceNode() instead of root.ReplaceNodes() can also result in an error since only the first update is actually used for the modified document - which might lead you to believe that nothing has changed at all, depending on the implementation.

Writing enumerable to csv file

I'm sure its very straightforward but I am struggling to figure out how to write an array to file using CSVHelper.
I have a class for example
public class Test
{
public Test()
{
data = new float[]{0,1,2,3,4};
}
public float[] data{get;set;}
}
i would like the data to be written with each array value in a separate cell. I have a custom converter below which is instead providing one cell with all the values in it.
What am I doing wrong?
public class DataArrayConverter<T> : ITypeConverter
{
public string ConvertToString(TypeConverterOptions options, object value)
{
var data = (T[])value;
var s = string.Join(",", data);
}
public object ConvertFromString(TypeConverterOptions options, string text)
{
throw new NotImplementedException();
}
public bool CanConvertFrom(Type type)
{
return type == typeof(string);
}
public bool CanConvertTo(Type type)
{
return type == typeof(string);
}
}

To further detail the answer from Josh Close, here what you need to do to write any IEnumerable (including arrays and generic lists) in a recent version (anything above 3.0) of CsvHelper!
Here the class under test:
public class Test
{
public int[] Data { get; set; }
public Test()
{
Data = new int[] { 0, 1, 2, 3, 4 };
}
}
And a method to show how this can be saved:
static void Main()
{
using (var writer = new StreamWriter("db.csv"))
using (var csv = new CsvWriter(writer))
{
var list = new List<Test>
{
new Test()
};
csv.Configuration.HasHeaderRecord = false;
csv.WriteRecords(list);
writer.Flush();
}
}
The important configuration here is csv.Configuration.HasHeaderRecord = false;. Only with this configuration you will be able to see the data in the csv file.
Further details can be found in the related unit test cases from CsvHelper.
In case you are looking for a solution to store properties of type IEnumerable with different amounts of elements, the following example might be of any help:
using CsvHelper;
using CsvHelper.Configuration;
using System.Collections.Generic;
using System.IO;
using System.Linq;
namespace CsvHelperSpike
{
class Program
{
static void Main(string[] args)
{
using (var writer = new StreamWriter("db.csv"))
using (var csv = new CsvWriter(writer))
{
csv.Configuration.Delimiter = ";";
var list = new List<AnotherTest>
{
new AnotherTest("Before String") { Tags = new List<string> { "One", "Two", "Three" }, After="After String" },
new AnotherTest("This is still before") {After="after again", Tags=new List<string>{ "Six", "seven","eight", "nine"} }
};
csv.Configuration.RegisterClassMap<TestIndexMap>();
csv.WriteRecords(list);
writer.Flush();
}
using(var reader = new StreamReader("db.csv"))
using(var csv = new CsvReader(reader))
{
csv.Configuration.IncludePrivateMembers = true;
csv.Configuration.RegisterClassMap<TestIndexMap>();
var result = csv.GetRecords<AnotherTest>().ToList();
}
}
private class AnotherTest
{
public string Before { get; private set; }
public string After { get; set; }
public List<string> Tags { get; set; }
public AnotherTest() { }
public AnotherTest(string before)
{
this.Before = before;
}
}
private sealed class TestIndexMap : ClassMap<AnotherTest>
{
public TestIndexMap()
{
Map(m => m.Before).Index(0);
Map(m => m.After).Index(1);
Map(m => m.Tags).Index(2);
}
}
}
}
By using the ClassMap it is possible to enable HasHeaderRecord (the default) again. It is important to note here, that this solution will only work, if the collection with different amounts of elements is the last property. Otherwise the collection needs to have a fixed amount of elements and the ClassMap needs to be adapted accordingly.
This example also shows how to handle properties with a private set. For this to work it is important to use the csv.Configuration.IncludePrivateMembers = true; configuration and have a default constructor on your class.

Unfortunately, it doesn't work like that. Since you are returning , in the converter, it will quote the field, as that is a part of a single field.
Currently the only way to accomplish what you want is to write manually, which isn't too horrible.
foreach( var test in list )
{
foreach( var item in test.Data )
{
csvWriter.WriteField( item );
}
csvWriter.NextRecord();
}
Update
Version 3 has support for reading and writing IEnumerable properties.