We Keep Coding

Using C# and Dapper/SQLite core packages how do I print the SQLite database data and how do I do the same for specific data entries?

I am very new to C# (been learning for approximately 6 months in my free time).
I have produced a script which can store data in an SQLite Database that seems to work fine for the minute. I am having trouble printing the entire table and/or specific data from the table. When I get to the point when it should print off the data it gives me the following:
enter image description here
This is the solution name and the data class.
**
Here is main program code relating to loading data from database:**
private void LoadGymList()
{
Console.WriteLine("Load Gym List Working");
List<GymEntry> entries = new List<GymEntry>();
entries = SQLiteDataAccess.LoadEntry();
Console.WriteLine(entries[0]);
}
**Here is the SQLiteDataAccess code for loading entries:
**
namespace GymAppLists
{
public class SQLiteDataAccess
{
public static List<GymEntry> LoadEntry()
{
using (IDbConnection cnn = new SQLiteConnection(LoadConnectionString()))
{
var output = cnn.Query<GymEntry>("select * from GeneralLog", new DynamicParameters());
return output.ToList();
}
}
Only other files are app.config and a basic class for 'id' and 'date' which are the only two database columns in the single table of the database.
I tried to print individual indexes of the list to see if it would print those, but it simply gave me the previous output. I am stumped as to why this is working this way. It is clearly accessing the database, but it must be formatted incorrectly or I am not using the correct method to access the specifics of the data.
If I print the list.count, it provides me with the correct number of rows in the db for example.
I imagine this is a very simple fix, any advice would be greatly appreciated.
thank you,
JH.

You are only writing the first element of the entries list (Console.WriteLine(entries[0])) and also you are only printing the object itself. Instead use a loop and print the properties. ie:
foreach(var x in entries)
{
Console.WriteLine($"Id:{x.Id}, Date:{x.Date}");
}

c# CSVHelper read CSV with variable headers

First time using the csvReader - note it requires a custom class that defines the Headers found in the CSV file.
class DataRecord
{
//Should have properties which correspond to the Column Names in the file
public String Amount { get; set; }
public String InvoiceDate { get; set; }......
}
The example given then uses the class such:-
using (var sr = new StreamReader(#"C:\\Data\\Invoices.csv"))
{
var reader = new CsvReader(sr);
//CSVReader will now read the whole file into an enumerable
IEnumerable<DataRecord> records = reader.GetRecords<DataRecord>();
//First 5 records in CSV file will be printed to the Output Window
foreach (DataRecord record in records.Take(5))
{
Debug.Print("{0} {1}, {2}", record.Amount, record.InvoiceDate, ....);
}
Two questions :-
1. The app will be loading in files with differing headers so I need to be able to update this class on the fly - is this possible & how?
(I am able to extract the headers from the CSV file.)
CSV file is potentially multi millions of rows (gb size) so is this the best / most efficient way of importing the file.
Destination is a SQLite DB - debug line is used as example.
Thanks

The app will be loading in files with differing headers so I need to be able to update this class on the fly - is this possible & how?
Although it is definetely possible with reflecion or third part libraries, creating an object for a row will be inefficient for such a big files. Moreover, using C# for such a scenario is a bad idea (unless you have some business data transformation). I would consider something like this, or perhaps a SSIS package.

Entity framework - how long should it take to add 75k rows? [duplicate]

This question already has answers here:
Fastest Way of Inserting in Entity Framework
(32 answers)
Closed 9 years ago.
I have a following setup:
Table1 - 150k rows currently, need to add another 25k rows,
Table2 - 300k rows, need to add another 50k.
I add rows in a loop, like that (EF code first):
using (var context = new MyDb())
{
loop() //25k rounds
{
var t3 = new Table2();
var t2 = new Table2();
var t1 = new Table1{prop1 = t2, prop2 = t3};
context.Table1.Add(t1);
context.SaveChanges();
}
}
So, the loop will take 25k turns in total, and my performance measurement shows me, that it will add those rows in roughly ten hours! And now I have no idea if it is because this is a normal time for this kind of operation with EF, or because there is something wrong with my dbms?

Have a look at this answer to a similar question
Fastest Way of Inserting in Entity Framework
The problem is that everytime you go round the loop you are saving the changes to the DB, this results in 25k db writes rather than just one

EF is not designed for efficient bulk update/insert. I would recommend using another method like BCP (command-line utility to import from text files), SqlBulkCopy form C# (which is basically a .NET wrapper for BCP, or an ETL tool like SSIS.
If you are generating the data in your program (not copying it form another locaiton) then you might try building an INSERT SQL command and using SqlCommand.

Linq to Excel with multiple tabs [duplicate]

This question already has answers here:
How do I create an Excel (.XLS and .XLSX) file in C# without installing Microsoft Office?
(47 answers)
Closed 9 years ago.
I use Entity Framework together with Linq to fill my gridviews.
I would like to get the same data I get from the Linq in an excel.
Also a second tab in the same excel sheet should come from another Linq query.
Whats the easiest and best way to do this?

There are two parts to this. The first is to serialise your linq collection to CSV. You can serialise an enumerable method like this:
public static string ToCsv<T>(string separator, IEnumerable<T> objectlist)
{
Type t = typeof(T);
FieldInfo[] fields = t.GetFields();
string header = String.Join(separator, fields.Select(f => f.Name).ToArray());
StringBuilder csvdata = new StringBuilder();
csvdata.AppendLine(header);
foreach (var o in objectlist)
csvdata.AppendLine(ToCsvFields(separator, fields, o));
return csvdata.ToString();
}
That method won't help you with deep objects but it will get you to CSV. The second stage is to build the XSLX format (assuming that 2 CSVs won't be enough). For that I would recommend the Open XML SDK from Microsoft to build the two worksheets from your CSV string. I also like this answer for building an Excel spreadsheet.

Storing the RecordString with the FileHelper Class

We are using FileHelpers 2.0 in our project. I have my record defined and have the data being imported correctly. After getting my array of generic objects:
var engine = new FileHelperEngine<UserRecord>();
engine.ErrorManager.ErrorMode = ErrorMode.SaveAndContinue;
UserRecord[] importedUsers = engine.ReadFile(_filepath);
After getting the records that errored due to formatting issues, I am iterating through the importedUsers array and doing validation to see if the information being imported is valid.
If the data is not valid, I want to be able to log the entire string from the original record from my file.
Is there a way to store the entire "RecordString" in the UserRecord class when the FileHelperEngine reads each of the records?

We do that often at work handling the BeforeRead event and storing it in a field mOriginalString that is marked this way:
[FieldNotInFile]
public string mOriginalString;
You must use the last version of the library from here:
http://teamcity.codebetter.com/repository/download/bt65/20313:id/FileHelpers_2.9.9_ReleaseBuild.zip
Cheers