We Keep Coding

Composite join on two CSV files in C#

Starting from a table of daily fruit prices
fruits.csv
Day,Name,Kind,Price
2019-09-04,"apple","red",63.09
2019-09-04,"apple","yellow",52.14
2019-09-04,"orange","navel",41.18
2019-09-04,"orange","blood",41.18
2019-09-03,"apple","red",63.07
2019-09-03,"apple","yellow",52.11
2019-09-03,"orange","navel",41.13
2019-09-03,"orange","blood",41.13
I'd like to insert the reference prices by name and kind
fruit_ref_prices.csv
Name,Kind,Reference_Price
"apple","red",60.00
"apple","yellow",50.00
"orange","navel",40.00
"orange","blood",42.00
to result in the following table
Day,Name,Kind,Price,Reference_Price
2019-09-04,"apple","red",63.09,60.00
2019-09-04,"apple","yellow",52.14,50.00
2019-09-04,"orange","navel",41.18,40.00
2019-09-04,"orange","blood",41.18,42.00
2019-09-03,"apple","red",63.07,60.00
2019-09-03,"apple","yellow",52.11,50.00
2019-09-03,"orange","navel",41.13,40.00
2019-09-03,"orange","blood",41.13,42.00
The solution should be simple using C#'s built-in SQL-like syntax, and I'm sure the answer lies in one of the following tutorial pages:
Join clause
Perform custom join operations
Join by using composite keys
but I'm having a hard time identifying the syntax of this language.
In my attempt below instead of writing
join fruit_ref in fruit_refs on fruit.name equals fruit_ref.name
I should be able to write
join fruit_ref in fruit_refs on fruit.name equals fruit_ref.name
and fruit.kind equals fruit_ref.kind
but the Boolean expression is not accepted. Why?
My attempt is:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data;
using System.IO;
namespace MyConsoleApplication
{
class Program
{
const string root = #"c:\path\to\here\";
const string file1_in = root + #"fruits.csv";
const string file2_in = root + #"fruit_ref_prices.csv";
static void Main(string[] args)
{
Fruit_Basket fruit_basket = new Fruit_Basket(file1_in, file2_in);
fruit_basket.PrintFruits();
}
}
public class Fruit
{
public DateTime day { get; set; }
public string name { get; set; }
public string kind { get; set; }
public decimal price { get; set; }
public Fruit(DateTime newFruit_day,
string newFruit_name,
string newFruit_kind,
decimal newFruit_price)
{
this.day = newFruit_day;
this.name = newFruit_name;
this.kind = newFruit_kind;
this.price = newFruit_price;
}
}
public class Fruit_Ref
{
public string name;
public string kind;
public decimal reference_price;
public Fruit_Ref(string newName, string newKind, decimal newRef_Price)
{
this.name = newName;
this.kind = newKind;
this.reference_price = newRef_Price;
}
}
public class Fruit_Basket {
public List<Fruit> fruits { get; set; }
public List<Fruit_Ref> fruit_refs { get; set; }
public Fruit_Basket(string file1_in, string file2_in) {
build_fruit_list(file1_in);
build_fruit_ref_list(file2_in);
}
public void build_fruit_list(string file_in)
{
fruits = new List<Fruit>();
int count = 0;
StreamReader reader = new StreamReader(file_in);
string line = "";
while ((line = reader.ReadLine()) != null)
{
if (++count > 1)
{
string[] splitLine = line.Split(new char[] { ',' }).ToArray();
var newFruit_day = DateTime.Parse(splitLine[0]);
var newFruit_name = splitLine[1];
var newFruit_kind = splitLine[2];
var newFruit_price = decimal.Parse(splitLine[3]);
Fruit newFruit = new Fruit(newFruit_day,
newFruit_name,
newFruit_kind,
newFruit_price);
fruits.Add(newFruit);
}
}
reader.Close();
}
public void build_fruit_ref_list(string file_in)
{
fruit_refs = new List<Fruit_Ref>();
int count = 0;
StreamReader reader = new StreamReader(file_in);
string line = "";
while ((line = reader.ReadLine()) != null)
{
if (++count > 1)
{
string[] splitLine = line.Split(new char[] { ',' }).ToArray();
var newFruit_name = splitLine[0];
var newFruit_kind = splitLine[1];
var newFruit_ref_price = decimal.Parse(splitLine[2]);
Fruit_Ref newFruit_ref = new Fruit_Ref(newFruit_name,
newFruit_kind,
newFruit_ref_price);
fruit_refs.Add(newFruit_ref);
}
}
reader.Close();
}
public void PrintFruits()
{
var innerJoinQuery =
from fruit in fruits
join fruit_ref in fruit_refs on fruit.name equals fruit_ref.name
select new { Day = fruit.day, Name = fruit.name, Kind = fruit.kind,
Price = fruit.price, Reference_Price = fruit_ref.reference_price };
Console.WriteLine($#"""Date"",""Name"",""Kind"",""Price"",""Ref Price""");
foreach (var i in innerJoinQuery)
{
Console.WriteLine($#"{i.Day},{i.Kind},{i.Price},{i.Reference_Price}");
}
}
}
}

You could also refactor your code to use the CsvHelper NuGet package for reading/writing CSV files.
First, You can make these classes to reflect the fruits, fruit references and final fruit structure.
public class Fruit
{
public string Day { get; set; }
public string Name { get; set; }
public string Kind { get; set; }
public string Price { get; set; }
}
public class FruitReferencePrice
{
public string Name { get; set; }
public string Kind { get; set; }
public string Reference_Price { get; set; }
}
public class FruitFinal
{
public string Day { get; set; }
public string Name { get; set; }
public string Kind { get; set; }
public string Price { get; set; }
public string ReferencePrice { get; set; }
public override string ToString()
{
return $"Day={Day},Name={Name},Kind={Kind},Price={Price},Reference_Price={ReferencePrice}";
}
}
Then you can make two methods to return the rows of each CSV file into List<Fruit> and List<FruitReferencePrice>.
private static IEnumerable<Fruit> BuildFruitList(string csvFilePath)
{
if (!File.Exists(csvFilePath))
{
throw new FileNotFoundException("Could not locate CSV at path " + csvFilePath, csvFilePath);
}
try
{
using var fileReader = File.OpenText(csvFilePath);
using var csv = new CsvReader(fileReader);
return csv.GetRecords<Fruit>().ToList();
} catch (Exception ex)
{
Console.WriteLine(ex.Message);
return Enumerable.Empty<Fruit>();
}
}
private static IEnumerable<FruitReferencePrice> BuildFruitReferenceList(string csvFilePath)
{
if (!File.Exists(csvFilePath))
{
throw new FileNotFoundException("Could not locate CSV at path " + csvFilePath, csvFilePath);
}
try
{
using var fileReader = File.OpenText(csvFilePath);
using var csv = new CsvReader(fileReader);
return csv.GetRecords<FruitReferencePrice>().ToList();
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
return Enumerable.Empty<FruitReferencePrice>();
}
}
Then you can perform a grouped join and output the merged result.
var path1 = "PATH\\fruits.csv";
var path2 = "PATH\\fruit_ref_prices.csv";
var fruitList = BuildFruitList(path1);
var fruitReferencePrices = BuildFruitReferenceList(path2);
var groupedJoin = from fruit in fruitList
join fruit_ref in fruitReferencePrices
on new { fruit.Name, fruit.Kind } equals new { fruit_ref.Name, fruit_ref.Kind }
select new FruitFinal
{
Day = fruit.Day,
Name = fruit.Name,
Kind = fruit.Kind,
Price = fruit.Price,
ReferencePrice = fruit_ref.Reference_Price
};
foreach (var fruit in groupedJoin)
{
Console.WriteLine(fruit.ToString());
}
Merged results:
Day=2019-09-04,Name=apple,Kind=red,Price=63.09,Reference_Price=60.00
Day=2019-09-04,Name=apple,Kind=yellow,Price=52.14,Reference_Price=50.00
Day=2019-09-04,Name=orange,Kind=navel,Price=41.18,Reference_Price=40.00
Day=2019-09-04,Name=orange,Kind=blood,Price=41.18,Reference_Price=42.00
Day=2019-09-03,Name=apple,Kind=red,Price=63.07,Reference_Price=60.00
Day=2019-09-03,Name=apple,Kind=yellow,Price=52.11,Reference_Price=50.00
Day=2019-09-03,Name=orange,Kind=navel,Price=41.13,Reference_Price=40.00
Day=2019-09-03,Name=orange,Kind=blood,Price=41.13,Reference_Price=42.00

Please change the equals clause as on new { fruit.name, fruit.kind } equals new { fruit_ref.name, fruit_ref.kind }
Why you require this
The query has two anonymous types (one for left table and one for right table). So to compare those anonymous types, the linq statement should use new keyword
Query :
var innerJoinQuery = from fruit in fruits
join fruit_ref in fruit_refs on new { fruit.name, fruit.kind } equals new { fruit_ref.name, fruit_ref.kind }
select new { Day = fruit.day, Name = fruit.name, Kind = fruit.kind,
Price = fruit.price, Reference_Price = fruit_ref.reference_price };

how to get Json nested properties to primary one

I have below scenario:
This is my class structure :
public class User
{
public string FirstName { get; set; }
public string LastName { get; set; }
public System.Collections.ObjectModel.Collection<Likes> Likes { get; set; }
}
public class Likes
{
public string Sport { get; set; }
public string Music { get; set; }
public string Food { get; set; }
public string Place { get; set; }
}
When I serialize object of User class then it will generate the below json string :
{"FirstName":"Naresh",
"LastName":"Parmar",
"Likes": [{"Sport":"Cricket",
"Music":"Classic",
"Food":"Gujarati",
"Place":"India"}]
}
I want to generate above json string like below:
{"FirstName":"Naresh",
"LastName":"Parmar",
"Sport":"Cricket",
"Music":"Classic",
"Food":"Gujarati",
"Place":"India"
}
I want the nested properties as primary one.
Any help would be appreciated.
Thanks in advance..
EDIT:
{"FirstName":"Naresh",
"LastName":"Parmar",
"Sport":"Cricket,Chess,Football",
"Music":"Classic",
"Food":"Gujarati",
"Place":"India"
}

It's really bad practice, since the code i'll post bellow doesn't have great maintainability, however if that's what you looking for, you can use this. Another class that have the format that you'd like, and have a method that adds a list of likes to the format you've required. That the class you should serialize to JSON:
class NestedUser
{
public string FirstName { get; set; }
public string LastName { get; set; }
public string Sport { get; set; }
public string Music { get; set; }
public string Food { get; set; }
public string Place { get; set; }
public void AddLikes(System.Collections.ObjectModel.Collection<Likes> likes)
{
foreach (Likes like in likes)
{
Sport += like.Sport + ",";
Music += like.Music + ",";
Food += like.Food + ",";
Place += like.Place + ",";
}
if (Sport != string.Empty)
{
Sport = Sport.Substring(0, Sport.Length - 1);
}
if (Music != string.Empty)
{
Music = Music.Substring(0, Music.Length - 1);
}
if (Food != string.Empty)
{
Food = Food.Substring(0, Food.Length - 1);
}
if (Place != string.Empty)
{
Place = Place.Substring(0, Place.Length - 1);
}
}
}

Since it's not only limited to Likes objects I'd suggest using dynamic objects. So the User class I propose is as follows:
public class User
{
public string FirstName { get; set; }
public string LastName { get; set; }
public dynamic Details { get; set; }
public User()
{
Details = new ExpandoObject();
}
public void AddSingleDetail(string key, string value)
{
var dict = this.Details as IDictionary<string, Object>;
if (dict.ContainsKey(key))
{
dict[key] += "," + value;
}
else
{
dict[key] = value;
}
}
public void AddDetails(object detailsObject)
{
var type = detailsObject.GetType();
foreach (var prop in type.GetProperties())
{
AddSingleDetail(prop.Name, prop.GetValue(detailsObject).ToString());
}
}
}
You can use it for adding single proerpties or adding an object as a whole. I used reflection to get all the property name and values and add them to the user details.
Sample usage:
static void Main(string[] args)
{
var user1 = new User() { FirstName = "Homer", LastName = "Simpson" };
user1.AddSingleDetail("Sport", "Bowling");
user1.AddSingleDetail("Sport", "Sleeping");
user1.AddSingleDetail("Food", "Donut");
user1.AddSingleDetail("Music", "Rock");
string flattenedHomer1 = ConvertUserToFlattenedJson(user1);
var user2 = new User() { FirstName = "Homer", LastName = "Simpson" };
var likes1 = new Likes() { Food = "Donut", Music = "Rock", Place = "Springfield", Sport = "Bowling" };
var likes2 = new Likes() { Food = "Steaks", Music = "Metal", Place = "Evergreen Terrace", Sport = "Sleeping" };
var proStuff = new ProfessionalStuff() { Title = "Boss" };
user2.AddDetails(likes1);
user2.AddDetails(likes2);
user2.AddDetails(proStuff);
string flattenedHomer2 = ConvertUserToFlattenedJson(user2);
}
And the method performing the JSON conversion is:
public static string ConvertUserToFlattenedJson(User u)
{
dynamic flatUser = new ExpandoObject();
flatUser.FirstName = u.FirstName;
flatUser.LastName = u.LastName;
var dict = u.Details as IDictionary<string, Object>;
foreach (var like in dict)
{
((IDictionary<string, Object>)flatUser)[like.Key] = like.Value;
}
string json = Newtonsoft.Json.JsonConvert.SerializeObject(flatUser);
return json;
}
In my sample above user2 is converted to the following JSON string which I believe is what you are looking for:
{
"FirstName": "Homer",
"LastName": "Simpson",
"Sport": "Bowling,Sleeping",
"Music": "Rock,Metal",
"Food": "Donut,Steaks",
"Place": "Springfield,Evergreen Terrace",
"Title": "Boss"
}
While concatenating strings you can check for null or duplicate values. I didn't handle that part.
For the sake of completeness, here's the ProfessionalStuff class I made up:
public class ProfessionalStuff
{
public string Title { get; set; }
}
Hope this helps.

How merge two lists of different objects?

Using C# with LINQ, how can I merge two lists of different objects, say, Seminar and Conference?
They have some common and some different fields/properties and do not share unique id.
class Seminar
{
int id,
DateTime joinDate,
string name
}
class Conference
{
Guid confNumber,
DateTime joinDate
Type type
}
I have a list of:
List<Seminar>
List<Conference>
I need to merge them into a super List:
List<Object>
A code snippet would be great help.

If you just want a single List<object> containing all objects from both lists, that's fairly simple:
List<object> objectList = seminarList.Cast<object>()
.Concat(conferenceList)
.ToList();
If that's not what you want, then you'll need to define what you mean by "merge".

Following code works fine for me, if this is your definition of Merge
One solution
List<A> someAs = new List<A>() { new A(), new A() };
List<B> someBs = new List<B>() { new B(), new B { something = new A() } };
List<Object> allS = (from x in someAs select (Object)x).ToList();
allS.AddRange((from x in someBs select (Object)x).ToList());
Where A and B are some classes as follows
class A
{
public string someAnotherThing { get; set; }
}
class B
{
public A something { get; set; }
}
Another Solution
List<A> someAs = new List<A>() { new A(), new A() };
List<B> someBs = new List<B>() { new B(), new B { something = string.Empty } };
List<Object> allS = (from x in someAs select (Object)new { someAnotherThing = x.someAnotherThing, something = string.Empty }).ToList();
allS.AddRange((from x in someBs select (Object)new { someAnotherThing = string.Empty, something = x.something}).ToList());
Where A and B are having class definition as
class A
{
public string someAnotherThing { get; set; }
}
class B
{
public string something { get; set; }
}

Simple method of pure code
internal class Person
{
public int Id { get; set; }
public string UserName { get; set; }
}
internal class User
{
public string FirstName { get; set; }
public string LastName { get; set; }
}
internal class UserPerson
{
public int Id { get; set; }
public string UserName { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
}
private static void Main(string[] args)
{
Person[] people = new Person[3] { new Person { Id = 1, UserName = "AliUserName" }, new Person { Id = 2, UserName = "MortezaUserName" }, new Person { Id = 3, UserName = "SalarUserName" } };
User[] users = new User[4] { new User { FirstName = "ali", LastName = "Barzegari" }, new User { FirstName = "Morteza", LastName = "Sefidi" }, new User { FirstName = "Salar", LastName = "Pirzadeh" }, new User { FirstName = "Babak", LastName = "Hasani" } };
UserPerson[] userPeople = new UserPerson[people.Length > users.Length ? people.Length : users.Length];
if (people.Length > users.Length)
for (int i = 0; i < people.Length; i++)
{
userPeople[i] = new UserPerson
{
Id = people[i].Id,
UserName = people[i].UserName,
FirstName = users.Length <= i ? "" : users[i].FirstName,
LastName = users.Length <= i ? "" : users[i].LastName
};
}
else
for (int i = 0; i < users.Length; i++)
{
userPeople[i] = new UserPerson
{
Id = people.Length <= i ? 0 : people[i].Id,
UserName = people.Length <= i ? "" : people[i].UserName,
FirstName = users[i].FirstName,
LastName = users[i].LastName
};
}
Console.ReadLine();
}

Entity Framework - Conditionally include related entities

I maintain an API that, based on a request for a list of people, returns a different result set based on the request. For example, some API clients want to get a list of people and a list of their interactions, others want people and a list of their metadata. All this can be specified int he request to the API method that returns people.
This does not appear to work:
using (var dbcontext = new ExampleEntities())
{
var query = dbcontext.People.AsQueryable();
//determined in earlier application logic based on request
if(includeMetadata)
{
query = query.Include("metadata");
}
//determined in earlier application logic based on request
if(includeInteractions)
{
query = query.Include("interactions");
}
/* ...SNIP... */
}
What I don't want to do is this:
var query = dbcontext.People.Include("Metadata").Include("interactions");
which will mean every request to get a person will include ALL their related entities, even if the requesting API client does not need them.
I also don't want to code every possible combination of logic:
if(includeMetadata && includeInteractions)
{
var query = dbcontext.People.Include("Metadata").Include("interactions");
}
else if(includeMetadata)
{
var query = dbcontext.People.Include("Metadata");
}
else if(includeInteractions)
{
var query = dbcontext.People.Include("Interactions");
}
else
{
var query = dbcontext.People;
}
This will result in hard-to-maintain code, however, I realize I could code generate this if needed.

You can chain the IQueryable's
using (var dbcontext = new ExampleEntities())
{
var query = dbcontext.People.AsQueryable();
if(includeMetadata)
{
query = query.Include("metadata");
}
if(includeInteractions)
{
query = query.Include("interactions");
}
}

Your first example should work if you replace u with query
u = u.Include("metadata");
with
query = query.Include("metadata");

Works fine here... checking the sql statements with the EF 6 Log handler
[TestClass]
public void SomeTestClass
{
[TestMethod]
public void ShouldLoadOnlyRequiredCollections()
{
Database.SetInitializer(new DropCreateDatabaseAlways<HomesContext>());
var db = new HomesContext();
Assert.IsFalse(db.Homes.Any());
var home = db.Homes.Create();
db.Homes.Add(home);
home.Staff.Add(new Staff { Name = "wilma" });
home.Staff.Add(new Staff { Name = "betty" });
home.Residents.Add(new Resident { Name = "fred" });
home.Residents.Add(new Resident { Name = "barney" });
db.SaveChanges();
db = null;
Database.SetInitializer(new DropCreateDatabaseIfModelChanges<HomesContext>());
var sb = new StringBuilder();
db = new HomesContext();
db.Database.Log = ((s) => { sb.Append(s + "\r"); });
Assert.IsTrue(db.Homes.Any());
string log;
log = sb.ToString();
Assert.IsTrue(sb.ToString().Contains("FROM [dbo].[Homes]"));
sb = new StringBuilder(); //ok get residents
var q = db.Homes.Include("Residents");
Assert.IsTrue(string.IsNullOrEmpty(sb.ToString()));
var lst = q.ToList();
log = sb.ToString();
Assert.IsTrue(sb.ToString().Contains("[dbo].[Homes]"));
Assert.IsTrue(sb.ToString().Contains("[dbo].[Residents]"));
Assert.IsTrue(!sb.ToString().Contains("[dbo].[Staff]"));
sb = new StringBuilder(); //get staff
q = db.Homes.Include("Staff");
Assert.IsTrue(string.IsNullOrEmpty(sb.ToString()));
lst = q.ToList();
log = sb.ToString();
Assert.IsTrue(log.Contains("[dbo].[Homes]"));
Assert.IsTrue(!log.Contains("[dbo].[Residents]"));
Assert.IsTrue(log.Contains("[dbo].[Staffs"));
sb = new StringBuilder(); //get residents and staff
q = db.Homes.Include("Staff");
q = q.Include("Residents");
lst = q.ToList();
log = sb.ToString();
Assert.IsTrue(log.Contains("[dbo].[Homes]"));
Assert.IsTrue(log.Contains("[dbo].[Residents]"));
Assert.IsTrue(log.Contains("[dbo].[Staffs]"));
}
}
public class HomesContext:DbContext
{
public DbSet<Home> Homes { get; set; }
}
public class Home
{
public Home()
{
Staff = new List<Staff>();
Residents = new List<Resident>();
}
public int HomeId { get; set; }
public string HomeName { get; set; }
public int MaxResidents { get; set; }
public int MaxStaff { get; set; }
public int CurrentResidents { get; set; }
[NotMapped]
public int CurrentStaff { get; set; }
public IList<Staff> Staff { get; set; }
public IList<Resident> Residents { get; set; }
}
public class Staff
{
public int StaffId { get; set; }
public string Name { get; set; }
public int HomeId { get; set; }
public Home Home { get; set; }
}
public class Resident
{
public int ResidentId { get; set; }
public string Name { get; set; }
public int HomeId { get; set; }
public Home Home { get; set; }
}

ForEach on IEnumerable<dynamic> type data

I am getting data in the form of :
[0]: { rkey = "100asb", pkey = "100ap0", ckey = {1/6/2013 3:28:09 AM} }
[1]: { rkey = "yxq4c", pkey = "100iis", ckey = {1/6/2013 6:38:48 PM} }
.
.
.
I have to write another method that has access to the values rkey, pkey and ckey. Unfortunately I can't access them in a linear ForEach or Parallel.ForEach. I have searched, but I have not found a solution to access my parameters. Some people suggested to convert this to a Dictionary, but I am not sure if that is a good path to take. It has to be much simpler than that.
the code that I have written is like this:
var films = myrollup.GetCompletedMovies(dtstart, dtend).Result;
using (var session = factory.OpenSession())
{
using (var transaction = session.BeginTransaction())
{
Parallel.ForEach(films, currentFilm =>
{
dynamic f = currentFilm;
lock (myrollup)
{
var user = User.GetAsync(f.pkey).Result;
var record = new FilmAnalytics()
{
UserID = currentFilm.pkey,
FilmID = currentFilm.rkey,
UserName = user.FirstName,
UserLastName = user.LastName,
UserAlias = user.Alias,
UserEmail = user.Email,
UserFacebook = user.FbEmail,
Dateofcompletion = currentFilm.ckey
};
session.SaveOrUpdate(record);
}
});
transaction.Commit();
}
}
The variable films produces the IEnumerable<dynamic> type data.

Works for me:
static void Main(string[] args) {
IEnumerable<dynamic> films = new dynamic[] { new { rkey = 1, rval = "val1" }, new { rkey = 2, rval = "val2" } };
foreach (var film in films.Select(f => new { RKey = (int)f.rkey,RValue = (string)f.rval }))
Console.WriteLine(film.RKey + ":" + film.RValue);
}
This way I can transform dynamics to strongly typed objects and then I can do whatever I want with them

You have to create a class like
public class Thing
{
public string rkey { get; set; }
public string pkey { get; set; }
public DateTime ckey { get; set; }
}
and then make the task like this:
public Task<IEnumerable<Thing>> ....
then I can have access to pkey, rkey, and ckey.