LINQ query grouping without duplication of data - c#

So I want to display an output that is group by two fields: SubsidiaryCode and AssetCreatedDate. My problem is it displays the grouping values redundantly.
I suspect it duplicates because of my Detail class.
What I want is:
But it displays like this:
LINQ query:
public DateTime FromDate { get; set; }
public DateTime ToDate { get; set; }
public IList<AssetListTemplate> List = new List<AssetListTemplate>();
public IList<AssetListTemplate> GetList()
{
using (var ctx = LinqExtensions.GetDataContext<NXpert.FixedAsset.DataAccess.FixedAssetDataContext>("AccountingDB"))
{
var list = (from x in ctx.DataContext.AssetRegistryEntities
where x.SubsidiaryCode2 != "" && x.SubsidiaryCode2.ToUpper().Contains("y-") && x.AssetCreatedDate>=FromDate && x.AssetCreatedDate <= ToDate
group new { x.SubsidiaryCode2, x.AssetCreatedDate,x.AssetCategoryID } by x into groupedList
select new AssetListTemplate
{
IsSelected = false,
SubsidiaryCode = groupedList.Key.SubsidiaryCode2,
AssetCreatedDate = groupedList.Key.AssetCreatedDate,
AssetCategory = groupedList.Key.AssetCategoryID
}
).OrderBy(x => x.SubsidiaryCode).ThenBy(y => y.AssetCreatedDate).ToList();
List = list;
foreach (var item in List)
{
var details = (from x in ctx.DataContext.AssetRegistryEntities
join y in ctx.DataContext.AssetCategoryEntities on x.AssetCategoryID equals y.AssetCategoryID
join z in ctx.DataContext.FixedAssetOtherInfoEntities on x.AssetCode equals z.AssetCode
where x.SubsidiaryCode2 == item.SubsidiaryCode
select new Details
{
AssetCode = x.AssetCode,
AssetCodeDesc = y.AssetCategoryDesc,
AssetDesc = x.AssetCodeDesc,
DepInCharge = z.DepartmentInCharge,
SerialNo = x.SerialNumber,
ModelNo = x.ModelNumber
}).ToList();
item.Details = details;
}
return List;
}
}
}
public class AssetListTemplate
{
public bool IsSelected { get; set; }
public string SubsidiaryCode { get; set; }
public DateTime? AssetCreatedDate { get; set; }
public string AssetCategory { get; set; }
public List<Details> Details { get; set; }
}
public class Details {
public string AssetCode { get; set; }
public string AssetCodeDesc { get; set; }
public string AssetDesc { get; set; }
public string DepInCharge { get; set; }
public string SerialNo { get; set; }
public string ModelNo { get; set; }
}
SQL Query:
SELECT Are_SubsidiaryCode2[SubsidiaryCode],Are_AssetCreatedDate[AssetCreatedDate],Are_AssetCategoryID[AssetCategory]
FROM E_AssetRegistry
WHERE Are_SubsidiaryCode2<>''
AND Are_SubsidiaryCode2 LIKE '%Y-%'
GROUP BY Are_SubsidiaryCode2
,Are_AssetCreatedDate
,Are_AssetCategoryID
ORDER BY AssetCreatedDate ASC

You don't seem to be using the grouping for any aggregate function , so you could make life simpler by just using distinct:
from x in ctx.DataContext.AssetRegistryEntities
where x.SubsidiaryCode2.Contains("y-") && x.AssetCreatedDate>=FromDate && x.AssetCreatedDate <= ToDate
select new AssetListTemplate
{
IsSelected = false,
SubsidiaryCode = groupedList.Key.SubsidiaryCode2,
AssetCreatedDate = groupedList.Key.AssetCreatedDate.Value.Date,
AssetCategory = groupedList.Key.AssetCategoryID
}
).Distinct().OrderBy(x => x.SubsidiaryCode).ThenBy(y => y.AssetCreatedDate).ToList();
Side note, you don't need to assign a list to a clas variable and also return it; I'd recommend just to return it. If you're looking to cache the results, make the class level var private, assign it and return it first time and just return it the second time (use the null-ness of the class level var to determine if the query has been run)
Expanding on the comment:
You don't need to store your data in a public property and also return it. Don't do this:
public class Whatever{
public string Name {get;set;}
public string GetName(){
var name = "John";
Name = name;
return name;
}
Typically we would either return it:
public class Whatever{
public string GetName(){
var name = MySlowDatabaseCallToCalculateAName();
return name;
}
//use it like:
var w = new Whatever();
var name = w.GetName();
Or we would store it:
public class Whatever{
public string Name {get;set;}
public void PopulateName(){
Name = MySlowDatabaseCallToCalculateAName();
}
//use it like
var w = new Whatever();
w.PopulateName();
var name = w.Name;
We might have something like a mix of the two if we were providing some sort of cache, like if the query is really slow and the data doesn't change often, but it is used a lot:
public class Whatever{
private string _name;
private DateTime _nameGeneratedAt = DateTime.MinValue;
public string GetName(){
//if it was more than a day since we generated the name, generate a new one
if(_nameGeneratedAt < DateTime.UtcNow.AddDays(-1)){
_name = MySlowDatabaseCallToCalculateAName();
_nameGeneratedAt = DateTime.UtcNow; //and don't do it again for at least a day
}
return _name;
}
This would mean that we only have to do the slow thing once a day, but generally in a method like "Get me a name/asset list/whatever" we wouldn't set a public property as well as return the thing; it's confusing for callers which one to use - they want the name; should they access the Name property or call GetName? If the property was called "MostRecentlyGeneratedName" and the method called "GenerateLatestName" it would make more sense - the caller can know they might call Generate..( first, and then they could use MostRecently.. - it's like a caching; the calling class can decide whether to get the latest, or reuse a recently generated one (but it does introduce the small headache of what happens if some other operation does a generate in the middle of the first operation using the property..)..
..but we probably wouldn't do this; instead we'd just provide the Generate..( method and if the caller wants to cache it and reuse the result, it can

Related

Not Able to assign value retrieved from SQL database to other variable

I'm retrieving a list from a SQL database with following code :
var matInList = db.MaterialInLists
.Where(m => m.PartName == "ABC" &&
m.Month == "July").ToList();
and I'm trying to get parameters of list with foreach loop like
foreach (var quanIn in matInList)
{
string quant = quanIn.Quantity;
string Date = quanIn.Date;
totalIn = totalIn + Int32.Parse(quanIn.Quantity);
}
Problem is in foreach loop only 3rd line gets the correct data i.e. totalIn.
"quanIn.Quantity" & "quanIn.Date" show correct value in debug mode but it is not able to assign its value to left hand side variable.
MaterialInList class is this:
public partial class MaterialInList
{
public int ID { get; set; }
public string PartName { get; set; }
public string Quantity { get; set; }
public string Date { get; set; }
}

LINQ return type and use in the other function

I am trying to get a result from LINQ query and pass it to different function to use there.
This is my first function that called from browser. Then, GetAddressByIncIDis called.
// GET: Hello/5
public void GetModel(int? incID)
{
var incAddress = GetAddressByIncID(incID);
var siteID = GetSiteIDbyAddress(incAddress);
LoadSite(siteID);
}
private ?returntype? GetAddressByIncID(int? incID)
{
var incAddress = (from address in db.Incs
where address.IncID == incID
select new
{
IncID = address.IncID,
Number = address.Number,
Direction = address.Direction,
StreetName = address.StreetName,
Zip = address.Zip,
City = city.CityDesc,
State = state.StateDesc
}).FirstOrDefault();
return incAddress;
}
At this point, I could get a query result that I wanted. I only needed one or null so I set FirstOrDefault().
Within this function, I am able to access values inside of incAddress, like incAddress.IncID, and I want to do this in other function when it passed to them. Because its type is Anonymous Type, I am not sure what return type should I use.
Once this is returned, I pass it to other function that called GetSiteIDbyAddress to find out SiteID. This result may be multiple.
private ?returntype? GetSiteIDbyAddress(string incAddress)
{
var searchaddress = (from address in db.Addresses
where PrimaryAddressNumber == incAddress.Number && Direction == incAddress.Direction && StreetName == incAddress.StreetName
select new
{
SiteID = address.SiteID
});
return searchaddress;
}
Would you please give me a suggestion on both first and second function's return type? I searched and tried (list, array, etc) but could not find a great solution for it.
EDIT:
With suggestions, I have edited my code to looks below. Now I am able to use values from IncAddress to pass to the other function. However, the other problem arise.
public class IncAddress
{
public int IncID { get; set; }
public string Number { get; set; }
.
.
}
public class IncSite
{
public int? SiteID { get; set; }
.
.
}
private IncAddress GetAddressByIncID(int incID)
{
var incAddress = (from address in db.Incs
where address.IncID == incID
select new IncAddress
{
IncID = address.IncID,
Number = address.Number,
Direction = address.Direction,
StreetName = address.StreetName,
Zip = address.Zip,
City = city.CityDesc,
State = state.StateDesc
}).FirstOrDefault();
return incAddress;
}
private IncSite GetSiteIDbyAddress(IncidentAddress incidentAddress)
{
var searchaddress = (from address in db.Addresses
where PrimaryAddressNumber == incAddress.Number
select new IncSite
{
SiteID = address.SiteID
});
return searchaddress;
}
On GetSiteIDbyAddress, I get an error says:
Cannot implicitly convert type 'System.Linq.IQueryable <
HelloWorld.Controllers.HelloController.IncSite > ' to
'HelloWorld.Controllers.HelloController.IncSite'. An explicit
conversion exists (are you missing a cast?)
I think it is because I did not put FirstOrDefault() because I expect multiple record will be found by this query. How can I complete this function to get a multiple results?
You can do this in two ways, either by creating a new class Let it be some Address with those selected values as fields and in this case return type will respect to Address(Address if you use FirstOrDefault, List if you use .ToList() and so on). Or you can specify the return type as dynamic in this case no other changes is needed. But I suggest you to create a Class if possible and avoid the usage of dynamic in this case, since
it is a lot slower and won't have the same compile time error checks
as a proper class
updates as per David's comment
Case 1 :
//definition should be something like this
public class Address
{
public string IncID { get; set; }
public string Number { get; set; }
public string Direction { get; set; }
public string StreetName { get; set; }
public string Zip { get; set; }
public string City { get; set; }
public string State { get; set; }
}
And the Method should be like this:
private Address GetAddressByIncID(int? incID)
{
var incAddress = (from address in db.Incs
where address.IncID == incID
select new Address
{
IncID = address.IncID,
Number = address.Number,
Direction = address.Direction,
StreetName = address.StreetName,
Zip = address.Zip,
City = city.CityDesc,
State = state.StateDesc
}).FirstOrDefault();
return incAddress;
}
Case 2
private dynamic GetAddressByIncID(int? incID)
{
var incAddress = // Query here
return incAddress;
}

Inner join not working when use equal with %, What an alternative way to use it like like %

I have Medals class, I call a service that return all Medal class fields except for the two fields ArabicName and ISOCode; Which I have to bring them from another table class "CountrysTBLs" , I made this join code:
The Medal class:
public class Medals {
public int bronze_count { get; set; }
public string country_name { get; set; }
public int gold_count { get; set; }
public string id { get; set; }
public int place { get; set; }
public int silver_count { get; set; }
public int total_count { get; set; }
public string ArabicName { get; set; } // this field not return by service
public string ISOCode { get; set; } // this field not return by service
}
The code:
var cntrs = db.CountrysTBLs.ToList();
List<Medals> objs = call_Service_Of_Medals_Without_ISOCode();
IEnumerable<Medals> query = from obj in objs
join cntry in cntrs on obj.country_name equals '%' + cntry.CommonName + '%'
select new Medals
{
ArabicName = cntry.ArabicName,
ISOCode = cntry.ISOCode,
country_name = obj.country_name,
place = obj.place,
gold_count = obj.gold_count,
silver_count = obj.silver_count,
bronze_count = obj.bronze_count,
total_count = obj.total_count
};
I get no result?!
How to fix that? Is there is any way to bring the two fields (ISOCode, ArabicName) without even use the inner join, and in same time best performance?
You want something like this to achieve LIKE functionality
List<Medals> medals = new List<Medals>();
var list = medals.Select(x => new
{
obj = x,
country = countries.FirstOrDefault(c => c.CommonName.Contains(x.country_name))
});
or something like this (if you want to just enrich each medal)
foreach (var medal in medals)
{
var country = countries.FirstOrDefault(c => c.CommonName.Contains(x.country_name));
medal.ISOCode = country.ISOCode;
medal.ArabicName = country.ArabicName;
}
Do note that this is not as performant as a Dictionary<string,Coutnry> of countries where the key is the country name, but as you need a LIKE comparison you would need to bring in a custom data structure such as Lucene index for fast lookups. But check first, if the lists are small enough, it probably won't be a problem. Otherwise, why not make the Medal.Country_Name and Country.Name the same? So you can do quick Dictionary (hashtable) lookups

Combine two lists and match values

Inside the "Distributions" variable is a key called "Deadline" which contains a date.
I would like to add "RealDeadline = i.Deadline,". All the other lines works fine, but I just cant find a way to add the last thing.
The match has to be made on AssignmentId which is the key for the whole combine. basically if the HandInData.Where(...) could just add the value of "Deadline" from "Distributions", that would do the trick..
var HandInData = db.Handins.ToList();
var Distributions = db.Distributes.ToList();
var AssignNames = HandInData.Where(a => Distributions.Any(x => x.AssignmentId == a.AssignmentId));
var StudentsHandedInDataFeed = AssignNames.Select(i => new {
*RealDeadline = i.Deadline, (this is not working..)*
Help = i.NeedHelp,
Done = i.Done,
AssName = i.Assignment.AssignmentName,
Student = i.Student.StudentName,
DeadlineInTimeformat = i.Assignment.AssignmentDeadline,
HandedInInTimeformat = i.HandedInDate,
Deadline = i.Assignment.AssignmentDeadline.ToString(),
HandedIn = i.HandedInDate.ToString()
});
public class Handin {
public int HandinId { get; set; }
public int StudentId { get; set; }
public int AssignmentId { get; set; }
public bool? Done { get; set; }
public bool? NeedHelp { get; set; }
[DisplayFormat(DataFormatString = "{0:dd/MM/yyyy}")]
public DateTime? HandedInDate { get; set; }
public virtual Student Student { get; set; }
public virtual Assignment Assignment { get; set; }
}
You need to join the two lists. You can do it with LINQ syntax like this:
var StudentsHandedInDatFeed =
from h in HandInData
join d in Distributions on h.AssignmentId equals d.AssignmentId
select new {
RealDeadline = d.Deadline,
Help = h.NeedHelp,
// etc
};
The join will only include values from HandInData where there is a matching value in Distributions, so this takes care of your Where(a => Distributions.Any(... code as well.

LINQ to Objects Performance - Huge dataset for long running process

I have a detail list (200,000 records) which was pulled from database and I need to find the locations for each detail and below is the code which is looping through the detail list and assigning location to the list. This loop is taking more than 15 minutes to execute but if don’t populate the Locations property then it takes less than a minute.
How can I optimize this code?
class Program
{
static void Main(string[] args)
{
List<Details> databaseDetailList = GetDetailsFromdatabase();
List<Location1> databaseLocation1List = GetLocations1Fromdatabase();
List<Location2> databaseLocation2List = GetLocations2Fromdatabase();
List<Details> detailList = new List<Details>();
foreach (var x in databaseDetailList)
{
detailList.Add(new Details
{
DetailId = x.DetailId,
Code = x.Code,
//If I comment out the Locations then it works faster
Locations = new LocationIfo {
Locations1 = databaseLocation1List
.Where(l=>l.DetailId == x.DetailId && l.Code == x.Code).ToList(),
Locations2 = databaseLocation2List
.Where(l => l.DetailId == x.DetailId && l.Code == x.Code).ToList()
}
});
}
}
private static List<Details> GetDetailsFromdatabase()
{
//This returns 200,000 records from database
return new List<Details>();
}
private static List<Location1> GetLocations1Fromdatabase()
{
//This returns 100,000 records from database
return new List<Location1>();
}
private static List<Location2> GetLocations2Fromdatabase()
{
//This returns 100,000 records from database
return new List<Location2>();
}
}
public class Details
{
public string DetailId { get; set; }
public string Code { get; set; }
public LocationIfo Locations { get; set; }
}
public class LocationIfo
{
public List<Location1> Locations1 { get; set; }
public List<Location2> Locations2 { get; set; }
}
public class Location1
{
public int LocationId { get; set; }
public string DetailId { get; set; }
public string Code { get; set; }
public string OtherProperty { get; set; }
}
public class Location2
{
public int LocationId { get; set; }
public string DetailId { get; set; }
public string Code { get; set; }
public string OtherProperty { get; set; }
}
What you're doing conceptually here is a Join. Using the proper operations will ensure that it executes much more effectively. Ideally you would even be doing the Join on the database side of things, rather than after pulling all of the data down into lists, but even if you do pull all of the data down, joining it in memory using Join will be much more efficient.
var query = from detail in databaseDetailList
join location1 in databaseLocation1List
on new { detail.DetailId, detail.Code }
equals new { location1.DetailId, location1.Code }
into locations1
join location2 in databaseLocation2List
on new { detail.DetailId, detail.Code }
equals new { location2.DetailId, location2.Code }
into locations2
select new Details
{
Code = detail.Code,
DetailId = detail.DetailId,
Locations = new LocationIfo
{
Locations1 = locations1.ToList(),
Locations2 = locations2.ToList(),
}
};
This is a very similar problem I faced a couple of days ago (in Ruby).
Finally the best solution I found was to convert the List(or Array) into a Dictionary(or Hash)
Create a Dictionary with your searching fields concatenated as a key and your list item as as value:
var dict1 = databaseLocation1List.ToDictionary(x => x.DetailId.ToString() + x.Code.ToString(), x);
And then the search in the Dictionary by key will be very fast:
Locations1 = dict1[x.DetailId.ToString() + x.Code.ToString()].ToList()
This is a bit hacky - but you could use lookups:
If it's the look-up portion of your code that is taking the longest - and you can't change the database code, as suggested with a Join - you could consider indexing your Location return-types in an in-memory Lookup.
Build a Lookup key, that's a combination of your two values - this will need to be specifically unique, dependent on your requirements.
DetailId-Code - "123123-343"
private static ILookup<int, Location1> GetLocations1Fromdatabase()
{
//This returns 100,000 records from database
return new List<Location2>()
.ToLookup(l => l.DetailId + "-" + l.Code);
}
Then:
Locations1 = databaseLocation1List[l.DetailId + "-" + l.Code].ToList()

Categories