Misbehaving LINQ joins - c#

I have a pretty large linq query driving a section of my system, and i am trying to add in a new feature, but the joins seem to be misbehaving.
The old query used to bring back a list of videos which i used to create a view model and only list each video once, with all of the metadata displayed, VisibleStaff, VisiblePlayers and VisibleTeams are all IEnumerable.
Since adding the par tthat drives VisibleStaff, any video with entries in VideosLinkings where the VideoInType flag is set to staff, displays once for each entry, rather than once, and giving me a list of staff members as metadata for VisibleStaff.
I think i am missing a grouping somewhere, but i have tried multiple groups in multiple places and cannot seem to get it right.
Does anyone have any idea where my joins have gone wrong and how i would return a single Video and multiple staff in each VideoModel?
Full Query
from video in Videos
where
video.ClubID == ClubId &&
(VideoFilter.Category == 0 || video.VideoCategoryID == VideoFilter.Category)
join userStaff in Database.Users on video.AddedByUserID.Value equals userStaff.UserID
join videoInTeams in VideoInTeams on video.VideoID equals videoInTeams.VideoID into teamsForVideo
join playerInVideo in Database.PlayersVideos on video.VideoID equals playerInVideo.VideoId into
playersForVideo
join soapVideoLink in Database.VideosLinkings on new {a = video.VideoID, b = VideoInType.SOAPNote}
equals new {a = soapVideoLink.VideoId, b = soapVideoLink.VideoInType} into soapVideoLinks
join staffVideoLink in Database.VideosLinkings on new {a = video.VideoID, b = VideoInType.Staff}
equals new {a = staffVideoLink.VideoId, b = staffVideoLink.VideoInType} into
staffVideoLinks
from svl in staffVideoLinks.DefaultIfEmpty()
join staff in Staff on svl.VideoInKeyId equals staff.StaffID into visibleStaff
let soapLinks = soapVideoLinks.Any(f => f.VideoInKeyId != -1)
let oldExtension =
video.H264Version == "Uploaded"
? ".mp4"
: (video.FlashVersion == "Uploaded" ? ".flv" : video.FileExtension)
where
VideoFilter.ShowSoapVideos || (VideoFilter.ShowSoapVideos == false && soapLinks == false)
orderby video.DateTimeUploaded descending
select new VideoModel
{
Video = video,
Category = video.VideoCategory,
Staff = userStaff.Staff,
ShowDeleteOption = VideoFilter.ShowDeleteOption,
VisibleTeams = teamsForVideo.Select(f => f.Team),
VisiblePlayers = playersForVideo.Select(f => f.Player),
Thumbnail =
video.ThumbnailURL != "" && video.ThumbnailURL != null
? video.ThumbnailURL
: "/Images/Videos/noimage.png",
IsNew = false,
IsMedicalVideo = soapLinks,
VisibleStaff = visibleStaff,
IsStaffVideo = staffVideoLinks.Any()
}
The New Lines
join staffVideoLink in Database.VideosLinkings on new {a = video.VideoID, b = VideoInType.Staff}
equals new {a = staffVideoLink.VideoId, b = staffVideoLink.VideoInType} into
staffVideoLinks
from svl in staffVideoLinks.DefaultIfEmpty()
join staff in Staff on svl.VideoInKeyId equals staff.StaffID into visibleStaff

Related

MVC Linq Invalid length parameter passed to the LEFT or SUBSTRING function

Good day. I have a Linq code that when run, shows this error
Invalid length parameter passed to the LEFT or SUBSTRING function.
But when I try to put all the same values into SQL, it works fine. Is there something wrong with my LINQ code?
THE OUTPUT IN SQL
OUTPUT
CODE
var list = (from R in db.vwEtracs_Receipt
join RI in db.vwEtracs_ReceiptItem on R.objid equals RI.parentid
join IA in db.vwEtracs_IncomeAccount on RI.acctid equals IA.objid
join M in db.vwtbl_Motor
on new { motor_no = R.remarks.Substring(5), operator_id = R.payerId }
equals new { motor_no = M.motor_no, operator_id = M.operator_id } into M_join
from M in M_join.DefaultIfEmpty()
join F in db.tbl_Franchise on R.objid equals F.or_id into F_join
from F in F_join.DefaultIfEmpty()
join B in db.tbl_Make on M.brand_id equals B.id
where
IA.objid == "FTFA00000238" &&
R.voidId == null &&
R.remarks != null
orderby R.txndate descending
select new PayedViewModel
{
application_no = F.application_no,
remarks = R.remarks,
serialno = R.serialno,
payername = R.payername,
payeraddress = R.payeraddress,
motor_no = M.motor_no,
chassis_no = M.chassis_no,
plate_no = M.plate_no,
brand_name = B.Make,
motor_id = M.motor_id,
year = R.txndate.Value.Year.ToString(),
mtop = F.mtop,
franchise_id = F.franchise_id
}
).Distinct();
Usually the error shows in this line, motor_no = R.remarks.Substring(5) because when I change it to 4, the code runs smoothly. I tried manually checking all the data in the db but found nothing suspicious nor anything that will give it negative value.
At this point I don't know what is wrong with my code or db. Thank you.
It may happen because your database have string which it's length less than 5
make sure that all R.remarks.length are more than 5
may be you need to do check or something like this :
R.remarks.Length > 5 ? R.remarks.Substring(5) : R.remarks

horrible and big LINQ statement optimisation

I have to to a rather large request to a database to fetch a bunch of data, it's however taking a noticeable time to run. is there some way to increase the performance on this? preemptive apologies for the ugly code (I did have a version that segmented this into multiple smaller functions but that was even slower)
from contact in _database.OBJECTCONTACT
where contact.OBJECTCONTACTOWNER.Any(o => o.OBJECTID == id && o.OBJECTTYPE == type) && contact.ACTIVE >= 1 && CheckViewAccess(contact)
group contact by (contact.OBJECTCONTACTPROJECT.Any() ? contact.OBJECTCONTACTPROJECT.First().OBJECTPROJECT.PROJECTNAME : "General") into projectGroup
select new ProjectViewModel()
{
ProjectName = projectGroup.Key,
ContactGroups = (from g in _database.OBJECTGROUP
where g.GROUPTYPE == "CONTACT" && ContactsModule.CheckUserRole("View", g.OBJECTTYPE, g.GROUPNAME)
select new ContactGroupViewModel()
{
CanEdit = ContactsModule.CheckUserRole("Edit", g.OBJECTTYPE, g.GROUPNAME),
GroupId = g.OBJECTGROUPID,
GroupName = g.GROUPNAME,
Contacts = (from c in projectGroup
join l in _database.OBJECTCONTACTLOCATION on c.OBJECTCONTACTLOCATIONID equals l.OBJECTCONTACTLOCATIONID into lgrp from loc in lgrp.DefaultIfEmpty(null)
orderby c.NAME
select new ContactViewModel()
{
Id = (int)c.OBJECTCONTACTID,
Name = c.NAME,
Description = c.DESCRIPTION,
ContactInformation = CreateContactInfoViewmodels(c),
Owners = c.OBJECTCONTACTOWNER.Where(owner => owner.OBJECTTYPE == "AIRPORT")
.Select(owner => ContactOwnerViewModel.FromOwnerId(owner.OBJECTID, owner.OBJECTTYPE)).ToList(),
Projects = c.OBJECTCONTACTPROJECT.Select(proj => proj.OBJECTPROJECT).ToList(),
Typename = GetTypeName(c),
TypeId = c.OBJECTCONTACTTYPEID ?? 0,
ContactGroupId = c.OBJECTGROUPID,
ContactGroup = g.GROUPNAME,
Editable = CheckAccessBool("EDIT", c),
Location = loc != null ? new LocationViewModel()
{
Address = loc.ADDRESS,
GoogleMapLink = loc.GMAPADDRESS,
LocationId = loc.OBJECTCONTACTLOCATIONID,
LatLon = Tuple.Create(loc.LATITUDE, loc.LONGITUDE)
} : null,
}).ToList()
}).ToList()
}).ToList();
I think I should be able to use joins to move the entire DB fetch code to the top (theoretically improving perfomance) but I am having trouble finding the syntax which would suit my needs
Thanks everyone for coming with suggestions. I am in a situation where I'm not able to do much with the database itself so I'm making the best of what I have. my hands a bit tied in regards to the tools at my disposal (also fairly old codebase I think it's EF 5 or something like that)
this version moves the DB transaction to the top (so that is fewer fetches) and does a lot of data manipulation at the bottom.
// general object is created above
var res = (from contact in _database.OBJECTCONTACT.AsEnumerable() // as enumerable used to allow for defaultifempty in join (minor damage to performance)
join oGroup in _database.OBJECTGROUP on contact.OBJECTGROUPID equals oGroup.OBJECTGROUPID into og from objectGroup in og.DefaultIfEmpty(defaultValue: general)
where contact.OBJECTCONTACTOWNER.Any(o => o.OBJECTTYPE == type && o.OBJECTID == id)
// ReSharper disable once PossibleNullReferenceException (it's taken care of by check using .any() )
group new {contact, objectGroup } by (contact.OBJECTCONTACTPROJECT.Any() ? contact.OBJECTCONTACTPROJECT.FirstOrDefault().OBJECTPROJECT.PROJECTNAME : "General") into pGroup
orderby pGroup.Key == "General" ? pGroup.Key : "􏿽" descending
select new ProjectViewModel()
{
ProjectName = pGroup.Key,
ProjectId = pGroup.FirstOrDefault() != null ? (pGroup.FirstOrDefault().contact.OBJECTCONTACTPROJECT.FirstOrDefault() != null ? pGroup.FirstOrDefault().contact.OBJECTCONTACTPROJECT.FirstOrDefault().OBJECTPROJECTID : -1) : -1,
ContactGroups = (from c in pGroup
group c by c.objectGroup into grp
let canEdit = ContactsModule.CheckUserRole("EDIT", grp.Key.OBJECTTYPE, grp.Key.GROUPNAME)
orderby grp.Key.SORTORDER descending
select new ContactGroupViewModel()
{
GroupName = grp.Key.GROUPNAME,
GroupId = grp.Key.OBJECTGROUPID,
CanEdit = canEdit,
Contacts = grp.Select(item => new ContactViewModel()
{
Id = (int)item.contact.OBJECTCONTACTID,
Name = item.contact.NAME,
Description = item.contact.DESCRIPTION,
Editable = canEdit,
ContactInformation = item.contact.OBJECTCONTACTNUMBER.OrderByDescending(num => num.ISMAININFO).Select(num => new ContactInfoViewmodel()
{
Data = num.NUMBERDATA,
IsMain = num.ISMAININFO > 0,
Type = num.OBJECTCONTACTNUMBERTYPE.NAME
}).ToList()
}).ToList()
}).ToList()
}).ToList();
this seems to (on average) take about a 4th of the time the original query needed (still a noticeable time due to the size of database but within acceptable limits)

C# LINQ join 2 tables with where clause

I am trying to join 2 tables while filtering with where clause. The data looks like this:
Category
Name Selected
A 0
B 1
C 0
SubCategory
Name ParentCategory Selected
s1 A 0
s2 B 1
s3 B 0
Expected results:
ParentCatName SubCatName SubCatSelected
B s2 1
B s3 0
Actual results:
ParentCatName SubCatName SubCatSelected
B s2 1
B s3 1 <-- should be 0
The code I am using is this:
IEnumerable<Category> result =
from s in subcategories
join c in categories
on s.Parent equals c.Name into t
from r in t.DefaultIfEmpty()
where r == null ? false : r.Selected == true
select new Category
{
Name = s.Name,
Parent = (r == null ? string.Empty : r.Name),
Selected = r.Selected
};
EDIT: Something that helped me get clarity was to temporarily rewrite this to see the resulting data structures...
var result =
from s in subcategories
join c in categories
on s.Parent equals c.Name into t
from r in t.DefaultIfEmpty()
select new
{
s, r
};
Then I came up with the answer to the filtering of selected categories. See my answer below..
Don't overcomplicate the things. What you are trying to achieve is to filter subcategories by the selected categories. You can get the desired result with the following simple query
var result = from s in subcategories
join c in categories on s.Parent equals c.Name
where c.Selected
select s;
It looks like you're setting it wrong. If r == null then you're setting it to false, otherwise you're setting it to true here: r.Select == true.
Just by reading your query it looks like you may not need that where clause at all.
You probably want something like this:
IEnumerable<Category> result =
from s in subcategories
join c in categories
on s.Parent equals c.Name into t
from r in t.DefaultIfEmpty()
select new Category
{
Name = s.Name,
Parent = (r == null ? string.Empty : r.Name),
Selected = r.Selected
};
Or if you need to do the null check then do this:
IEnumerable<Category> result =
from s in subcategories
join c in categories
on s.Parent equals c.Name into t
from r in t.DefaultIfEmpty()
where r != null //I added the null check here
select new Category
{
Name = s.Name,
Parent = (r.Name), //I removed the null check here
Selected = r.Selected
};
I think, your t value inculudes categoies list. t must be include subcategory list and then you can take selected value of subcategory. So you always get selected value as 1 pls try this:
IEnumerable<SubCategory> result =
from c in categories
join s in subcategories
on c.Name equals s.Parent into t
from r in t.DefaultIfEmpty()
where r == null ? false : r.Selected == true
select new SubCategory
{
Name = s.Name,
Parent = (r == null ? string.Empty : r.Name),
Selected = r.Selected
};
OBS:I'm not try this now. But I think works.
Okay. So I did some more looking and came up with this...
IEnumerable<Category> result =
from s in subcategories
join c in categories.Where(f => f.Selected)
on s.Parent equals c.Name into t
from r in t.DefaultIfEmpty()
where r == null ? false : true
select new Category
{
Name = s.Name,
Parent = s.Name,
Selected = s.Selected,
};
To filter join to only selected parent category, I have added lambda expression right to that data.

LINQ to SQL Right Outer Join

I have a SQL Query that I'm having trouble converting to LINQ query:
SELECT DISTINCT Nodes.NodeName, NodeConfig.IPAddresses, NodeConfig.InSCOM, NodeConfig.InOrion, NodeConfig.OrionCustomerName, NodeConfig.OrionApplication, NodeConfig.NodeID
FROM Tags INNER JOIN
TagToNode ON Tags.TagID = TagToNode.TagID RIGHT OUTER JOIN
NodeConfig INNER JOIN
Nodes ON NodeConfig.NodeID = Nodes.NodeID ON TagToNode.NodeID = NodeConfig.NodeID
WHERE (NodeConfig.Session = '7/3/2014 1:46:33 PM') AND (NodeConfig.InSCOM = 0)
That returns 1076 rows.
I tried to write the LINQ equivalent:
var list1 = (from t in mldb.Tags
join tn in mldb.TagToNodes on t.TagID equals tn.TagID into tagJoin
from tj in tagJoin.DefaultIfEmpty()
join nc in mldb.NodeConfigs on tj.NodeID equals nc.NodeID
join n in mldb.Nodes on nc.NodeID equals n.NodeID
where (nc.Session == #"7/3/2014 1:46:33 PM") && (nc.InSCOM == 0)
select new { Customer = nc.OrionCustomerName, DeviceName = n.NodeName, DeviceType = nc.OrionApplication, IPAddress = nc.IPAddresses, NodeID = n.NodeID }).Distinct().ToList();
That returns 183 rows.
I have tried converting the query to inner joins as suggested by some when I searched for solutions on this site. The original query implements a SQL "RIGHT OUTER JOIN" which from what I've read left/right isn't supported in LINQ but joins can be done.
The tables that I'm pulling from have primary keys as follows:
[DataServiceKey(new string[] { "NodeID", "TagID" })]
public partial class TagToNode { }
[DataServiceKey(new string[] { "NodeID" })]
public partial class Node { }
[DataServiceKey(new string[] { "TagID" })]
public partial class Tag { }
[DataServiceKey(new string[] { "ConfigID" })]
public partial class NodeConfig { }
The relationship is that Nodes have many NodeConfigs, and many Nodes are Tagged with many tags.
Can someone help me with the query logic?
OK so I took a step back and restructured the query. I created an inner join first, and then left-joined that to the main query.
I have a list of computers in a data base that are "tagged" in my application. I wanted to be able to search the tag names in the database via the many-to-many relationship between a tag and a device, where I have a join table in the middle named "TagToNode".
The distinct selection just weeds out the dupes in the end, the idea is to get ALL of the computers (nodes) even if they're not tagged with anything.
LINQ
var tags = (from tn in mldb.TagToNodes
join t in mldb.Tags on tn.TagID equals t.TagID
select new { tn.TagID, tn.NodeID, t.TagName, t.AssocUser });
return (from nc in mldb.NodeConfigs
join n in mldb.Nodes on nc.NodeID equals n.NodeID
join t in tags on n.NodeID equals t.NodeID into nj
from tg in nj.DefaultIfEmpty()
where nc.Session == sc.SessionName && n.NodeActive == 1 && ((tg.TagName.Contains(sc.SearchTerm) && (tg.AssocUser.Contains(windowsId) || (tg.AssocUser == null || tg.AssocUser == ""))) || (n.NodeName.Contains(sc.SearchTerm)) || (nc.OrionCustomerName.Contains(sc.SearchTerm)) || (nc.IPAddresses.Contains(sc.SearchTerm)))
select new NodeInfo { Customer = nc.OrionCustomerName, DeviceName = n.NodeName, DeviceType = nc.OrionApplication, IPAddress = nc.IPAddresses, NodeID = n.NodeID }).Distinct().ToList();

More efficient way of loading children of entity objects in linq to entity query

I have a rather complex linq to entity query that I'm performing, in the end, I have a result set. I loop through that result set, build business objects and return that list of business objects. it's pretty quick, the problem is that 2 of the child properties are complex objects with their own child objects. for every business object in my loop, I then have to make 2 DB calls to fill its child object. Those 2 calls slow down the overall process, is there a better way to do this? noob to EF here. (EF 4,SQL Server 2008,c#)
Get a result set:
var newresult = from r in result // result is another complex query
join subedit in
(from sa in context.Security_Access
join g in context.Security_UserGroup on sa.EntityID equals g.GroupID
where (sa.PrivledgeID == xx) && g.UserID == userId
select new { user = g.UserID, linkid = sa.LinkID }).Distinct() on new { aid = r.AssetId } equals new { aid = subedit.linkid } into theSubEdit
from subEditAccess in theSubEdit.DefaultIfEmpty()
join subdownload in
(from sa in context.Security_Access
join g in context.Security_UserGroup on sa.EntityID equals g.GroupID
where (sa.PrivledgeID == xx|| sa.PrivledgeID == yy) && g.UserID == userId
select new { user = g.UserID, linkid = sa.LinkID }).Distinct() on new { aid = r.AssetId } equals new { aid = subdownload.linkid } into theSubDownload
from subDownloadAccess in theSubDownload.DefaultIfEmpty()
join subView in
(from sa in context.Security_Access
join g in context.Security_UserGroup on sa.EntityID equals g.GroupID
where (sa.PrivledgeID == xx|| sa.PrivledgeID == yy|| sa.PrivledgeID == 101) && g.UserID == userId
select new { user = g.UserID, linkid = sa.LinkID }).Distinct() on new { aid = r.AssetId } equals new { aid = subView.linkid } into theSubView
from subViewAccess in theSubView.DefaultIfEmpty()
select new { r, EditAccess = (int?)subEditAccess.user, DownloadAccess = (int?)subDownloadAccess.user, ViewAccess = (int?)subViewAccess.user };
I then loop through that result set:
foreach (var asset in newresult)
{
// and build a new business object, set its properties
BoAsset boAsset = new BoAsset();
boAsset.HasEditRights = (asset.EditAccess > 0);
boAsset.HasDownloadRights = (asset.DownloadAccess > 0);
boAsset.HasViewRights = (asset.ViewAccess > 0);
boAsset.Description = asset.r.Description;
boAsset.DetailedDescription = asset.r.DetailedDescription;
boAsset.Keywords = asset.r.Keywords;
boAsset.Notes = asset.r.Notes;
boAsset.Photographer = asset.r.Photographer;
boAsset.PhotographerEmail = asset.r.PhotographerEmail;
boAsset.Notes = asset.r.Notes;
boAsset.Author = asset.r.Author;
// these 2 properties i've commented out are
// complex objects/entities, setting them the way I am
// requires me to call 2 separate methods which make 2 DB trips
// per business object.
//boAsset.Domains = GetAssetDomains(asset.r.AssetId);
//boAsset.DomainEntries = GetAssetCustomDomains(asset.r.AssetId);
myListofObjects.Add(boAsset);
}
return myListofObjects;
Is there a better way?
Just add this .Include("Domains").Include("DomainEntries") to your Linq in in context.Security_Access That should get rows from those tables all in one go.
So your "inner" queries would look like:
from sa in context.Security_Access.Include("Domains").Include("DomainEntries")
join g in context.Security_UserGroup on sa.EntityID equals g.GroupID
where (sa.PrivledgeID == xx) && g.UserID == userId
select new { ...
Here is the documentation from MS: http://msdn.microsoft.com/en-us/library/bb738708.aspx
If you want to improve your performance use compile queries !
You can check the example here.
static readonly Func<AdventureWorksEntities, Decimal,
IQueryable<SalesOrderHeader>> s_compiledQuery2 =
CompiledQuery.Compile<AdventureWorksEntities, Decimal, IQueryable<SalesOrderHeader>>((ctx, total) =>
from order in ctx.SalesOrderHeaders.Include("Orders") where order.TotalDue >= total select order);
MSDN
AND
You can Introduce Include suppose to select all the employees along with their departments . If you have a navigational property, you won't need a join at all. You can use Include like this:
List<Employee> employeesWithDepartments = CreateObjectSet<Employee>().
Include(e => e.Department).
ToList();

Categories