We're currently migrating our production platform to Azure, and as such I need to move over all of our support tools. Previously, we relied heavily on data adapters and stored procedures, and many of these stored procedures performed cross-database joins.
With our migration to Azure, none of these cross-database joins are operational. I tried moving our data adapters to Entity Framework, but I cannot seem to make them work. Instead, I get an error stating that cross-context joins are not allowed. Many of these queries rely on data from multiple databases, so I'm just trying to figure out what the best method of approach is to get this operational.
I've looked at several of the other questions asking for similar solutions, but none of them seem terribly applicable to my solution.
For example, here's one of the more simple queries in SQL:
USE CustomerDB1234
SELECT DISTINCT u.[UserID]
,u.[UserLogin]
,u.[UserPhoneNumber]
,u.[UserPasswordHash]
, ISNULL(gl.[gl_login_name],'* no global login ID') AS [gl_login_name]
,gl.[gl_password_hash]
,gl.[gl_GUID]
,gl.[gl_Email_Validated]
,u.[usr_unit_set_id]
,oob.[oob_org_id]
FROM [dbo].[User] u WITH (NOLOCK)
LEFT JOIN [dbo].[OrganizationObjectBridge] oob WITH (NOLOCK) ON oob.[oob_object_type_id] = 9 AND oob.[oob_object_id]= u.UserID
LEFT JOIN [MainServer].[MainDb].[dbo].[GlobalLoginCustomerBridge] glcb WITH (NOLOCK) ON glcb.[glcbr_user_id] = u.UserID
AND glcb.[glcbr_customer_id] = dbo.efnGetCustomerID()
LEFT JOIN [MainServer].[MainDb].[dbo].[GlobalLogin] gl WITH (NOLOCK) ON gl.[gl_id] = glcb.[glcbr_gl_id]
WHERE ([UserID] = #userID OR #userID IS NULL)
AND ([UserDisabled] = #isDisabled OR #isDisabled IS NULL)
ORDER BY [gl_login_name]
And in Linq, it would look similar to:
List<User2> userList = new List<User2>();
using (var e = new eContext())
using (var context = new CustomerContext(CustomerID))
{
var databaseConnections = e.DatabaseConnectionStrings;
var customer = e.Customers.Select(n => new
{
ID = n.CustomerID,
Name = n.CustomerName,
Email = n.CustomerEmail,
Website = n.CustomerWWW,
Logo = n.CustomerLogo,
DatabaseConnectionName = databaseConnections.FirstOrDefault(d => d.DatabaseConnectionID == n.DatabaseConnectionID).DatabaseConnectionName,
DatabaseConnectionString = databaseConnections.FirstOrDefault(d => d.DatabaseConnectionID == n.DatabaseConnectionID).DatabaseConnectionString1,
AccountNumber = n.CustomerAcctNumber
}).FirstOrDefault(n => n.ID == CustomerID);
userList = context.Users
.Join(e.GlobalLoginCustomerBridges,
u => u.UserID,
glcb => glcb.glcbr_user_id,
(u, glcb) => new { u, glcb })
.Where(n => n.glcb.glcbr_customer_id == CustomerID)
.Join(e.GlobalLogins,
glcb => glcb.glcb.glcbr_gl_id,
gl => gl.gl_id,
(glcb, gl) => new { glcb, gl })
.Join(context.OrganizationObjectBridges,
glcb => glcb.glcb.u.UserID,
oob => oob.oob_object_id,
(glcb,oob) => new {glcb, oob})
.Where(n=>n.oob.oob_object_type_id == 9)
.Select(n => new
{
ID = n.glcb.glcb.u.UserID,
GlobalLogin = n.glcb.gl.gl_login_name,
FirstName = n.glcb.glcb.u.UserFirstName,
MiddleName = n.glcb.glcb.u.UserMiddleName,
LastName = n.glcb.glcb.u.UserLastName,
GUID = n.glcb.gl.gl_GUID,
UserID = n.glcb.gl.gl_id,
HasSHA256Hash = n.glcb.gl.gl_password_hash_sha256 == null,
Customer = customer,
Organization = context.Organizations
.Select(o => new
{
ID = o.org_id,
Name = o.org_name,
ParentID = o.org_parent_id,
ExternalID = o.org_external_id,
Default = o.org_default,
Logo = o.org_logo,
URL = o.org_url,
PeerGroupID = o.org_peer_grp_id,
ExternalInfo = o.org_external_info
}).Cast<Organization2>().FirstOrDefault(o=>o.ID == n.oob.oob_org_id)
}).Cast<User2>().ToList();
}
Two approaches:
Perform the joins in the application in memory. Depending on the query this is more or less efficient. Also, the code changes required range from tiny to terrible.
Merge databases together. Having many databases without a physical reason for that is an anti-pattern. Databases are not logical units of application modularization. Document modularization through schemas or table name prefixes. Databases are physical units of deployment.
Third approach: Run a SQL Server in a VM.
By using the information posted by the other answer, I came up with a solution that appears to work. Granted, it doesn't appear nearly as efficient or as concise as cross-database joins, but it gets the job done.
using (var context = new CustomerContext(CustomerID))
using (var e = new eContext())
{
var globalUserList = e.GlobalLoginCustomerBridges
.Join(e.GlobalLogins,
glcb => glcb.glcbr_gl_id,
gl => gl.gl_id,
(glcb, gl) => new { glcb, gl })
.Where(n => n.glcb.glcbr_customer_id == CustomerID)
.Select(n => new User2
{
ID = (int)n.glcb.glcbr_user_id,
GlobalLogin = n.gl.gl_login_name,
GUID = n.gl.gl_GUID
}).ToList();
var customer = e.Customers
.Join(e.DatabaseConnectionStrings,
c => c.DatabaseConnectionID,
d => d.DatabaseConnectionID,
(c, d) => new { c, d })
.Select(n => new Customer2
{
ID = n.c.CustomerID,
Name = n.c.CustomerName,
DatabaseConnectionName = n.d.DatabaseConnectionName,
DatabaseConnectionString = n.d.DatabaseConnectionString1,
GUID = n.c.cust_guid,
}).ToList().FirstOrDefault(n => n.ID == CustomerID);
var orgs = context.Organizations
.Select(o => new Organization2
{
ID = o.org_id,
Name = o.org_name,
}).ToList();
var users = context.Users
.Select(n => new User2
{
ID = n.UserID,
FirstName = n.UserFirstName,
}).ToList();
var userList = users
.Join(globalUserList,
u => u.ID,
gl => gl.ID,
(u, gl) => new { u, gl })
.Join(context.OrganizationObjectBridges,
u => u.u.ID,
oob => oob.oob_object_id,
(u, oob) => new { u, oob })
.Where(o => o.oob.oob_object_type_id == 9)
.Select(n => new User2
{
ID = n.u.u.ID,
GlobalLogin = n.u.gl.GlobalLogin,
FirstName = n.u.u.FirstName,
GUID = n.u.gl.GUID,
Customer = customer,
Organization = orgs.FirstOrDefault(o => o.ID == n.oob.oob_org_id)
}).Where(n => !isDisabled != null && n.Disabled == isDisabled).ToList();
return userList;
}
Related
I receive this error, and I tried a lot to solve it, but I receive other errors, is there a solution?
using (var contextDb1 = new Db1Context(System.Web.HttpContext.Current.Session["DB1ConnectionString"].ToString(), false))
{
using (var contextDb2 = new Db2Context(System.Web.HttpContext.Current.Session["DB2ConnectionString"].ToString(), false))
{
var messagesList = contextDb2.Messages
.Select(m => new MessagesViewModel
{
UserName = contextDb1.UsersInfo.FirstOrDefault(u=>u.Id == m.UserId).UserName,
MessageId = m.MessageId,
MessageText = m.MessageText,
DateTime = m.DateTime
})
.ToList();
return messagesList;
}
}
You cannot query two different databases via the same LINQ Query. But you can use intermediate result to execute two queries to databases and then combine result.
using (var contextDb1 = new Db1Context(System.Web.HttpContext.Current.Session["DB1ConnectionString"].ToString(), false))
using (var contextDb2 = new Db2Context(System.Web.HttpContext.Current.Session["DB2ConnectionString"].ToString(), false))
{
var rawMessages = contextDb2.Messages
.Select(m => new
{
m.UserId
m.MessageId,
m.MessageText,
m.DateTime
})
.ToList();
var userIds = rawMessages.Select(x => xu.UserId);
var usersInfo = contextDb1.UsersInfo.Where(u => userIds.Contains(u.Id))
.Select(u => new
{
UserId = u.Id,
UserName = u.UserName
});
var messageQuery =
from m in rawMessages
join u in usersInfo.AsEnumerable() on m.UserId equals u.UserId into gj
from u in gj.DefaultIfEmpty()
select new MessagesViewModel
{
UserName = u?.UserName,
MessageId = m.MessageId,
MessageText = m.MessageText,
DateTime = m.DateTime
};
var messagesList = messageQuery.ToList();
return messagesList;
}
I'm having trouble working with Entity Framework and PostgreSQL, does anybody know how to join two tables and use the second table as a where clause?
The select I want to do in Entity Framework would be in SQL:
SELECT ai.id, ai.title, ai.description, ai.coverimageurl
FROM app_information ai
INNER JOIN app_languages al on al.id = ai.languageid
WHERE al.languagecode = 'es'
Currently I have this
appInformationToReturn = context.app_information
.Join(context.app_language, ai => ai.languageid,
al => al.id, (ai, al) => new AppInformation()
{
id = ai.id,
title = ai.title,
description = ai.description,
coverimageurl = ai.coverimageurl
})
.Where()
.FirstOrDefault();
I don't know how to build the where clause.
Like this:
appInformationToReturn = context.app_information
.Join(context.app_language, ai => ai.languageid,
al => al.id, (ai, al) => new
{
id = ai.id,
title = ai.title,
description = ai.description,
coverimageurl = ai.coverimageurl,
lang = al.languagecode
}).Where(x=>x.lang == "es")
.Select(x=> new AppInformation()
{
id = x.id,
title = x.title,
description = x.description,
coverimageurl = x.coverimageurl
})
.FirstOrDefault();
try this:
var item = (
from ai in context.app_information
join al in context.app_language on ai.languageid equals al.id
where (al.languagecode == "es")
select new AppInformation
{
id = ai.id,
title = ai.title,
description = ai.description,
coverimageurl = ai.coverimageurl
}).FirstOrDefault();
or try shorter
var item = context.app_information
.Where(ai => ai.app_language.languagecode == "es")
.Select(ai => new AppInformation
{
id = ai.id,
title = ai.title,
description = ai.description,
coverimageurl = ai.coverimageurl
})
.FirstOrDefault();
I have a linq query that returns users/employees with their corresponding supervisors.
List<OrgChartViewModel> OrgChart = new List<OrgChartViewModel>();
var userlist = (from u in users
select new OrgChartViewModel
{
id = u?.id.ToString(),
pid = u?.pid.ToString(),
name = u?.name,
title = u?.title,
img = u?.img
}).OrderBy(x => x.name).ToList();
OrgChart.AddRange(userlist);
return Json(OrgChart.DistinctBy(x => x.id));
However, I need a query that returns the supervisor and their children along with their sub-children without having a sub-array/nested array.
What I have tried :
List<OrgChartViewModel> OrgChart = new List<OrgChartViewModel>();
var userlist = (from u in users
select new OrgChartViewModel
{
id = u?.id.ToString(),
pid = u?.pid.ToString(),
name = u?.name,
title = u?.title,
img = u?.img
}).OrderBy(x => x.name).ToList();
OrgChart.AddRange(userlist);
foreach (var ul in userlist)
{
var userlist2 = (from u in users
select new OrgChartViewModel
{
id = u?.id.ToString(),
pid = u?.pid.ToString(),
name = u?.name,
title = u?.title,
img = u?.img
}).Where(x => x.pid == ul.id).OrderBy(x => x.name).ToList();
OrgChart.AddRange(userlist2);
}
return Json(OrgChart.DistinctBy(x => x.id));
But this only returns the first layer of sub-children. What I want to achieve is to return unlimited layers of sub-children without having sub-arrays.
I'm getting log of last logged users and getting rid of duplicate
var log = Database.GetCollection<LoggerMongoDocument>(Consts.MongoDb.LoggerCollection).AsQueryable()
.Where(d => d.Description == "OK")
.OrderByDescending(s => s.Date).ToList();
var Users = log.GroupBy(i => i.UserAuthName).Select(group => group.First());
Then I've searched collection in mongodb to find proper users using foreach
var ListOfActivity = new List<MongoCursor<UserAuth>>();
foreach (var item in Users)
{
ListOfActivity.Add(Database.GetCollection(Consts.MongoDb.UserAuth).FindAs<UserAuth>(Query.EQ("Email", item.UserAuthName))
.SetFields(f => f.Roles, f => f.Id, f => f.UserName, f => f.Email));
}
After that I want to assign all users to a variable, but I can only assign one user using select
var accounts = ListOfActivity.First().Select(x => new AccountExtended
{
Id = x.Id,
Email = x.Email,
Roles = x.Roles,
CreatedDate = x.CreatedDate,
UserName = x.UserName,
});
How can I assign all users to a variable accounts?
I have a query that is currently far too slow.
I am trying to search a Code (a string) on the main page that will bring the user the relevant info.
Eg. The user can search a code from the main page and this will search for the code in Job, Work Phase, Wbs, Work Element, EA, Jobcard and Estimate and return the relevant info.
I make a number of trips to the database to collect the data i need when I believe it can be done in just one.
I have a number of tables that are all linked:
Contracts, Jobs, WorkPhases, Wbss, Engineering Activities, Jobcards and Estimates.
Contracts have a list of Jobs,
Jobs have a list of Workphases,
Workphases have a list of Wbss etc
Is there a quicker way to do this?
public Result Handle(Query query)
{
query.Code = query.Code ?? string.Empty;
var result = new Result();
//result.SetParametersFromPagedQuery(query);
result.Items = new List<Item>();
if (query.SearchPerformed)
{
var contracts = _db.Contracts.AsEnumerable().Where(x => x.Code == query.Code);
result.Items = result.Items.Concat(contracts.Select(x => new Item()
{
Code = x.Code,
Id = x.Id,
Name = x.Name,
Type = MainPageSearchEnum.Contract,
ContractName = x.Name,
Url = string.Format("Admin/Contract/Edit/{0}", x.Id)
})).ToList();
var jobs = _db.Jobs.AsEnumerable().Where(x => x.Code == query.Code);
result.Items = result.Items.Concat(jobs.Select(x => new Item()
{
Code = x.Code,
Id = x.Id,
Name = x.Name,
ContractName = x.Contract.Name,
Type = MainPageSearchEnum.Job,
Url = string.Format("Admin/Job/Edit/{0}", x.Id)
})).ToList();
//var workPhases = _db.WorkPhases.AsEnumerable().Where(x => x.ContractPhase.Code.ToLower() == query.Code.ToLower());
var workPhases = _db.WorkPhases.AsEnumerable().Where(x => x.ContractPhase.Code == query.Code);
result.Items = result.Items.Concat(workPhases.Select(x => new Item()
{
Code = x.ContractPhase.Code,
Id = x.Id,
Name = x.ContractPhase.Name,
Type = MainPageSearchEnum.WorkPhase,
Url = string.Format("Admin/WorkPhase/Edit/{0}", x.Id)
})).ToList();
var wbss = _db.WBSs.AsEnumerable().Where(x => x.Code == query.Code);
result.Items = result.Items.Concat(wbss.Select(x => new Item()
{
Code = x.Code,
Id = x.Id,
Name = x.Name,
Type = MainPageSearchEnum.WBS,
Url = string.Format("Admin/WBS/Edit/{0}", x.Id)
})).ToList();
var eas = _db.EngineeringActivities.AsEnumerable().Where(x => x.Code == query.Code);
result.Items = result.Items.Concat(eas.Select(x => new Item()
{
Code = x.Code,
Id = x.Id,
Name = x.Name,
Type = MainPageSearchEnum.EA,
Url = string.Format("Admin/EngineeringActivity/Edit/{0}", x.Id)
})).ToList();
var jcs = _db.Jobcards.AsEnumerable().Where(x => x.Code == query.Code);
result.Items = result.Items.Concat(jcs.Select(x => new Item()
{
Code = x.Code,
Id = x.Id,
Name = x.Name,
Type = MainPageSearchEnum.EA,
Url = string.Format("Admin/JobCard/Edit/{0}", x.Id)
})).ToList();
var estimates = _db.Estimates.AsEnumerable().Where(x => x.Code == query.Code);
result.Items = result.Items.Concat(estimates.Select(x => new Item()
{
Code = x.Code,
Id = x.Id,
Name = x.Name,
Type = MainPageSearchEnum.Estimate,
Url = string.Format("Estimation/Estimate/Edit/{0}", x.Id)
})).ToList();
}
return result;
}
Disclaimer: I'm the owner of the project Entity Framework Plus
This library has a Query Future feature which allows batching multiple queries in a single roundtrip.
Example:
// using Z.EntityFramework.Plus; // Don't forget to include this.
var ctx = new EntitiesContext();
// CREATE a pending list of future queries
var futureCountries = ctx.Countries.Where(x => x.IsActive).Future();
var futureStates = ctx.States.Where(x => x.IsActive).Future();
// TRIGGER all pending queries in one database round trip
// SELECT * FROM Country WHERE IsActive = true;
// SELECT * FROM State WHERE IsActive = true
var countries = futureCountries.ToList();
// futureStates is already resolved and contains the result
var states = futureStates.ToList();
Wiki: EF+ Query Future
Have you tried the Union / UnionAll operator?
It's purpose is exactly like you wish - combine the identical data from different sources.
Furthermore due to the concept of deferred execution your query will only be executed when you actually iterate over the result (or call a method that does that, for example - .ToList()
var contractsQuery = _db.Contracts.AsEnumerable().Where(x => x.Code == query.Code).Select(x=>new {Code=x.Code, Id=x.Id, ...});
var jobsQuery = _db.Jobs.AsEnumerable().Where(x => x.Code == query.Code).Select(x=>new{Code=x.Code, Id=x.Id, ...});
var workPhasesQuery = _db.WorkPhases.AsEnumerable().Where(x => x.ContractPhase.Code == query.Code).Select(x=>new{Code=x.Code, Id=x.Id, ...});
// and so on
var combinedQuery = contractsQuery.UnionAll(jobsQuery).UnionAll(workPhasesQuery ).UnionAll(...
var result = combinedQuery.ToList();
A similar question is Union in linq entity framework
Another code sample can be found here
Please notice that this is exactly the same concept of manipulating data as in T-SQL union, and under the covers you will get an sql query using a union operator
Yes there most certainly is a way to query multiple tables. You can use the Include() method extension for your query. for instance:
var examplelist = _db.Contracts.(v => v.id == "someid" && v.name == "anotherfilter").Include("theOtherTablesName").ToList();
You can include as many tables as you like this way. This is the recommended method.
You can also use the UnionAll() method but you'd have to define your queries separately for this