I have not found any good material for querying multiple databases using LINQ. I have been using connection strings to change between databases and display data for user, now i want to implement a search function which query's all databases and returns a list instead of selecting the database before hand.
Here's what I've thrown together so far. which returns one list from one database which obviously is not what i want.
public ActionResult getCustomers(string cust)
{
List<trakman_Entities> teInstances = new List<trakman_Entities>();
IEnumerable<customer> customers = null;
for (var i = 1; i < ConfigurationManager.ConnectionStrings.Count; i++)
{
if (ConfigurationManager.ConnectionStrings[i].ConnectionString.ToLower().Contains("metadata"))
{
string con = ConfigurationManager.ConnectionStrings[i].ConnectionString;
teInstances.Add(new trakman_Entities(con));
}
}
foreach (trakman_Entities entitiy in teInstances)
{
customers = entitiy.customers.Where(c => c.code.StartsWith(cust));
}
foreach(customer c in customers)
{
Response.Write(c.code);
Response.Write(c.name);
}
Response.End();
return View(customers);
}
The problem is that you keep reassigning the customers variable in your foreach loop:
foreach (trakman_Entities entitiy in teInstances)
{
// overwrites on each iteration!
customers = entitiy.customers.Where(c => c.code.StartsWith(cust));
}
Instead, consider:
var customers = teInstances.SelectMany(e => e.customers.Where(c => c.code.StartsWith(cust)))
.ToList();
Or, do do the whole thing using a single LINQ query:
// start with the list of connection string settings cast to IEnumerable<T>
var customers = ConfigurationManager.ConnectionStrings.Cast<ConnectionStringSettings>()
// filter to the relevant connection strings
.Where(s => s.ConnectionString.ToLower().Contains("metadata"))
.SelectMany(s => {
// for each connection string, select a data context
using( var context = new trakman_entities(s.ConnectionString)) {
// for each context, select all relevant customers
return context.customers.Where(c => c.code.StartsWith(cust)).ToArray();
} // and dispose of the context when we're done with it
})
.ToList();
Related
I have a class which is used to carry filtering data. I want to retrieve data from database based on the filtering options.
My FilteringDto class:
public class FilteringDto
{
public int id { get; set; }
public string search_text { get; set; }
}
I want to retrieve data from CafeTableGroup table. This is what my query looks like:
using (ISession session = SessionFactory.OpenSession)
{
using (ITransaction transaction = session.BeginTransaction())
{
groups = session.CreateCriteria<CafeTableGroup>().List<CafeTableGroup>();
if (string.IsNullOrEmpty(filters.search_text))
{
groups = groups.Where(a => a.field_1.Like(filters.search_text)).ToList();
}
if (filters.id != 0)
{
groups = groups.Where(a => a.field_2== filters.id).ToList();
}
transaction.Commit();
}
}
But I have a problem here. In order to get filtered data, first it retrieves all the data in table and then filters it based on the condition. Is there any way that I can do it using a single query and retrieve only the filtered data not all the datas? Thanks in advance.
Problem in your code is .List<CafeTableGroup>(); which is causing materializing the instances too early. Just delay the call to List.
I am not using your exact example. Also, my code uses IQueryOver instead of CreateCriteria. You can achieve this with code something like below:
public IList<Table1Entity> GetList(FilterParams filterParams = null, PageParams pageParams = null)
{
IList<Table1Entity> instance = null;
Conjunction conjTable1 = Restrictions.Conjunction();
Conjunction conjTable2 = Restrictions.Conjunction();
if(filterParams == null)
filterParams = new FilterParams();
if(!string.IsNullOrEmpty(filterParams.Date))
conjTable1.Add(Restrictions.Eq(Projections.Property<Table1Entity>(x => x.Date), filterParams.Date));
if(!string.IsNullOrEmpty(filterParams.FromTime))
conjTable1.Add(Restrictions.Eq(Projections.Property<Table1Entity>(x => x.FromTime), filterParams.FromTime));
if(!string.IsNullOrEmpty(filterParams.ToTime))
conjTable1.Add(Restrictions.Eq(Projections.Property<Table1Entity>(x => x.ToTime), filterParams.ToTime));
if(!string.IsNullOrEmpty(filterParams.Id))
conjTable1.Add(Restrictions.Eq(Projections.Property<Table1Entity>(x => x.Id), Guid.Parse(filterParams.Id)));
if(!string.IsNullOrEmpty(filterParams.Pid))
conjTable2.Add(Restrictions.Eq(Projections.Property<Table2Entity>(x => x.Pid), Guid.Parse(filterParams.Pid)));
IQueryOver<Table1Entity> query = NHSession.QueryOver<Table1Entity>()
.Where(conjTable1)
.JoinQueryOver(x => x.Table2)
.And(conjTable2);
if(pageParams != null)
query = query.Skip(pageParams.SkipRecords).Take(pageParams.TakeRecords);
instance = query.List();
return instance;
}
This also demonstrates how to implement joins and paging.
So I have this Entity Framework query in which the SELECT statement which i can copy and run in sql server ctx.MainOrders returns 2 very different records.
However when i run this LINQ query with .AsEnumerable() and .Select(z => new MainOrder() i'm getting the EXACT SAME record twice! :/
This class is a POCO
public class MainOrder
Here is my query
public List<MainOrder> GetMainOrder(string hashString)
{
try
{
using (var ctx = new ClearContext())
{
var query = ctx.MainOrders
.Where(z => z.MainId == queryHashMain)
.AsEnumerable()
.Select(z => new MainOrder()
{
Email = z.Email,
AuthorizationFirstName = z.AuthorizationFirstName,
AuthorizationLastName = z.AuthorizationLastName,
Btn = z.Btn,
AccountNumber = z.AccountNumber,
UtilityTypeName = z.UtilityTypeName
}).ToList();
return query;
return new List<MainOrder>();
}
}
In your comments you mention a VIEW , that is the issue I bet.
the ctx.Mainorders query will show 2 records in sql server as it is a View in which MainId
Add in method AsNoTracking for view
var query = ctx.MainOrders.AsNoTracking()
Have searched ant tested many examples in this forum but can't get a fully working method.
I am using linq to bulk insert a list of entity classes (RemoteReadings).
Due to unique constraints I need to filter out any items already inserted.
Uniqiuness is composed of 2 columns meterid and datetime in RemoteReadings table.
// approx 5000 records (I need to do this in batches of 2000 due to a
// constraint in L2S,but can do this after get this working)
List<RemoteReading> lst = createListFromCSV();
// Option 1:
// This does not work as am comparing memory list to db list. I need to use contains() method.
// Actually am trying to accomplish this infollowing examples.
List<RemoteReading> myLst = (from ri in db.RemoteReadings
from l in lst
where l.meterid = ri.meterid
&& l.date = r.date
select ri).ToList();
////
// Option2:
// Get the list from DB that are in memory lst
List<RemoteReading> myLst = (from ri in db.RemoteReadings
where
// where in this list by comparing meaterid and datemeaured
(from l in lst
select
/// help here !
///
select ri).ToList<RemoteInterconnectorReading>();
// Option3:
// Get the list from lst that are not in database
// I am bit confused here !
// Tried also to remove from list any duplicates:
List<RemoteReading> result = List<RemoteReading>)myLst.Except(lst).ToList<RemoteReading>();
// Ultimately
db.RemoteReading.InsertAllOnSubmit(result);
db.submitChanges();
Any help please?
Due to limitations in EF, we can't join DB query with in-memory list. Also, Contains can only be used with primitive list. So we need to make some efforts to find the duplicates on two columns.
var newItems = createListFromCSV();
var meterIds = newItems.Select(n=> n.meterid).Distinct().ToList();
var dates = newItems.Select(n=> n.date).Distinct().ToList();
var probableMatches = (from ri in db.RemoteReadings
where (meterIds.Contains(ri.meterids)
|| dates.Contains(ri.date)
select new {ri.merterid, ri.date}).ToList();
var duplicates = (from existingRi in probaleMatches
join newRi in newItems
on new {existingRi.meterid, existingRi.date}
equals {newRi.meterid, newRi.date}
select newRi).ToList();
var insertList = newItems.Except(duplicates).ToList();
db.RemoteReadings.Insert(insertList); // or whatever
With the great help of aSharma and some other tweaks, I finally got a working and tested method. As my lists contain over 5000 items I had to execute in batches to override the 2112 SQL RPC call limitation. Added some comments and credits :)
/// List<RemoteReadings> contains a list of database Entity Classes RemoteReadings
public List<RemoteReadings> removeDublicatesFirst(List<RemoteReadings> lst)
{
try
{
DataClasses1DataContext db = new DataClasses1DataContext();
var meterIds = lst.Select(n => n.meterId).Distinct().ToList();
var dates = lst.Select(n => n.mydate).Distinct().ToList();
var myfLst = new List<RemoteReadings>();
// To avoid the following SqlException, Linq query should be exceuted in batches as follows.
//{System.Data.SqlClient.SqlException
// The incoming tabular data stream (TDS) remote procedure call (RPC) protocol stream is incorrect.
// Too many parameters were provided in this RPC request. The maximum is 2100.
foreach (var batch in dates.Batch(2000))
{
// Gets a list of possible matches from DB.
var probableMatches = (from ri in db.RemoteReadingss
where (meterIds.Contains(ri.meterId)
&& batch.Contains(ri.mydate))
select new { ri.meterId, ri.mydate }).ToList();
// Join the probableMatches with the lst in memory on unique
// constraints meterid.date to find any duplicates
var duplicates = (from existingRi in probableMatches
join newRi in lst
on new
{
existingRi.meterId,
existingRi.mydate
}
equals new { newRi.meterId, newRi.mydate }
select newRi).ToList();
//Add duplicates in a new List due to batch executions.
foreach (var s in duplicates)
{
myfLst.Add(s);
}
}
// Remove the duplicates from lst found in myfLst;
var insertList = lst.Except(myfLst).ToList();
return insertList;
}
catch
(Exception ex)
{
return null;
}
}
// Found this extension Class to divide IEnumerable in batches.
// http://stackoverflow.com/a/13731854/288865
public static class MyExtensions
{
public static IEnumerable<IEnumerable<T>> Batch<T>(this IEnumerable<T> items,
int maxItems)
{
return items.Select((item, inx) => new { item, inx })
.GroupBy(x => x.inx / maxItems)
.Select(g => g.Select(x => x.item));
}
}
I'm using the following code to store a list of connectionID's in a List<string>:
List<string> connectionIds =
connectedUsers.Where(x => x.UserId == userId).Select(x => x.ConnectionId).ToList();
I need to update the Database to set the connection status to false when it finds the corresponding ID's. So far, I am using the following code:
if (connectionIds.Any())
{
foreach (string connectionId in connectionIds)
{
Connection curConn = db.Connection.FirstOrDefault(x => x.ConnectionID == connectionId);
if (curConn != null)
curConn.Connected = false;
db.SaveChanges();
}
}
However, this makes a call to the DB for each connection... Is there any simple way to update the connection in an easier process?
You can use the Contains method. This will result in a single query for loading the connection objects. Once you have the result of the query, loop through the results to update the Connected flag, and then save the changes.
List<string> connectionIds = ...;
if (connectionIds.Any()) {
var data = db.Connection
.Where(x => connectionIds.Contains(x.ConnectionID))
.ToList();
foreach (var item in data) {
item.Connected = false;
}
db.SaveChanges();
}
You want to perform batch updates in one SQL statement with EF. EF doesn't support this in a straightforward manner. See here for a possible solution.
I was working this weekend on parallelizing a section of code using Tasks to run all the queries I needed for a dashboard page.
What I have now is many copy/paste methods with almost exactly the same query and a different line at the very end of the method.
Is there a way to write a query against one object context then detach it and pass to a method?
I want to do something like this:
using(DbContext db = new DbContext)
{
var query = db.cars.where(x => x.make == "Ford");
int handleCounts = getDoorHandleCounts(query);
}
public int getDoorHandleCounts(type? query)
{
using(DbContext db = new DbContext())
{
return query.where(x => x.partType == "DoorHandle").Count();
}
}
Any ideas?
Keep in mind all my count() methods are launched from a Task array so they'll be running in parallel. I need a new object context to run each count query on.
I've done some googling and thought about trying to use a pre-compiled query and call it from different object context's, but my real query is kind of complex with allot of if blocks to determine the where condition. Can you compile a query that isn't really simple?
You can't detach and attach a query from/to a context. However, you could reuse the first expression:
Expression<Func<Car,bool>> InitialSelection()
{
return x => x.make == "Ford";
}
public int GetDoorHandleCounts()
{
using(DbContext db = new DbContext())
{
return db.Cars()
.Where(InitialSelection())
.Where(x => x.partType == "DoorHandle").Count();
}
}
And in your task:
int handleCounts = getDoorHandleCounts();
This only works if the initial query is "simple", i.e. does not contain joins and predicates on joined sets that you should repeat over and over in each getX method.
As an alternative, you could initialize a context and feed it to a method that returns a query:
public IQueryable<Car> GetInitialQuery(DbContext db)
{
return db.Cars().Join(....)
.Where(x => x.make == "Ford")
.Where(....);
}
public int GetDoorHandleCounts()
{
using(DbContext db = new DbContext())
{
return GetInitialQuery(db)
.Where(x => x.partType == "DoorHandle").Count();
}
}
Maybe I'm misunderstanding the question, but wouldn't this do what you're looking for?
using(DbContext db = new DbContext)
{
var carsResult = db.cars.where(x => x.make == "Ford");
int handleCounts = getDoorHandleCounts(carsResult);
}
public int getDoorHandleCounts(IEnumerable<Car> result)
{
return result.where(x => x.partType == "DoorHandle").Count();
}
Edit: never mind, I only now saw your mention of the Task array.
Change
public int getDoorHandleCounts(type? query)
to
public int getDoorHandleCounts(IQueryable<cars> query)
And replace cars with the of objects the query will return.
Edit
I would just recommend passing value you're looking to filter on, like this:
{
...
int handleCounts = getDoorHandleCounts("Ford");
}
public int getDoorHandleCounts(string Make)
{
using(DbContext db = new DbContext())
{
return db.cars.where(x => x.make == Make && x.partType == "DoorHandle").Count();
}
}