I want to calculate the rows of a related table:
MainTable tbl = tblInfo(id);
var count = tbl.Related_Huge_Table_Data.Count();
The problem is: this takes too long (about 20 seconds) to execute, although when I run this query in Sql Server it executes below one second. How can I optimize this query in linq? I also tried to use stored procedure but no luck.
This is the tblInfo method:
public MainTable tblInfo(int id)
{
MyDataContext context = new MyDataContext();
MainTable mt = (from c in context.MainTables
where c.Id == id
select c).SingleOrDefault();
return mt;
}
I used LinqToSql and classes was generated by LinqToSql.
By running SingleOrDefault() you execute the query and have to deal with results in memory after that. You need to stay with IQueryable until your query is fully constructed.
The easiest way to answer "how many child records this parent record has" is to approach it from the child side:
using (var dx = new MyDataContext())
{
// If you have an association between the tables defined in the context
int count = dx.Related_Huge_Table_Datas.Where(t => t.MainTable.id == 42).Count();
// If you don't
int count = dx.Related_Huge_Table_Datas.Where(t => t.parent_id == 42).Count();
}
If you insist on the parent side approach, you can do that too:
using (var dx = new MyDataContext())
{
int count = dx.MainTables.Where(t => t.id == 42).SelectMany(t => t.Related_Huge_Table_Datas).Count();
}
If you want to keep a part of this query in a function like tblInfo, you can, but you can't instantiate MyDataContext from inside such function, otherwise you will get an exception when trying to use the query with another instance of MyDataContext. So either pass MyDataContext to tblInfo or make tblInfo a member of partial class MyDataContext:
public static IQueryable<MainTable> tblInfo(MyDataContext dx, int id)
{
return dx.MainTables.Where(t => t.id == id);
}
...
using (var dx = new MyDataContext())
{
int count = tblInfo(dx, 42).SelectMany(t => t.Related_Huge_Table_Datas).Count();
}
Try this
MyDataContext context = new MyDataContext();
var count=context.Related_Huge_Table_Data.where(o=>o.Parentid==id).Count();
//or
int count=context.Database.SqlQuery<int>("select count(1) from Related_Huge_Table_Data where Parentid="+id).FirstOrDefault();
If you wish to take full advantage of your SQL Database's performance, it may make sense to query it directly rather than use Linq. Should be reasonably more performent :)
var Related_Huge_Table_Data = "TABLENAME";//Input table name here
var Id = "ID"; //Input Id name here
var connectionString = "user id=USERNAME; password=PASSWORD server=SERVERNAME; Trusted_Connection=YESORNO; database=DATABASE; connection timeout=30";
SqlCommand sCommand = new SqlCommand();
sCommand.Connection = new SqlConnection(connectionString);
sCommand.CommandType = CommandType.Text;
sCommand.CommandText = $"COUNT(*) FROM {Related_Huge_Table_Name} WHERE Id={ID}";
sCommand.Connection.Open();
SqlDataReader reader = sCommand.ExecuteReader();
var count = 0;
if (reader.HasRows)
{
reader.Read();
count = reader.GetInt32(0);
}
else
{
Debug.WriteLine("Related_Huge_Table_Data: No Rows returned in Query.");
}
sCommand.Connection.Close();
Try this:
MyDataContext context = new MyDataContext();
var count = context.MainTables.GroupBy(x => x.ID).Distict().Count();
The answer of GSerg is the correct one in many case. But when your table starts to be really huge, even a Count(1) directly in SQL Server is slow.
The best way you can get round this is to query the database stats directly, which is impossible with Linq (or I don't know of).
The best thing you can do is to create a static sub (C#) on your tables definition witch will return the result of the following query:
SELECT
SUM(st.row_count)
FROM
sys.dm_db_partition_stats st
WHERE
object_name(object_id) = '{TableName}'
AND (index_id < 2)
where {TableName} is the database name of your table.
Beware it's an answer only for the case of counting all records in a table!
Is your linq2sql returning the recordset and then doing the .Count() locally, or is it sending SQL to the server to do the count on the server? There will be a big difference in performance there.
Also, have you inspected the SQL that's being generated when you execute the query? From memory, Linq2Sql allows you to inspect SQL (maybe by setting up a logger on your class?). In Entity Framework, you can see it when debugging and inspecting the IQueryable<> object, not sure if there's an equivalent in Linq2Sql.
Way to view SQL executed by LINQ in Visual Studio?
Alternatively, use the SQL Server Profiler (if available), or somehow see what's being executed.
You may try following:-
var c = from rt in context.Related_Huge_Table_Data
join t in context.MainTables
on rt.MainTableId ==t.id where t.id=id
select new {rt.id};
var count=c.Distict().Count();
Related
I have Master and Detail classes:
class Master
{
public int ID { get; set; }
public string Name { get; set; }
public List<Detail> Details { get; set; }
}
class Detail
{
public Description { get; set; }
public Amount { get; set; }
}
I use below approach and working fine now.
List<Master> result = new List<Master>();
// SQL Connection
string sqlCommand = "SELECT * FROM Master LEFT JOIN Detail on Master.ID = Detail.ID";
using (System.Data.SqlClient.SqlDataReader dr = db.DbDataReader as System.Data.SqlClient.SqlDataReader)
{
if (dr.HasRows)
{
Master LastMaster = null;
while (dr.Read())
{
if (LastMaster == null || Convert.ToInt(dr["ID"]) != LastMaster.ID)
{
Master h = new Master();
h.ID = Convert.ToInt(dr["ID"]);
h.Name = Convert.ToString(dr["Name"]);
result.Add(h);
LastMaster = h;
}
if (dr["Description"] == DBNull.Value)
continue;
if (h.Detail == null)
h.Detail = new List<Detail>();
Detail d = new Detail();
d.Description = dr["Description"] as string;
d.Amount = Convert.ToDouble(dr["Amount"]);
LastMaster.Detail.Add(d);
......
}
}
.....
}
Is there any better approach to fill list of list objects in C# ? I appreciate any suggestion. Thanks.
You can use Dapper (a micro ORM) for your scenario. Below is a sample code
const string createSql = #"
create table #Users (Id int, Name varchar(20))
create table #Posts (Id int, OwnerId int, Content varchar(20))
insert #Users values(99, 'Sam')
insert #Users values(2, 'I am')
insert #Posts values(1, 99, 'Sams Post1')
insert #Posts values(2, 99, 'Sams Post2')
insert #Posts values(3, null, 'no ones post')";
using(var connection = new SqlConnection("database connection string"))
{
connection.Execute(createSql);
try
{
const string sql =#"select * from #Posts p
left join #Users u on u.Id = p.OwnerId
Order by p.Id";
var data = connection.Query<Post, User, Post>(sql, (post, user) => { post.Owner = user; return post; }).ToList();
}
catch(Exception ex){}
}
Ibram commented about EF and Dapper and Abu gave an example for Dapper (but I'm not sure it demos generating a graph with a single master and multiple detail per master, as you have - dapper can do so if you want to explore it)
In EF we could do something like:
install EF core power tools - as you have a db already we will use it to generate classes from. This operation can just be done with the command line but EFCPT makes a lot of operations easier
right click your project, choose EF Core Power Tools .. Reverse Engineer
fill in a new connection string detail
choose the database objects you wish to turn into classes
set other options as appropriate (you can find out more about them later, maybe only use the pluralize one for now, if your db tables are like Orders, Customers, Companies and you want your classes called Order/Customer/Company (classes should not have plural names). Tick on "put connectionstring in code" for now- you can remove it to config file later
finish. Eventually you'll get some classes and a context that has a load of code in OnModelCreating that lays out a description of everything in the tables, the columns, keys, relationships..
Now you can run some query like:
var c = new YourContext();
var ms = c.Masters.Include(m => m.Details).ToList();
That's basically the equivalent of what you posted
You can get more trick by shaping a more involved linq query:
var q = c.Masters.Include(m => m.Details)
.Where(m => m.Name.StartsWith("Smith"))
.OrderBy(m => m.Name);
var ms = q.ToList();
It will be translated into something like
SELECT * FROM master join detail on ...
WHERE name LIKE 'Smith%'
ORDER BY m.name
You can see the generated query if you inspect the DebugView property of q
You could make changes:
ms[0].Details.Clear(); //causes delete of all details for this master
ms[1].Details.Add(new Detail { someprop = some value}); //causes insert of new details for this master
ms[2].Name = "Hello"; //causes update of this master name
c.SaveChanges(); //carries out the above, in sql, to affect the db
When you manipulate the returned objects and save, EF will delete/insert/update as appropriate to sync the db to what happened to the objects. It is important that you understand that EF tracks what happens to all the objects it creates, so that it can do this
When would you use EF and when would you use Dapper? Well, it doesn't have to be mutually exclusive; you can use them in the same project. Generally I'd say use EF (or some other ORM like it - nHibernate is another popular one, works on a similar concept of translating linq expressions to sql and tracking the data back into an object) for stuff where the sql is so simple that it's a productivity boost to not have to write it, track it, and write the changes back. What it is not intended for, is forming as hoc queries that don't map well to client side objects. For that you can use Dapper, or you could form client side objects and add them to EF's model and then run raw sql that populates them. Dapper is fast, because it doesn't do any of that tracking changes, mapping or wiring up complex object graphs; you do all that manually. Dapper makes a convenient abstraction over raw sql and creates classes, but EF goes much further; it comes at a cost - EF is highly convenient but much more heavy weight.
Without writing an entire foreach loop is there a way to do a Update/Set in LINQ to Entities?
Using EF 6.x
Simple update query:
UPDATE stop_detail
SET cap_unique_id = b.Delivery_Location_Id
FROM order_detail b
WHERE Stop_Detail.CAP_Unique_Id IS NULL AND ((b.customer_id = 20 OR b.customer_id = 291) AND b.id = stop_detail.order_detail_id AND stop_type = 1)
all the context name are the same.
I normally end up writing about 30 lines of C# code to do this and I know there has to be a better way!
Whether you can and whether you should are two different things.
Here's how you can.
Example from EF6 Raw SQL Queries
using (var context = new BloggingContext())
{
context.Database.ExecuteSqlCommand(
"UPDATE dbo.Blogs SET Name = 'Another Name' WHERE BlogId = 1");
}
Hint: you probably shouldn't
I have a C# based api and I send queries to a mysql server. I wonder how can i read the id from a select to a table on C# Note that I am using MySql.Data.MySqlClient;
My code until the execute is this one below. But in this step I wonder how can I retrieve the desired id. I used ExecuteNotQuery but it seems it does not fit on what I need.
string connectionString = #"server=x.x.x.x;userid=xxxx;password=xxxxxx;database=testdatabase";
string getLastStoryIdQuery = "SELECT MAX(ID) FROM testdatabase.test";
MySqlCommand getLastTestIdCommand = new MySqlCommand(getLastStoryIdQuery, mySqlConnection);
int lastId = getLastStoryIdCommand.ExecuteNonQuery();
How can I retrieve the result as an Integer or in worst case as a string response? Thank you in advance. :)
int lastId = Convert.ToInt32(getLastStoryIdCommand.ExecuteScalar());
You can find the documentation on MySqlCommand here: https://dev.mysql.com/doc/dev/connector-net/8.0/html/T_MySql_Data_MySqlClient_MySqlCommand.htm
The method ExecuteNonQuery returns the number affected by the query, while ExecuteScalar returns the first column of the first row. You can also use ExecuteReader to get a datareader so that you can read a resultset the database produces.
In practice, I rarely use DbCommand/DbReader anymore and prefer to just use Dapper for database access in most cases where performance isn't absolutely critical. It simplifies parameter creation, and object filling which serves the vast majority of my use cases.
Dapper would look like this:
string connectionString = #"server=x.x.x.x;userid=xxxx;password=xxxxxx;database=testdatabase";
string getLastStoryIdQuery = "SELECT MAX(ID) FROM testdatabase.test";
int lastId;
using(var conn = new MySqlConnection(connectionString))
{
lastId=conn.Query<int>(getLastStoryIdQuery).First();
// you can also do the following in this instance, but you will use the
// above for results that return multiple rows or multiple columns
//lastId=conn.ExecuteScalar<int>(getLastStoryIdQuery);
// Here is how you use parameters:
// var something = conn.ExecuteScalar<int>("SELECT id FROM testdatabase.test WHERE id=#param",new {param = 10});
// This gets multiple columns and rows into a List<person> (assuming you have a person class with fname,lname,dob properties):
// var people = conn.Query<person>("SELECT fname,lname,dob FROM persons WHERE dob>#start", new {start=new DateTime(2000,1,1)}).ToList();
}
So far I know two ways to query the database. One of it is by using MySQL's Libraries; By declaring "MySqlCommand", "MySqlDataReader", "MySqlConnection" and utilizing the inbuilt methods.
And another is through using the Entity Framework (Which is my preferred option).
I've however run into a bit of a problem with this later method. I personally blame my lack of knowledge. It would seem that when conveying data to the end user, Entity-Framework favours the ObservableCollection<> or List<>.
For me, this means that in the View, I'd be using a Datagrid or List control. Normally this would be fine.
But then, what happens if I don't want a datagrid control in the mix? What if I want the query's result conveyed in a textboxes? How can I do the below without sacrificing the usage of the entity framework? Is it even possible?
string config = "server=localhost; userid = root; database = databaseName";
MySqlConnection con = new MySqlConnection(config);
MySqlDataReader reader = null;
// Run the select query
string query = "SELECT * FROM students WHERE id = " +id;
MySqlCommand command = new MySqlCommand(query, con);
con.Open();
reader = command.ExecuteReader();
while (reader.Read())
{
// Put the results in the appropriate strings so I can databind them
string studentName = (string)reader["studentName"];
string studentNum = (string)reader["studentNum"];
.....
}
con.Close();
How could I do this utilising the entity framework?
EF equivalents for this SQL query:
// note, that this leads to SQL injections,
// use parametrized queries instead!
"SELECT * FROM students WHERE id = " +id
will be:
var student = context.Students.Find(id) // this will lookup local data as well
or:
var student = context.Students.Single(_ => _.Id == id)
Both will return single Student instance, not a sequence/collection, and you can bind your UI to its properties or access them as usual:
student.StudentName
Try to something like that:
public static Studens GetStudent(int id)
{
return context.Studens.SingleOrDefault(x=>x.Id == id);
}
I'm doing an EF query with a list which contains many elements and I seem to be coming across the same issue listed here (where SQL Server throws an error because there are too many elements in the IN statement). I was wondering if there is another way to do it.
Would ExecuteStoreQuery or ExecuteStoreCommand work if the command set up a temporary table?
Thanks
Yep, the best way to select from a large list of keys is to use a temp table.
http://explainextended.com/2009/08/18/passing-parameters-in-mysql-in-list-vs-temporary-table/
If you're using MsSQL and C# then SqlBulkCopy will get your list of keys in the fastest.
public void bulkCopy(String tmpTableName, DataTable table)
{
using (SqlBulkCopy bulkCopy =
new SqlBulkCopy((SqlConnection)connection))
{
bulkCopy.DestinationTableName = tmpTableName;
bulkCopy.WriteToServer(table);
}
}
Then have a stored procedure to match to the temp table by key.
If that list data comes from a database query, do not call .ToList() on your source query. Instead pass the IQueryable to the query.
var statuses = context.Statuses.Where(o => o.IsActive).Select(o => o.Id);
var data = context.Orders.Where(o => statuses.Contains(o.StatusId));
Alternatively you can use something like this
var sessionId = Guid.NewGuid();
foreach (var s in statusList)
insert into SearchValues(sessionId, s); // pseudo code
var statuses = context.SearchValues.Where(o => o.SessionId == sessionId).Select(o => o.Id);
var data = context.Orders.Where(o => statuses.Contains(o.StatusId));
delete from SearchValues where SessionId == #sessionId // pseudo code
You might want to do the insert and delete using SQL queries (and not EF context operations) for optimal performance.