speed difference dblinq results vs sql query in prompt - c#

I've setup a database to be used with dblinq.
CREATE TABLE 'quotes' (
'DBDate' int(8) unsigned NOT NULL,
'TickerID' int(11) unsigned NOT NULL,
'Open' double(12,4) NOT NULL,
'High' double(12,4) DEFAULT NULL,
'Low' double(12,4) DEFAULT NULL,
'Close' double(12,4) DEFAULT NULL,
'AdjClose' double(12,4) DEFAULT NULL,
'Volume' int(11) unsigned NOT NULL,
PRIMARY KEY ('TickerID','DBDate'),
CONSTRAINT 'quotes_ibfk_1' FOREIGN KEY ('TickerID') REFERENCES 'tickers' ('TickerID') ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=latin1 COLLATE=latin1_general_ci CHECKSUM=1 DELAY_KEY_WRITE=1 ROW_FORMAT=DYNAMIC
the above is the mysql table schedule
The MySQL server is on a different machine.
When I run this mysql query on my test machine (so not the same machine as the server)
SELECT a.*, b.* FROM quotes a INNER JOIN quotes b ON a.DBDate =
b.DBDate AND a.TickerID=956 and b.TickerID=957 order by a.dbdate asc;
I'll get an output as expected:
2934 rows in set (0.03 sec)
but when I want to get the same result in my C# envirement using DBLinq like this:
var tradeAbleA = (from a in _db.Quotes where a.TickerID == 956 select a);
var tradeAbleB = (from a in _db.Quotes where a.TickerID == 957 select a);
var myDataSet = (from a in tradeAbleA.AsEnumerable() join b in tradeAbleB.AsEnumerable() on a.DbdAte equals b.DbdAte orderby a.DbdAte ascending select new { a, b }).ToList();
it takes over a second to get the list filled. This is way too long. How can I speed this up? (I need it in a list)
regards,
Matthijs

Shouldn't your Sql translate to the following linq?
var myDataSet = (from a in _db.Quotes
join b in db.Quotes on a.DbdAte equals a.DbdAte
where a.TickerID == 956 && b.TickerID == 957
orderby a.DbdAte ascending select new { a, b }).ToList();
In your current version, you create the query for a and b seperately, and by calling .AsEnumerable() on them in the 3rd of your linq expressions you force them to be evaluated. You move the results in memory, which then Linq to Objects joins for you (which can be expensive). You then order the remaining items in memory.
The above should allow you to pass all these steps through to the query provider which tend to be much faster.

Related

Sql query to get newest comment added joined 3 tables

I am trying to make a sql query, that gets me the registration_timestamp of the newest comment.
By supplying a category id.
I have three tables. ( seen below with the fields that should be needed)
Ctm_Comments{
Id,
Page_ID,
Registration_Timestamp
}
Ctm_Forum_Categories{
Id
}
Ctm_Forum_Posts{
Id,
FK_Category_ID
}
I have tried the following, and it returns zero results.
var query = from p in Ctm_Forum_Posts
join c in Ctm_Forum_Categories on p.FK_Categori_ID equals c.Id
join ctm in Ctm_Comments on p.Id equals ctm.Page_ID
where c.Id == 1
select ctm.Reqistration_timestamp;
SQL Queries like these are not my strong suit, so i hope someone here can help out.
Ended up with this, based on the response from accepted answer.
var query = (from comments in Ctm_Comments
join posts in Ctm_Forum_Posts on comments.Page_ID equals posts.Id
join category in Ctm_Forum_Categories on posts.FK_Categori_ID equals category.Id
where category.Id == 1
orderby comments.Reqistration_timestamp descending
select comments.Reqistration_timestamp).FirstOrDefault();
SQL (MS SQL) Query needed is
SELECT TOP 1 [Registration_Timestamp]
FROM [dbo].[Ctm_Comments] AS C
INNER JOIN [dbo].[Ctm_Forum_Posts] AS P ON C.Page_ID = P.Id
INNER JOIN [dbo].[Ctm_Forum_Categories] AS CAT ON CAT.Id = P.Category_ID
WHERE CAT.Id = 1
ORDER BY C.Registration_Timestamp DESC
and this is if we accept that PageID (of Comments Table) is the Post Id. Otherwise, you are missing the PostId Column in the table of Comments which should be the join point
Run the Script below in SQL Server Studio for verification
CREATE TABLE [dbo].[Ctm_Comments] ( [Id] [int] NULL,[Page_ID] [int] NULL,[Registration_Timestamp] [datetime] NULL) ON [PRIMARY]
CREATE TABLE [dbo].[Ctm_Forum_Categories] ( [Id] [int] NULL) ON [PRIMARY]
CREATE TABLE [dbo].[Ctm_Forum_Posts] ( [Id] [int] NULL,[Category_ID] [int] NULL) ON [PRIMARY]
INSERT INTO [dbo].[Ctm_Comments] VALUES (1, 1, '2020-10-23 13:12:55')
INSERT INTO [dbo].[Ctm_Comments] VALUES (2, 1, '2020-10-26 12:12:55')
INSERT INTO [dbo].[Ctm_Comments] VALUES (3, 1, '2020-10-26 12:25:55')
INSERT INTO [dbo].[Ctm_Comments] VALUES (4, 1, '2020-10-26 13:12:55')
INSERT INTO [dbo].[Ctm_Forum_Categories] VALUES (1)
INSERT INTO [dbo].[Ctm_Forum_Posts] VALUES (1, 1)
SELECT TOP 1 [Registration_Timestamp]
FROM [dbo].[Ctm_Comments] AS C
INNER JOIN [dbo].[Ctm_Forum_Posts] AS P ON C.Page_ID = P.Id
INNER JOIN [dbo].[Ctm_Forum_Categories] AS CAT ON CAT.Id = P.Category_ID
WHERE CAT.Id = 1
ORDER BY C.Registration_Timestamp DESC
DROP TABLE [dbo].[Ctm_Comments]
DROP TABLE [dbo].[Ctm_Forum_Categories]
DROP TABLE [dbo].[Ctm_Forum_Posts]
the Result is 2020-10-26 13:12:55.000
When you fix the "my query returns 0 results" part, I'd suggest something like this:
var mostRecentCommentTimestamp = query.Max();
But as you've only selected timestamps, this can only tell you the max timestamp, nothing else about the comment
If you want the whole most recent comment swap the select for an order by descending on the timestamp and take the first*, or install morelinq and use their MaxBy
*Edit, like this:
var query = from p in Ctm_Forum_Posts
join c in Ctm_Forum_Categories on p.FK_Categori_ID equals c.Id
join ctm in Ctm_Comments on p.Id equals ctm.Page_ID
where c.Id == 1
orderby ctm.Reqistration_timestamp descending
select ctm;
var firstComment = query.First();
All this said, at the moment you say your query produces no results; you need to fix that (the joins are wrong, or there is no category 1, or the db is missing data) before you can get a max/orderby of anything

ASP.NET MVC 5 - LINQ Query to Select Data from Database

I'm working on a school project in ASP.NET MVC 5. The project is about creating a social network. After the user logs in, he will see all public posts on his newsfeed.
I am having issues, though, in showing the public posts' data from the database.
This is the script of the database :
create table Utilizador(
id_utilizador integer not null identity(1,1),
nome varchar(50) not null,
apelido varchar(50) not null,
username varchar(15) not null unique,
pass varchar(50) not null,
email varchar(50) not null unique,
sexo char(1) not null CHECK (sexo IN('M', 'F')),
paĆ­s varchar(50) not null,
imagem_perfil varchar(50) not null,
data_nascimento date not null,
estado int not null default 2, --0->Bloqueado 1-Activo, 2-por activar
primary key (id_utilizador),
check (email LIKE '%#%.%')
)
create table Post(
id_post integer not null identity(1,1),
texto varchar(400) not null,
primary key(id_post)
)
create table Publish_Post(
id_post integer not null,
id_utilizador integer not null,
data timestamp not null,
primary key(id_post),
foreign key(id_post) references Post(id_post),
foreign key(id_utilizador) references Utilizador(id_utilizador)
)
create table Privacy(
id_privacidade integer not null identity(1,1), --> 1 public, 2 private
nome varchar(50) not null,
primary key(id_privacidade)
)
create table Have_Privacy(
id_post integer not null,
id_privacidade integer not null,
primary key(id_post),
foreign key(id_post) references Post(id_post),
foreign key(id_privacidade) references Privacidade(id_privacidade)
)
Let me explain why I create the database the way I do:
The user creates and publishes some posts that have will have a privacy value (1 or 2). After the user logs in, all public posts(1) should appear on his newsfeed.
So far I have this LINQ query in C#:
var id_posts = from p in db.Posts
select p.texto;
ViewBag.Posts = id_posts;
Can someone help me?
Thanks in advance :)
Just do this
var id_posts = from p in db.Posts
join hp in db.Have_Privacy on p.id_post equals hp.id_post
join prv in db.Privacy on hp.id_privacidade equals prv.id_privacidade
where prv.nome = 'Private'
select p.texto;
Tell how it goes
Why not just add a field in Post called isprivate with boolean type of BIT that determines if it's private or not and then use query for provided data with where clause:
var id_posts = from p in db.Posts
where isprivate == false
select p.texto;
If you want to have more than 2 types of privacy and just stick with DB schema you provided, you can go with a JOIN:
If id decides it is private:
var id_posts = from p in db.Posts
join hp in db.Have_Privacy on p.id_post equals hp.id_post
where hp.id_privacidade = 1
select p.texto;
If name decides it is private:
var id_posts = from p in db.Posts
join hp in db.Have_Privacy on p.id_post equals hp.id_post
join prv in db.Privacy on hp.id_privacidade equals prv.id_privacidade
where prv.nome = 'Private'
select p.texto;
Also please note that naming tables in one language and columns in other is considered as bad design. It's hard for others (in this example me) to read it, even if I know what it should mean.
Two last queries use your schema with no changes implemented.

C# linq-SQL and SQL null join return different result

I'm using left (null) join to check which data does table A have and table B does not.
SQL is:
select a.* from a
left join b
on a.sid=b.sid
and a.pid=b.pid
where b.sid = null
and a.pid='r'
this returns 0 row, that is as expected.
but, C# linq-SQL :
from a in entities.a
join b in entities.b
on new { sid = (int)a.sid, pid = a.pid}
equals new { sid = (int)b.sid, pid = b.pid}
into j
from x in j.DefaultIfEmpty()
where x.sid== null
&& a.pid=="r"
select a
this return thousands of rows.
so anyone tell me why..?
thanks!
#Hogan is right. Your LINQ statement is very different from the SQL one.
The combination join + into is called a group join, and has no SQL equivalent. It puts in each j a collection of b. It does not filter the results in a, and therefore, you get as many j's as there are in entitities.a. However, the equals clause is always false because C# creates two different anonymous types at compile time. So, each j becomes an empty Enumerable, and DefaultIfEmtpy returns a collection with a single empty instance of b (all properties null), so the condition b.sid == null is true for all rows.
They are very different. This sql is closer:
select a.* from a
left join b
on isnull(cast(a.sid as varchar(max)),'')+'#'+isnull(cast(a.pid as varchar(max)),'')==
isnull(cast(b.sid as varchar(max)),'')+'#'+isnull(cast(a.pid as varchar(max)),'')
where b.sid == null
and a.pid=='r'
It is possible that the SQL is running with ANSI_NULLS ON , which may return zero records.
and the linq is running by default on ANSI_NULLS OFF.
To Check this execute sql query in sql server with both
Set ANSI_NULLS ON
select a.* from a
left join b
on a.sid=b.sid
and a.pid=b.pid
where b.sid = null
and a.pid='r'
and then with
Set ANSI_NULLS OFF
select a.* from a
left join b
on a.sid=b.sid
and a.pid=b.pid
where b.sid = null
and a.pid='r'
you may get different results
Also the linq you are using is almost right except you don't need to type cast with (int)
from a in entities.a
join b in entities.b
on new { sid = a.sid, pid = a.pid}
equals new { sid = b.sid, pid = b.pid} into j
from x in j.DefaultIfEmpty()
where x.sid== null
&& a.pid=="r"
select a
OR alternatively
from a in entities.a
join b in entities.b
on a.sid equals b.sid into j
from x in j.DefaultIfEmpty()
where a.pid == b.pid && x.sid== null
&& a.pid=="r"
select a

Forcing linq to perform inner joins

I'm trying to force Linq to preform an inner join between two tables. I'll give an example.
CREATE TABLE [dbo].[People] (
[PersonId] [int] NOT NULL,
[Name] [nvarchar](MAX) NOT NULL,
[UpdatedDate] [smalldatetime] NOT NULL
... Other fields ...
)
CREATE TABLE [dbo].[CompanyPositions] (
[CompanyPositionId] [int] NOT NULL,
[CompanyId] [int] NOT NULL,
[PersonId] [int] NOT NULL,
... Other fields ...
)
Now I'm working with unusual database as there's a reason beyond my control for people to be missing from the People table but have a record in CompanyPositions. I want to filter out CompanyPositions with missing People by joining the tables.
return (from pos in CompanyPositions
join p in People on pos.PersonId equals p.PersonId
select pos).ToList();
Linq sees this join as redundant and removes it from the SQL it generates.
SELECT
[Extent1].[CompanyPositionId] AS [CompanyPositionId],
[Extent1].[CompanyId] AS [CompanyId],
....
FROM [dbo].[CompanyPositions] AS [Extent1]
However it's not redundant in my case. I can fix it like this
// The min date check will always be true, here to force linq to perform the inner join
var minDate = DateTimeExtensions.SqlMinSmallDate;
return (from pos in CompanyPositions
join p in People on pos.PersonId equals p.PersonId
where p.UpdatedDate >= minDate
select pos).ToList();
However this now creates a needless where clause in my SQL. As a purest I'd like to remove this. Any idea's or does the current database design tie my hands?
Since PersonId is declared NOT NULL (and I assume it is declared as an FK to People) then I'm not sure how you could have a CompanyPosition with a person that is not assigned; and Linq can't see how you can eiter, which is why as you have observed Linq considers the join redundant.
If you're using LinqToSql, you can use LoadWith similar to this:
var context = new MyDataContext();
var options = new DataLoadOptions();
options.LoadWith<People>(x => x.CompanyPositions);
context.LoadOptions = options;
I don't know how to force linq to use a join. But the following statment should give you the required result.
return (from pos in CompanyPositions
where (p in People select p.PersonId).Contains(pos.PersonId)
select pos).ToList();
ClientSide transformation:
(
from pos in CompanyPositions
join p in People on pos.PersonId equals p.PersonId
select new {pos, p}
).ToList().Select(x => x.pos);
More direct filtering:
from pos in CompanyPositions
where pos.People.Any()
select pos

Nested Select in LLBLGen

So, I have the following tables:
Using LLBLGen 2.6 (Adapter Version - No Linq), SQL Server, and .NET 3.5, how would I write the following query?
SELECT o.ObjectID
FROM Object o
INNER JOIN ObjectDetail d ON i.ObjectID = d.ObjectID
WHERE d.CreatedDate = ( SELECT MAX(CreatedDate)
FROM ObjectDetail
WHERE ObjectID = o.ObjectID
)
There will be more filtering, however it's not relevant to this, like if I had an ObjectDetailType and I wanted the max ObjectDetail row for a certain type.
Also, it doesn't have to select o.ObjectID, any / all columns will be fine.
Solved it
PredicateExpression.AddWithAnd(
new FieldCompareSetPredicate(
ObjectDetailFields.CreatedDate,
null,
ObjectDetailFields.CreatedDate.SetAggregateFunction(AggregateFunction.Max),
null,
SetOperator.Equal,
(ObjectFields.ObjectID == ObjectDetailsFields.ObjectID)
)
);

Categories