In SQL one can write a query that searches for a name of a person like this:
SELECT * FROM Person P WHERE P.Name LIKE N'%ike%'
This query would run with unicode characters (assuming that the Name column and database were setup to handle unicode support).
I have a similar query in HQL that is run by Hibernate (NHibernate). The generated query looks like:
SELECT P FROM SumTotal.TP.Models.Party.Person P join P.Demographics PD WHERE (PD.LastName LIKE '%カタカ%' )
Unfortunately, placing a 'N' in front of the literal in the HQL results in an error. I've tried escaping the unicode characters in the string and still no success.
The database is accepting and saving unicode characters from Hibernate. I've been able to successfully populate an object with a unicode string, save it with Hibernate, and verify it in the database. It would seem to me as a little odd that I cannot use unicode strings in custom queries (or I'm also assuming named queries).
Is this a known issue or limitation of Hibernate (Nhibernate)? How do you use unicode in HQL?
Several sites suggest using the Criteria queries. Due to constraints in the framework that I'm working in, this is not possible.
Have you tried with parameters:
IList<Person> people = session
.CreateQuery("from Person p where p.Name like :name")
.SetParameter("name", "%カタカ%")
.List<Person>();
They also have the advantage to protect your query against SQL injection.
I found a solution that works. I highly doubt it is the best solution. It is however the solution that I'm going to implement until I can authorize a rewrite of the entire query building section of the software that I'm working on.
In the instance:
SELECT P FROM SumTotal.TP.Models.Party.Person P join P.Demographics PD WHERE (PD.LastName LIKE '%カタカ%')
The where clause contains this literal:
'%カタカ%'
This literal can be broken up into nchars which Hibernate (Nhibernate) will unknowingly pass through to the SQL it generates. Odd, but it works. Thus the previous query could be written as:
SELECT P FROM SumTotal.TP.Models.Party.Person P join P.Demographics PD WHERE (PD.LastName LIKE '%' + nchar(0x30AB) + nchar(0x30BF) + nchar(0x30AB)+ '%')
This solution is far from optimal because it would require going through each character and determining if it was a multibyte character. However, in the case of where this code lives in its app it is used in a dynamic query generator that processes multiple different criteria under different operations. In the example I give it is looking for unicode anywhere in the string. It is possible that this function may be returning the where clause portion for a column equaling a specific numeric value or it could be looking for the starting characters of a string. The method that builds the operator uses a operator type and the term to hand back a string. I could rewrite that but it would be a large task. The above fix will allow me to process a string passed into this method. I offer this solution because it does work, but darin's answer is probably the best way I could find.
This problem arises because NHibernate tries to access the column without the type length. Hence in the HBM.xml its very important to mention length for Legacy databases otherwise it will fail for UNIcode related stuff.
Thanks,
Thani
Related
We recently discovered a bug in our system whereby any serial numbers that have been entered in lowercase have not been processed correctly.
To correct this, we need to add a one off function that will run through the database and re-process all items with lower case serial numbers.
In linq, is there a query I can run that will return a list of such items?
Note: I am not asking how to convert lowercase to uppercase or reverse, which is all google will return. I need to generate a list of all database entries where the serial number has been entered in lowercase.
EDIT: I am using Linq to MS SQL, which appears to be case insensitive.
Yes, there is. You can try something like this:
var result = serialnumber.Any(c => char.IsLower(c));
[EDIT]
Well, in case of Linq to Entities...
As is stated here: Regex in Linq (EntityFramework), String processing in database, there's few ways to workaround it.
Change database table structure. E.g. create table Foo_Filter which will link your entities to filters. And then create table Filters
which will contain filters data.
Execute query in memory and use Linq to Objects. This option will be slow, because you have to fetch all data from database to memory
Note: link to MSDN documentation has been added by me.
For example:
var result = context.Serials.ToList().Where(sn => sn.Any(c => char.IsLower(c)));
Another way is to use SqlMethods.Like Method
Finally, i'd strongly recommend to read this: Case sensitive search using Entity Framework and Custom Annotation
I'm from old school where DB had all data access encapsulated into views, procedures, etc. Now I'm forcing myself into using LINQ for most of the obvious queries.
What I'm wondering though, is when to stop and what practical? Today I needed to run query like this:
SELECT D.DeviceKey, D.DeviceId, DR.DriverId, TR.TruckId, LP.Description
FROM dbo.MBLDevice D
LEFT OUTER JOIN dbo.DSPDriver DR ON D.DeviceKey = DR.DeviceKey
LEFT OUTER JOIN dbo.DSPTruck TR ON D.DeviceKey = TR.DeviceKey
LEFT OUTER JOIN
(
SELECT LastPositions.DeviceKey, P.Description, P.Latitude, P.Longitude, P.Speed, P.DeviceTime
FROM dbo.MBLPosition P
INNER JOIN
(
SELECT D.DeviceKey, MAX(P.PositionKey) LastPositionKey
FROM dbo.MBLPosition P
INNER JOIN dbo.MBLDevice D ON P.DeviceKey = D.DeviceKey
GROUP BY D.DeviceKey
) LastPositions ON P.PositionKey = LastPositions.LastPositionKey
) LP ON D.DeviceKey = LP.DeviceKey
WHERE D.IsActive = 1
Personally, I'm not able to write corresponing LINQ. So, I found tool online and got back 2 page long LINQ. It works properly-I can see it in profiler but it's not maintainable IMO. Another problem is that I'm doing projection and getting Anonymous object back. Or, I can manually create class and project into that custom class.
At this point I wonder if it is better to create View on SQL Server and add it to my model? It will break my "all SQL on cliens side" mantra but will be easier to read and maintain. No?
I wonder where you stop with T-SQL vs LINQ ?
EDIT
Model description.
I have DSPTrucks, DSPDrivers and MBLDevices.
Device can be attached to Truck or to Driver or to both.
I also have MBLPositions which is basically pings from device (timestamp and GPS position)
What this query does - in one shot it returns all device-truck-driver information so I know what this device attached to and it also get's me last GPS position for those devices. Response may look like so:
There is some redundant stuff but it's OK. I need to get it in one query.
In general, I would also default to LINQ for most simple queries.
However, when you get at a point where the corresponding LINQ query becomes harder to write and maintain, then what's the point really? So I would simply leave that query in place. It works, after all. To make it easier to use it's pretty straight-forward to map a view or cough stored procedure in your EF model. Nothing wrong with that, really (IMO).
You can firstly store Linq queries in variables which may help to make it not only more readable, but also reusable.
An example maybe like the following:
var redCars = from c in cars
where c.Colour == "red"
select c;
var redSportsCars = from c in redCars
where c.Type == "Sports"
select c;
Queries are lazily executed and not composed until you compile them or iterate over them so you'll notice in profiler that this does produce an effecient query
You will also benifit from defining relationships in the model and using navigation properties, rather than using the linq join syntax. This (again) will make these relationships reusable between queries, and more readable (because you don't specify the relationships in the query like the SQL above)
Generally speaking your LINQ query will be shorter than the equivalent SQL, but I'd suggest trying to work it out by hand rather than using a conversion tool.
With the exception of CTEs (which I'm fairly sure you can't do in LINQ) I would write all queries in LINQ these days
I find when using LINQ its best to ignore whatever sql it generates as long as its retrieving the right thing and is performant, only when one of those doesn't work do I actually look at what its generating.
In terms of the sql it generates being maintainable, you shouldn't really worry about the SQL being maintainable but more the LINQ query that is generating the SQL.
In the end if the sql is not quite right I believe there are various things you can do to make LINQ generate SQL more along the lines you want..to some extent.
AFAIK there isn't any inherent problem with getting anonymous objects back, however if you are doing it it multiple places you may want to create a class to keep things neater.
I have a setup on SQL Server 2008. I've got three tables. One has a string identifier as a primary key. The second table holds indices into an attribute table. The third simply holds foreign keys into both tables- so that the attributes themselves aren't held in the first table but are instead referred to. Apparently this is common in database normalization, although it is still insane because I know that, since the key is a string, it would take a maximum of 1 attribute per 30 first table room entries to yield a space benefit, let alone the time and complexity problems.
How can I write a LINQ to SQL query to only return values from the first table, such that they hold only specific attributes, as defined in the list in the second table? I attempted to use a Join or GroupJoin, but apparently SQL Server 2008 cannot use a Tuple as the return value.
"I attempted to use a Join or
GroupJoin, but apparently SQL Server
2008 cannot use a Tuple as the return
value".
You can use anonymous types instead of Tuples which are supported by Linq2SQL.
IE:
from x in source group x by new {x.Field1, x.Field2}
I'm not quite clear what you're asking for. Some code might help. Are you looking for something like this?
var q = from i in ctx.Items
select new
{
i.ItemId,
i.ItemTitle,
Attributes = from map in i.AttributeMaps
select map.Attribute
};
I use this page all the time for figuring out complex linq queries when I know the sql approach I want to use.
VB http://msdn.microsoft.com/en-us/vbasic/bb688085
C# http://msdn.microsoft.com/en-us/vcsharp/aa336746.aspx
If you know how to write the sql query to get the data you want then this will show you how to get the same result translating it into linq syntax.
Let us say I have a database of Terms and a list of strings, is this a good (efficient) idea? It works smoothly, but I'm not sure it is scalable or the most efficient.
var results =
from t in Terms
join x in Targets on t.Term equals x
select t;
Here Terms is a database table with index table Term. Targets is an IEnumerable of strings. Terms might hold millions, Targets between 10-20 strings. Any thoughts?
Ultimately what matters, as far as efficiency is concerned, is if the query that is executed against the database is efficient. To see this, you can either use SQL Profiler or find an application that will show you SQL generated by linq-to-sql.
If you use SQL Profiler, be sure to have it look for stored procedures, as Linq-to-sql uses the exec_sql procedure to execute queries.
If you need to join two tables on one key, as in your example, there's no other way to express it than an actual join. What you have is as efficient as it CAN get.
However, change the select to return only the fields you're interested in, and make sure you trim them, because sql databases like to return char fields with trailing spaces, and they take time to process and transfer across the network.
Hmm, I didn't know you could join a local collection in like that. Perhaps that's a .Net 4.0 feature?
I have frequently issued queries like this:
IQueryable<Term> query =
from t in Terms
where Targets.Contains(t.Term)
select t;
There's a few caveats.
The variable x must be a List<string> reference. The variable x may not be an IList<string> reference.
Each string in the list is translated into a sql parameter. While linq to sql will happily translate many thousands of strings into parameters (I've seen 50k parameters), Sql Server will only accept ~2100. If you exceed this limit, you'll get a sql exception.
nvarchar vs varchar indexes.
What's the best way to convert search terms entered by a user, into a query that can be used in a where clause for full-text searching to query a table and get back relevant results? For example, the following query entered by the user:
+"e-mail" +attachment -"word document" -"e-learning"
Should translate into something like:
SELECT * FROM MyTable WHERE (CONTAINS(*, '"e-mail"')) AND (CONTAINS(*, '"attachment"')) AND (NOT CONTAINS(*, '"word document"')) AND (NOT CONTAINS(*, '"e-learning"'))
I'm using a query parser class at the moment, which parses the query entered by users into tokens using a regular expression, and then constructs the where clause from the tokens.
However, given that this is probably a common requirement by a lot of systems using full-text search, I'm curious as to how other developers have approached this problem, and whether there's a better way of doing things.
How to implement the accepted answer using .Net / C# / Entity Framework...
Install Irony using nuget.
Add the sample class from:
http://irony.codeplex.com/SourceControl/latest#Irony.Samples/FullTextSearchQueryConverter/SearchGrammar.cs
Write code like this to convert the user-entered string to a query.
var grammar = new Irony.Samples.FullTextSearch.SearchGrammar();
var parser = new Irony.Parsing.Parser(grammar);
var parseTree = parser.Parse(userEnteredSearchString);
string query = Irony.Samples.FullTextSearch.SearchGrammar.ConvertQuery(parseTree.Root);
Perhaps write a stored procedure like this:
create procedure [dbo].[SearchLivingFish]
#Query nvarchar(2000)
as
select *
from Fish
inner join containstable(Fish, *, #Query, 100) as ft
on ft.[Key] = FishId
where IsLiving = 1
order by rank desc
Run the query.
var fishes = db.SearchLivingFish(query);
This may not be exactly what you are looking for but it may offer you some further ideas.
http://www.sqlservercentral.com/articles/Full-Text+Search+(2008)/64248/
In addition to #franzo's answer above you probably also want to change the default stop word behaviour in SQL. Otherwise queries containing single digit numbers (or other stop words) will not return any results.
Either disable stop words, create your own stop word list and/or set noise words to be transformed as explained in SQL 2008: Turn off Stop Words for Full Text Search Query
To view the system list of (English) sql stop words, run:
select * from sys.fulltext_system_stopwords where language_id = 1033
I realize it's a bit of a side-step from your original question, but have you considered moving away from SQL fulltext indexes and using something like Lucene/Solr instead?
The easiest way to do this is to use dynamic SQL (I know, insert security issues here) and break the phrase into a correctly formatted string.
You can use a function to break the phrase into a table variable that you can use to create the new string.
A combination of GoldParser and Calitha should sort you out here.
This article: http://www.15seconds.com/issue/070719.htm has a googleToSql class as well, which does some of the translation for you.