I'm using LINQ to Entities to get some data from a database. Below is my query.
var location = from l in dbContext.Locations
join e in dbContext.Equipment on l.ID equals e.LocationID into rs1
from e in rs1.DefaultIfEmpty()
where ids.Contains(l.ID)
select new
{
EquipmentClass = e,
LocationID = l.ID,
LocationName = l.Name,
EquipmentName = e == null ? null : e.Name,
Description = e == null ? null : e.Description,
InServiceStatus = e == null ? false : e.InServiceStatus,
EquipmentType = e.EquipmentType.Name
};
foreach (var item in location)
{
// some logic
}
In the code above, ids is a list of ints that I pass in to filter the results. When I get the results, I see that one of the return records has a null EquipmentClass. I had performed some null checking but realized that I forgot to do null checking on one of the properties. Now I would expect to get a null reference exception on EquipmentType = e.EquipmentType.Name but I don't. To my surprise, it works just fine, and is set to null. My EquipmentClass has a property type of EquipmentType, which is another class. EquipmentType has a Name property which is a String.
Why doesn't this throw a null reference exception?
Just as a test, I removed the null check from InServiceStatus = e == null ? false : e.InServiceStatus and it fails with an invalid operation exception upon running a foreach loop using the query.
Does this mean that I only need to do null checking on non-nullable values?
Update:
foreach (var item in location)
{
var p = item.EquipmentClass.EquipmentType.Name;
}
Added this right after the query. On the assignment of p, I get a null reference exception. I'm not sure how it even gets that far, as it should fail on the first line of the foreach loop. Without the line declaring variable p, I do not get a null reference exception. If anyone can explain what is happening I would be grateful. Just for reference, the values of item.EquipmentClass and item.EquipmentType are both null by the time the foreach loop starts.
Update2:
I found this link where it seems that someone has an almost identical issue using LINQ to SQL. I get the gist of the answer but don't fully understand its potential impact on my two questions above.
If you write a LINQ query against IQueryable, what happens is that the methods that you are calling behind the scenes (such as Select, Where, etc.) don't do anything more than just recording how you have called them, i.e. they record the predicate expressions and carry over a LINQ provider. As soon as you start iterating the query, the provider is asked to execute the query model. So basically, the provider uses the expression model to give you a result of the expected type.
The provider is by no means required to actually compile or even execute the code (model) you delivered as an expression. In fact, the whole point of LINQ to SQL or LINQ to Entities is that the provider does not do this and translate the code expression to SQL instead.
Therefore, your query is actually rendered as an SQL query and the result of that query is translated back. Therefore, the variable e that you see in the query is not necessarily really created but only used for the LINQ provider to compile an SQL query. However, most database servers have null propagations.
Run the same query against LINQ to Objects and you will get your missing NullReferenceException.
Your update helped me understand your real concern. The concept you need to know in regards to LINQ queries is deferred execution in LINQ.
Please go through below links for more details:
What are the benefits of a Deferred Execution in LINQ?
Linq - What is the quickest way to find out deferred execution or not?
Now what happens in your case? You've stored your query in location variable. That particular step is just the initialization part. It doesn't really executes the query on your database through your ORM layer. This is how you can test this.
Put a break point on the line of code where you're initializing location variable with LINQ query. When debugger stops in Visual Studio, then go to SQL Server Management Studio (SSMS) and start a SQL Server Profiler session.
Now press F10 in Visual Studio to step over the code statement. At this point of time you'll see absolutely no query execution in profiler session as shown below:
That is all because LINQ query didn't get executed at all till this point of time.
Now you reach to your below line of code:
foreach (var item in location)
{
var p = item.EquipmentClass.EquipmentType.Name;
}
The moment you enter inside foreach loop, the LINQ query gets fired and you'll see the corresponding log in trace session in SQL Server profiler. So a LINQ query doesn't get fired unless it is being enumerated. This is called deferred execution i.e. the runtime defers the execution until enumeration. If you never enumerate location variable in your code then query execution will never happen at all.
So to answer your query, exception will come only when query gets fired. Not before that!
Update 1: You're saying that - I do not get a null reference exception. Yes! you will not get null reference exception until you reach to the record whose corresponding RHS joined record is missing. Have a look at below code for better understanding:
class Program
{
static void Main(string[] args)
{
var mylist1 = new List<MyClass1>();
mylist1.Add(new MyClass1 { id = 1, Name1 = "1" });
mylist1.Add(new MyClass1 { id = 2, Name1 = "2" });
var mylist2 = new List<MyClass2>();
mylist2.Add(new MyClass2 { id = 1, Name2 = "1" });
var location = from l in mylist1
join e in mylist2 on l.id equals e.id into rs1
from e in rs1.DefaultIfEmpty()
//where ids.Contains(l.ID)
select new
{
EquipmentClass = e,
InServiceStatus = e == null ? 1 : e.id,
EquipmentType = e.id
};
foreach (var item in location)
{
}
}
}
class MyClass1
{
public int id { get; set; }
public string Name1 { get; set; }
}
class MyClass2
{
public int id { get; set; }
public string Name2 { get; set; }
}
So, Now when I start iterating the location variable, it doesn't breaks in first iteration. It breaks in second iteration. It breaks when it fails to obtain the record/object corresponding to MyClass1 object having id 2 present in mylist1. mylist2 doesn't have any object with id 2. Hope this helps!
your e.EquipmentType.Name is null and it is getting assigned to EquipmentType and it is perfectly fine to assign null to a nullable type and you are using DefaultIfEmpty() which will initialize the elements with their default value if it fails to match any condition and in this case your e.ElementType.Name is getting set to null which is fine I guess. use ToList() that will throw the exception.
I hope I am making some sense and you people might have discussed this.
I'm using entity framework to connect with the database. I've one little problem:
I've one table which have one varbinary(MAX) column(with filestream).
I'm using SQL request to manage the "Data" part, but EF for the rest(metadata of the file).
I've one code which has to get all files id, filename, guid, modification date, ... of a file. This doesn't need at all the "Data" field.
Is there a way to retrieve a List but without this column filled?
Something like
context.Files.Where(f=>f.xyz).Exclude(f=>f.Data).ToList();
??
I know I can create anonymous objects, but I need to transmit the result to a method, so no anonymous methods. And I don't want to put this in a list of anonymous type, and then create a list of my non-anonymous type(File).
The goal is to avoid this:
using(RsSolutionsEntities context = new RsSolutionsEntities())
{
var file = context.Files
.Where(f => f.Id == idFile)
.Select(f => new {
f.Id, f.MimeType, f.Size, f.FileName, f.DataType,
f.DateModification, f.FileId
}).FirstOrDefault();
return new File() {
DataType = file.DataType, DateModification = file.DateModification,
FileId = file.FileId, FileName = file.FileName, Id = file.Id,
MimeType = file.MimeType, Size = file.Size
};
}
(I'm using here the anonymous type because otherwise you will get a NotSupportedException: The entity or complex type 'ProjectName.File' cannot be constructed in a LINQ to Entities query.)
(e.g. this code throw the previous exception:
File file2 = context.Files.Where(f => f.Id == idFile)
.Select(f => new File() {Id = f.Id, DataType = f.DataType}).FirstOrDefault();
and "File" is the type I get with a context.Files.ToList(). This is the good class:
using File = MyProjectNamespace.Common.Data.DataModel.File;
File is a known class of my EF datacontext:
public ObjectSet<File> Files
{
get { return _files ?? (_files = CreateObjectSet<File>("Files")); }
}
private ObjectSet<File> _files;
Is there a way to retrieve a List but without this column filled?
Not without projection which you want to avoid. If the column is mapped it is natural part of your entity. Entity without this column is not complete - it is different data set = projection.
I'm using here the anonymous type because otherwise you will get a
NotSupportedException: The entity or complex type 'ProjectName.File'
cannot be constructed in a LINQ to Entities query.
As exception says you cannot project to mapped entity. I mentioned reason above - projection make different data set and EF don't like "partial entities".
Error 16 Error 3023: Problem in mapping fragments starting at line
2717:Column Files.Data in table Files must be mapped: It has no
default value and is not nullable.
It is not enough to delete property from designer. You must open EDMX as XML and delete column from SSDL as well which will make your model very fragile (each update from database will put your column back). If you don't want to map the column you should use database view without the column and map the view instead of the table but you will not be able to insert data.
As a workaround to all your problems use table splitting and separate the problematic binary column to another entity with 1 : 1 relation to your main File entity.
I'd do something like this:
var result = from thing in dbContext.Things
select new Thing {
PropertyA = thing.PropertyA,
Another = thing.Another
// and so on, skipping the VarBinary(MAX) property
};
Where Thing is your entity that EF knows how to materialize. The resulting SQL statement shouldn't include the large column in its result set, since it's not needed in the query.
EDIT: From your edits, you get the error NotSupportedException: The entity or complex type 'ProjectName.File' cannot be constructed in a LINQ to Entities query. because you haven't mapped that class as an entity. You can't include objects in LINQ to Entities queries that EF doesn't know about and expect it to generate appropriate SQL statements.
You can map another type that excludes the VarBinary(MAX) column in its definition or use the code above.
you can do this:
var files = dbContext.Database.SqlQuery<File>("select FileId, DataType, MimeType from Files");
or this:
var files = objectContext.ExecuteStoreQuery<File>("select FileId, DataType, MimeType from Files");
depending on your version of EF
I had this requirement because I have a Document entity which has a Content field with the content of the file, i.e. some 100MB in size, and I have a Search function that I wanted to return the rest of the columns.
I chose to use projection:
IQueryable<Document> results = dbContext.Documents.Include(o => o.UploadedBy).Select(o => new {
Content = (string)null,
ContentType = o.ContentType,
DocumentTypeId = o.DocumentTypeId,
FileName = o.FileName,
Id = o.Id,
// etc. even with related entities here like:
UploadedBy = o.UploadedBy
});
Then my WebApi controller passes this results object to a common Pagination function, which applies a .Skip, .Take and a .ToList.
This means that when the query gets executed, it doesn't access the Content column, so the 100MB data is not being touched, and the query is as fast as you'd want/expect it to be.
Next, I cast it back to my DTO class, which in this case is pretty much exactly the same as the entity class, so this might not be a step you need to implement, but it's follows my typical WebApi coding pattern, so:
var dtos = paginated.Select(o => new DocumentDTO
{
Content = o.Content,
ContentType = o.ContentType,
DocumentTypeId = o.DocumentTypeId,
FileName = o.FileName,
Id = o.Id,
UploadedBy = o.UploadedBy == null ? null : ModelFactory.Create(o.UploadedBy)
});
Then I return the DTO list:
return Ok(dtos);
So it uses projection, which might not fit the original poster's requirements, but if you're using DTO classes, you're converting anyway. You could just as easily do the following to return them as your actual entities:
var dtos = paginated.Select(o => new Document
{
Content = o.Content,
ContentType = o.ContentType,
DocumentTypeId = o.DocumentTypeId,
//...
Just a few extra steps but this is working nicely for me.
For EF Core 2
I implemented a solution like this:
var files = context.Files.AsNoTracking()
.IgnoreProperty(f => f.Report)
.ToList();
The base idea is to turn for example this query:
SELECT [f].[Id], [f].[Report], [f].[CreationDate]
FROM [File] AS [f]
into this:
SELECT [f].[Id], '' as [Report], [f].[CreationDate]
FROM [File] AS [f]
you can see the full source code in here:
https://github.com/aspnet/EntityFrameworkCore/issues/1387#issuecomment-495630292
I'd like to share my attempts to workaround this problem in case somebody else is in the same situation.
I started with what Jeremy Danyow suggested, which to me is the less painful option.
// You need to include all fields in the query, just make null the ones you don't want.
var results = context.Database.SqlQuery<myEntity>("SELECT Field1, Field2, Field3, HugeField4 = NULL, Field5 FROM TableName");
In my case, I needed a IQueryable<> result object so I added AsQueryable() at the end. This of course let me add calls to .Where, .Take, and the other commands we all know, and they worked fine. But there's a caveat:
The normal code (basically context.myEntity.AsQueryable()) returned a System.Data.Entity.DbSet<Data.DataModel.myEntity>, while this approach returned System.Linq.EnumerableQuery<Data.DataModel.myEntity>.
Apparently this means that my custom query gets executed "as is" as soon as needed and the filtering I added later is done afterwards and not in the database.
Therefore I tried to mimic Entity Framework's object by using the exact query EF creates, even with those [Extent1] aliases, but it didn't work. When analyzing the resulting object, its query ended like
FROM [dbo].[TableName] AS [Extent1].Where(c => ...
instead of the expected
FROM [dbo].[TableName] AS [Extent1] WHERE ([Extent1]...
Anyway, this works, and as long as the table is not huge, this method will be fast enough. Otherwise you have no option than to manually add the conditions by concatenating strings, like classic dynamic SQL. A very basic example in case you don't know what I'm talking about:
string query = "SELECT Field1, Field2, Field3, HugeField4 = NULL, Field5 FROM TableName";
if (parameterId.HasValue)
query += " WHERE Field1 = " + parameterId.Value.ToString();
var results = context.Database.SqlQuery<myEntity>(query);
In case your method sometimes needs this field, you can add a bool parameter and then do something like this:
IQueryable<myEntity> results;
if (excludeBigData)
results = context.Database.SqlQuery<myEntity>("SELECT Field1, Field2, Field3, HugeField4 = NULL, Field5 FROM TableName").AsQueryable();
else
results = context.myEntity.AsQueryable();
If anyone manages to make the Linq extensions work properly like if it was the original EF object, please comment so I can update the answer.
I'm using here the anonymous type because otherwise you will get a
NotSupportedException: The entity or complex type 'ProjectName.File'
cannot be constructed in a LINQ to Entities query.
var file = context.Files
.Where(f => f.Id == idFile)
.FirstOrDefault() // You need to exeucte the query if you want to reuse the type
.Select(f => new {
f.Id, f.MimeType, f.Size, f.FileName, f.DataType,
f.DateModification, f.FileId
}).FirstOrDefault();
And also its not a bad practice to de-normalize the table into further, i.e one with metadata and one with payload to avoid projection. Projection would work, the only issue is, need to edit any time a new column is added to the table.
I tried this:
From the edmx diagram (EF 6), I clicked the column I wanted to hide from EF and on their properties you can set their getter and setter to private. That way, for me it works.
I return some data which includes a User reference, so I wanted to hide the Password field even though it's encrypted and salted, I just didn't want it on my json, and I didn't want to do a:
Select(col => new {})
because that's a pain to create and maintain, especially for big tables with a lot of relationships.
The downside with this method is that if you ever regenerate your model, you would need to modify their getter and setter again.
Using Entity Framework Power Tools you can do the following in efpt.config.json:
"Tables": [
{
"ExcludedColumns": [
"FileData"
],
"Name": "[dbo].[Attachment]",
"ObjectType": 0
}
]
I searched vigorously for an answer to this, but I was unable to find (or understand) a solution. I have a query where I am joining to another table that may or may not have associated records. Here is the query:
var educationUniversity = result.new_educationUniversity.Select(c => new
{ c.majorDegree, c.dateEnd, c.dateStart, c.institutionName,
c.degreeProgramCompletionStatus, c.institutionAddress,
attachmentId = c.attachmentId ?? 0,
fileName = c.new_attachments.fileName ?? "No Attachment"}).ToList();
The first table is "new_educationUniversity" and it holds details of a user's college or university degree. The user may or may not have uploaded an attachment (which is stored in the "new_attachments" table). The attchmentID is the primary key in the "new_attachments" table and a foreign key in the "new_educationUniversity" table. EF sees the relationship.
I am binding the results to a repeater but the code fails on the line above if there is no related attachment. Everything works fine if there is an attachment or if I remove the reference to the fileName.
Above, I am handling if the fileName is NULL (or at least I am trying to), but I suspect my issue is that the record simply doesn't exist, which is different from NULL I guess. I've tried using something like: c.new_attachments.fileName.SingleOrDefault() or DefaultIfEmpty() with no luck.
As an analogy, imagine you have a CUSTOMERS table and an ORDERS table. You want to query the following:
-customer last name
-customer first name
-customer most recent order Id
However, you have registered customers who have never purchased anything. I'm sure I am doing something completely noobish, so any assistance is GREATLY appreciated. Thanks!
So you have a few things going on here:
You can get a null reference exception on your reference to c.new_attachments.fileName. If new_attachments is null, then this will throw an exception. Including the null-coalescing operator (the ??) won't help, because you're trying to access the property of something which can be null.
Unless you're lazy-loading (generally bad), then there's no reason for you to be trying to create a dynamically typed object to send to your repeater. Just pass the result.new_educationUniversity object directly.
What's the solution?
I'd create a partial class to add a new property to your new_educationUniversity class. Add a null-reference-safe property reference to determine the file name of the new_attachments property of new_educationUniversity. Then, bind the repeater to your new property. Something like:
public partial class new_educationUniversity {
public String AttachmentFileName {
get {
if (new_attachments == null)
return "";
else
return new_attachments.fileName;
}
}
}
The following line of code seems to be working.
var educationUniversity = result.new_educationUniversity.Select
(c => new { c.majorDegree, c.dateEnd, c.dateStart, c.institutionName,
c.degreeProgramCompletionStatus, c.institutionAddress, attachmentId = c.attachmentId ?? 0,
fileName = (c.new_attachments == null ? "***NO ATTACHMENT***" : c.new_attachments.fileName)}).ToList();
I don't fully understand what this line means:
fileName = (c.new_attachments == null ? "***NO ATTACHMENT***" : c.new_attachments.fileName)}
I assume it says "if nothing is there replace with the string "NO ATTACHMENT" otherwise use what is in the DB"
Is this acceptable? So far, so good. Thanks!
I have some issue querying my local sqlite-database. Below is the current code:
public static Dictionary<Guid, string> GetHashedComanyMembers(List<Guid> IdsToSearch)
{
using (var dbContext = new DbEntities(GetConnectionString()))
{
var test = dbContext.<myTable>.Where(cm => IdsToSearch.Contains(cm.IdToFind));
}
}
The variable test contains an emtpt dataset. And now to my question.
If is set a variable
var firstGuid = IdsToSearch[0]
and i change the next line to
var test = dbContext.<myTable>.Where(cm => cm.IdToFind == firstGuid);
test contains the two sets i want to have.
If i let me show the (first) generated statement and run it on the database directly it also fails to find something. But the (second) statement succeeds.
So does anyone have any idea where i'm going wrong or could point out the error?
Thanks in advance, Steve
The first version:
SELECT
[Extent1].[ValueId] AS [ValueId],
[Extent1].[ValueFirstName] AS [ValueFirstName],
[Extent1].[ValueLastName] AS [ValueLastName],
[Extent1].[ValueLastChanged] AS [ValueLastChanged],
[Extent1].[ValueLastChangedBy] AS [ValueLastChangedBy]
FROM [tblValue] AS [Extent1]
WHERE '28826de0-27ea-42ab-9104-234ee289de16' = [Extent1].[ValueId]
Second:
SELECT
[Extent1].[ValueId] AS [ValueId],
[Extent1].[ValueFirstName] AS [ValueFirstName],
[Extent1].[ValueLastName] AS [ValueLastName],
[Extent1].[ValueLastChanged] AS [ValueLastChanged],
[Extent1].[ValueLastChangedBy] AS [ValueLastChangedBy]
FROM [tblValue] AS [Extent1]
WHERE [Extent1].[ValueId] = #p__linq__0
Edit
If i call th ToList() method on my table before the Where() everything is fine because the whole table-content will be enumerated an i call the Contains on this list. But that can't be the solution to Select the whole table first.
So i think the problem is the conversation done by Linq because the generated statements checks agains the Guid as string but it is internally stored as Blob.
In the second statement the guid is not set directly. There is an placeholder that will be replaced by DBMS? And this query is alright.
So any ideas where to start?
I've just started learning how to use the Entity Framework to write a very simple C# network monitoring program - this is a learning exercise to try and "drive home" what I've only read about to date. I'm also new to C# and LINQ (just to complicate things further.)
I believe I have the data model suitably normalised but I may be wrong. Visual Studio generates a conceptual model that looks OK. I've pluralised the associations and EntitySets where necessary, but I'm struggling to perform what I think is a fairly basic query/projection on the data.
The database contains 3 tables:
[Server] - A server defined by the user that should be pinged.
ServerID - primary key
HostAddress - IP or hostname
[Result] - A result containing data about the last server test
ResultID - primary key
ServerID - foreign key on [Server].[ServerID]
StateID - an integer used to lookup one of 3 possible Server states
TimeStamp - Time stamp of last ping
[State] - A lookup table containing an integer -> string mapping.
StateID - a unique key
StateLabel - human-readable string like "unreachable" or "OK" or "timeout"
I have manually populated the database using a few simple entries - just enough to give me something to work with.
For starters, I would like to present all of the Result data in a ListView on a WinForm. The ListView contains the following static columns:
State | Server Address | Last checked
In theory, the ListView's data needs to be generated by projecting(?) across each of the 3 tables:
The "State" column should display the human-readable [State].[StateLabel] linked from [Result].[StateID]
The "Server Address" column should display [Server].[HostAddress] linked from [Result].[ServerID]
The "Last Checked" column should display [Result].[TimeStamp]
Since I have no need for the object materialisation and/or change-tracking features of ObjectServices, am I correct in thinking it would be more efficient/correct to use Entity SQL/EntityClient and DbDataReader? If so, what would a suitable Entity SQL query look like?
For what it's worth, I tried using LINQ to Entities and anonymous types in a method but was thwarted by a lack of understanding on a suitable return type:
var results = from r in _context.Result
select new
{
State = (from s in _context.State
where s.StateId == r.StateId
select s.StateLabel),
r.ServerReference.Value.HostAddress,
r.TimeStamp
};
return results.ToList(); // <- No can do.
Thanks for your help!
Steve
Well you won't be able to return a list of anonymous types unless you cast them to object and have the signature define the return type as List<object> (or suitable interface). Your other issue is that the subquery for State will actually return an IQueryable instead of a single entry (you can use the First extension method with EF to get the first matching item.) Although if you have the foreign key relationship the model should have setup an navigation property for the state as well and you should be able to use the property attached instead of a subquery. So if you would like to have this as a method call that returns a list of objects you will have to create a type that represents the transform or downcast to object. Otherwise you could do it at the form level (this all depends on your needs) where you are attempting to bind the list.
public List<object> GetStuff()
{
var results = from r in _context.Result
select new
{
State = r.StateNavigationProperty.StateLabel, //If FK
State = _context.State.First(state => state.StateId == r.StateId), //If Not FK
HostAddress = r.ServerReference.Value.HostAddress,
TimeStamp = r.TimeStamp
};
return results.Cast<object>().ToList();
}
...
myListView.DataSource = GetStuff();
And like I said the other alternative is to either create a class for the transform or bind the list directly to the query.
public class SimpleStuff
{
public string State { get; set; }
public string HostAddress { get; set; }
public DateTime TimeStamp { get; set; }
}
Then just add the class to the select new ala select new SimpleStuff and change the method signature to reflect the class and remove the cast in the return.