So we have a WCF Data Service exposing OData with an Entity Framework (v5.6) model.
Since we wanted to filter the data based on authentication and access control, some Query Interceptors were added. At first, all they had to do was get the user's scope and match that to an entity's property - something like this:
[QueryInterceptor("Entity")]
public Expression<Func<Entity, bool>> EntitiesInterceptor()
{
// Get user scope
string userScope = GetUserScope();
return entity => entity.Scope.StartsWith(userScope));
}
This worked as expected, and everything was well with the world.
Later on, we wanted to evolve the interceptor to allow matching with a list of scopes. So we changed the code to something like this:
// Get user scopes
string[] validScopes = GetUserScopes();
return entity => validUnits.Any(scope => entity.Scope.StartsWith(scope));
This seemed to work as long as our OData requests didn't expand beyond one level of relations. So, for instance, stuff like:
/ODataService.svc/Entity?$expand=OtherEntity
/ODataService.svc/Entity?$expand=OtherEntity,AnotherEntity
...would process just fine. However, when we start querying deeper relations:
/ODataService.svc/Entity?$expand=OtherEntity/YetAnotherEntity
We start getting an 'Object reference not set to an instance of an Object' error from System.Entity.Data. The stack trace is really long and mentions lots of classes on the System.Data namespace. Sadly, it does not mention any line in our code.
Link to full stack trace
It seems to us that the problem is on how EF is converting our lambda expressions to queries. We've tried replacing the .Any() with other equivalents like .Count(expression) > 0 in hopes that the conversion would work, but alas, the result was the same.
So, can anyone think of a different way to approach the problem? Has somebody faced the same issue? Or is it something that we're doing wrong?
Thanks in advance for reading the huge wall of text above.
Related
Good evening!
I am currently working on the backend of my application and I need to get a list of all properties of a certain datatype but only the ones in the current page.
listFiltersCms = _umbraco.ContentAtRoot().SelectMany(c => c.Descendants<DataFilters>()).ToList();
This line above gathers all the filters from all the pages, but I want the filters from a specific page (can be currentPage).
I have tried something like this:
var listFiltersCms = _umbraco.AssignedContentItem.SelectMany(c => c.Descendants<DataFilter>()).ToList();
But without any luck :( Any ideas?
Not entirely sure what you mean by "the backend of my application" - are you inside Umbraco or on a public website?
I ask because there is a fairly straightforward way of getting all info on a specific datatype, but it is not really meant to be used on public facing websites. It can be, but it might be taxing on the system as I believe it queries the database directly, which is not ideal.
Anyway, you might want to take a look at the DataTypeService:
https://our.umbraco.com/Documentation/Reference/Management/Services/DataTypeService/Index-v8
And here you can see what options you have for using it (not very explanatory, but I spot a few methods you could probably look into):
https://our.umbraco.com/apidocs/v8/csharp/api/Umbraco.Core.Services.IDataTypeService.html
If you decide to use the service in a scenario where there will be a lot of requests, maybe consider caching the output to spare the database.
The way I solved this issue was by getting the page in the correct model. In my case all I had to do was:
(_umbraco.AssignedContentItem as PageModel).FilterCustomMeasures;
I have some code which makes calls to a SharePoint Managed Metadata Service that starts off like this :
var clientContext = new ClientContext("http://mysharepointsite/")
{ AuthenticationMode = ClientAuthenticationMode.Default};
var taxonomySession = TaxonomySession.GetTaxonomySession(clientContext);
var termStore = taxonomySession.GetDefaultSiteCollectionTermStore();
which I have no problem with. However, after this we have :
clientContext.Load(termStore,
store => store.Name,
store => store.Groups.Include(
group => group.Name,
group => group.TermSets.Include(
termSet => termSet.Name,
termSet => termSet.Terms.Include(
term => term.Name)
)
)
);
Can anyone please help me understand what is going on here?
At first I thought this was some kind of LINQ query, but then I would expect the class to have the line using System.Linq;, which it does not.
I just noticed that in Visual Studio there is some IntelliSense which says the call is structured like this : void ClientruntimeContext.Load<T>(T clientObject, params System.Linq.Expressions.Expression<Func<T, object>>[] retrievals) - which makes it seem like it is using Linq in some way
I understand that the code is in some way "loading" the termstore data in the managed metadata service from the given sharepoint site, but I don't quite understand what exactly that syntax is doing.
I got the code sample from here, and it does exactly what I want it to do but I would feel a lot more comfortable if I actually understood that syntax!
The documentation was also not particularly helpful, as it just defines Load()s parameters as <T>, which could be anything!
Any advice or recommended reading is much appreciated, thank you!
ClientRuntimeContext.Load<T> Method
The second parameter of this method specifies what properties of target client object (first parameter) should be retrieved using lambda expressions.
Examples
In the following query, all the properties except collection properties such as TermStore.Groups of TermStore client object will be retrieved
ctx.Load(termStore);
In the next query only the explicitly specified list of properties (TermStore.Name, TermStore.Groups) will be retrieved for TermStore client object:
ctx.Load(termStore, store => store.Name, store => store.Groups);
The next question arise, how to specify what properties of collection client object to retrieve, Include<TSource>(IQueryable<TSource>, \[\]) method method comes to the rescue here.
Include<TSource>(IQueryable<TSource>, \[\]) method
This method is used to limit what properties are returned from a collection of objects ( for performance purposes)
Example
The following expression
ctx.Load(termStore, store => store.Groups.Include( g => g.Name));
tells to construct the query to return TermStore client object which includes TermStore.Groups property but not the default properties of Group client object, only Group.Name property.
I'm really new to API design and MVC concepts, but as far as I can tell, something like GET /api/products should return a list of products and GET /api/products/1 should return a single product. In terms of speed my feeling is that /api/products should return less information i.e. just id and name, whereas /api/products/1 should return more i.e. id, name, and description.
As far as I can see, the best way to handle this is to make certain fields of the product class not be returned in the /api/products endpoint. This is especially necessary in the case of /api/products?fields=name . I'm using ASP.Net Web Api 2 and have tried the following:
http://www.nuget.org/packages/WebApi.PartialResponse/ - installing this package caused an assembly version error.
Adding ShouldSerialize methods on the Product fields. For reasons I won't go into here, this method is a little cumbersome.
Looked at ASP.NET Web API partial response Json serialization but there doesn't seem to be a conclusive answer there.
ASP.NET WebApi and Partial Responses suggests using a product class with all nullable fields. I'm not sure I understand exactly what to do there.
Is there any simple way to do what I'm trying to do?
Otherwise could you suggest a better API design than what I'm doing?
You could also use WebApi.PartialResponse (http://www.nuget.org/packages/WebApi.PartialResponse/). It's a package I wrote which uses LINQ to JSON (Json.NET) to manipulate the returned objects. It uses the fields syntax used by Google in their API's, eg.:
fields=items/id,playlistItems/snippet/title,playlistItems/snippet/position
fields=items(id,snippet/title,snippet/position)
fields=items(id,snippet(title,position))
You can find more information on the GitHub project page: https://github.com/dotarj/PartialResponse.
I'd recommend using separate classes to map to when returning a list of entities.
Particularly as the problem is not just what you return to the user, but also what you select from the database.
So, make getting and entity return a Product object, and getting a list of entities return a ProductLink object or something similar.
Edit
As per jtlowe's comment, if you have many different methods returning slight variations of product properties, use anonymous classes (though I'd question whether this is necessarily a good design).
Consider something like this in your action
return from p in this.context.Products
select new
{
p.Id,
p.Name,
p.SKU
};
This:
Only selects the columns you need from the database.
Needs no additional classes defined for new variations of the product
This doesn't make it easy to pass the result of this statement around to other methods because you can only return it as IEnumerable, object or dynamic. If you are putting this in the controller then it may be good enough. If you are implementing a repository pattern, you'll be unable to return strongly typed lists if you use anonymous types.
Stumpled over this topic and just want to share my feelings - maybe it helps others :) I recommend to use something like OData.
You can implement it so that you can write /api/products?$select=Id,Name,Price
some advantages:
with OData you can use further functions, like $filter, $orderby to work with filters and sort it
$skip, $top, $count to get a nice paging
more $-functions :)
you can directly apply it to a IQueryable<T>. Why is this great? You reduce the result not just in the response of your API, but you even reduce the result your database generates, which makes your application much faster. - and you don't even have to change your query
some disadvantages:
you can't filter directly on columns that are calculated
setting it up will take a little time
hint: sometimes it's better to just use ODataQueryOptions<T> in the parameter instead of complete implementation.
I have a local and development environment, each very similar to each other (OS differences aside). They both run the same project which is making use of the Entity Framework. I use LINQpad quite a bit to interrogate data on both environments -- but where required I have access to SQL etc. etc.
So, this is all started with a very strange occurance. Within my business logic layer, I make a call to get a list of Contacts and then marshall that list into a custom type. The custom-type calls for an Initial index based on the name of each of the contacts.
For reference, this is the code that performs the marshalling:-
private static IEnumerable<AlphabetisedContact> _getGroupedContacts(int clientid)
{
return _getLiteContacts(clientid).GroupBy(c => c.Name[0]).Select(
g => new AlphabetisedContact { Initial = g.Key, Contacts = g.ToList() }).OrderBy(g => g.Initial);
}
So, this all seems to work fine. Except, it never returns any Contact with the first initial of a. I decided to try and debug this and using LINQpad found a weirdness. Whether this has anything to do with my code not returning a contacts I don't know (??), but this was the weirdness I found:-
Local Machine:-
Development Machine:-
For the less eagle-eyed of you, the Entity Set Name returned by EF, seems to be different. With the development machine, they're returned with underscores between words -- which is not how the EDMX was setup. For example, it is Name on local, Contact_name on dev. Again, this may have absolutely nothing to do with why I can't get a contacts. I don't get errors on the dev box, contacts are returned etc. etc. but I can't get a contacts.
Can anyone offer some assistance/advice/guidance on how to fix this? It's become a case of "wood-for-the-trees" now...
Help appreciated.
Turns out this was a catastrophic boo-boo on my part. Please see answer to this on this SO:
Why Won't This LINQ Return A Particular Initial?
Thanks to all who took the time to take a peek - and at least a stab at answer.
;)
Consider this code:
var svc = new ProductDesignerEntities(AppSettings.ProductDesignerServiceLocation);
var scenariox = svc.Scenarios
.Where(o => o.ID == id)
.FirstOrDefault();
var scenario = svc.Scenarios
.Expand(o => o.SomeNavigationProperty)
.Where(o => o.ID == id)
.FirstOrDefault();
When this code runs, scenario.SomeNavigationProperty will be unpopulated. Commenting out the code which populates scenariox fixes it. Why is this? Is it possible to fix this through configuration of the service context or the data service, or will I have to change the design of my code in some fashion?
The alternatives that I can think of are all inferior in some capacity:
Create a ProductDesignerEntities instance per action. This kills within-request caching almost entirely.
Create a ProductDesignerEntities instance per controller. Less annoying with a base controller class, but this kills caching between controllers, and wouldn't fix the issue if different actions in the same controller needed different sets of navigation properties from the same table.
Manually populate the property if it is empty. Difficult to consistently implement a manual solution like this; bound to be a source of bugs.
Always manually populate the property. Kind of defeats the purpose of navigation properties.
Ensure that all uses of the table within a request have the same .Expand() list. Extremely annoying and begging for hard-to-fix bugs.
I'm starting to lean toward the first alternative. It seems to have the least issues, despite the additional overhead of a new connection for each instance :/
EDIT: I managed to get the service queries to run through Fiddler, and they look correct. But the navigation property on scenario still winds up empty unless I comment out the scenariox code.
After doing further research, it appears that you're supposed to create a new ProductDesignerEntities instance within each use context anyway, so that's the solution I'll be going with. It bugs me a lot that I wasn't able to properly fix the problem, though!
I'm going to write it off as either a very weird configuration issue, or a bug in WCF data services, because according to Fiddler, it does actually perform the request for the additional data; it just doesn't show up in the returned object.