We Keep Coding

Filtering with Web API

I have an application with several Web API controllers and I now I have a requirement which is to be able to filter GET results by the object properties. I've been looking at using OData but I'm not sure if it's a good fit for a couple reasons:
The Web API controller does not have direct access to the DataContext, instead it gets data from our database through our "domain" layer so it has no visibility into our Entity Framework models.
Tying into the first item, the Web API deals with lightweight DTO model objects which are produced in the domain layer. This is effectively what hides the EF models. The issue here is I want these queries to be executed in our database but by the time the Web API method gets a collection from the domain layer all of the objects in the collection have been mapped to these DTO objects, so I don't see how the OData filter could possibly do it's job when the objects are once-removed from EF in this way.
This item may be the most important one: We don't really want to allow arbitrary querying against our Web API/Database. We just sort of want to leverage this OData library to avoid writing our own filters, and filter parsers/builders for every type of object that could be returned by one of our Web API endpoints.
Am I on the wrong track based on #3? If not, would we be able to use this OData library without significant refactoring to how our Web API and our EF interact?

I haven't had experience with OData, but from what I can see it's designed to be fed a Context and manages the interaction and returning of those models. I am definitely not a fan of returning Entities in any form to a client.
It's an ugly situation to be in, but when faced with this, my first course of action is to push back to the clients to justify their searching needs. The default request is almost always "Well, it would be nice to be able to search against everything." My answer to that is that I don't want to know what you want, I want to know what you need because I don't want to give you a loaded gun to shoot your own foot off with and then have you blame me because the system came grinding to a halt. Searching is a huge performance killer if it's too open-ended. It's hard to test for accuracy/relevance, and efficiently index for 100% of possible search cases when users only need 25% of those scenarios. If the client cannot tell you what searching they will need, and just want everything because they might need it, then they don't need it yet.
Personally I stick to specific search DTOs and translate those into the linq expressions.
If I was faced with a hard requirement to implement something like that, I would:
Try to push for these searches/reports to be done off a reporting replica that is synchronized with the live database. (To minimize the bleeding when some idiot managers fire up some wacky non-indexed search criteria so that it doesn't tie up the production DB where people are trying to do work.)
Create a new bounded DbContext specific for searching with separate entity definitions that only expose the minimum # of properties to represent search criteria and IDs.
Hook this bounded context into the API and OData. It will return "search results". When a user selects a search result, use the ID(s) against the API to load the applicable domain, or initiate an action, etc.
no. 1. is optional, a nice to have provided they can live with searches not "seeing" updated criteria until replicated. (I.e. a few seconds to minutes depending on replication strategy/size) Normally these searches are used for reporting-type queries so I'd push to keep these separate from the normal day-to-day searching options that users use. (I.e. an advanced search option or the like.)

$expand outside IQueryable

I seem to have an unusual need that I need to be able to $expand objects, but I can not do that using standard IQUeryable.
I am using EntityFramework and AutoMapper to project from database entities to API data objects that I then expose via OData. This has the problem that it limits what I can do - to those elements that AutoMapper can ProjectTo.
This generally is not a problem - if the items in the Api object are part of the database. I now have certain objects where I must (sometimes optionally) add data that comes from other sources. We talk generally about data only held in memory (like an object's error details, which include the stack trace, or some runtime stats).
As such, I need to find a way to manipulate (filter) queries and - more important actually - to handle $expand in single entity GET operations (though support for multi get is also welcome, but there I can possibly handle this by using ODataQueryDetails).
For single item queries, though, I have a controller method in the form like:
Get ([FromODataUri] Guid key) {
which gives me no access to the ODataQueryDetails at all. Anyone knows how to get those query details in this case?
Documentation on the whole "customize the query" element is not particularly big - and generally seems to assume someone is "Just playing with some IQueryable interface" - not taking into account that you may need a multi stage processing or do something that mixes in memory and database data.
Note: Web API OData: How do you $expand on a single entity? is NOT a duplicate - that was a single issue with a parameter name (key required as name) and does not go deep enough.

If you simply add a parameter to Get as follows, Web API parameter binding will supply a value to the method. Let Thing be the entity type handled by the controller.
public IHttpActionResult Get([FromODataUri] Guid key, ODataQueryOptions<Thing> opts)
Note that you need to specify the generic version of ODataQueryOptions or you will get an exception.
This will get you an object representing the query options for the current request, but you will not be able to modify the options (none of the interesting properties have public setters). There seems to be a lot of developer demand for the ability to intercept and modify query options, but there is no out-of-the-box solution at the moment. See the open issue on Github that is currently targeting the 5.1.0 release. See OData V4 modify $filter on server side for current best practice on modifying query options in a controller method.

WCF Data services conditional query interceptor

I have a wcf data service that is being consumed by my applications within my organization. It contains the following entities:
Client
User
Membership
The data is set-up in as multi-tenant and currently requires what we call a client string, or the string representation of the clients unique id, to be passed-in as part of the path and I extract that and add that in a query interceptor as a filter. This has worked beautifully up until recently when we had a new client come on-board that has a parent clientstring with many children. So the result is a query that includes a list of clientids passed-in for all queries as an "in (.....)". Not too bad but when this client now attempts to query memberships they have 9k+ clientids passed-in. The resultant query seems to run fairly quickly when executed alone but the entire db transaction is taking >30 seconds consistently.
I would ultimately like to find a way to bypass this filter for such queries if a specific userid is provided. The URL looks like this:
http://service.com/clienta/Users(1234)/memberships
If a specific id is given for users or memberships I can avoid the filter as the effectively achieves the same end-result. What we can't have is someone doing something like this:
http://service.com/clienta/memberships
Without any filter.
I realize this may be scattered and require much more detail as this was a bit rushed. Please ask any questions. Any help or guidance is much appreciated.

Can you use a query interceptor to intercept the request then if your condition is met to avoid the filtering, simply ignore it?
You could also move to a Service Method and pass in your arguments and do just about anything you want to do.

proper design of web service and associated queries

I am setting up a web service and i'm unsure about the most efficient design..
The web service is going to be called to populate a remote website with a list of people and their associated information. So, in the database there will be a "Person" table, then other tables for nicknames, addresses, phone numbers, etc. There may be 50 "People" returned for the page.
I would think I'd want to keep the # of web service calls to a minimum.. So is the best way to just have one call, GetPeople(), which returns all People and their information in one xml? Or would I want to maybe have GetPeople() return just a list of People, then have a GetPeopleInfo() call for each person?
Thanks in advance

Will you have an opportinuty to talk a bit more with the folks using the web service? The best answer would be depend on a number of factors, including how they are using it.
The simplest soution would be to create one web service method GetPeople() that executes a query joining the Person table with the rest of the tables and returnes all of the data in one flat table. If the client is just going to display all this information in a single list/table this would be the best option all around.
If, on the other hand, the client is going to generate a list of people and then have a click through to get more detailed information on a separate detail page, they might want a GetPeople() method that just returns the names/ids and a GetPeopleInfo() that pulls back the detail. If this isn't going to cause a performance problem on your system, this should be relatively straight forward.
I would probably build both - create a GetPeople() method that brings back all the data {as long as there isn't so much data that transportation becomes and issue} and a GetPersonInfo() method that allows they to pull back details on a specific person. This offers your client the greatest flexibility with not too much additional effort on your part.

You need to think about how your service is going to be used.
Will the users require all of that information most/all of the time? If so then it's appropriate for GetAllPeoplesDetails() to return everything
If they're going to only rarely need all the info then GetPeople() should return just a list of People.
If you're not sure then you could provide both GetAllPeoplesDetails() and GetPeople() and let the end user make that decision

Giving your users options especially when they are disperssed is your best option in my opinion. Having options such as:
GetPeopleList() : return list of people names
GetPeopleAll() : Get list of all people and all info
GetPersonInfo() : Get info of 1 person
Just my opinion but by giving your people a choice your making your application more usefull to them.

I think your GetPeople() call is appropriate. You can return a very lean result set and the end user can GetPeopleInfo() if they need it. If the result set is large, you may want to provide a paging option and result count in your method that allows the caller to pull back a certain amount of rows at a time.

XML is fine.
JSON is better.
Dave Ward at Encosia uses DataTransferObjects to serialize his "Person" objects.
I keep linking to this article because it's done so well, and this is a common problem.
Hope this helps, JP

returning data via webservice c#

i know this is an age old question, but this is my scenario
This is in C# 2.0
Have a windows application which has a datagridview control. This needs to be populates by making a webservice call.
I want to achive the same functionality on the data if i were to use a direct connection and if i were using datasets, namely like paging and applying filters to returned data. i know returning datasets is a bad idea and am looking to find a good solution.

I might look at ADO.NET Data Services, aka Astoria, in VS2008 SP1.
This allows you to expose data over a web-service (WCF exposing ATOM, IIRC) - but you don't need to know all these details: the tooling worries about that for you: you just get regular IQueryable<T> sources on a data-context (not quite the same as the LINQ-to-SQL data-context, but same concept).
The good thing here is that a LINQ query (such as filtering (Where), paging (Skip/Take) etc) can get composed all the way from the client, through the web-service, and down to the LINQ-enabled data store (LINQ-to-SQL or Entity Framework, etc). So only the right data comes over the wire: if you ask for the first 10 rows (of 20000) ordered by Name, then that is what you get: 10 rows out of the database; 10 rows over the wire, no messing.

Write a custom class (MyDataItem) that'll hold your data. Then you can pass a List<MyDataItem> or some collection of MyDataItem and bind to your grid.
Paging, filtering, etc. would need to be implemented by you.

There is no way to get the binding behavior you automatically get with a DataSet if you are going through a Web Services data layer. You would have to create your own proxy class that supports all the databinding functions and persists them through your web service calls. Depending on your application's environment, you may want to batch up modifications to avoid excess round trips to the web services.

fallen888 has it right - you will need to create a collection class of List or a DataTable, fill it with the output from the webservice data stream, and handle paging and filtering yourself yourself.