We Keep Coding

Can EF Core return IQueryable from Stored Procedure / Views / Table Valued Function?

We're in need of passing ODATA-V4 query search, order by clauses to Database directly.
Here is the case:
There are joins among tables and we invoke (inline) table valued
functions using SQL to get desired records.
ODATA where clauses needs to be applied on the result-set, then we
apply pagination Skip, Take and Order By.
We started with Dapper, however Dapper supports only IEnumerable, Thus Dapper would bring entire records from DB then only OData (Query Options ApplyTo) pagination will apply, spoiling the performance gain :-(
[ODataRoute("LAOData")]
[HttpGet]
public IQueryable<LAC> GetLAOData(ODataQueryOptions<LAC> queryOptions)
{
using (IDbConnection connection = new SqlConnection(RoutingConstants.CascadeConnectionString))
{
var sql = "<giant sql query";
IQueryable<LAC> iqLac = null;
IEnumerable<LAC> sqlRes = connection.Query<LAC>(sql, commandTimeout: 300);
**IQueryable<LAC> iq = sqlRes.AsQueryable();
iqLac = queryOptions.ApplyTo(iq) as IQueryable<LAC>;
return iqLac;**
}
}
Most of the example we see on Stored procedure, Views support apparently returns List.
https://hackernoon.com/execute-a-stored-procedure-that-gets-data-from-multiple-tables-in-ef-core-1638a7f010c
Can we configure EF Core 2.2 to return IQueryable so that ODATA could
further filter out and then yield only desired counts say 10.?

Well, yes and no. You can certainly return an IQueryable, and you're already doing so, it seems. And, you can certainly further query via LINQ on that IQueryble, in memory.
I think what you're really asking, is if you can further query at the database-level, such only the ultimate result set you're after is returned from the database. The answer to that is a hard no. The stored procedure must be evaluated first. Once you've done that, all the result have been returned from the database. You can further filter in memory, but it's already too late for the database.
That said, you should understand that OData is fundamentally incompatible with the idea of using something like a stored procedure. The entire point is to describe the query via URL parameters - the entire query. You could use a view instead, but stored procedures should not be used along-side OData.

EF cannot return IQueryable from a Stored Procedure because the database engine itself doesn't provide a mechanism for selectively querying or manipulating execution of the script, you can't for instance do the following in SQL:
SELECT Field1, Field2
EXEC dbo.SearchForData_SP()
WHERE Field2 IS NOT NULL
ORDER BY Field3
The Stored Procedure is a black box to the engine, and because of this there are certain types of expressions and operations that you can use in SPs that you cannot use in normal set based SQL queries or expressions. For instance you can execute other stored procedures. SPs must be executed in their entirety, before you can process the results.
If the database engine itself cannot do anything to optimise the execution of Stored Procedures, its going to be hard for your ORM framework to do so.
This is why most documentation and examples around executing SPs via EF returns a List as that makes it clear that the entire contents of that list is in memory, casting that List to IQueryable with .AsQueryable() doesn't change the fact that the data is maintained within that List object.
There are joins among tables and we invoke (inline) table valued functions using SQL to get desired records.
What you are describing here is similar to what OData and EF try to offer you, mechanisms for composing complex queries. To take full advantage of OData and EF your should consider replicating or replacing your TVFs with linq statements. EF is RDBMS agnostic so it tries to use and enforce generic standards that can be applied to many database engines, not just SQLSERVER. When it comes to CTEs, TVFs and SPs the implementation and syntax in each database engine becomes a lot more specific even to specific versions in some cases. Rather than trying to be everything to everyone, the EF team has to enforce some limits so they can maintain quality of the services they have offered for us.
There is a happy medium that can be achieved however, where you can leverage the power of the two engines:
Design your SPs so that the filtering variables are passed through as parameters and restrict dependence on Stored Procedures to scenarios where the structure of the output is as efficient as you would normally need. You can then expose the SP as an Action endpoint in OData and the called can pass the parameter values directly through to the SP.
You can still wrap the response in an IQueryable<T> and decorate this action with the EnableQuery attribute, this will perform in memory $select, $expand and simple $filter operations but the service will still load the entire recordset into memory before constructing the response payload. This mechanism can still reduce bandwidth between the server and the client, just not between the database and the service layer.
Make different versions of your SP if you need to have different result structures for different use cases.
Use TVFs or Views only when the query is too complex to express easily using linq or you need to use Table Hints, CTEs, Recursive CTEs or Window Functions that cannot be easily replicated in Linq.
In many cases where CTEs (non recursive) are used, the expression can be easier to construct in Linq.
To squeeze the most performance from indexes you can use Table Hints in SQL, because we don't have tight control over how our Linq expressions will be composed into SQL it can take a lot of work to construct some queries in a way that the database can optimise them for us. In many scenarios, as with CTEs above, going through the process of rewriting your query in Linq can help avoid scenarios where you would traditionally have used table hints.
There are limits, when you want or need to take control, using specialised SQL Server concepts that EF doesn't support, you are making a conscious decision to have one and not the other.
I do not agree that OData and Stored Procedures are fundamentally incompatible there are many use cases where the two get along really well, you have to find the balance though. If you feel the need to pass through query options such as $select, $expand $filter, $top, $skip... to your Stored Procedure, either change your implementation to be constructed purely in Linq (so no SP) or change the client implementation so that you pass through formal parameters that can be handled directly in the SP.

Insert complex objects from C# into SQL Server without Entity Framework and data tables

How to insert complex objects from C# into SQL Server without using Entity Framework and data tables (I thought about Dapper.Contrib but it is only for entities).
I'm using stored procedures and Dapper for now, but I can insert only one object and also - only with dynamic parameters so it not what I looking for.
For example:
DynamicParameters p = new DynamicParameters();
p.Add("ID", user.UserInformation.ID);
p.Add("DELEGATORID", user.UserRole.DelegateID ?? string.Empty);
p.Add("APPROVER", user.UserRole.Approver);
p.Add("DELEGATOR", user.UserRole.IsDelegator);
p.Add("SEARCHTEXT", searchText);
and then insert. But I need to do 2 more inserts.
For example - I want to get 3 complex objects to insert in the stored procedure and then execute 3 stored procedure for each object
Thanks.

Ideally, you'd use an ORM that takes care of this but one option is to use raw SQL strings in that you build the entire query as a string (or in several strings), parameterize it and execute it. A quick google gives and example here; https://codereview.stackexchange.com/questions/59863/transaction-handling-for-multiple-sql-statements. Important points of note are;
Values are parameterized
The statements execute as a transaction (and so will all get rolled back in event of an issue

How to insert complex objects from C# into SQL Server without using Entity Framework and data tables
SQL Server can parse XML, and newer versions can also parse JSON. So the simplest way to insert shaped data into SQL Server without an ORM is to send an XML or JSON document and parse it on the server.

Do LINQ "queries" get compiled into actual SQL stored procedures?

I'm an ASP.NET n00b and I read in my ASP.NET book that your C# LINQ code gets made into actual SQL queries when you compile your project. I can't find anything on the internet to verify this; however, if it's true, I'm wondering if stored procedures are created from LINQ queries; and if not, what's the point of LINQ? I'm confused about why someone who knows how to write SQL would invest time learning something that's "valid C# and looks like SQL but isn't quite SQL" ...
I saw a related thread LINQ queries vs Stored procedures but it was written in 2011.

No, LINQ queries to databases do not get converted to stored procedures. They would be converted to SQL if the underlying database language is SQL, for example SQL Server / Oracle.
You are confusing ORM and LINQ. LINQ is more than querying database. It is also about querying general collections like arrays/List<T> (IEnumerable<T>).

Adding to above answer, LINQ is feature of .NET Framework which are strongly typed which aren't specific to Entity Framework (ORM) and even INDEPENDENT to database. There are different LINQ provider i.e. LINQ to Entities is responsible to translating LINQ to appropriate query on data and get result back.
So this will shine for someone who don't know SQL yet and who wants to deal table and data in terms of object.

Why I can't see my T-SQL string generated by a queryable

I'm noticed that some queries in my application are so slowly, so that's the reason why I want to know what is trying to accomplish my queries in LINQ TO SQL through entity framework.
In some sites, I realized that if you put your mouse over the IQueryable variable, you can see the T-SQL generated and at this moment, I can't see that.
I'd like to know if I'm doing a wrong configuration in my Entity Framework model

For Entity Framework you can see generated SQL query by inspecting your context Log property, or you can cast your IQueryable to System.Data.Objects.ObjectQuery and use method ToTraceString().

I want to suggest a different approach: Look at the real query in SQL Profiler. You can see all queries executed including parameter values. You can copy the query including parameter assignments to SSMS to debug it.

Getting data into and out of SQL tables with C#

Getting data into and out of SQL tables with C#
I built a C# app that puts data into a SQL table.
I am currently using the SqlConnection class in .net.
Got started with this article.
http://www.codeproject.com/KB/database/sql_in_csharp.aspx
I had thought about looking at Linq to SQL but have not done it before and wasn't sure how to get up and going.
Well today I ran across a bug where I was trying to add data that include a "'" in it which broke my insert statement. I got to doing some research and am starting to wonder what other problems are lurking. What if I try to add the data "drop table"?
My question is, is there a better model for data insertion? Do I look at Linq? Or do I check all my data before it is entered?

You discovered SQL Injection Attacks. It is not good policy to just append user supplied data to a SQL query for exactly the reason you specified. Any user of your system could try to steal or corrupt your data by injecting some SQL.
The way to deal with it is described in the link, but basically you specify the parameters and let the supplied classes handle properly escaping the data so that if someone passes "Drop Table", it will simply be entered as data.
Here's a great example from CodeBetter.com
SqlCommand command = connection.CreateCommand();
command.CommandText = "SELECT * FROM Customers WHERE CustomerID = #CustomerID";
command.Parameters.Add(
new SqlParameter("#CustomerID", SqlDbType.NChar, 5)).Value = customerID;
Alternatively, feel free to use LINQ to SQL. It will handle this for you and is much easier to work with from a developer perspective. You can drag and drop your database into your code and it will completely map every table. Then you can write LINQ's version of SQL statements right in your code where you'll get code completion and Compile time checking for errors. This SO question will get you started.
Here's some simple LINQ code that lets you read a customer from the database, write his/her name to the screen, then update his personalized greeting (all safe from SQL injection):
Customer myCustomer = (
from cust in myDatabase.Customers
where cust.CustomerID == userPassedCustomerID
select cust).Single();
Console.WriteLine(myCustomer.FullName);
myCustomer.PersonalizedGreeting = userPassedGreeting;
myDatabase.SubmitChanges();

The best way to work with database queries that need to be mingled with user input is to parameterized them. Now LINQ to SQL will do this for you but plain old ADO.NET lets you do it too (as the article I linked describes).
This works whether you are calling stored procedures or creating dynamic queries in your application to send to your RDBMS.

It appears you were doing some dynamic SQL, i.e. creating SQL statements on the fly, using various string manipulations in C# and then running these.
This way of doing is very versatile but introduces the risk of SQL injection. This was an accidental situation, self inflicted, but assuming that some of the elements used in building the SQL statement are provided by way of html fields, an malicious user could craft a particular string to effectively "DROP TABLE" or worse...
There are many ways to work around this situation, the most common one being to use parametrized SQL templates. With this technique, the variable parts of a SQL statement are provided to SQL in separate arguments (called parameter at the level of SQL). The ADO Command object is the vehicle used for invoking parametrized queries.
LINQ to SQL could also be used to handle this type of thing as well.