I have the following code:
ILookup<string, costcenterdata> areas = p_data.CostcenterDatas.ToLookup(p => OracleDataProvider.GetAreaForClient(p_data.ClientID), p => p);
it takes acollection of pocos and generates a lookup. For determining the key it goes to an ORACLE table and performs a select foo from data where bar.
Nothing fancy. This is done in GetAreaFromClient which does open conncection, read, close connection with ODP.net. No Entity Framework or other newfangled stuff.
But sometimes it seems to produce a lock. the code above waits endlessly and in the end, we have to cancel the connection in the database.
Can it be tha the toLockup somehow produces a parallel access that may or may not produce this lock?
Oracle's locking/blocking is more akin to sql server's snapshot isolation, so there shouldn't be any blocking of a simple select.
Are you sure that method is only doing a select or that it's connection wasn't used for something else?
In any case a dba should be able to help you determine why the connection is blocked. You could also try looking yourself if you have permissions with something like:
select
ao.object_name,
ao.object_type,
s.sid,
s.username,
s.status,
s.osuser,
s.machine,
s.program,
s.module,
s.wait_time,
s.seconds_in_wait,
lo.inst_id,
s.*, lo.*
from
gv$locked_object lo ,
gv$session s,
all_objects ao
where
s.sid = lo.session_id
and
lo.object_id = ao.object_id
and s.username = 'connectionUserName';
Related
I want to ask about the execution timing of a LinqToSql query.
From my understanding refer to this MSDN blog. It seems that a LinqToSql query will only execute when
IQueryable's property is accessed
IQueryable's function (which is not returning IQueryable/IEnumeration type) is called
However, I did an experiment like that:
var ents = from ent in dal.ents
select ent;
string s1 = ents.first().Value1; // I got 1 here
Console.ReadLine(); // When the system is waiting for my input. I change the same record manually in DB, I change Value2 of ent from 2 to 3 here.
string s2 = ents.first().Value2 // I got 2 here.
Why am I still getting "2" for s2?
From my understanding, ents.first().Value2 should connect to the DB again and get the new Value2. Why am I still getting the old value?
As soon as you get Value1 on this line, the call is made to the db
string s1 = ents.first().Value1;
Then it keeps the object in memory (along with Value2). It doesn't call the database again when you try to access Value2.
Finally, I think I found the working principal behind.
L2S is really working like
LinqToSql query will only CONNECTION TO DB AND EXECUTE when
IQueryable's property being accessed
IQueryable's function (which is not returning IQueryable / IEnumeration type) being called
But in additions, after L2S fetch data from DB for each record at the first time. It will cache the record by its PK.
Eventually, on each further fetch. It will check if the record has been fetched before.
yes, It will use the cached version of the record instead of the DB version.
If no, It will use the DB version.
P.S. The lifetime of the cached records will last until the DBContext being released.
I have updated my ASP.NET app from NpgSQL 2.2.5 to 3.0.1. In the breaking changes it's specified that they have removed the Preload Reader support. So I remove it from the string connection.
Testing my web app, I got the error "An operation is already in progress." specially in the linq query like this:
var plugins =
from p in _pluginRepository.GetPlugins() // this method return this: GetAll().OrderBy(p => p.Created)
join e in _userPluginRepository.GetByUserId(user.Id).ToList() on p.Id equals e.Plugin.Id into pe
from e in pe.DefaultIfEmpty()
select new PluginViewModel
{
Active = e != null,
Name = p.Translations.ToUserLanguage(loggedInUser),
Key = p.Key,
PluginId = p.Id,
SettingId = e == null ? 0 : e.Id,
ExpireDate = e != null && e.ExpireDate.HasValue ? e.ExpireDate.Value : (DateTime?) null,
Grants = e == null ? UserPluginGrants.None.GetHashCode().ToString() : e.Grants.GetHashCode().ToString()
};
To solve this error, I have to append a ToList after the GetPlugins method.
Is this the correct behavior to use without Preload Reader? Why?
In Npgsql 2.x, using the Preload Reader made Npgsql pull the entire result set of the query from the database into the memory of your application. This freed the connection and allowed another command to be executed while still traversing the resultset of the first query. In other words, it allowed you to program as if you could execute multiple queries concurrently (known sometimes as MARS), although behind the scenes this was implemented inefficiently.
Adding a ToList() does exactly the same thing - pull everything into client memory, only it happens in your application code instead of in the database driver. So it's definitely an acceptable way to port your application from Npgsql 2.x to 3.x.
Now, if the result set being pulled (in this case GetPlugins) is small, this is a perfectly valid approach. If it's big, however you should look into alternatives. In your example, the join could be sent to the database, making your Linq expression translate into a single SQL query and eliminating the need for multiple queries (ORMs such as Entity Framework can usually do this for you). A more extreme solution would be to use multiple database connections, but that is heavier and also problematic if you're using transactions.
Note that there's an issue open for implementing true MARS in Npgsql (although it isn't likely to be implemented very soon): https://github.com/npgsql/npgsql/issues/462
I had the following:
List<Message> unreadMessages = this.context.Messages
.Where( x =>
x.AncestorMessage.MessageID == ancestorMessageID &&
x.Read == false &&
x.SentTo.Id == userID ).ToList();
foreach(var unreadMessage in unreadMessages)
{
unreadMessage.Read = true;
}
this.context.SaveChanges();
But there must be a way of doing this without having to do 2 SQL queries, one for selecting the items, and one for updating the list.
How do i do this?
Current idiomatic support in EF
As far as I know, there is no direct support for "bulk updates" yet in Entity Framework (there has been an ongoing discussion for bulk operation support for a while though, and it is likely it will be included at some point).
(Why) Do you want to do this?
It is clear that this is an operation that, in native SQL, can be achieved in a single statement, and provides some significant advantages over the approach followed in your question. Using the single SQL statement, only a very small amount of I/O is required between client and DB server, and the statement itself can be completely executed and optimized by the DB server. No need to transfer to and iterate through a potentially large result set client side, just to update one or two fields and send this back the other way.
How
So although not directly supported by EF, it is still possible to do this, using one of two approaches.
Option A. Handcode your SQL update statement
This is a very simple approach, that does not require any other tools/packages and can be performed Async as well:
var sql = "UPDATE TABLE x SET FIELDA = #fieldA WHERE FIELDB = #fieldb";
var parameters = new SqlParameter[] { ..., ... };
int result = db.Database.ExecuteSqlCommand(sql, parameters);
or
int result = await db.Database.ExecuteSqlCommandAsync(sql, parameters);
The obvious downside is, well breaking the nice linqy paradigm and having to handcode your SQL (possibly for more than one target SQL dialect).
Option B. Use one of the EF extension/utility packages
Since a while, a number of open source nuget packages are available that offer specific extensions to EF. A number of them do provide a nice "linqy" way to issue a single update SQL statement to the server. Two examples are:
Entity Framework Extended Library that allows performing a bulk update using a statement like:
context.Messages.Update(
x => x.Read == false && x.SentTo.Id == userID,
x => new Message { Read = true });
It is also available on github
EntityFramework.Utilities that allows performing a bulk update using a statement like:
EFBatchOperation
.For(context, context.Messages)
.Where(x => x.Read == false && x.SentTo.Id == userID)
.Update(x => x.Read, x => x.Read = true);
It is also available on github
And there are definitely other packages and libraries out there that provide similar support.
Even SQL has to do this in two steps in a sense, in that an UPDATE query with a WHERE clause first runs the equivalent of a SELECT behind the scenes, filtering via the WHERE clause, then applying the update. So really, I don't think you need to be worried about improving this.
Further, the reason why it's broken into two steps like this in LINQ is precisely for performance reasons. You want that "select" to be as minimal as possible, i.e. you don't want to load any more objects from the database into in memory objects than you have to. Only then do you alter objects (in the foreach).
If you really want to run a native UPDATE on the SQL side, you could use a System.Data.SqlClient.SqlCommand to issue the update, instead of having LINQ give you back objects that you then update. That will be faster, but then you conceptually move some of your logic out of your C# code object model space into the database model space (you are doing things in the database, not in your object space), even if the SqlCommand is being issued from your code.
I have bought the premium version of LINQPad. I thought it would also be possible to perform cross data base queries with DevForce models.
There are two ways to do this. The simplest is the drag-and-drop
approach: hold down the Ctrl key while dragging additional databases
from the Schema Explorer to the query editor. To access those
additional databases in your queries, use database.table notation,
e.g., Northwind.Regions.Take(100). The databases that you query must
reside on the same server.
The second approach is to list the extra database(s) that you want to
query in the connection properties dialog. This dialog also lets you
choose databases from linked servers. Here's how to proceed:
Add a new LINQ to SQL connection.
Choose Specify New or Existing Database and choose the primary database that you want to query.
Click the Include Additional Databases checkbox and pick the extra database(s) you want to include. You can also choose databases from
linked servers in this dialog.
Source
But obviously there isn't any way, is it? Anyone a solution for this?
Cross-database querying works only with standard SQL Server connections, with databases on the same server or on linked servers. The main rationale is to ensure server-side joining (otherwise you'd end up pulling entire tables back to the client whenever you joined).
I have considered adding a feature to LINQPad to allow arbitrary cross-database queries, because sometimes it would be useful even with client-side joining. However, getting this to work with custom data contexts (such as DevForce or Entity Framework) turned out to be really tricky, and so the feature ended up in the "too-hard basket". A major problem was dealing with namespace/assembly/app.config conflicts.
Bear in mind that there's nothing to stop you from pressing F4 and adding a reference to an assembly containing an additional datacontext. Of course, you'd have to manually instantiate the second data context, but that shouldn't be a huge deal. You'll still get autocompletion, and you'll still be able to see its schema in the tree view if you create a separate connection for it. And functionally, that's what you'd end up with anyway, if LINQPad supported multi-connection queries.
What's special about LINQPad's cross-database querying support for SQL Server is that it does something you couldn't otherwise do simply by adding a reference to another assembly, which is to allow efficient cross-database querying by leveraging server-side joins.
You can instantiate as many contexts as you like to disparate SQL instances and execute pseudo cross database joins, copy data, etc. Note, joins across contexts are performed locally so you must call ToList(), ToArray(), etc to execute the queries using their respective data sources individually before joining. In other words if you "inner" join 10 rows from DB1.TABLE1 with 20 rows from DB2.TABLE2, both sets (all 30 rows) must be pulled into memory on your local machine before Linq performs the join and returns the related/intersecting set (20 rows max per example).
//EF6 context not selected in Linqpad Connection dropdown
var remoteContext = new YourContext();
remoteContext.Database.Connection.ConnectionString = "Server=[SERVER];Database="
+ "[DATABASE];Trusted_Connection=false;User ID=[SQLAUTHUSERID];Password="
+ "[SQLAUTHPASSWORD];Encrypt=True;";
remoteContext.Database.Connection.Open();
var DB1 = new Repository(remoteContext);
//EF6 connection to remote database
var remote = DB1.GetAll<Table1>()
.Where(x=>x.Id==123)
//note...depending on the default Linqpad connection you may get
//"EntityWrapperWithoutRelationships" results for
//results that include a complex type. you can use a Select() projection
//to specify only simple type columns
.Select(x=>new { x.Col1, x.Col1, etc... })
.Take(1)
.ToList().Dump(); // you must execute query by calling ToList(), ToArray(),
// etc before joining
//Linq-to-SQL default connection selected in Linqpad Connection dropdown
Table2.Where(x=>x.Id = 123)
.ToList() // you must execute query by calling ToList(), ToArray(),
// etc before joining
.Join(remote, a=> a.d, b=> (short?)b.Id, (a,b)=>new{b.Col1, b.Col2, a.Col1})
.Dump();
localContext.Database.Connection.Close();
localContext = null;
I am often comparing data in tables in different databases. These databases do not have the same schema. In TSQL, I can reference them with the DB>user>table structure (DB1.dbo.Stores, DB2.dbo.OtherPlaces) to pull the data for comparison. I like the idea of LINQPad quite a bit, but I just can't seem to easily pull data from two different data contexts within the same set of statements.
I've seen people suggest simply changing the connection string to pull the data from the other source into the current schema but, as I mentioned, this will not do. Did I just skip a page in the FAQ? This seems a fairly routine procedure to be unavailable to me.
In the "easy" world, I'd love to be able to simply reference the typed datacontext that LINQPad creates. Then I could simply:
DB1DataContext db1 = new DB1DataContext();
DB2DataContext db2 = new DB2DataContext();
And work from there.
Update: it's now possible to do cross-database SQL Server queries in LINQPad (from LINQPad v4.31, with a LINQPad Premium license). To use this feature, hold down the Control key while dragging databases from the Schema Explorer to the query window.
It's also possible to query linked servers (that you've linked by calling sp_add_linkedserver). To do this:
Add a new LINQ to SQL connection.
Choose Specify New or Existing Database and choose the primary database you want to query.
Click the Include Additional Databases checkbox and pick the linked server(s) from the list.
Keep in mind that you can always create another context on your own.
public FooEntities GetFooContext()
{
var entityBuilder = new EntityConnectionStringBuilder
{
Provider = "Devart.Data.Oracle",
ProviderConnectionString = "User Id=foo;Password=foo;Data Source=Foo.World;Connect Mode=Default;Direct=false",
Metadata = #"D:\FooModel.csdl|D:\FooModel.ssdl|D:\FooModel.msl"
};
return new FooEntities(entityBuilder.ToString());
}
You can instantiate as many contexts as you like to disparate SQL instances and execute pseudo cross database joins, copy data, etc. Note, joins across contexts are performed locally so you must call ToList(), ToArray(), etc to execute the queries using their respective data sources individually before joining. In other words if you "inner" join 10 rows from DB1.TABLE1 with 20 rows from DB2.TABLE2, both sets (all 30 rows) must be pulled into memory on your local machine before Linq performs the join and returns the related/intersecting set (20 rows max per example).
//EF6 context not selected in Linqpad Connection dropdown
var remoteContext = new YourContext();
remoteContext.Database.Connection.ConnectionString = "Server=[SERVER];Database="
+ "[DATABASE];Trusted_Connection=false;User ID=[SQLAUTHUSERID];Password="
+ "[SQLAUTHPASSWORD];Encrypt=True;";
remoteContext.Database.Connection.Open();
var DB1 = new Repository(remoteContext);
//EF6 connection to remote database
var remote = DB1.GetAll<Table1>()
.Where(x=>x.Id==123)
//note...depending on the default Linqpad connection you may get
//"EntityWrapperWithoutRelationships" results for
//results that include a complex type. you can use a Select() projection
//to specify only simple type columns
.Select(x=>new { x.Col1, x.Col1, etc... })
.Take(1)
.ToList().Dump(); // you must execute query by calling ToList(), ToArray(),
// etc before joining
//Linq-to-SQL default connection selected in Linqpad Connection dropdown
Table2.Where(x=>x.Id = 123)
.ToList() // you must execute query by calling ToList(), ToArray(),
// etc before joining
.Join(remote, a=> a.d, b=> (short?)b.Id, (a,b)=>new{b.Col1, b.Col2, a.Col1})
.Dump();
remoteContext.Database.Connection.Close();
remoteContext = null;
I do not think you are able to do this. See this LinqPad request.
However, you could build multiple dbml files in a separate dll and reference them in LinqPad.
Drag-and-drop approach: hold down the Ctrl key while dragging additional databases
from the Schema Explorer to the query editor.
Use case:
//Access Northwind
var ID = new Guid("107cc232-0319-4cbe-b137-184c82ac6e12");
LotsOfData.Where(d => d.Id == ID).Dump();
//Access Northwind_v2
this.NORTHWIND_V2.LotsOfData.Where(d => d.Id == ID).Dump();
Multiple databases are as far as I know only available in the "paid" version of LinqPad (what I wrote applies to LinqPad 6 Premium).
For more details, see this answer in StackOverflow (section "Multiple database support").