Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Closed 9 years ago.
Improve this question
If I have same table and same multiple database servers. How to connect to multiple database servers, obtain those records from each database server and then display the first 10 of the combined results ?
Say for instance you are querying the multiple instances using different connection strings for a sample table Orders, you could try the following:
var orders = ConfigurationManager.ConnectionStrings.Cast<ConnectionStringSettings>()
// filter to the relevant connection strings
.Where(s => s.ConnectionString.ToLower().Contains("metadata"))
.SelectMany(s => {
// for each connection string, select a data context
using(var context = new NorthwindEntities(s.ConnectionString)) {
// for each context, select all relevant orders
return context.Orders.ToArray();
} // and dispose of the context when done with it
})
.Take(10)
.ToList();
Here are a couple solutions off the top of my head.
Solution 1:
1 - Create a staging database / table on server A.
2 - Import all data from all servers into table.
3 - Query table to get results.
Solution 2:
1 - Create a Linked server for each server B .. Z on server A.
2 - Create Query using 4 part notations on linked servers.
Overall, solution 2 can be slow since you are using distributed transactions.
Solution 1 allows you to store the aggregated results that can be indexed (for speed) and can be queried multiple times.
As for importing the data from server to server, just pick a way to do it. There are two many solutions are there to get into the particulars.
Related
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 7 years ago.
Improve this question
I am adding multiple entities in the database using AddRange in Entity Framework:
foreach (string tagNumber in notPresent)
{
element = new TagMaster { Name = Guid.NewGuid().ToString(), IsActive = true };
element.TagCollections.Add(new TagCollection { TagNumber = tagNumber });
newTagMasters.Add(element);
}
dbContext.TagMasters.AddRange(newTagMasters);
dbContext.SaveChanges();
What I was expecting is that by adding the complete collection in context using AddRange method, there would be a single query that will be sent to database. But to my surprise, I see multiple insert statements for each record to be inserted.
Any Insights?
The problem you are running in is that sadly the entity framework commands know NO bulk inserts. Instead they generate 1 statement per line that you want to insert.
There is no workaround to this.
The only possiblity to get 1 single statement that does all the inserts is to use specific classes or libraries. As example here SqlBulkCopy which needs no external lib to be downloaded to work.
Here a link to the msdn site:
https://msdn.microsoft.com/de-de/library/system.data.sqlclient.sqlbulkcopy(v=vs.110).aspx
The usage is quite easy. You only give the constructor your connection (after opening it beforehand!) and tell it what it shall write to teh server and what the destination table name is. Then you only need to close the connection afterwards again.
sqlcon.Open();
using (SqlBulkCopy sqlBulkCopyVariable= new SqlBulkCopy(sqlcon))
{
sqlBulkCopyVariable.BulkCopyTimeout = 600; // 10 minutes timeout
sqlBulkCopyVariable.DestinationTableName = "targetTableName";
sqlBulkCopyVariable.WriteToServer(MyData);
}
sqlcon.Close();
The WriteToServer takes DataTable, DataReader or even arrays of DataRow. The exact implementation there would depend on how you want to give the data to it. So far from my personal experience: That class is quite fast and generates only 1 single statement. BUT it is only there for SqlClients. Thus if you have a different client you need to look up which class or external library would be best fitting for you.
I am afraid insertions through Linq is not optimized as you would expect. It does that by multiple insert statements as you observed. You could instead bypass Linq in those cases and use the bulk copying alternatives instead (ie: for MS SQL server use SqlBulkCopy class, for postgreSQL use Copy etc).
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
I've a complicated SQL query. I have to retrieve that from C#. What is the best method to retrieve complicated SQL queries? (Like QueryByAttribute, FetchXML, QueryExpression etc.)
Here is my code:
SELECT R.Name
FROM role R
WHERE R.roleid IN
(SELECT roleid
FROM systemuserroles SR
WHERE SR.systemuserid IN
(SELECT S.systemuserid
FROM systemuser S
WHERE S.new_departmentid3 =
(SELECT new_departmentid3
FROM systemuser S
WHERE S.systemuserid = '8B8825F9-6B27-E411-8BA9-000C29E0C100')))
Thanks for the replies.
If you are using an on-premise system I would use the Filtered Views and create a SQL query against them directly. This is by far the best performing option and is fully supported.
If this isn't an option because you are using CRM Online then FetchXML will give you the best performance available.
I'm going to disagree with Ben on a few issues:
The Filtered View contains security checks within them, and are designed to be able to run reports for users (they are not technically supported, but I have had only one breaking change over the course of 12 rollups, and it was realtively minor). The Non-Filtered Views are nearly identical, except that they don't contain all of the extra joins to ensure the user has access to query the information they are attempting to. So in this aspect, the Non-Filtered Views are going to give you the best possible performance, but I would recommend only using it when it makes a big performance difference, and only for reports. (Theoretically you could go directly to the Tables, but this is seems much more likely to be changed by Microsoft with any given Roll Up).
The best possible performance for large data requests available for online is not Fetch-Xml, but actually Odata since the payload is much smaller with O-Data than the Fetch Xml. However there are some technical limitations to using oData (You wouldn't be able to do your current query in one call due to it having too many joins, but you could do it in two).
P.S. I think this is an easier to read equivalent SQL statement:
SELECT R.Name
FROM role R
INNER JOIN systemuserroles SR ON R.roleId = SR.roleId
INNER JOIN SystemUser S ON SR.systemuserid = S.systemuserid
INNER JOIN systemuser S2 ON S.new_departmentid3 = S2.new_departmentid3
WHERE S2.systemuserid = '8B8825F9-6B27-E411-8BA9-000C29E0C100'
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 8 years ago.
Improve this question
I am trying to do a fuzzy match of records in two Account table using .NET Entity Framework.
I wrote some code like this but it has bad performance like 1 min a record.
ARSalesforceEntities arsf = new ARSalesforceEntities(); //dbcontext
Salesforce_FFEntities ffsf = new Salesforce_FFEntities(); //dbcontext
var araccounts = arsf.Accounts; //dbset contains 400000 records
var ffaccounts = ffsf.Accounts; //dbset contains 6000 records
IDCONV byName = new IDCONV();
IDCONV byAddress = new IDCONV();
foreach (var ffaccount in ffaccounts)
{
Console.WriteLine(++count);
foreach (var araccount in araccounts)//this line goes every slow like 1 min
{
Basically, I am comparing the records in two tables with complicated logic.
How can I greatly improve the performance of the code?
Thank you
This line:
var ffaccounts = ffsf.Accounts;
is what's hitting you hard. You're basically assigning the IQueryable to a variable which, when accessed in your inner loop, re-queries the database everytime. I imagine simply adding ToList() on the end will drastically improve your performance:
var ffaccounts = ffsf.Accounts.ToList();
That's assuming of course that it's acceptable to materialise the 6000 rows into memory.
If not, then you might want to consider writing the logic in SQL and doing all the work in the DB instead...
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 9 years ago.
Improve this question
I need to select a product for a user based on other data in the database.
If the data is filtered out on the database that will require less data to be send to the server.
User (Id)
Product (code)
Access (User_Id, code) // Matching users to object codes
Will this query execute on the database sending back the minimal amout of data?
var products = QueryOver.Of<Access>()
.Where(a => a.User_Id == User.Id())
.Select(Projections.Property<Acces>(a => a.Code));
var access = QueryOver.Of<Product>()
.WithSubquery.WhereProperty(h => h.Code)
.In(products)
.Future();
This is very reasonable way how to filter data. The result of your queries would look like one SELECT against the DB:
SELECT ...
FROM Product
WHERE Code IN (SELECT Code FROM Access WHERE UserId = #userId)
So, this will for sure be executed on the DB Server, less data will be transfered, and what's more, it also would allow you to do the correct paging (if needed) - this scenario is the way how to filter parent over its one-to-many relations (find Parents which child has...)
Maybe check these Join several queries to optimise QueryOver query, NHibernate QueryOver - Retrieve all, and mark the ones already "selected"
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Closed 9 years ago.
Improve this question
I have two tables:
**Users**
UserID | UserName | Password
**Task**
TaskId | Hours | UserID (Empty as of now)
I need to assign UserID in Tasks table to tasks so that all users gets tasks of even hours. I have about 5000 tasks in database and Hours column value range from 1 to 30.
How can it be done with SQL Server query OR LINQ?
This sounds like the partition problem (see here), where you are trying to assign the tasks so the sum of the hours for each user is the same.
In some situations, the problem is easily solvable (for instance, if all tasks are 1 hour in length). In other situations, the problem has no solution (for instance, if there are more users than tasks). As a hint, when a problem has such extreme variations, it probably cannot be solved by using a SQL query.
Of course, you can represent the data in tables. And, you can use a cursor over some query with complicated logic, and call this SQL.
The Wikipedia page has descriptions of several different possible algorithms. Good luck.