Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 8 years ago.
Improve this question
I am trying to do a fuzzy match of records in two Account table using .NET Entity Framework.
I wrote some code like this but it has bad performance like 1 min a record.
ARSalesforceEntities arsf = new ARSalesforceEntities(); //dbcontext
Salesforce_FFEntities ffsf = new Salesforce_FFEntities(); //dbcontext
var araccounts = arsf.Accounts; //dbset contains 400000 records
var ffaccounts = ffsf.Accounts; //dbset contains 6000 records
IDCONV byName = new IDCONV();
IDCONV byAddress = new IDCONV();
foreach (var ffaccount in ffaccounts)
{
Console.WriteLine(++count);
foreach (var araccount in araccounts)//this line goes every slow like 1 min
{
Basically, I am comparing the records in two tables with complicated logic.
How can I greatly improve the performance of the code?
Thank you
This line:
var ffaccounts = ffsf.Accounts;
is what's hitting you hard. You're basically assigning the IQueryable to a variable which, when accessed in your inner loop, re-queries the database everytime. I imagine simply adding ToList() on the end will drastically improve your performance:
var ffaccounts = ffsf.Accounts.ToList();
That's assuming of course that it's acceptable to materialise the 6000 rows into memory.
If not, then you might want to consider writing the logic in SQL and doing all the work in the DB instead...
Related
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 5 years ago.
Improve this question
I'm trying to do something like -
List<int> accountList = new List<int>();
accountList .Add(1);
accountList .Add(27);
var rec = _db.Accounts.Where(a=> accountList.Contains(a.accountId)).Take(10);
My code is a little more complicated than this - there are several other conditions in the where clause, but this is the bit that is causing problems - nothing gets returned even when there are matching values.
Basically I want it to retrieve all the records where accountId matches a value in my list.
Any pointers?
The sample above is giving me a cant convert lambda error.
You are missing a bit on your contains version
var hold2 = _db.Accounts.Where(a => find.Contains(a.accountDd)).Take(10).ToList();
Have you tried using any
var hold2 = _db.Accounts.Where(a => accountList.Any(m => m == a.accountId)).Take(10).ToList();
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 5 years ago.
Improve this question
I've always used Contains method to gain SQL IN functionality to my LINQ queries.
But this time, very strange thing occured.
I've got an array of string like this :
string[] FilenamesToParse = {"JOHN_X200-", "DOE_X300-", "FOO_X300_M-"};
Then I've used this array just like below:
var result = (from dps in appProcessList
where FilenamesToParse.Contains(dps.FileName)
select dps.Devices).ToList()
Above query resulted with 0 result but I'm sure that there are filenames contains words defined in FilenamesToParse array.
So I've tried below snippet and Contains worked.
foreach (var applicationProcess in appProcessList)
{
if (applicationProcess.FileName.Contains(FilenamesToParse[0]))
{
}
}
Where am I wrong here ?
Thanks in advance.
Your two appraoches aren´t similar. In your linq you´re iterating your FilenamesToParse-array and check if any of its elements exactly matches dps.FileName, whereby in the second one you iterate appProcessList and check if its FileName-property contains the first FilenamesToParse.
The following would be the linq-approach similar to your loop:
var result = (from dps in appProcessList
where FilenamesToParse.Any(x => dps.FileName.Contains(x))
select dps.Devices).ToList()
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 7 years ago.
Improve this question
I am adding multiple entities in the database using AddRange in Entity Framework:
foreach (string tagNumber in notPresent)
{
element = new TagMaster { Name = Guid.NewGuid().ToString(), IsActive = true };
element.TagCollections.Add(new TagCollection { TagNumber = tagNumber });
newTagMasters.Add(element);
}
dbContext.TagMasters.AddRange(newTagMasters);
dbContext.SaveChanges();
What I was expecting is that by adding the complete collection in context using AddRange method, there would be a single query that will be sent to database. But to my surprise, I see multiple insert statements for each record to be inserted.
Any Insights?
The problem you are running in is that sadly the entity framework commands know NO bulk inserts. Instead they generate 1 statement per line that you want to insert.
There is no workaround to this.
The only possiblity to get 1 single statement that does all the inserts is to use specific classes or libraries. As example here SqlBulkCopy which needs no external lib to be downloaded to work.
Here a link to the msdn site:
https://msdn.microsoft.com/de-de/library/system.data.sqlclient.sqlbulkcopy(v=vs.110).aspx
The usage is quite easy. You only give the constructor your connection (after opening it beforehand!) and tell it what it shall write to teh server and what the destination table name is. Then you only need to close the connection afterwards again.
sqlcon.Open();
using (SqlBulkCopy sqlBulkCopyVariable= new SqlBulkCopy(sqlcon))
{
sqlBulkCopyVariable.BulkCopyTimeout = 600; // 10 minutes timeout
sqlBulkCopyVariable.DestinationTableName = "targetTableName";
sqlBulkCopyVariable.WriteToServer(MyData);
}
sqlcon.Close();
The WriteToServer takes DataTable, DataReader or even arrays of DataRow. The exact implementation there would depend on how you want to give the data to it. So far from my personal experience: That class is quite fast and generates only 1 single statement. BUT it is only there for SqlClients. Thus if you have a different client you need to look up which class or external library would be best fitting for you.
I am afraid insertions through Linq is not optimized as you would expect. It does that by multiple insert statements as you observed. You could instead bypass Linq in those cases and use the bulk copying alternatives instead (ie: for MS SQL server use SqlBulkCopy class, for postgreSQL use Copy etc).
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
I try to group by StringId and then by Name
var a = myList
.GroupBy(item => new TcGroupByKey()
{StringId = item.Id, ChannelName = item.ChannelName}).ToList();
I have created this inner class TcGroupByKey
so I can pass the result to TrackingChannelRow ctor
and will get strongly type argument and not object
public TrackingChannelRow(ManageTcModel.TcGroupByKey tcGroupByKey,
IGrouping<ManageTcModel.TcGroupByKey, TrackingChannelItem> subChannels,
IEnumerable<Manager.TrackingChannels.Model.ToolbarItem> toolbars,
IEnumerable<Manager.TrackingChannels.Model.BundleItem> bundles)
{
But the group by doesn't work.
What am I doing wrong?
I think that what you're looking for is to group by composite key...
Something similar to what Marc described in this post Linq to SQL with Group by
The key is to use 'anonymous' types - not the named one like you did - that LINQ can translate into SQL, otherwise there's an issue and that code can only run 'post SQL' (so you enumerate first then group but lose on the SQL integration, performance).
So, in your case it'd be something like this...
new {StringId = item.Id, ChannelName = item.ChannelName}
...instead of new TcGroupByKey() {StringId = item.Id, ChannelName = item.ChannelName}
There is similar problem with doing a 'Select' into anonymous (works) or named type (doesn't - unless, as I mentioned above, you separate the SQL and C#-only expressions part, a bit simplified).
hope this helps
You are asking linq to group your list by an object he doesn't recognize.
What's wrong with:
var a = myList.GroupBy(Id).GroupBy(ChannelName).ToList();
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 9 years ago.
Improve this question
I wish to make all my save transactions at the same time.
I'm using Subsonic 3.0 with active record. I've seen some post with MyClassCollection.Save();
However when i "Run Custom tool" i don't get collections. Obviously i'm missing a trick here?
There is of course another way to do this, using BatchQuery. I had an issue with this. Appreciate any thoughts, as you can see i've very new too all this.
var provider = ProviderFactory.GetProvider("MyProvider");
var batch = new BatchQuery(provider);
foreach (var cdnEntry in cdnEntries)
{
var query1 = new Insert(provider).Into<Clip>
("Author").Values(
cdnEntry.Author);
batch.QueueForTransaction(query1);
}
batch.ExecuteTransaction(); // nothing happens as query count alway = 0
I obviously need to read on the different patterns available for subsonic. Tho i have got the batchquery working. The above code works. I had few silly sql typo's giving me grief.