I have an existing table for Users in my database that contains around one million record . my database is MS sql server 2016 . I am working code first . Now I will add an additional column for that table which will contain an invitaion code .
I need to update the value of all existing users to have a unique invitaion code for that new column.
I need to make something that will run one time only (may be in the seed method) as this data will be updated and any new user will be created in future will have its invitaion code while registering .
So this will be applied for old users , what is the best way to do that regarding the performance and speed and i have too much records in that table , is it seed method that will run at start of the app or what ?
I think a script SQL would be the best solution, you run it ones and you are done.
A seed method would be a solution if you are in early development mode, so you may delete records and re-run the seed method, but if your database is stable a script would be better in my opinion.
1) Create a migration with the added column (you'll have to mark it as nullable)
2) Create another migration where you
Populate the column with whatever values you want.
You do this by adding a custom sql statement
Sql(#"UPDATE YourTableName set YourColumnName = insert logic here")
3) Create another migration modifying the column to be non-nullable.
You may be able to add the custom sql statement at the end of the first migration or the beginning of the last migration.
Related
There is mssql table in an external customer network. The aim is to create and reflect the same table in the local server. External mssql table, of course, can be changed (data) every hour and somehow I have to check for changes and reflect that changes in local table when new rows are added/deleted or updated. Is there any efficient way to do it? Additionaly i know that this table will have thousands of records. First of all, I thought about some windows service application but have no idea what approach to do, I do not think datatable/dataset with regards to so much records is fine as i remember memory out of exception in past. Any ideas?
The way I would go about it is to create triggers on the existing tables that upon insert, update and delete would insert into a new sync table (or a sync table per existing table) which would mark the change pending synchronization. Your C# code would read from this table on a schedule, apply changes to the local DB and delete the rows from the 'pending' table.
For example, this is how Azure SQL Data Sync works; it creates a table per existing table in the source table and then checks all these tables. I think, depending on how many tables you have and the structure etc, you could write something like JSON in just the one table instead, and it would be easier to check one table than plenty (obviously this depends on how many actual tables we're talking about).
How to clear this table: __EFMigrationsHistories
I don't want to delete my migration or something, I explicitly want to clear this table by code.
Edit:
I would try to explain a little bit why i want to do this! I want to call on every startup the same (and the only one) migration.
And this migration loops trough all my models and call's the onUpdateMethod, so every model can handle his update by itself.
If you want to Clear Data within SQL here is the Query:
DELETE FROM [TableName]
If you want to clear Data Within your application run this query using Entity Framework like below:
context.Database.ExecuteSqlCommand("TRUNCATE TABLE [TableName]");
The TRUNCATE TABLE statement is a fast, efficient method of deleting all rows in a table. TRUNCATE TABLE is similar to the DELETE statement without a WHERE clause. However, TRUNCATE TABLE is faster and uses fewer system and transaction log resources.
I am developing an utility that transfers the data from one structure into another data structure which is then stored in a SQl Server database controlled by Entity-Framework.
Each time I change the target database model I am doing the following;
Deleting the data from the target database.
Running "dbcc checkident("table", reseed)" on each table (many tables) to reset the ID values.
Reseeding the Database using "update-database"
After reading the following http://www.entityframeworktutorial.net/code-first/database-initialization-strategy-in-code-first.aspx I think I have two options. I don't think I want to roll my own initialiser.
The first option is to use one of the drop create strategies during development , (dropCreateifmodelChanges, or DropCreateAlways) Will the Drop Create process delete the existing target data, and reseed the Identity counter?
The second option is to somehow include the deletion, and the ID reset in the seed function. Can this be done? If so - how?
Is there a better option? Does anyone have any advice?
Me too had a same sort of situation and I have chosen the manual method which you have mentioned.B'cos it gave me more control over the automatic method.
But you can use the DropCreateDatabaseAlways strategy if you like.Yes,it creates fresh database.That means all the PK values will be reseeded.You can use this strategy only when you're developing the app on dev environment.
I need to insert large ammount of data each day into my SQL Server database. The data is inserted from the file, where rows represent data from this year, and some of these rows may be changed, while new rows are added to the file each day, so I need to check if some row is changed in the file and to update database and always to insert new rows.
So, what approach do you recommend (clear database/bulk insert, read line-by-line and insert in C#, ssis, etc.) ?
Based on your comments, I would just drop the table and reload the CSV every night using SSIS, then create fresh indexes as part of the nightly job.
If each CSV contains all of the relevant information, then this is the simplest way to go.
No reason to fool around with update/merge logic that I can see.
Plus, given your aversion to SSIS, a straight table load with index creation should be very easy to implement in a C# script.
SSIS Route:
First, build the first load as demonstrated here.
Next, right click on the table is SSMS and generate the create script for that table.
Then, create an Execute SQL task in SSIS that runs before the load task. That SQL task will run the following 2 pieces of code drop table <your table name>, followed by the create table script you copied earlier.
Finally (and optionally), create an Execute SQL task that runs after the data load task that will create any needed indexes. Since I know nothing about your data, I'd recommend a nonclustered index that includes all of the columns that you use for parameters in your report, such as CREATE NONCLUSTERED INDEX IX_SalesPerson_SalesQuota_SalesYTD ON Sales.SalesPerson (SalesQuota, SalesYTD);
While not a perfectly tuned solution, it should suffice for what you are trying to do and be easy to maintain.
I may be able to add screenshots later.
There are different approach depends on how data is being used by application for the moment I would suggest following steps:
Create schedule a job to upload data into Temp Table(s)
Bulk Insert into Temp Table
Insert New rows form Temp Table to Main Table
Update existing rows from Temp to Main Table
or You can delete existing rows by comparing Temp Table to Main Table and Then Build Insert every thing from Temp Table to Main Table.
Sounds like you need an "upsert". SQL server supports the MERGE statement that achieves this. You could feed your csv data into a temp table & then MERGE it into your destination table w/that syntax. Probably SSIS would let you set that up in a neat job.
In my application I am running a process that will run update commands (sometimes more than 10000) on over 100 different tables. I am using entity framework which for updating can be incredibly slow - on the order of 40+ minutes to update 13000 records by updating the entities and then calling saveChanges() after a sizeable batch of updates.
A Merge command wont work because I would need a temp table for every table updating and a stored procedure doesn't seem feasible either. So I started looking at UpdateCommand, passing it a data table, and I am having two problems. The first is this.
da.UpdateCommand.Parameters.Add("#YourField", SqlDbType.SmallDateTime).SourceColumn = "YourField"
There is no way to determine the dbtype of destination generically. So how do I map the columns into the parameters of the update? Secondly, I don't want to upsert, I just want to update and if there is no matching record, just ignore it and proceed. I know I can keep doing the updates on fail using
da.ContinueUpdateOnError = False
but I cant seem to find a way to prevent it from inserting on a record not found. Any help is greatly appreciated. Thank you!
If SQL replication is not an option you can use our SQL Data Compare Command Line tool - you can easily configure it to ignore the rows that exist on only one side and only sync (that is update) the rows that exist on both sides (you can use the GUI if you wish to review the process or use the command line utility if you want to schedule it to happen automatically).