How do I import an XML file into a SQL Server Database using C# and .NET in a windows application?
The table name and column names of the SQL Server have to be automatically created (I saw somewhere that they were mapping explicitly every column of the XML table to the already created SQL table's columns).
Appreciate your help.
Sam.
I kind of think this is two questions. So it kind of gets two answers.
1) You would need to parse out the name of the nodes to create column names. Depending on how "deep" your XML is, it may need multiple tables. You'd then connect to your database and run sql statements through code (SqlConnection and SqlCommand objects) to create the tables with the appropriate columns. (CREATE TABLE Blah COLUMNS Bleh, for instance). There is also a SQL Specific API (I think) the name of which I can't recall that will let you do that in a slightly more object-oriented way. The gist is the same: marshal your table and column names, create your table(s) with the appropriate columns.
2) You then have a couple of options- the best would probably be to marshal DataSets from the XML (plenty of examples exist) and then use XML Bulk Insert to upload the whole kit-and-caboodle to SQL all at once.
Let me just say, though, that this is a really ugly solution. SSIS would probably be able to do the same thing for more easily if that's an option.
Related
I would like to turn a XSD file into a SQL Server table. Is there anyway to do this without using the tool XSD2DB because I already tried it and does not work .
In fact what I would like to do is to persist information of an object in the database. I already have a class that I created from XSD and it has about 400 properties. I don't want to have to manually create 400 columns in a table database ...
I've already tried to create a code that does an INSERT, but I don't think it is the best way to do it; it would be huge ugly and inefficient. I could use Entity Framework however with the tutorials that I've seen on the Internet you have to create a Data Model based on a previously existing table in the database ... Is there any other way to do this ?
Thank you all!
I have two databases. One of them belongs to a CRM software and is the source.
The other one will be the destination used by a tool I'm developing.
The destination will contain a table ADDRESSES with a subset of the columns of a table of the same name in the source database.
What is the best (most efficient) way to copy the data between those databases (btw: they're on different SQL Server instances if that's important).
I could write a loop which does INSERT into the destination for each row obtained from the source but I don't think that this is efficient.
My thoughts and information:
The data won't be altered on its way from source to destination
It will be altered on its way back
I don't have the complete structure of the source but I know which fields I need and that they're warranted to be in the source (hence, access to the rows obtained from source isn't possible using the index of columns)
I can't use LINQ.
Anything leading me in the right direction here is appreciated.
Edit:
I really need a C# way to copy the data. I also need to know how to merge the copied rows back to the source. Is it really necessary (or even best practise) to do this row after row?
Why write code to do this?
The single fastest and easiest way is just to use SQL Server's bcp.exe utility (bcp: Bulk Copy Program).
Export the data from the source server.
Zip it or tar it if it needs it.
FTP it over to where it needs to go, if you need to move it to another box.
Import it into the destination server.
You can accomplish the same thing via SQL Server Management Studio in a number of different ways. Once you've defined the task, it can be saved and it can be scheduled.
You can use SQL Server's Powershell objects to do this as well.
If you're set on doing it in C#:
write your select query to get the data you want from the source server.
execute that and populate a temp file with the output.
execute SQL Server's bulk insert statement against the destination server to insert the data.
Note: For any of these techniques, you'll need to deal with identity columns if the target table has them. You'll also need to deal with key collisions. It is sometimes easier to bulk load the data into a perma-temp table first, and then apply the prerequisite transforms and manipulations to get it to where it needs to go.
According to your comment on Jwrit's answer, you want two way syncs.
If so, you might want to look into Microsoft Sync Framework.
We use it to sync 200+ tables on Premise SQL to SQL Azure and SQL Azure to SQL Azure.
You can use purely C#. However, it might offer a lot more than you want, or it might be over kill for a small project.
I'm just saying so that you can have different option for your project.
If these databases exist on two servers you can setup a link between the servers by executing sp_addlinkedserver there are instructions for setting this up here. This may come in handy if you plan on regularly "Sharing" data.
http://msdn.microsoft.com/en-us/library/ff772782.aspx
Once the servers are linked a simple select statement can copy the rows from one table to another
INSERT INTO db1.tblA( Field1,Field2,Field2 )
SELECT Field1,Field2,Field2 FROM db2.tblB
If the Databases are on the same instance you only need to execute similar SQL to the above
If this is one time - the best bet is normally SSIS (SQL server integration services), unless there are complex data transformations - you can quickly and easily do column mappings and have it done (reliably) in 15 mins flat......
I am looking for an idea or some direction. I have to transfer data from one database to another both are structurally and schema wise same.
Its a complete database with maybe 70 tables and having relationship between tables at different levels. Even though i ' m going to mess up the identity when i move across database but as of now i am ok with it.
Idea which i thought was to load required data from all table into an XML and then create connection to second database and push data from this XML its kind of repeated and not best way at all. So looking for direction.
Can i use entity framework for this somehow??
I cannot use SSIS for this it has to be C# Sorry.
You can create a linked server as stated in the comments to your question. You seemed to indicate that you know how to do this, but in case not, you can do this from SQL Server Management Studio by drilling down to "Server Objects > Linked Servers" beneath your source database on the source database server, then right-click, "New Linked Server", etc.
Then you would use a statement like this, for example, from your C# code:
insert into DestServer.DBName.dbo.TableName
select * from SourceServer.DBName.dbo.TableName
Assuming you are connected to 'SourceServer' and that 'SourceServer' maintains a linked server object pointing to 'DestServer'. Note: you don't actually need to use the fully-qualified name for the table on 'SourceServer', but I've put it there for clarity. I.E. you could also do this:
insert into DestServer.DBName.dbo.TableName
select * from TableName
Don't forget to setup the permissions properly in your linked server object so that your query can write to the table in the destination server. You can do this any number of ways, and often (because I work in a small environment where it's maintained by just me and a couple other folks) I just use the "sa" login:
Yes, can use linked servers in .NET.
You just use the 4 part name.
What you can do in TSQL in SSMS you can do in a .NET SQLcommand.
My experience is that you get better performance connecting to the server you are writing to.
I have a VERY large (50 million+ records) dataset that I am importing from an old Interbase database into a new SQL Server database.
My current approach is:
acquire csv files from the Interbase database (done, used a program called "FBExport" I found somewhere online)
The schema of the old database doesn't match the new one (not under my control), so now I need to mass edit certain fields in order for them to work in the new database. This is the area I need help with
after editing to the correct schema, I am using SqlBulkCopy to copy the newly edited data set into the SQL Server database.
Part 3 works very quickly, diagnostics shows that importing 10,000 records at once is done almost instantly.
My current (slow) approach to part 2 is I just read the csv file line by line, and lookup the relevant information (ex. the csv file has an ID that is XXX########, whereas the new database has a separate column for each XXX and ########. ex2. the csv file references a model via a string, but the new database references via an ID in the model table) and then insert a new row into my local table, and then SqlBulkCopy after my local table gets large.
My question is: What would be the "best" approach (perfomance wise) for this data-editing step? I figure there is very likely a linq-type approach to this, would that perform better, and how would I go about doing that if it would?
If step #3’s importing is very quick, I would be tempted to create a temporary database whose schema exactly matches the old database and import the records into it. Then I’d look at adding additional columns to the temporary table where you need to split the XXX######## into XXX and ########. You could then use SQL to split the source column into the two separate ones. You could likewise use SQL to do whatever ID based lookups and updates you need to ensure the record relationships continue to be correct.
Once the data has been massaged into a format which is acceptable, you can insert the records into the final tables using IDENTITY_INSERT ON, excluding all legacy columns/information.
In my mind, the primary advantage of doing it within the temporary SQL DB is that at any time you can write queries to ensure that record relationships using the old key(s) are still correctly related to records using the new database’s auto generated keys.
This is of coursed based on me being more comfortable doing data transformations/validation in SQL than in C#.
what is the best way For Transfer Data From One DataBase (sql server 2008) To Another db(sql server 2008) With Different Schema?
is there a program for doing that ?
thanks 4 your future asnwer
best regards
Depending on how complex your needs are, you may want to look into using SSIS.
I would use SSIS (which comes with sql server) to do this. This will not be an easy or quick task however. Mapping from one schema to another can be very time consuming and you will almost certainly find data that doesn't fit from one schema to the other that you have to deal with. For example if the field is varchar in the first schema and datetime in the second, you will almost certainly have to exclude bad values (like "ASAP") which can be a problem especially if the field is a required field. In fact you will have to deal with those types of mismathces as well - if it wasn't required and now is, what value do you put in there for the records where it currently is null. What do you do when field sizes don't match up, you may have to truncate data. If you have different lookup values from one to the other, you need a conversion table. You may need a conversion table to relate old and new id fields so that you can properly load child tables. This is a very complex task.
Link the source server to the dest server.
Then you can, from the dest server, executes your very own queries such as
INSERT INTO Destination_Table (col1, col2, col3)
SELECT colA, colB, colC
FROM [SRC_MACHINE\SRC_SERVER].Source_Database.dbo.Source_Table
If you're comfortable with writing queries, you can include all your data transformation (if you need any) in the nested SELECT, and play with JOINs and stuff to make the selected data structure fit the destination schema.
Doable even if both servers are running SQL Server Express (which is not shipped with SSIS).
Documentation on Linking Servers