I have two databases in MySQL and SQL Server, and I want to create tables in SQL Server and copy all rows from the table in MySQL into the new table in SQL Server.
I can create table in SQL Server same as MySQL, with this code:
List<String> TableNames = new List<string>();
{
IDataReader reader=
ExecuteReader("SELECT Table_Name FROM information_schema.tables WHERE table_name LIKE 'mavara%'",MySql);
while (reader.Read()) {
TableNames.Add(reader[0].ToString());
}
reader.Close();
}
foreach (string TableName in TableNames) {
IDataReader reader =
ExecuteReader("SELECT Column_Name,IS_NULLABLE,DATA_TYPE FROM information_schema.columns where TABLE_Name='" + TableName + "'",MySql);
List<string[]> Columns = new List<string[]>();
while (reader.Read()) {
string[] column = new string[3];
column[0] = reader[0].ToString();
column[1] = reader[1].ToString();
column[2] = reader[2].ToString();
Columns.Add(column);
}
reader.Close();
// create table
string queryCreatTables= "CREATE TABLE [dbo].[" + TableName + "](\n";
foreach(string[] cols in Columns)
{
queryCreatTables +="["+ cols[0] + "] " + cols[2] + " ";
if (cols[1] == "NO")
queryCreatTables += "NOT NULL";
// else
// queryCreatTables += "NULL";
queryCreatTables += " ,\n ";
}
queryCreatTables += ")";
System.Data.SqlClient.SqlCommand smd =
new System.Data.SqlClient.SqlCommand(queryCreatTables, MsSql);
System.Data.SqlClient.SqlDataReader sreader = smd.ExecuteReader();
sreader.Close();
but I have problem to copy rows from one table into another table.
for select query, I use Idatareader, but I don't know how insert rows to another table.
For inserting rows from one table into another table please refer the below sample query
INSERT INTO Store_Information (store_name, Sales, Date)
SELECT store_name, sum(Sales), Date
FROM Sales_Information
The algorithm is as follows:
1. For each table in source database
2. Get a list of columns for that table
3. Create table in destination database
4. SELECT * FROM the table in source
5. For each row in data
6. Generate INSERT statement and execute on destination database
The information you need for a column is Name, Type, Length, etc.
Then you generate the insert statement by iterating on the columns
var insertStatement = "INSERT INTO " + tableName + " VALUES( ";
foreach( var column in columns )
insertStatement += "#" + column.Name + ",";
insertStatement[insertStatement.Length-1] = ')';
var command = new SqlCommand( insertStatement, MsSql );
// iterate over the columns again, but this time set values to the parameters
foreach( var column in columns )
command.Parameters.AddWithValue( "#"+column.Name, currentRow[column.Name] );
But I have a problem to copy rows from one table into another table.
You can use the SqlDataAdapter.UpdateBatchSize to perform Batch Updates/Inserts with a DataAdapter against the database to copy the data from one table to the other. After you get all records from the first MYSQL table using something like:
//Get the rows from the first mysql table as you did in your question
DataTable mysqlfirstTableRowsTobeCopied = GetDataTableFromMySQLTable();
Then you have to create a commend text that do the INSERT something like:
cmd.CommandText = "INSERT INTO TableName Column_Name, ... VALUES(...)";
Or you can use a stored procedure:
CREATE PROCEDURE sp_BatchInsert ( #ColumnName VARCHAR(20), ... )
AS
BEGIN
INSERT INTO TableName VALUES ( #ColumnNamne, ...);
END
Then:
DataTable mysqlfirstTableRowsTobeCopied = GetDataTableFromMySQLTable();
SqlConnection conn = new SqlConnection("Your connection String");
SqlCommand cmd = new SqlCommand("sp_BatchInsert", conn);
cmd.CommandType = CommandType.StoredProcedure;
cmd.UpdatedRowSource = UpdateRowSource.None;
// Set the Parameter with appropriate Source Column Name
cmd.Parameters.Add("#ColumnName", SqlDbType.Varchar, 50,
mysqlfirstTableRowsTobeCopied.Columns[0].ColumnName);
...
SqlDataAdapter adpt = new SqlDataAdapter();
adpt.InsertCommand = cmd;
// Specify the number of records to be Inserted/Updated in one go. Default is 1.
adpt.UpdateBatchSize = 20;
conn.Open();
int recordsInserted = adpt.Update(mysqlfirstTableRowsTobeCopied);
conn.Close();
This code actually is quoted from a full tutorial about this subject in the codeproject, you can refer to it for more information:
Multiple Ways to do Multiple Inserts
Assuming that you already have a datatable with rows that you want to merge with another table...
There are two ways to do this...again, assuming you've already selected the data and put it into a new table.
oldTable.Load(newTable.CreateDataReader());
oldTable.Merge(newTable);
Usually that's sth. you can do in SQL directly: INSERT INTO table FROM SELECT * FROM othertable;
For insertion of all record into new table(If the second table is not exist)
Select * into New_Table_Name from Old_Table_Name;
For insertion of all records into Second Table(If second table is exist But the table structure should be same)
Insert into Second_Table_Name from(Select * from First_Table_Name);
Just in case it can help someone, I've found an easy way to do it in C# using SqlDataAdapter. It automatically detects the structure of the table so you don't have to enumerate all the columns or worry about table structure changing. Then bulk inserts the data in the destination table.
As long as the two tables have the same structure, this will work. (For example, copying from a production table to a dev empty table.)
using(SqlConnection sqlConn = new SqlConnection(connectionStringFromDatabase))
{
sqlConn.Open();
using(var adapter = new SqlDataAdapter(String.Format("SELECT * FROM [{0}].[{1}]", schemaName, tableName), sqlConn))
{
adapter.Fill(table);
};
}
using(SqlConnection sqlConn = new SqlConnection(connectionStringDestination))
{
sqlConn.Open();
// perform bulk insert
using(var bulk = new SqlBulkCopy(sqlConn, SqlBulkCopyOptions.KeepIdentity|SqlBulkCopyOptions.KeepNulls, null))
{
foreach(DataColumn column in table.Columns)
{
bulk.ColumnMappings.Add(column.ColumnName, column.ColumnName);
}
bulk.DestinationTableName = String.Format("[{0}].[{1}]", schemaName, tableName);
bulk.WriteToServer(table);
}
}
Related
I'm trying to insert a dataset into an SQL database but I am having difficulties passing my dataset as an argument to my DB class. I am not sure if it is allowed to pass as an argument. If not, what are my alternatives?
The way I create my dataset:
public static void getLogs() {
string path = Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData) + #"\someDir";
SQLiteConnection cn = new SQLiteConnection("Data Source=" + path + ";Version=3;New=False;Compress=True;");
cn.Open();
SQLiteDataAdapter sd = new SQLiteDataAdapter("SELECT * FROM table", cn);
DataSet ds = new DataSet();
sd.Fill(ds);
cn.Close();
db.InsertLogs(Form1.adminID, Form1.deviceID, ds);
}
My database class and insert method looks like the following:
public void InsertLogs(string user_id, string device_id, DataSet history)
{
string query = "INSERT INTO table (column1, column2, column3, column4, column5, column6, column7) VALUES (#value1, #value2, #value3, #value4, #value5, #value6, #value7);";
if (OpenConnection() == true)
{
foreach (DataTable table in history.Tables)
{
foreach (DataRow row in table.Rows)
{
MySqlCommand cmd = new MySqlCommand(query, connection);
cmd.Parameters.AddWithValue("#value1", int.Parse(user_id));
cmd.Parameters.AddWithValue("#value2", int.Parse(device_id));
cmd.Parameters.AddWithValue("#value3", row[0]);
cmd.Parameters.AddWithValue("#value4", row[1]);
cmd.Parameters.AddWithValue("#value5", row[2]);
cmd.Parameters.AddWithValue("#value6", row[3]);
cmd.Parameters.AddWithValue("#value7", row[4]);
cmd.ExecuteNonQuery();
}
}
CloseConnection();
}
}
Thank you
you can loop through datatables in a dataset and can pass a datatable as a stored procedure paramater,
found an example here
1.- Go to SQL Server, under your DB name go to "programmability\Types\User-Defined Table Types, right click and create a new one:
USE DBNAME
GO
-- Create the data type
CREATE TYPE ValuesToInsert AS TABLE
(
Value1 INT NOT NULL,
Value2 INT NOT NULL,
Value3 VARCHAR(20)
)
GO
2.- Create a SP to receive the table as parameter, parameter must be the new User-Defined table type created in step 1
USE [DBNAME]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
--
CREATE PROCEDURE [dbo].[spImportData]
#DataImported dbo.ValuesToInsert READONLY
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
-- Insert statements for procedure here
INSERT INTO [dbo].[TableName] (Value1, Value2, Value3)
SELECT Value1, Value2, Value3
FROM #DataImported
3.- Pass a datatable from your code to DB, in this case using Dapper.net as following:
DataTable dtExcelData = new DataTable();
//Fill dtExcelData and pass as parameter
ParametersCollection param = new ParametersCollection();
param.Add(CreateParameter("#DataImported", dtExcelData));
ExecuteDataSet("spImportData", CommandType.StoredProcedure, param);
I'm getting error while fetching the data
Exception of type 'System.OutOfMemoryException' was thrown
Error getting line is dataAdapter.Fill(dataTable);
(2826000) records count in my table.
here is the code i'm using.
var dataTable = new DataTable();
var DicTableNameAndValues = new Dictionary<string, object>();
using (var connection = new SqlConnection(ConnectionString))
{
connection.Open();
var dataQuery = "SELECT * FROM " + table;
using (var command = new SqlCommand(dataQuery, connection))
{
var dataAdapter = new SqlDataAdapter(command);
dataAdapter.Fill(dataTable);
var result = dataTable.AsEnumerable().Skip(skip).Take(pageSize).ToList().Select(c => c.ItemArray);
DicTableNameAndValues.Add(table, result);
}
}
You can get a certain number of records as required by your application instead of all the records. You can create a stored procedure something to the effect of the following.
--Create a stored procedure with following parameters
DECLARE #skip int, #pagesize int
--Added testing values
SELECT #skip = 4, #pagesize = 3
--Give #tbl with your table name.
/*If you already have an identity key then probably row_number function might not
be required. But if the records are getting deleted as well then row_number is a
better option*/
SELECT * FROM
(SELECT ROW_NUMBER() over(ORDER BY Your_Col) AS ROWNUM, * FROM #tbl) as tbl
WHERE ROWNUM BETWEEN (#skip * #pagesize) + 1 and (#skip + 1) * (#pagesize)
Hope this helps.
I have a DataTable with about 6000 rows created from a query to SQL Server DB. Now I try to add the data into a MS-Access table. Everything seems to work. No exceptions are raised but no data is a being added. Here is the code I'm using:
DataTable t = new DataTable();
// fill data table
// Create DB - works!
var Name = DateTime.Now.ToString("H_mm_ss");
string strAccessConnectionString = #"Provider = Microsoft.Jet.OLEDB.4.0;" +
#"Data Source=" + "C:\\chhenning\\" + Name + ".mdb;";
ADOX.CatalogClass Catalog = new ADOX.CatalogClass();
Catalog.Create(strAccessConnectionString);
OleDbConnection accessConnection = new OleDbConnection(strAccessConnectionString);
accessConnection.Open();
// Create Table - works!
OleDbCommand Command = accessConnection.CreateCommand();
Command.CommandText = "Create Table Test( "
+ " ID_ int not null "
+ " , Year_ int not null "
+ " , Value_ float not null )";
Command.ExecuteNonQuery();
// Add data into table - does not work!
var adapter = new OleDbDataAdapter();
adapter.SelectCommand = new OleDbCommand("select * from Test", accessConnection);
var cbr = new OleDbCommandBuilder(adapter);
cbr.GetInsertCommand();
var Rows = adapter.Update(t);
I have made sure that my DataTable has data in and that both the DataTable and MS-Access have the same columns with the same data types.
Can someone spot what wrong with code? What are the steps I do to investigate the problem further?
The adapter.Update(table) method works looking at the RowState of the rows in the DataTable.
If the rows have RowState == DataRowState.Unchanged, the method will not perform any update.
I suppose that you load the datatable from the SqlServer database without making any change and thus the RowState is Unchanged for every row. Try to loop on the Rows and call
foreach(DataRow r in t.Rows)
r.SetAdded();
This will force the RowState on each column to Added and then the InsertCommand on the DataAdapter will execute the insert
I'm looking at the example here:
http://msdn.microsoft.com/en-US/library/y06xa2h1(v=vs.80).aspx
string s = "primaryKeyValue";
DataRow foundRow = dataSet1.Tables["AnyTable"].Rows.Find(s);
if (foundRow != null)
{
MessageBox.Show(foundRow[1].ToString());
}
else
{
MessageBox.Show("A row with the primary key of " + s + " could not be found");
}
They don't specify where does dataSet1 come from and does this represent some database?
I'm trying to use this example in my code to find unique rows but I can't seem to implement this syntax. I'm only using connection string to open connection to SQL and I use SqlDataAdapter to perform functions...
EDIT:
SqlConnection myConnection = new SqlConnection("Data Source=server; Initial Catalog=Dashboard; Integrated Security=SSPI; Persist Security Info=false; Trusted_Connection=Yes");
SqlDataAdapter da = new SqlDataAdapter();
try
{
//Opens the connection to the specified database
myConnection.Open();
//Specifies where the Table in the database where the data will be entered and the columns used
da.InsertCommand = new SqlCommand("INSERT INTO DashboardLibAnswer(Id,Date,Time,Question,Details,Answer,Notes,EnteredBy,WhereReceived,QuestionType,AnswerMethod,TransactionDuration)"
+ "VALUES(#Id,#Date,#Time,#Question,#Details,#Answer,#Notes,#EnteredBy,#WhereReceived,#QuestionType,#AnswerMethod,#TransactionDuration)", myConnection);
//Specifies the columns and their variable type where the data will be entered
//Special note: Conversion from String > DateTime will cause exceptions that will only import some part of data and not everything
da.InsertCommand.Parameters.Add("#Id", SqlDbType.NVarChar);
da.InsertCommand.Parameters.Add("#Date", SqlDbType.Text);
da.InsertCommand.Parameters.Add("#Time", SqlDbType.Text);
da.InsertCommand.Parameters.Add("#Question", SqlDbType.Text);
da.InsertCommand.Parameters.Add("#Details", SqlDbType.Text);
da.InsertCommand.Parameters.Add("#Answer", SqlDbType.Text);
da.InsertCommand.Parameters.Add("#Notes", SqlDbType.Text);
da.InsertCommand.Parameters.Add("#EnteredBy", SqlDbType.NVarChar);
da.InsertCommand.Parameters.Add("#WhereReceived", SqlDbType.NVarChar);
da.InsertCommand.Parameters.Add("#QuestionType", SqlDbType.NVarChar);
da.InsertCommand.Parameters.Add("#AnswerMethod", SqlDbType.NVarChar);
da.InsertCommand.Parameters.Add("#TransactionDuration", SqlDbType.NVarChar);
//Using the global variable counter this loop will go through each valid entry and insert it into the specifed database/table
for (int i = 0; i < counter; i++)
{
//Iterates through the collection array starting at first index and going through until the end
//and inserting each element into our SQL Table
DataSet dashboardDS = new DataSet();
da.Fill(dashboardDS, "DashboardLibAnswer");
DataTable dt = dashboardDS.Tables["DashboardLibAnswer"];
foreach (DataColumn col in dt.Columns)
{
if (col.Unique)
{
da.InsertCommand.Parameters["#Id"].Value = collection.getIdItems(i);
da.InsertCommand.Parameters["#Date"].Value = collection.getDateItems(i);
da.InsertCommand.Parameters["#Time"].Value = collection.getTimeItems(i);
da.InsertCommand.Parameters["#Question"].Value = collection.getQuestionItems(i);
da.InsertCommand.Parameters["#Details"].Value = collection.getDetailsItems(i);
da.InsertCommand.Parameters["#Answer"].Value = collection.getAnswerItems(i);
da.InsertCommand.Parameters["#Notes"].Value = collection.getNotesItems(i);
da.InsertCommand.Parameters["#EnteredBy"].Value = collection.getEnteredByItems(i);
da.InsertCommand.Parameters["#WhereReceived"].Value = collection.getWhereItems(i);
da.InsertCommand.Parameters["#QuestionType"].Value = collection.getQuestionTypeItems(i);
da.InsertCommand.Parameters["#AnswerMethod"].Value = collection.getAnswerMethodItems(i);
da.InsertCommand.Parameters["#TransactionDuration"].Value = collection.getTransactionItems(i);
da.InsertCommand.ExecuteNonQuery();
}
}
//Updates the progress bar using the i in addition to 1
_worker.ReportProgress(i + 1);
} // end for
//Once the importing is done it will show the appropriate message
MessageBox.Show("Finished Importing");
} // end try
catch (Exception exceptionError)
{
//To show exceptions thrown just uncomment bellow line
//rtbOutput.AppendText(exceptionError.ToString);
} // end catch
//Closes the SQL connection after importing is done
myConnection.Close();
}
if you populate a dataset from your data adapter, you'll be able to follow the same logic -
http://msdn.microsoft.com/en-us/library/bh8kx08z(v=vs.71).aspx
It might be worth showing what you actually have to get more specific help
EDIT
I think I'm understanding what you want - if you fill your datatable from the already populated table, just check the item doesn't already exist before adding it - i.e.
if (dt.Rows.Find(collection.getIdItems(i)) == null)
{
// add your new row
}
(just to be sure I knocked together a quick test - hopefully this helps):
// MyContacts db has a table Person with primary key (ID) - 3 rows - IDs 4,5,6
SqlConnection myConnection = new SqlConnection("Data Source=.; Initial Catalog=MyContacts; Integrated Security=SSPI; Persist Security Info=false; Trusted_Connection=Yes");
SqlDataAdapter da = new SqlDataAdapter();
da.SelectCommand = new SqlCommand("select * from Person", myConnection);
myConnection.Open();
DataSet dashboardDS = new DataSet();
da.Fill(dashboardDS, "Person");
dashboardDS.Tables[0].PrimaryKey = new[] { dashboardDS.Tables[0].Columns["ID"]};
List<int> ids = new List<int> {4, 6, 7};
foreach (var id in ids)
{
if (dashboardDS.Tables[0].Rows.Find(id) == null)
{
Console.WriteLine("id not in database {0}", id); //i.e. 7
}
}
You will first need to open a connection to your database. This is an excellent source for connection strings: The Connection String Reference.
Then you will need to fill the dataset with data from some table. Since we are only interested in the schema information we are only selecting one row (SELECT TOP 1 ...).
Then we can go through the columns and check their Unique property (Boolean):
string connString =
"server=(local)\\SQLEXPRESS;database=MyDatabase;Integrated Security=SSPI";
string sql = #"SELECT TOP 1 * FROM AnyTable";
using (SqlConnection conn = new SqlConnection(connString)) {
conn.Open();
SqlDataAdapter da = new SqlDataAdapter(sql, conn);
using (DataSet ds = new DataSet()) {
da.Fill(ds, "AnyTable");
DataTable dt = ds.Tables["AnyTable"];
foreach (DataColumn col in dt.Columns) {
if (col.Unique) {
Console.WriteLine("Column {0} is unique.", col.ColumnName);
}
}
}
}
UPDATE #1
Sorry, I missunderstood your question. The above example returns unique columns, not unique rows. You can get unique (distinct) rows by using the DISTINCT keyword in SQL:
SELECT DISTINCT field1, field2, field3 FROM AnyTable
You can then fill the data table the same way as above.
Usually the word "unique" is used for unique constraints and unique indexes in database jargon. The term "distinct" is used for rows which are different.
UPDATE #2
Your updated question seems to suggest that you don't want find unique rows, but that you want to insert unique rows (which is the exact opposite).
Usually you would select distinct items from a collection like this. However it is difficult to answer your question accurately, since we don't know the type of your collection.
foreach (var item in collection.Distinct()) {
}
UPDATE #3
The easiest way to insert distinct values in the SQL Server table is to filter the rows at their origin, when reading them from the CSV-File; even before splitting them.
string[] lines = File.ReadAllLines(#"C:\Data\MyData.csv");
string[][] splittedLines = lines
.Distinct()
.Select(s => s.Split(','))
.ToArray();
Now you have distinct (unique) splitted lines that you can insert into the SQL Server table.
I have a Session, which is list int, and I need to make a query that will take from a database only those rows that have the PK value that exists in Session.
I was thinking of doing it with the IN function, or making a new datatable with 1 collumn and values from the Session and doing a double join, probably left...
I just dont know how to make a table from a list.
What I have so far:
String ConnString = "Data Source=BRACO-PC\SQL1;Initial Catalog=DiplomskiSQL1SQL;Integrated Security=True";
SqlConnection Conn = new SqlConnection(ConnString);
Conn.Open();
DataTable ukosarici = new DataTable();
SqlDataAdapter da = new SqlDataAdapter("Select Proizvodi.ime, TipProizvoda.tip, Proizvodi.dimenzije, Proizvodi.cijena from Proizvod LEFT JOIN TipProizvoda On Proizvod.tip=TipProizvoda.id_t WHERE Proizvod.id_p IN ", Conn);
SqlCommandBuilder cmd = new SqlCommandBuilder(da);
da.Fill(ukosarici);
GridView1.DataSource = ukosarici;
GridView1.DataBind();
Conn.Close();
Create a temporary table or table variable, insert the ints into it using INSERT or BULK INSERT, do a join in the SQL query then drop the temp table or table variable.
There are many ways you could do this, but one of my preferred methods is to serialize the list to a CSV, e.g. '1,3,5,33'. I then use a custom SQL Table function to de-serialize the list and filter in the database:
SELECT * FROM mytable t
JOIN dbo.ufn_CSVtoTextList('1,3,5,33' , ',') csv
ON csv.[Entry] = t.Id
The ufn_CSVtoTextList function CREATE script is below:
CREATE FUNCTION [dbo].[ufn_CSVToTextlist] ( #StringInput nVARCHAR(max) ,#SepChar nchar(1) = ',')
RETURNS #OutputTable TABLE ( [Entry] nVarchar(255), [index] int identity (0,1) )
AS
BEGIN
DECLARE #Entry nVarChar(255)
WHILE LEN(#StringInput) > 0
BEGIN
SET #Entry = LEFT(#StringInput,
ISNULL(NULLIF(CHARINDEX(#SepChar, #StringInput) - 1, -1),
LEN(#StringInput)))
SET #StringInput = SUBSTRING(#StringInput,
ISNULL(NULLIF(CHARINDEX(#SepChar, #StringInput), 0),
LEN(#StringInput)) + 1, LEN(#StringInput))
INSERT INTO #OutputTable ( [Entry] )
VALUES ( #Entry )
END
RETURN
END
Try by changing your SqlDataAdapter Call as follows
List<int> list ; // Assign with your session int list values
List<string> l2 = list.ConvertAll<string>(delegate(int i) { return i.ToString(); });
string query = "Select Proizvodi.ime, TipProizvoda.tip, Proizvodi.dimenzije, Proizvodi.cijena from Proizvod LEFT JOIN TipProizvoda On Proizvod.tip=TipProizvoda.id_t WHERE Proizvod.id_p IN (";
query = query + string.Join(",", l2.ToArray()) + ")";
SqlDataAdapter da = new SqlDataAdapter(query, Conn);