CSV - Import file to a Sql database through code - c#

I need to load data from text file/csv file to a SQL Server database. I used the code shown below to load data and is loading to the database the problem is the data in second column may contain space but I use space for separate the column data.
i.e.
200007 XXXX Check XXXX yyy 50
200013 YYYY Check ZZZZ yyy 50
200022 nnnn 25Mg 30 Tabs
200042 mmmm 30 Mg 30 Tabs
I need to store the first ID number in the first column and the remaining text in second column:
string str = Properties.Settings.Default.con;
SqlConnection con = new SqlConnection(str);
SqlCommand cmd = new SqlCommand();
try
{
cmd.Connection = con;
con.Open();
cmd.CommandText = "IF NOT EXISTS (SELECT * FROM sysobjects WHERE name='TEMP_AUTO' AND xtype='U')" +
"CREATE TABLE TEMP_AUTO (" +
"ID varChar(10) NULL," +
"NAME varChar(50) NULL," +
"DATE TIMESTAMP NULL," +
")";
cmd.ExecuteNonQuery();
string query1 = "INSERT INTO [dbo].[TEMP_AUTO]([ID],[NAME]) VALUES (#num1, #num2)";
cmd.CommandText = query1;
string[] allLines = File.ReadAllLines(txtFilePath.Text);
for (int i = 0; i < allLines.Length; i++)
{
cmd.Parameters.Clear();
string[] items = allLines[i].Split(new char[] { ' ' });
cmd.Parameters.AddWithValue("#num1", items[0]);
cmd.Parameters.AddWithValue("#num2", items[1]);
cmd.ExecuteNonQuery();
}
MessageBox.Show("Successfully saved your data");
}
finally
{
cmd.Dispose();
con.Close();
}

A possible solution might be this:
string[] allLines = {
"200007 XXXX Check XXXX yyy 50",
"200013 YYYY Check ZZZZ yyy 50",
"200015 ",
"2541111"
};
for (int i = 0; i < allLines.Length; i++)
{
string param1 = null;
string param2 = null;
int spaceIndex = allLines[i].IndexOf(' ');
if (spaceIndex > 0)
{
param1 = allLines[i].Substring(0, spaceIndex);
if (spaceIndex < allLines[i].Length - 1)
{
param2 = allLines[i].Substring(spaceIndex + 1, allLines[i].Length-1 - spaceIndex);
}
}
else
{
param1 = allLines[i];
}
Console.WriteLine("param1:{0} param2:{1}", param1, param2);
}

Use SSIS to map this file as long as it has a standard structure to SQL Table.

Is this a one time thing? Have you tried getting the data organized in Excel then using the SSMS import tool to bring it in? If you right click on the Database then Tasks > Import Data the wizard will appear when given the option to choose the source, choose Excel. Flat files is an option but if you can format it in Excel first that tends to work better.
As you click through you can adjust the column types and where the breaks are, much like importing into Excel itself.

Use the String.Split(Char[], Int32) method to split on the first occurrence of ' ' only. Eg
string[] items = allLines[i].Split(new char[] { ' ' }, 2);
Refs: MSDN and previous relevant question

Use the below code.
using (StreamReader sr = File.OpenText("txtFile.txt")) // Mention the path,if the file is not in application folder.
{
string str = String.Empty;<br/>
while ((str = sr.ReadLine()) != null)
{
string[] item = str.Split(' ');
SqlConnection con = new SqlConnection(str);
SqlCommand cmd = new SqlCommand();
string query1 = "INSERT INTO [dbo].[TEMP_AUTO]([ID],[NAME]) VALUES ('" + item[0] + "', '" + item[1] + "')";
// Do remain part<br/>
}
}

Related

Reading data from SQL Server and formatting it

I have a SQL Server database with a table QC and columns A, B, C, D, E,
Comment.
What I am trying to do is to read the data from the columns and display it in a label. But there is not always data in all columns.
Expected output is:
2.5|2.1
if there is data only in A and B column. But I get:
2.5|2.1| |||||
This is my code:
SqlCommand cmd = new SqlCommand("SELECT TOP 1 * FROM [TableQC] ORDER BY id DESC", conn);
SqlDataReader reader = cmd.ExecuteReader();
string temp = "";
string temp1 = "";
while (reader.Read())
{
temp += reader["A"].ToString() + "|";
temp += reader["B"].ToString() + "|";
temp += reader["C"].ToString() + "|";
temp += reader["D"].ToString() + "|";
temp += reader["E"].ToString() + "|";
temp1 += reader["Comment"].ToString();
//temp += "<br/>";
}
conn.Close();
label1.Text = temp;
label2.Text = temp1;
As Mong Zhu pointed out, check while reading the column values:
if (!string.IsNullOrEmpty(reader.GetString("A")))
This will test each column for null value and replace it with an empty string.
To avoid a dangling | delimiter at the end, work with an array of strings you the Join()
while (reader.Read())
{
// create an array big enough to hold all columns
object[] qc = new object[reader.FieldCount];
// iterate over all columns of your reader
for (int i = 0; i < reader.FieldCount; i++)
{
if (reader[i] == reader["Comment"])
{
label2.Text = reader.GetSqlString(i).IsNull ? null : reader.GetSqlString(i).Value;
}
else
{
// add to array
qc[i] = reader.GetValue(i);
}
}
label1.Text = string.Join("|", qc.OfType<string>());
}
However it seems to didn't tell the actual data type of columns A, B, etc. and assuming them to be of string/character data type turned out to be false.
Depending on your actual data types, you will have to edit the filtering Linq query to the actual type like qc.OfType<decimal>() or only filter for null values like qc.Where(v => !(v is DBNull)).
You can do this in the query. Basically you want concat_ws(), but that is not available in SQL Server. Instead:
SELECT TOP 1 STUFF( (COALESCE('|' + A, '') +
COALESCE('|' + B, '') +
COALESCE('|' + C, '') +
COALESCE('|' + D, '') +
COALESCE('|' + E, '') +
COALESCE('|' + Comment, '')
), 1, 1, ''
) as abcde
FROM [TableQC]
ORDER BY id DESC
Thanks you all for helping me and resolving my problem. This is the code that works for me:
SqlCommand cmd = new SqlCommand("SELECT TOP 1 * FROM [TableQC] ORDER BY id DESC", conn);
SqlDataReader reader = cmd.ExecuteReader();
while (reader.Read())
{
// create an array big enough to hold all columns
object[] qc = new object[reader.FieldCount];
// iterate over all columns of your reader
for (int i = 0; i < reader.FieldCount; i++)
{
if (reader[i] == reader["Comment"])
{
lblMessage1.Text = reader.GetSqlString(i).IsNull ? null : reader.GetSqlString(i).Value;
}
else
{
// add to array
qc[i] = reader.GetValue(i);
}
}
lblMessage.Text = string.Join("|", qc.OfType<double>());
}
conn.Close();
Thanks you a lot once more time.

format string with int64 value

i have a web service in which training number is assigning as per my database
there is a line
dates + "1"; this lines add 1 in date
like 27012017 will become 270120171
then i convert this into int64
newid = Convert.ToInt64(dates);
but now i want to add trainer id in this
so i updated my line with this
dates ="00"+trainerid +"-"+ dates + "1";
newid = Convert.ToInt64(dates);
the error is coming input string is not in correct format,
i know this is because of the addition of +"-"+
but i want to store the data in this format only
and my whole portion look like
DateTime dt = DateTime.Now;
string dates = dt.ToString();
dates = dates.Replace("-", "");
dates = dates.Substring(0, 8);
SqlCommand cmdbill = new SqlCommand("select top 1 * from listmybill where bill_id like #trid order by bill_id desc", con);
con.Open();
cmdbill.Parameters.AddWithValue("#trid", "%" + dates + "%");
SqlDataReader dr = cmdbill.ExecuteReader();
while (dr.Read())
{
value = dr["bill_id"].ToString();
}
con.Close();
con.Open();
SqlDataAdapter da = new SqlDataAdapter(cmdbill);
DataTable dat = new DataTable();
da.Fill(dat);
if (dat.Rows.Count > 0)
{
newid = Convert.ToInt64(value);
newid = newid + 1;
}
else
{
dates ="00"+trainerid +"-"+ dates + "1";
newid = Convert.ToInt64(dates);
}
con.Close();
what should i do here,
i want to enter the data like 001-270120171
and if i convert toInt64 into string,
there can be problem when a row found in the table
if (dat.Rows.Count > 0)
{
newid = Convert.ToInt64(value);
newid = newid + 1;
}
what i need to do now?
i want to store this into my database
I hardly think you will be doing mathematical calculations on this training number string. I suggest that you convert the int in your database into a varchar using alter statement. Then you you will be able to store the training number in whichever format you like.
You can change new id to string and do the following
if (dat.Rows.Count > 0)
{
newid = Convert.ToString(Convert.ToInt64(value)+1);
}
else
{
newid="00"+trainerid +"-"+ dates + "1";
}

Can't export large data from oracle to excel file using c#

I have a problem with extracting large data from oracle table to C#, and I
couldn't find the solution myself.
For this task I wrote a C# code, which loaded data from oracle procedure, which returns cursor, in excel file for the first time.
But when I tried to load bigger table (about 20 columns and 90 000 rows), it just didn't work.
Script doesn't fall with error, but data are not inserted into excel file.
I tried to load for 10 000 rows and then save the results, but again, only 30 000 rows were inserted.
I monitored the counter in loop, it is going correct and reach needed 90 000 and ExecuteNonQuery() always returned the value 10 000. But when I open excel file, there are only 30 000 rows there.
Can you please help me to catch the error, or may be somebody met the same problem, and can advise me what to do or what to read.
Thank you for any help!
I didn't write the connection string, but I think, it's correct, cause script works correctly with small datatable.
public static void Main()
{
string datetime = DateTime.Now.ToString("yyyy-MM-dd HH-mm-ss");
System.Threading.Thread.CurrentThread.CurrentUICulture = new System.Globalization.CultureInfo("en-US");
try
{
OleDbConnection Excel_OLE_Con = new OleDbConnection();
OleDbCommand Excel_OLE_Cmd = new OleDbCommand();
string qwe_constr = "connection string";
OracleConnection myADONETConnection = new OracleConnection(qwe_constr);
string connstring = "Provider=Microsoft.ACE.OLEDB.12.0;" + "Data Source=" + "E:\\qaz\\15.07.2016\\qwe" +
";" + "Extended Properties=\"Excel 12.0 Xml;HDR=YES;\"";
File.Delete("E:\\qaz\\15.07.2016\\qwe.xlsx");
//fill datatable with data for insert
myADONETConnection.Open();
OracleCommand cmd_proc = new OracleCommand();
cmd_proc.Connection = myADONETConnection;
cmd_proc.CommandType = System.Data.CommandType.StoredProcedure;
cmd_proc.CommandText = "procedure_name";
cmd_proc.Parameters.Add("p_show_del", OracleDbType.Int16).Value = 0;
cmd_proc.Parameters.Add("p_type", OracleDbType.Varchar2, 3).Value = "INV";
cmd_proc.Parameters.Add("p_errno", OracleDbType.Int16).Value = 157;
cmd_proc.Parameters.Add("outcur", OracleDbType.RefCursor).Direction = ParameterDirection.Output;
DataTable dt_with_data = new DataTable();
dt_with_data.Load(cmd_proc.ExecuteReader());
myADONETConnection.Close();
//string with column headers
string TableColumns = "";
foreach (DataColumn column in dt_with_data.Columns)
{
TableColumns += column + "],[";
}
// Replace most right comma from Columnlist
TableColumns = ("[" + TableColumns.Replace(",", " Text,").TrimEnd(','));
TableColumns = TableColumns.Remove(TableColumns.Length - 2);
//Use OLE DB Connection and Create Excel Sheet
Excel_OLE_Con.ConnectionString = connstring;
Excel_OLE_Con.Open();
Excel_OLE_Cmd.Connection = Excel_OLE_Con;
Excel_OLE_Cmd.CommandText = "Create table [sheet1] (" + TableColumns + ")";
Excel_OLE_Cmd.ExecuteNonQuery();
Excel_OLE_Con.Close();
//Write Data to Excel Sheet from DataTable dynamically
//string with command
Excel_OLE_Con.Open();
String sqlCommandInsert = "";
String sqlCommandValue = "";
foreach (DataColumn dataColumn in dt_with_data.Columns)
{
sqlCommandValue += dataColumn + "],[";
}
sqlCommandValue = "[" + sqlCommandValue.TrimEnd(',');
sqlCommandValue = sqlCommandValue.Remove(sqlCommandValue.Length - 2);
sqlCommandInsert = "INSERT into [sheet1] (" + sqlCommandValue + ") VALUES(";
int columnCount = dt_with_data.Columns.Count;
int i_qaz = 0;
foreach (DataRow row in dt_with_data.Rows)
{
i_qaz++;
Console.WriteLine(i_qaz.ToString());
string columnvalues = "";
for (int i = 0; i < columnCount; i++)
{
int index = dt_with_data.Rows.IndexOf(row);
columnvalues += "'" + dt_with_data.Rows[index].ItemArray[i].ToString().Replace("'", "''") + "',";
}
columnvalues = columnvalues.TrimEnd(',');
var command = sqlCommandInsert + columnvalues + ")";
Excel_OLE_Cmd.CommandText = command;
Excel_OLE_Cmd.ExecuteNonQuery();
}
}
catch (Exception exception)
{
// Create Log File for Errors
using (StreamWriter sw = File.CreateText("E:\\qaz\\15.07.2016\\qwe_" + datetime + ".log"))
{
sw.WriteLine(exception.ToString());
}
}
}
PS: Same question in Russian.

Split text and insert words in sql database

I want to select file contains .txt and split all strings in each text into array
then insert the divided words into sql database (word, counter) and count the repeated words in each text by counter that lead to unrepeated words in database tables>>
(1)-counter code is incorrect I found several errors in my code >> (i want to prevent a repeat of word at the same time calculate how many times repeated word in databases ,by using counter.)
(2)- my code has static path (just one text ),but I want user to select his file which he want to split. (browse.. button)
(3-)sql database could not show Arabic Words (?????)
namespace lib123
{
public partial class Form1 : Form
{
SqlConnection sqlConn;
SqlCommand sqlComm;
SqlDataAdapter sqlAdptr;
public Form1()
{
InitializeComponent();
sqlConn = new SqlConnection();
sqlComm = new SqlCommand();
sqlAdptr = new SqlDataAdapter();
sqlComm.Connection = sqlConn;
sqlComm.CommandType = CommandType.Text;
sqlConn.ConnectionString = "Data Source=007-PC\\SQLEXPRESS ;Initial Catalog= Email_DB;Integrated Security =True ";
}
private void Form1_Load(object sender, EventArgs e)
{
FillGrid();
}
private void button1_Click(object sender, EventArgs e)
{
if (sqlConn.State != ConnectionState.Open)
sqlConn.Open();
// sqlComm = sqlConn.CreateCommand();
StreamReader streamReader = new StreamReader(#"C:\Users\007\Desktop\spam-email\spamenglish.txt"); //get the file
string stringWithMultipleSpaces = streamReader.ReadToEnd(); //load file to string
streamReader.Close();
Regex r = new Regex(" +"); //specify delimiter (spaces)
string[] words = r.Split(stringWithMultipleSpaces); //(convert string to array of words)
int c = 1;
string strQry = "select ISNULL( max(id),0) as id from word_tb ";
sqlComm.CommandText = strQry;
int LastID = int.Parse(sqlComm.ExecuteScalar().ToString());
string x ;
String st = null;
for (int i = 0; i < words.Length; i++)
{
string y = words[i];
for (int j = 0; j <LastID; j++)
{
x = "select word from word_tb where id = j";
sqlComm.CommandText = x ;
if (x.Equals(y))
{
c = c + 1;
string sql = "INSERT INTO word_tb (count) VALUES ('" + c + "') where id = i";
sqlComm.CommandText = sql;
}
else
{
LastID = LastID + 1;
st += "INSERT INTO word_tb(id, word,count) VALUES('" + LastID + "', '" + words[i].ToString() + "','" + c + "');";
//st += "INSERT INTO word_tb(word) VALUES('" + words[i].ToString() + "');";
}
}
}
sqlComm.CommandType = CommandType.Text;
sqlComm.CommandText = st;
sqlComm.ExecuteNonQuery();
FillGrid();
}
private void FillGrid()
{
DataTable tbl = new DataTable();
string strQry = "select * from word_tb ";
sqlComm.CommandText = strQry;
sqlAdptr.SelectCommand = sqlComm;
sqlAdptr.Fill (tbl) ;
dataGridView1.DataSource = tbl;
}
private void dataGridView1_CellContentClick(object sender, DataGridViewCellEventArgs e)
{
}
//private void btnDelete_Click(object sender, EventArgs e)
//{
// string str = "DELETE FROM word_tb";
// sqlComm.CommandType = CommandType.Text;
// sqlComm.CommandText = str;
// sqlComm.ExecuteNonQuery();
// dataGridView1.Rows.Clear();
// }
}
}
1-counter code is incorrect I found several errors in my code?
you can use below code to split the File Text into words based on space as delimiter.
Code:
String strAllData = System.IO.File.ReadAllText(#"C:\Users\007\Desktop\spam-email\spamenglish.txt");
String[] words = strAllData.Split(' ');
2- my code has static path (just one text ),but I want user to select
his file which he want to split. (browse.. button) ?
you can use OpenFileDialog control to let user choose the file which he/she wants to work on.
Code:
OpenFileDialog fileDialog = new OpenFileDialog();
if (fileDialog.ShowDialog() == DialogResult.OK)
{
String strAllData = System.IO.File.ReadAllText(fileDialog.FileName);
String[] words = strAllData.Split(' ');
}
3 -sql database could not show Arabic Words (?????) ?
Sql Server can not show Uni Code characters when you fire a SELECT Query because you have created your table columns/feilds as varchar.
Solution: if you want to get the Arabic characters when you fire a SELECT query you should create your table columns to accept Uni Code characters using NVARCHAR datatype instead of VARCHAR. N stands for National language character set.
Step1 : create your table columns as NVARCHAR as below:
create table sample(
[name] [nvarchar](100) NOT NULL)
Step 2: Use N as prefix while inserting Data into NVARCHAR columns. N as prefix tells that all following characters are Uni Code characters.
Code:
INSERT INTO sample VALUES(N'لا أتكلم العربية');
Now if you fire a SELECT Query you will be able to see the Arabic Characters.

Using stream reader to catch values returned by sql statemtnt c#

guys i have an SQL statement returning more than 1 value.
I am trying to use the StreamReader to get the values into an array as below
string sql = "select distinct COLUMN_NAME from INFORMATION_SCHEMA.KEY_COLUMN_USAGE where TABLE_NAME=' " + table + "' and CONSTRAINT_NAME like 'PK_%'";
SqlConnection conn2 = new SqlConnection(cnstr.connectionString(cmbDatabase.Text));
SqlCommand cmd_server2 = new SqlCommand(sql);
cmd_server2.CommandType = CommandType.Text;
cmd_server2.Connection = conn2;
conn2.Open();
//reader_sql = new StreamReader();
SqlDataReader reader_sql = null;
string[] colName = new string[200];
reader_sql = cmd_server2.ExecuteReader();
while (reader_sql.Read());
for (int rr = 0; rr < 20; rr++)
{
colName[rr] = reader_sql["COLUMN_NAME"].ToString();
}
It is not working, what am I doing wrong guys ?
You've got a stray ; turning your while into a tight loop, so instead try:
while (reader_sql.Read())
for (int rr = 0; rr < 20; rr++)
{
colName[rr] = reader_sql["COLUMN_NAME"].ToString();
}
You get the exception because
while (reader_sql.Read());
should be
while (reader_sql.Read())
{
for (int rr = 0; rr < 20; rr++)
{
colName[rr] = reader_sql["COLUMN_NAME"].ToString();
}
}
Perhaps you should remove the semicolon at the end of Read
while (reader_sql.Read())
{
for (int rr = 0; rr < 20; rr++)
colName[rr] = reader_sql["COLUMN_NAME"].ToString();
}
However, if your intention is to retrieve the columns belonging to the primary key, your code is wrong because you add 20 times the same primary key column, then repeat the same for the remaining columns ending with an array of 20 strings all equals to the last column in the primary key set. I think you should change your code to use a List(Of String) instead of a fixed length array and let the reader loop correctly on the primary key columns retrieved
List<string> pks = new List<string>();
while (reader_sql.Read())
{
pks.Add(reader_sql["COLUMN_NAME"].ToString());
}
EDIT: I have just noticed that your query contains a space before the table name. The string concatenation then produces an invalid table name, the query is syntactically right but doesn't return any data
string sql = "select distinct COLUMN_NAME from INFORMATION_SCHEMA.KEY_COLUMN_USAGE " +
"where TABLE_NAME='" + table + "' and CONSTRAINT_NAME like 'PK_%'";
^ space removed here
And while you are at it, remove the string concatenation and use a parameterized query.....
string sql = "select distinct COLUMN_NAME from INFORMATION_SCHEMA.KEY_COLUMN_USAGE " +
"where TABLE_NAME=#tName and CONSTRAINT_NAME like 'PK_%'";
SqlCommand cmd_server2 = new SqlCommand(sql, connection);
connection.Open();
cmd_server2.Parameters.AddWithValue("#tName", table);

Categories