i want to parse a CSV file, store it in a list then insert values from that list to a database . Here is the sample of my code.
I'm still learning so if it is not clear then i will explain further. But the idea is to parse the csv file, row by row, and then insert each row in my database. Thank you in advance.
public class SKU : List<string[]>
{
public string SKU_ID { get; set; }
public string SKU_Name { get; set; }
public string Code { get; set; }
public string Product_Name { get; set; }
public string DistributionCenter_Name { get; set; }
internal static SKU ParseRow(string row)
{
var columns = row.Split(';');
return new SKU()
{
SKU_ID = columns[0],
SKU_Name = columns[1],
Code = columns[2],
Product_Name = columns[3],
DistributionCenter_Name = columns[4],
};
}
}
In the script i named each column like in the csv file and in my database.
My main is as following
class Programm
{
static void Main(string[] args)
{
var sku_info = ProcessCSV("skutest1.csv");
SqlConnection conn = new SqlConnection();
conn.ConnectionString = #"...";
foreach (var information in sku_info)
{
using SqlConnection connection = new SqlConnection(conn.ConnectionString);
string commandString = ("INSERT INTO SKU VALUES ('" + information.SKU_ID + " "+information.SKU_Name+" "+information.Code+" "+information.Product_Name+" "+information.DistributionCenter_Name+"')");
conn.Open();
SqlTransaction transaction = conn.BeginTransaction();
SqlCommand cmd = new SqlCommand(commandString, conn, transaction);
cmd.ExecuteNonQuery();
transaction.Commit();
}
Console.ReadKey();
}
private static List<SKU> ProcessCSV(string path)
{
return File.ReadAllLines("C:/.../skutest1.csv").Where(row => row.Length > 0).Select(SKU.ParseRow).ToList();
}
}
The fastest way to load a file on the server would be to use BULK INSERT, eg :
BULK INSERT someTable
FROM 'pathtofile`
WITH ( FORMAT = 'CSV' )
You can do something similar by using ADO.NET's SqlBulkCopy class to execute a BULK INSERT operation with data sent from the client. SqlBulkCopy expects either a DataTable or IDbDataReader parameter. You can use CsvHelper' CsvDataReader for this. The CsvDataReader parses a CSV file and produces the IDbDataReader interface needed by SqlBulkCopy.
The code could be as simple as this :
using var txtReader = File.OpenText(pathToCSV);
var reader = new CsvReader(txtReader,CultureInfo.InvariantCulture);
using var dbReader = new CsvDataReader(reader);
var bcp = new SqlBulkCopy(cns);
bcp.ColumnMappings.Add("sku_id","sku_id");
bcp.ColumnMappings.Add("sku_name","sku_name");
...
bcp.DestinationTableName = table;
bcp.WriteToServer(dbReader);
Without the mappings SqlBulkCopy will send the columns in the order they appear in the file. If that doesn't match the order of the table columns, you'll get an error or worse, get mixed up data
Here's a short tutorial on building a parameterized commandstring - this is a much safer way of inserting to your database.
Here's a sample of how you could parameterize your insert:
string commandString = (#"INSERT INTO SKU VALUES (#sku_id, #sku_name, #code, #product_name, #distributioncenter_name");
conn.Open();
SqlTransaction transaction = conn.BeginTransaction();
SqlCommand cmd = new SqlCommand(commandString, conn, transaction);
cmd.Parameters.Add(new SqlParameter("#sku_id", information.SKU_ID));
cmd.Parameters.Add(new SqlParameter("#sku_name", information.SKU_Name));
cmd.Parameters.Add(new SqlParameter("#code", information.Code));
cmd.Parameters.Add(new SqlParameter("#product_name", information.Product_Name));
cmd.Parameters.Add(new SqlParameter("#distributioncenter_name", information.DistributionCenter_Name));
cmd.ExecuteNonQuery();
transaction.Commit();
Related
I have a table with three columns
MyDate : DateiIme
MyBlob: blob
Id: String
I want to return MyDate and MyBlob as Json. There can be multiple records in the table.
public class MyData
{
public string? MyDate { get; set; };
public string? MyBlob { get; set; };
}
public async Task<string> GetQueryResult(string Id)
{
MyData data = new MyData();
List<MyData> MyList = new List<MyData>();
string sqlSelect = string.Format("Select MyDate, MyBlob from MyTablee WHERE Id = '{0}'", Id);
try
{
MySqlCommand sqlcmd = new MySqlCommand();
MySqlConnection connetcion = new MySqlConnection(connectionString);
sqlcmd.Connection = connetcion;
sqlcmd.CommandTimeout = 0;
sqlcmd.CommandType = CommandType.Text;
sqlcmd.CommandText = sqlSelect;
connetcion.Open();
using (connetcion)
{
int count = 0;
using (MySqlDataReader reader = sqlcmd.ExecuteReader())
{
while (reader.Read())
{
data.MyDate = reader.GetString(0);
data.MyBlob = reader.GetString(1);
MyList.Add(data);
}
}
}
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
var json = JsonSerializer.Serialize(results);
return json;
}
The output in the Postman is:
"{\"MyDate\":\"1/30/2023 9:16:40 PM\",\"MyBlob\":\"#MyBlob\"}"
I am not sure the blob data to JSON conversion is correct. Thank you.
It's probable that your table doesn't contain any BLOB data, but instead contains the literal string #MyBlob in that column.
The cause of this would be using a SQL statement like INSERT INTO MyTablee(MyDate, MyBlob) VALUES(NOW, '#MyBlob');, which inserts the literal text #MyBlob.
Make sure your insert code is constructed as follows:
using var command = connection.CreateCommand();
command.CommandText = "INSERT INTO MyTablee(MyDate, MyBlob) VALUES(NOW(), #blob);";
command.Parameters.AddWithValue("#blob", yourBlobDataHere);
command.ExecuteNonQuery();
In particular, there are no quotes around the parameter name in the INSERT statement, and command parameters are being used to send values, instead of string concatenation.
I have to write identical tables to two different Sqlite database. The tables rewrite existing data and ids, so I currently just delete the entire table and rewrite. Depending on the order that I write to the db, the C# SQLiteCommandBuilder does not update the second db called.
If I call Db_A before Db_B, then A gets written and B gets deleted and vice versa. Can anyone tell me why the second table enters the code to get deleted, but the Sqlite adapter never updates the second table? It doesn't throw an error either.
public static bool WriteDt_Name(DataTable dt)
{
using (connA = GetDbAConn())
{
SaveDataTable(connA, dt);
}
using (connB = GetDbBConn())
{
SaveDataTable(connB, dt);
}
return true;
}
public static void SaveDataTable(SQLiteConnection conn, DataTable dt)
{
string table = dt.TableName;
var cmd = conn.CreateCommand();
cmd.CommandText = string.Format("DELETE FROM {0}", table);
int val = cmd.ExecuteNonQuery();
cmd.CommandText = string.Format("SELECT * FROM {0}", table);
using (var adapter = new SQLiteDataAdapter(cmd))
{
using (SQLiteCommandBuilder builder = new SQLiteCommandBuilder(adapter))
{
adapter.Update(dt);
conn.Close();
}
}
}
public static SQLiteConnection GetDbAConn()
{
string base_dir = System.AppDomain.CurrentDomain.BaseDirectory;
string path = Directory.GetParent(Directory.GetParent(Directory.GetParent(Directory.GetParent(Directory.GetParent(base_dir).ToString()).ToString()).ToString()).ToString()).ToString();
path = path + "\\db\\DbA.sqlite;";
SQLiteConnection conn = new SQLiteConnection("Data Source=" + path + "Version=3;");
return conn;
}
I have tried splitting SaveDataTable into a SaveDt_A and SaveDt_B and calling it that way. I still get the same result.
I am passing a long list of employeeIds to employeeIdlist and I split them into a List. Using this list I am adding parameters to my query.
I am getting the following error
{"Must declare the scalar variable \"#EmployeeId\"."}
public List<versionInfo> GetVersion(string employeeIdlist)
{
DbHelper helper = new DbHelper();
List<versionInfo> empVerInfo = new List<versionInfo>();
using (SqlConnection conn = new SqlConnection(connString))
{
conn.Open();
using (SqlCommand getVersion = new SqlCommand())
{
getVersion.Connection = conn;
getVersion.CommandText = #"SELECT EmployeeId,Version
FROM [dbo].[EmployeeVersion]
WHERE EmployeeId in (#EmployeeId)";
getVersion.CommandType = CommandType.Text;
List<int> empIds = employeeIdlist.Split(',').Select(int.Parse).ToList();
StringBuilder sb = new StringBuilder();
int i = 0;
foreach (var emp in empIds)
{
// IN clause
sb.Append("#EmployeeId" + i.ToString() + ",");
// parameter
getVersion.Parameters.AddWithValue("#EmployeeId" + i.ToString(), emp);
i++;
}
// getVersion.Parameters.AddWithValue("#EmployeeId", employeeIdlist);
SqlDataReader rdr = getVersion.ExecuteReader();
while (rdr.Read())
{
versionInfo vi = new versionInfo();
vi.employeeId = helper.GetDb<int>(rdr, "EmployeeId");
vi.version = helper.GetDb<decimal>(rdr, "Version");
empVerInfo.Add(vi);
}
rdr.Close();
}
conn.Close();
}
return empVerInfo;
}
Remove the text after the IN
getVersion.CommandText = #"SELECT EmployeeId,Version
FROM [dbo].[EmployeeVersion]
WHERE EmployeeId in (";
then the foreach could build the full list of parameters and texts
foreach (var emp in empIds)
{
sb.Append("#EmployeeId" + i.ToString() + ",");
getVersion.Parameters.AddWithValue("#EmployeeId" + i.ToString(), emp);
i++;
}
after exiting the loop remove the last comma from the StringBuilder
sb.Length--;
finally, complete the command text appending the content of the StringBuilder and do not forget the closing parenthesys for the IN clause.
getVersion.CommandText += sb.ToString() + ")";
Now you can run the command with the correct IN clause and a matching list of parameters
If fails because your string query has one parameter named #EmployeeId and your Command object has many parameters with different names ("#EmployeeId1" is not equal to "#EmployeeId")
It seems like you are trying to apply this approach, which is a good idea.
You are two lines away of getting it to work:
Add this lines:
sb.Lenght--;
getVersion.CommandText = getVersion.CommandText.Replace("#EmployeeId",sb.ToString())
just before:
SqlDataReader rdr = getVersion.ExecuteReader();
After doing that your added parameters will match those #parameters existing in the sql string.
This is just another option. You can achieve the same result in 3 lines of code using Dapper ORM used in Stack Overflow.
You can download via NuGet.
public class VersionInfo
{
public int EmployeeId { get; set; }
public decimal Version { get; set; }
}
class Program
{
public static string connString = "...";
static void Main(string[] args)
{
var result = GetVersion(new List<int> {1, 2});
Console.ReadLine();
}
public static List<VersionInfo> GetVersion(IList<int> employeeIds)
{
using (IDbConnection conn = new SqlConnection(connString))
{
conn.Open();
var entities = conn.Query<VersionInfo>(
#"SELECT EmployeeId, Version from EmployeeVersion WHERE EmployeeId IN #EmployeeIds",
new {EmployeeIds = employeeIds});
return entities.ToList();
}
}
}
On your select statement you have to declare a value for your variable. I have made it an Integer. If it is a text value, then you can use varchar(25).
#"DECLARE #EmployeeId INT
SELECT EmployeeId,Version
FROM [dbo].[EmployeeVersion]
WHERE EmployeeId in (#EmployeeId)";
I'm creating a list <object> and here I'm adding my data from a database.
But I totally don't know how to deal with a list <object>.
public class ClassList
{
public int Name { get; set; }
public string Birthday { get; set; }
}
private void GetClassList()
{
SqlConnection conn = new SqlConnection(GlobalVar.ConnString);
SqlCommand cmd = new SqlCommand
("SELECT * " +
"FROM Class_Data ", conn);
conn.Open();
SqlDataReader dr = cmd.ExecuteReader();
try
{
List<ClassList> ClassList = new List<ClassList>();
while (dr.Read())
{
ClassList.Add(new ClassList()
{
Name = ___?___, //what should i write here
Birthday = ___?___ //and here
});
}
}
catch{}
finally
{
dr.Close();
conn.Close();
}
}
Having been looking for lots of information but still fail to work it out.
I will be very appreciated if someone could help!
This should do it:
List<ClassList> ClassList = new List<ClassList>();
while (dr.Read())
{
ClassList.Add(new ClassList()
{
Name = (int)dr["Name"], //the name of the db column
Birthday = dr["Birthday"].ToString();
});
}
As a side note, it maybe worth altering your query from SELECT *... to just selecting the columns you require, from readability/maintenance point of view.
Also, is Name really an int seems like it could be a string.
You can also refer to the documentation:
SqlDataReader.Item Property (String)
You can use column name that are return by your database query with dr like dr["ColumnName"]
ClassList.Add(new ClassList()
{
Name = dr["NameColum"], //what should i write here
Birthday = dr["BrithColum"] //and here
});
Something like that:
// You don't use "this" in the context
// It seems, that you want to return the collected data (List<ClassList>)
private static List<ClassList> GetClassList() {
// Put IDisposable into using
using (SqlConnection conn = new SqlConnection(GlobalVar.ConnString)) {
conn.Connect();
// You want to fetch two fields only, right? Not the entire table (*)
String sql =
#"select Name,
BirthDay
from Class_Data"
using (SqlCommand cmd = new SqlCommand(sql, conn)) {
List<ClassList> result = new List<ClassList>();
using (SqlDataReader dr = cmd.ExecuteReader()) {
while (dr.Read()) {
result.Add(new ClassList() {
Name = Convert.ToString(dr[0]),
Birthday = Convert.ToDateTime(dr[1])
});
}
}
return result;
}
}
}
You can access the column values with dr["ColumnName"].
What would be the sql to enter thousands of records into Oracle db with C#?
Use ODP.NET and array binding:
class Record {
public int Value1 { get; set; }
public int Value2 { get; set; }
}
Record[] records = GetRecords();
const string CMD_TEXT = "INSERT INTO TABLE (Col1, Col2) VALUES (:Col1, :Col2);"
using (var conn = new OracleConnection(connectionString))
using (var cmd = new OracleCommand(CMD_TEXT, conn)) {
cmd.BindByName = true;
// number of rows to insert
cmd.ArrayBindCount = records.Length;
// bind array of values to parameters
cmd.Parameters.Add(":col1", OracleDbType.Int32,
records.Select(r => r.Value1).ToArray()
);
cmd.Parameters.Add(":col2", OracleDbType.Int32,
records.Select(r => r.Value2).ToArray()
);
conn.Open();
cmd.ExecuteNonQuery();
conn.Close();
}