We Keep Coding

How to append pipe character in a comma delimited string?

I have a SSIS package which is pulling data from a SQL database and generating a comma delimited flat file. The flat file is to be used to import the data into a system and it is causing issues with the text field values in the file as it contains comma in the value.
I am now told to insert the pipe character | as the text qualifier for all text fields.
Example
1234,Smith, John,5678 should be 1234,|Smith, John|,5678
I followed the tutorial in this link to create my SSIS package for the export. It is using a Script Task with the Visual C# script pasted below, which I finding it difficult on how to modify to prepend and append the pipe character to the text values consisting the comma character.
I think it is this part where I will need to insert the pipe character, but I do not know the C# language enough to modify it as needed. Any help or reference resource would be really helpful and appreciated.
StreamWriter sw = null;
sw = new StreamWriter(FileFullPath, false);
// Write the Header Row to File
int ColumnCount = d_table.Columns.Count;
for (int ic = 0; ic < ColumnCount; ic++)
{
sw.Write(d_table.Columns[ic]);
if (ic < ColumnCount - 1)
{
sw.Write(FileDelimiter);
}
}
sw.Write(sw.NewLine);
// Write All Rows to the File
foreach (DataRow dr in d_table.Rows)
{
for (int ir = 0; ir < ColumnCount; ir++)
{
if (!Convert.IsDBNull(dr[ir]))
{
sw.Write(dr[ir].ToString());
}
if (ir < ColumnCount - 1)
{
sw.Write(FileDelimiter);
}
}
sw.Write(sw.NewLine);
}
string datetime = DateTime.Now.ToString("yyyyMMddHHmmss");
try
{
//Declare Variables
string DestinationFolder = Dts.Variables["User::DestinationFolder"].Value.ToString();
string FileDelimiter = Dts.Variables["User::FileDelimiter"].Value.ToString();
string FileExtension = Dts.Variables["User::FileExtension"].Value.ToString();
//USE ADO.NET Connection from SSIS Package to get data from table
SqlConnection myADONETConnection = new SqlConnection();
myADONETConnection = (SqlConnection)(Dts.Connections["DBConn"].AcquireConnection(Dts.Transaction) as SqlConnection);
//Read list of Tables with Schema from Database
string query = "SELECT Schema_name(schema_id) AS SchemaName,name AS TableName FROM sys.tables WHERE is_ms_shipped = 0";
//MessageBox.Show(query.ToString());
SqlCommand cmd = new SqlCommand(query, myADONETConnection);
DataTable dt = new DataTable();
dt.Load(cmd.ExecuteReader());
//Loop through datatable(dt) that has schema and table names
foreach (DataRow dt_row in dt.Rows)
{
string SchemaName = "";
string TableName = "";
object[] array = dt_row.ItemArray;
SchemaName = array[0].ToString();
TableName = array[1].ToString();
string FileFullPath =DestinationFolder +"\\"+ SchemaName+"_"+TableName + "_" + datetime+FileExtension;
//Get the data for a table into data table
string data_query = "Select * From ["+SchemaName+"].["+TableName+"]";
SqlCommand data_cmd = new SqlCommand(data_query, myADONETConnection);
DataTable d_table = new DataTable();
d_table.Load(data_cmd.ExecuteReader());
StreamWriter sw = null;
sw = new StreamWriter(FileFullPath, false);
// Write the Header Row to File
int ColumnCount = d_table.Columns.Count;
for (int ic = 0; ic < ColumnCount; ic++)
{
sw.Write(d_table.Columns[ic]);
if (ic < ColumnCount - 1)
{
sw.Write(FileDelimiter);
}
}
sw.Write(sw.NewLine);
// Write All Rows to the File
foreach (DataRow dr in d_table.Rows)
{
for (int ir = 0; ir < ColumnCount; ir++)
{
if (!Convert.IsDBNull(dr[ir]))
{
sw.Write(dr[ir].ToString());
}
if (ir < ColumnCount - 1)
{
sw.Write(FileDelimiter);
}
}
sw.Write(sw.NewLine);
}
sw.Close();
Dts.TaskResult = (int)ScriptResults.Success;
}
}
catch (Exception exception)
{
// Create Log File for Errors
using (StreamWriter sw = File.CreateText(Dts.Variables["User::LogFolder"].Value.ToString() + "\\" +
"ErrorLog_" + datetime + ".log"))
{
sw.WriteLine(exception.ToString());
Dts.TaskResult = (int)ScriptResults.Failure;
}
}

Is the package somewhat hard-code query written for getting data and what will be made available for the export process? If so, is the data ONLY for export process from the query? Or is it query data, show some report, then allow the export to comma separated list from same results?
I know, bunch of questions. But if your query is exclusively to pull the data for export, why not tack on the pipes as you query the data, something like
select
SomeIntegerColumn,
'|' + SomeStringField + '|' as SomeStringField,
'|' + AnotherString + '|' as AnotherString
SomeDateColumn,
etc
from
...
Then the data is already formatted for you ready to go.

How to add conditional data from a datatable into another datatable. [error: " no row at position 0] . C#

I am new to programming and got this job to create a tool to convert .DBF table into a .csv file.
so here is the scenario;
The dbf table 'Poles' contain four fields 'pole_id', 'guy_hoa_1', 'guy_hoa_2','guy_hoa_3' and 'guy_hoa_4'.
And the final csv file should show the value in two columns only:'PoleId' and 'HOA' respectively, where PoleID will be == pole_id and HOA= guy_hoa_1 + '|' +guy_hoa_2+'|' +guy_hoa_3 +'|'+ guy_hoa_4.
for example, the Poles table will have data like;
Sample data of Poles table
And, the ouput csv file should show data as follows;
Sample Output CSV file
*The pole_id is the main field and based on it the values of other fields will be selected.
So far I managed to write following code:
enter code here
enter code here
string str = textBox1.Text;
string path = str.Substring(0, str.LastIndexOf("\\") + 1);
string conn = "Provider=Microsoft.Jet.OLEDB.4.0; Data Source = '" + path + "';Extended Properties=dBase IV;User ID=Admin;Password=;";
OleDbConnection connection = new OleDbConnection();
connection.ConnectionString = conn;
connection.Open();
CheckConnectionLabel.Text = "Connected Successfully";
OleDbDataAdapter adapter = new OleDbDataAdapter(#"SELECT pole_id, guy_hoa_1, guy_hoa_2,guy_hoa_3,guy_hoa_4 FROM poles" + ".dbf", connection);
DataSet ds = new DataSet();
DataTable dt = new DataTable();
adapter.Fill(dt);
DataTable dt1 = dt.AsEnumerable()
.Where(r=> r.Field<string>("pole_id")!= null)
.Where(r=> r.Field<string>("pole_id")!=" ")
.CopyToDataTable();
DataTable dtTemp = new DataTable();
dtTemp.Columns.Add("PoleId", typeof(String));
dtTemp.Columns.Add("HOA", typeof(string));
string x = string.Empty;
for (int i=0;i< dt1.Rows.Count;i++)
{
if(dt1.Rows[i]["pole_id"]!= null || dt1.Rows[i]["pole_id"].ToString()!= "")
{
if(dt1.Rows[i]["guy_hoa_1"]!=null && dt1.Rows[i]["guy_hoa_1"].ToString()!="")
{
x =dt1.Rows[i]["guy_hoa_1"].ToString();
}
if(dt1.Rows[i]["guy_hoa_2"]!= null && dt1.Rows[i]["guy_hoa_2"].ToString()!="")
{
x = x + "|" + dt1.Rows[i]["guy_hoa_2"].ToString();
}
if(dt1.Rows[i]["guy_hoa_3"]!=null && dt1.Rows[i]["guy_hoa_3"].ToString()!= "")
{
x = x + "|" + dt1.Rows[i]["guy_hoa_3"].ToString();
}
if(dt1.Rows[i]["guy_hoa_4"]!=null && dt1.Rows[i]["guy_hoa_4"].ToString()!= "")
{
x = x + "|" + dt1.Rows[i]["guy_hoa_4"].ToString();
}
dtTemp.Rows[i]["PoleId"] = dt1.Rows[i]["poles_id"].ToString();
dtTemp.Rows[i]["HOA"] = x ;
}
}
connection.Close();
dataGridView1.DataSource = dtTemp;
}
catch (Exception ex)
{
MessageBox.Show("Error " + ex.Message);
}
}
enter code here
So, through above code I am connected to the dbf table and collected required data in 'dt' table. Then I filtered the data by removing the rows where pole_id was blank/null and put it in another 'dt1' table. Now my purpose was to check the conditions in dt1 table and then fill rows in dtTemp table which would later display the data in datagridview.
The Code is fetching the value of x till last IF statement correctly however nothing is getting filled up in dtTemp datatable and then showing this error.
Please help me and let me know where I am wrong... many thanks in advance!!

I got the solution as follows;
enter code here
object y = dt1.Rows[i]["pole_id"].ToString();
dtTemp.NewRow();
dtTemp.Rows.Add(y ,x);

Can I insert the data to a table which columns are unknown in mdb file from c#?

I think I have a weird doubt!!
I have created a table using C#[with a tool not programatically ] in mdb file, then I am inserting the values to that table, what the issue is I don't know how many columns are available in that table, but I wanna insert value from the datagridview..
Spire.DataExport.Access.AccessExport accessExport = new Spire.DataExport.Access.AccessExport();
accessExport.DataSource = Spire.DataExport.Common.ExportSource.DataTable;
accessExport.DataTable = this.dataGridView2.DataSource as System.Data.DataTable;
accessExport.DatabaseName = saveFileDialog1.FileName;
accessExport.TableName = "ExtractedTable";
accessExport.SaveToFile();
//OleDbCommand cmdt = new OleDbCommand("Create Table "+profiletablegrid. ", con);
string strDirectory = saveFileDialog1.FileName;
OleDbConnection conn = new OleDbConnection("Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + strDirectory);
conn.Open();
for (int i = 41; i < dataGridView2.Rows.Count; i++)
{
for (int j = 0; j < dataGridView2.Rows[i].Cells.Count; j++)
{
OleDbCommand cmdd = new OleDbCommand("Insert into sample values(#a,#b,#c,#d)", conn);
cmdd.Parameters.Add("#a", OleDbType.VarChar).Value = dataGridView2.Rows[i].Cells[j].Value.ToString();
cmdd.Parameters.Add("#b", OleDbType.VarChar).Value = dataGridView2.Rows[i].Cells[j].Value.ToString();
cmdd.Parameters.Add("#c", OleDbType.VarChar).Value = dataGridView2.Rows[i].Cells[j].Value.ToString();
cmdd.Parameters.Add("#d", OleDbType.VarChar).Value = dataGridView2.Rows[i].Cells[j].Value.ToString();
cmdd.ExecuteNonQuery();
}
}
So Since I know the columns I am inserting 4 values, but if I don't know how many columns are there, then how can i insert the value...
I can count the datagridview total columns, but how can I insert according to that?

Without knowing column Names or Number of Columns of a table in my experience it's not possible to insert data in to it. How ever you can use this work around to get column names of particular table then insert data into those columns.
The first thing you would do is make sure that no data gets returned:
SELECT TOP 0 your_table.* FROM your_table WHERE 1 = 2;
Now assuming you know how to set up a DataReader you would do the following:
using(var reader = command.ExecuteReader())
{
// This will return false - we don't care, we just want to make sure the schema table is there.
reader.Read();
var table = reader.GetSchemaTable();
foreach (DataColumn column in table.Columns)
{
Console.WriteLine(column.ColumnName);
}
}
Now you have column names so build up your insert statement.
Ok Consider you have n number of columns then your code will look like this.
List<string> colArr=new List<string>();
foreach (DataColumn column in table.Columns)
{
colArr.Add(column.ColumnName);
}
now build your sql in this way.
string colNames="";
string val="";
for (int i = 0; i < colArr.Count; i++)
{
if(i!=colArr.Count-1)
{
colNames+=col+",";
val+="Some Value,";
}
else
{
colNames+=col;
val+="Some Value";
}
}
string sqlQuery="Insert Into your_Table "+colNames+" ("+val+")";

assuming you are using OleDbConnection you can call
DataTable schema = connection.GetSchema("Columns");
to get the schema data of your Database ... in that table you will find each column of each table in the db ...
use that to build you SQL statement at runtime

convert datetime to string in sql select query

I am trying to display data from database on datagriview and export data from data grid view to csv, my sqlite database dateformat is DateTime
2012-02-20 16:42:10.000
displayed on datagridview is in the format of
20/02/2012 16:42:10
my select statment is , i want to display on datagridview datetime column as the same format which in database
m_dbConnection.Open();
SQLiteCommand myCommand = new SQLiteCommand();
myCommand.Connection = m_dbConnection;
myCommand.CommandText = "select CompanyId,CONVERT(VARCHAR,DateTime, 103) as date_issued,Serial,ShortDeviceId,MatricolaA,Upper(Targa),VerbaliRuleOnePoints,VerbaliMissedNotificationDescription from VerbaliData";
//myCommand.Connection = myConn;
DataTable data = new DataTable();
SQLiteDataAdapter myAdapter = new SQLiteDataAdapter(myCommand);
//myAdapter.SelectCommand = myCommand;
myAdapter.Fill(data);
dataGridView1.DataSource = data;
this.dataGridView1.Refresh();
if (dataGridView1.RowCount > 0)
{
string value = "";
DataGridViewRow dr = new DataGridViewRow();
StreamWriter swOut = new StreamWriter("I:/final test/finaltest12.csv");
//write header rows to csv
for (int i = 0; i <= dataGridView1.Columns.Count - 1; i++)
{
if (i > 0)
{
swOut.Write(",");
}
swOut.Write(dataGridView1.Columns[i].HeaderText);
}
swOut.WriteLine();
//write DataGridView rows to csv
for (int j = 0; j <= dataGridView1.Rows.Count - 1; j++)
{
if (j > 0)
{
swOut.WriteLine();
}
dr = dataGridView1.Rows[j];
for (int i = 0; i <= dataGridView1.Columns.Count - 1; i++)
{
if (i > 0)
{
swOut.Write(",");
}
// Datetime column content transformed in a formatted string....
if(i == 1)
{
object cellValue = dr.Cells[i].Value;
value = (cellValue == DBNull.Value ?
string.Empty : Convert.ToDateTime(cellValue).ToString("DD-MM-YYYY hh:mm:ss"));
}
value = dr.Cells[i].Value.ToString();
//replace comma's with spaces
value = value.Replace(',', ' ');
//replace embedded newlines with spaces
value = value.Replace(Environment.NewLine, " ");
swOut.Write(value);
}
}
swOut.Close();
}
m_dbConnection.Close();
}

You can use Dataformatstring
Add Dataformatstring like
Dataformatstring="yyyy-MM-dd HH:mm:ss.s" in bound field column

The easiest way is to not cast your DateTime object to a string in your SQL statement:
myCommand.CommandText = "select CompanyId, date_issued, Serial, ShortDeviceId, " +
"MatricolaA, Upper(Targa), VerbaliRuleOnePoints, VerbaliMissedNotificationDescription " +
"from VerbaliData";
I don't think "Upper(Targa)" is going to work, though.
Whatever, right? That's not the issue here.
Once you're done with that, handle your DateTime object in your code as you are parsing through the data:
if (i == 1)
{
object cellValue = dr.Cells[i].Value;
value = (cellValue == DBNull.Value) ?
string.Empty :
Convert.ToDateTime(cellValue).ToString("DD-MM-YYYY hh:mm:ss");
}
I hope that helps.

Add full row to DataTable MVC4

i'm new to MVC and I am trying to build a DataTable from a stored procedure response and pass it back to my View. For the rows I build a comma delimited string full of cell values.
The issue I am having is that the string is not getting parsed by the commas, and effectively it is passing the whole string into the first cell of each row.
What is the correct way to build up a row comprised of the individual values for each column? The number of columns, their names, and amount of records returned are all variable.
public ActionResult dataSet(string table, string key, string search)
{
SqlDataReader rdr = null;
SqlConnection con = new SqlConnection("Connection stuff");
SqlCommand cmd = new SqlCommand();
cmd = new SqlCommand("dbo.USP_getDataSet", con);
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.AddWithValue("#key", key);
cmd.Parameters.AddWithValue("#table", table);
cmd.Parameters.AddWithValue("#search", search);
con.Open();
DataTable theTable = new DataTable();
try
{
rdr = cmd.ExecuteReader();
while (rdr.Read())
{
int count = rdr.FieldCount;
string rowString = "";
int intRows = theTable.Columns.Count;
//Build columns on first pass through
if (intRows == 0){
for (int i = 0; i < count; i++){
theTable.Columns.Add(Convert.ToString(rdr.GetName(i).TrimEnd()), typeof(string));
}
}
//Grab all values for each column
for (int i = 0; i < count; i++){
rowString += '\"' + (Convert.ToString(rdr.GetValue(i)).TrimEnd()) + '\"' + ", ";
}
//Remove trailing delimiter
string finishedRow = rowString.Substring(0, rowString.Length - 2);
//Add the full row for each time through reader
theTable.Rows.Add(finishedRow);
}
}
finally
{
if (rdr != null)
{ rdr.Close(); }
if (con != null)
{ con.Close(); }
}
return View(theTable);
}

According to the documentation for the DataRowCollection.Add(params Object[] values) method, each value passed in will populate each cell. Since you are passing in a single value, it is the value of the cell.
You probably want:
var cells = new object[count];
for (int i = 0; i < count; i++)
{
cells[i] = rdr.GetString(i).Trim() + "\"
}
theTable.Rows.Add(cells)