We Keep Coding

More efficient way to iterate on my data (DbReader/DataSet)

I'm working on a project in C# that converts a database table to an XML-file with base64 encoded contents. Please bear with me, because C# is not my day-to-day programming language.
The code I've managed to come up with is this:
OdbcCommand DbCommand = DbConnection.CreateCommand();
DbCommand.CommandText = "SELECT * FROM " + dbTable;
OdbcDataReader DbReader = DbCommand.ExecuteReader();
int fCount = DbReader.FieldCount;
string[] colnames = new string[fCount];
output += "<" + dbTable + ">\n";
for (int i = 0; i < fCount; i++)
{
string fName = DbReader.GetName(i);
colnames[i] = fName.ToString();
}
while (DbReader.Read())
{
output += "\t<export_row>\n";
for (int i = 0; i < fCount; i++)
{
string col = "";
try
{
col = DbReader.GetString(i);
}
catch (Exception) { }
if (col.ToString().Length > 0 || i == 0)
{
output += "\t\t<" + colnames[i] + ">" + Base64Encode(col).ToString() + "</" + colnames[i] + ">\n"; ;
}
}
output += "\t</export_row>\n";
}
output += "</" + dbTable + ">\n";
The problem is, that even with a relatively small table, this causes the application to choke up and run extremely slowly. The obvious clue is that there's an enormous amount of iterations involved for each row, so I have been looking for a solution to this problem. I have tried using a DataSet, which seemed to increase performance slightly, but not significantly enough.
connection.Open();
adapter.Fill(dataSet);
output += "<" + dbTable + ">\n";
foreach (DataTable table in dataSet.Tables)
{
foreach (DataRow row in table.Rows)
{
output += "\t<export_row>\n";
foreach (DataColumn column in table.Columns)
{
output += "\t\t<" + column.ToString() + ">" + Base64Encode(row[column].ToString()).ToString() + "</" + column.ToString() + ">\n"; ;
}
output += "\t</export_row>\n";
}
}
output += "</" + dbTable + ">\n";
However, the problem remains that there is no other way than iterating through all the columns each and every time. Which begs the question: isn't there a more efficient way to do this? I'm not going to make a model for every table, because there are hundreds of tables in this database and the power would be the flexibility of transferring data in this way.
Can someone help me out, or point me in the right direction? For example, is there a way to extract both the column and the value at the same time? As in: foreach(row as key => value) or something. That would drastically reduce the amount of iterations required.
Thanks in advance for thinking along! There must be something (obvious) I missed.

The key is always not to write formatting of text formats yourself be it HTML, JSON, XML, YAML, or anything else. This is just asking for hard-to-find bugs and injections since you do not have control of the data or table names. For example, what happens if your data contains !, <, or >?
C# has numerous built-in XML tools and so does SQL where the formatting is done for you. Which one to use would depend on your other requirements or preferences.
SqlCommand.ExecuteXmlReader
string cmd = "SELECT * FROM " + myTable + " FOR XML AUTO";
using (SqlCommand k = new SqlCommand(cmd, c))
{
c.Open();
XmlReader xml = k.ExecuteXmlReader();
Console.WriteLine(xml);
c.Close();
}
DataTable.WriteXml
string ConString = "your connection string";
string CmdString = "SELECT * FROM " + myTable;
SqlConnection con;
SqlCommand cmd;
SqlDataAdapter sda;
DataTable dt;
using (con = new SqlConnection(ConString))
{
cmd = new SqlCommand(CmdString, con);
con.Open();
dt = new DataTable(tableName);
sda = new SqlDataAdapter(cmd);
sda.Fill(dt);
dt.WriteXml(tableName + ".xml");
con.Close();
}
DataSet.GetXml()
// Create a DataSet with one table containing
// two columns and 10 rows.
DataSet dataSet = new DataSet("dataSet");
DataTable table = dataSet.Tables.Add("Items");
table.Columns.Add("id", typeof(int));
table.Columns.Add("Item", typeof(string));
// Add ten rows.
DataRow row;
for(int i = 0; i <10;i++)
{
row = table.NewRow();
row["id"]= i;
row["Item"]= "Item" + i;
table.Rows.Add(row);
}
// Display the DataSet contents as XML.
Console.WriteLine(dataSet.GetXml());

Filtering database results with 3 textBoxes

I am writing a program in which the user can filter results from a database by 3 textboxes, however, the results are not being filtered correctly, because if one box is left empty, it doesn't display anything
private void textBox1_TextChanged(object sender, EventArgs e)
{
con = new SQLiteConnection(cs);
con.Open();
if ((textBox2.Text==""||textBox.Text3=="")&&textBox1.Text!="")
{
adapt = new SQLiteAdapter("select data1, data2 from DataTable where data1 like '" + textBox1.Text + "%'", con);
dt = new DataTable();
adapt.Fill(dt);
dataGridView1.Source = dt;
}
else if(textBox1.Text !="")
{
adapt = new SQLiteAdapter("select data1, data2 from DataTable where data1 like '" + textBox1.Text + "%' and data2 like '" + textBox2.Text + "%' and substr(data2,-2) like '" + textBox3.Text +"'", con);
dt = new DataTable();
adapt.Fill(dt);
dataGridView1.Source = dt;
}
con.close();
}
That is the code that I am using on one of the textboxes, for the other two it look almost the same, except I change the if clause conditions.
Do I have to write 9 different clauses for each textbox, so that I cover all the options? Is there a right way?

I would parameterize the query to prevent sql injection and use the IFNULL function to help you. This way you have one query to cover all scenarios. If any textbox is empty, the LIKE clause for that item will basically not filter anything out:
string qry = #"SELECT
data1,
data2
FROM DataTable
WHERE
data1 LIKE IFNULL(#data1, data1) AND
data2 LIKE IFNULL(#data2, data2) AND
SUBSTR(data2, -2) LIKE IFNULL(#data3, data3)";
To create the parameters get the textbox values, set the parameter value to null if the textbox is empty. Do this for all 3 textboxes:
string data1 = null;
if(!string.IsNullOrWhiteSpace(textbox1.Text))
{
data1 = textbox1.Text + "%";
}
SqlLiteCommand cmd = new SqlLiteCommand(qry, con);
SqlLiteParameter parData1 = new SqlLiteParameter("#data1", (object)data1 ?? DBNull.Value);
cmd.Parameters.Add(parData1);
Now you can execute that command.

In the Constructor or Form_Load wire up the TextBoxes change events to the one handler:
textBox1.TextChanged += textBox1_TextChanged;
textBox2.TextChanged += textBox1_TextChanged;
textBox3.TextChanged += textBox1_TextChanged;
And build up the WHERE clause dynamically instead of 9 conditions:
con = new SQLiteConnection(cs);
con.Open();
StringBuilder sb = new StringBuilder();
sb.Append("select * from DataTable where ");
foreach (Control c in this.Controls) {
TextBox t = c as TextBox;
if (t != null) {
if (t.Length > 0) {
//In design-time set the TextBox Tag property to the SQL column name
sb.Append(t.Tag.ToString() + " like '" + t.Text + "%' and ");
}
}
}
string SQL = sb.ToString();
if (SQL.Length > 0) {
SQL = SQL.Substring(0, SQL.Length-5);
}
adapt = new SQLiteAdapter(SQL, con);
dt = new DataTable();
adapt.Fill(dt);
dataGridView1.Source = dt;
You should also use a Stored Procedure or pay
Parameterized commands.

Special Character does not displays in gridview from SQL database with Parameter

Well I have tried all you people suggested but the problem remain same
let me tell you in brief:
Form Name : ​AuzineForum.aspx
Has 1 GridView which displays all the field from the database using ​ select * from QF
its working good , index is also working , the gridView Has Button and onClick I open new Form ​AuzineForumAnswer.aspx​ ..OK
I want to pick one record from ​AuzineForum.aspx​ and display on AuzineForumAnswer.aspx as well as it happens here in http://stackoverflow.com (we click on threads then the new page opens which has the question and answer on which we clicked previous) ...ok
so On AuzineForum.aspx's Button the code is
Button lb = (Button)sender;
GridViewRow row = (GridViewRow)lb.NamingContainer;
if (row != null)
{
int index = row.RowIndex; //gets the row index selected
Label AID1 = (Label)ForumQuesView.Rows[index].FindControl("AID1");
Label AID2 = (Label)ForumQuesView.Rows[index].FindControl("AID2");
Label AID3 = (Label)ForumQuesView.Rows[index].FindControl("AID3");
HyperLink Question = (HyperLink)ForumQuesView.Rows[index].FindControl("Question");
Label Questiontags = (Label)ForumQuesView.Rows[index].FindControl("Questiontags");
Label Askedby = (Label)ForumQuesView.Rows[index].FindControl("Askedby");
Response.Redirect(String.Format("AuzineForumAnswer.aspx?Question=" + Question.Text + "&Questiontags=" + Questiontags.Text + "&Askedby=" + Askedby.Text + "&AID1=" + AID1.Text + "&AID2=" + AID2.Text + "&AID3=" + AID3.Text, Server.UrlEncode(Question.Text), Server.UrlEncode(Questiontags.Text), Server.UrlEncode(Askedby.Text), Server.UrlEncode(AID1.Text), Server.UrlEncode(AID2.Text), Server.UrlEncode(AID3.Text)));
I had passes so many many parameter becuase of accuracy......
Now when I run it and click on the button so its open ​AuzineForumAnswer.AuzineForumAnswerand shows that record very well but problem occurs when the qtags field has "#" type of data like here tags( ​C#, GridView, etc) so, ​ when the tags field has the data includin "#" chracter then it gives "​Object refrence not set to Instance of an object " and if the qtags has normal data like ( specialcharacter gridview sql C ) then it open AuzineForumAnswer.​aspx​ and show data without error
the code behind of AuzineForumAnswer.aspx is below
protected void GetAllData()
{
string connection = System.Configuration.ConfigurationManager.ConnectionStrings["AuzineConnection"].ConnectionString;
using (SqlConnection sqlconn = new SqlConnection(connection))
{
using (SqlCommand sqlcomm = sqlconn.CreateCommand())
{
sqlcomm.CommandText = "Select * From QF where Question='" + Server.UrlDecode(Request.QueryString["Question"].ToString()) + "' And qtags='" + Server.UrlDecode(Request.QueryString["Questiontags"].ToString()) + "' And UserFullName='" + Server.UrlDecode(Request.QueryString["Askedby"].ToString()) + "' And AID1='" + Server.UrlDecode(Request.QueryString["AID1"].ToString()) + "' And AID2='" + Server.UrlDecode(Request.QueryString["AID2"].ToString()) + "' And AID3='" + Server.UrlDecode(Request.QueryString["AID3"].ToString()) + "'";
SqlDataAdapter sda = new SqlDataAdapter(sqlcomm);
DataTable dt = new DataTable();
sda.Fill(dt);
try
{
sqlconn.Open();
ForumQuesView.DataSource = dt;
ForumQuesView.DataBind();
ForumQuesView.AllowPaging = true;
}
catch (Exception ex)
{
Status.Text = ex.Message.ToString();
}
}
}
}
NOW I ALSO DO NOT UNDERSTAND What the problem here because only qtags and question is two field in which user allows to store data as they want, the ​question ​is text and qtags and all are the char field but problem is not in database the problem is here with the character #

Try changing your sql statement to include parameters and see if that works.
What you have now is not only difficult to maintain and causes errors but it’s prone to SQL injection attack quite easily.
sqlcomm.CommandText = "Select * From QF where Question=#Question And qtags=#Qtags And UserFullName=#UserName And AID1=#AID1 And AID2=#AID2 And AID3=#AID3";
sqlcomm.Parameters.Add(new SqlParameter("#Question", Server.UrlDecode(Request.QueryString["Question"])));
sqlcomm.Parameters.Add(new SqlParameter("#Qtags", Server.UrlDecode(Request.QueryString["Questiontags"])));
sqlcomm.Parameters.Add(new SqlParameter("#UserName", Server.UrlDecode(Request.QueryString["Askedby"])));
sqlcomm.Parameters.Add(new SqlParameter("#AID1", Server.UrlDecode(Request.QueryString["AID1"])));
sqlcomm.Parameters.Add(new SqlParameter("#AID2", Server.UrlDecode(Request.QueryString["AID2"])));
sqlcomm.Parameters.Add(new SqlParameter("#AID3", Server.UrlDecode(Request.QueryString["AID3"])))
;

As I know the query is fine even if you using # in the condition.
I doubt for a moment, then I tried this query
Select * From QF where Question='question 1'
And qtags='tag #1';
those query still run smooth and return the record.

Getting error while looping through dataset

I'm using VS 2012 C#. I'm trying to loop through datasets to get information from an Access database. I have a combobox that when an item is selected it should print out all the data for that given scenario. For some the the values in the combobox I get the error, "The SelectCommand property has not been initialized before calling 'Fill'." But for some of the other values I don't and the information is retrieved without an issue. Here is all the necessary code,
con2.Open();
ad2 = new OleDbDataAdapter(query, con2);
ad2.Fill(ds2, "AC_SCENARIO");
con2.Close()
try
{
string end = "ENDDATE";
string start = "START";
foreach (DataRow drRow in ds2.Tables[0].Rows)
{
for (int i = 0; i <= ds2.Tables[0].Columns.Count; i++)
{
string qual0 = ds2.Tables["AC_SCENARIO"].Rows[i]["QUAL0"].ToString();
string qual1 = ds2.Tables["AC_SCENARIO"].Rows[i]["QUAL1"].ToString();
string qual2 = ds2.Tables["AC_SCENARIO"].Rows[i]["QUAL2"].ToString();
string qual3 = ds2.Tables["AC_SCENARIO"].Rows[i]["QUAL3"].ToString();
string qual4 = ds2.Tables["AC_SCENARIO"].Rows[i]["QUAL4"].ToString();
//HERE IS A SAMPLE QUERY
if (qual0 != null && (string)comboBox1.SelectedItem == qual0)
{
ad.SelectCommand = new OleDbCommand("SELECT b.RSV_CAT, b.SEQNUM, b.LEASE, b.WELL_ID, a.QUALIFIER, a.KEYWORD, a.EXPRESSION FROM [AC_ECONOMIC] a INNER JOIN [AC_PROPERTY] b on a.PROPNUM=b.PROPNUM WHERE a.KEYWORD = '"
+ end + "' AND (a.QUALIFIER = '" + qual0 + "' OR a.QUALIFIER IS NULL) AND NOT a.EXPRESSION Like '%[/#]%'", con);
}
ds.Clear();
ad.Fill(ds); //The SelectCommand property has not been initialized before calling 'Fill'.
//ERROR OCCURS HERE
con.Open();
ad.SelectCommand.ExecuteNonQuery();
con.Close();
EDIT: HERE IS THE con CODE
String filePath = textBox1.Text;
con = new OleDbConnection(#"Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + filePath);
I have more queries such as those listed for another dataset and then I merge the two datasets together and output them to a datagridview. The error occurs on the line ad.Fill(ds); If anyone can explain this issue to me or has any help that would be great.

Columns collection of DataTable is zero based index and is one less then Count, the loop should be one less then number of columns as first column is at 0 index and last column is columns count-1. You can use < instead of <= in loop condition to iterate from zero to Count-1.
Change
for (int i = 0; i <= ds2.Tables[0].Columns.Count; i++)
To
for (int i = 0; i < ds2.Tables[0].Columns.Count; i++)
Creation of con is also not present in the code you have in OP

the problem is probably here
if (qual0 != null && (string)comboBox1.SelectedItem == qual0)
In some scenarios the if loop turns out to be true and ad.SelectCommand will get initialized.
EDIT
Are you sure the Select Command is forming correctly for every values of qual0.
Please try using named parameters while forming the query . the qual variable might be having some ' in it that prevent your query getting formed correctly..
EDIT
qual0.Replace("'","\"");

You've not provided a default case for if qual0 is null.
I'm going to assume that you do this multiple times and it's not just the one qual variable you check and build a statement for, if that's the case then you might be better off refactoring to something like this:
if (qual0 != null && (string)comboBox1.SelectedItem == qual0)
{
ad.SelectCommand = new OleDbCommand("SELECT b.RSV_CAT, b.SEQNUM, b.LEASE, b.WELL_ID, a.QUALIFIER, a.KEYWORD, a.EXPRESSION FROM [AC_ECONOMIC] a INNER JOIN [AC_PROPERTY] b on a.PROPNUM=b.PROPNUM WHERE a.KEYWORD = '"
+ end + "' AND (a.QUALIFIER = '" + qual0 + "' OR a.QUALIFIER IS NULL) AND NOT a.EXPRESSION Like '%[/#]%'", con);
}
if (ad.SelectCommand != null)
{
if (!String.IsNullOrEmpty(ad.SelectCommand.CommandText))
{
ds.Clear();
ad.Fill(ds);
}
}

Dropdownlist and Datareader

After trying many solutions listed on the internet I am very confused now. I have a C#/SQL web application for which I am simply trying to bind an ExecuteReader command to a Dropdownlist so the user can select a value. This is a VS2008 project on an XP OS.
How it works is after the user selects a table, I use this selection as an input parameter to a method from my Datamatch.aspx.cs file. Then this Datamatch.aspx.cs file calls a method from my ADONET.cs class file. Finally this method executes a SQL procedure to return the list of columns from that table. (These are all tables in Adventureworks DB). I know that this method returns successfully the list of columns if I execute this SP in SSMS. However, I'm not sure how to tell if it works in VS or not.
This should be simple. How can I do this? Here is some of my code. The T-sQL stored proc:
CREATE PROCEDURE [dbo].[getColumnNames]
#TableName VarChar(50) AS
BEGIN
SET NOCOUNT ON;
SELECT col.name 'COLUMN_NAME' FROM sysobjects obj
INNER JOIN syscolumns col ON obj.id = col.id
WHERE obj.name = #TableName
END
It gives me desired output when I execute following from SSMS: exec getColumnNames 'AddressType'
And the code from Datamatch.aspx.cs file currently is:
private void CreateDropDownLists()
{
SqlDataReader dr2 = ADONET_methods.DisplayTableColumns(targettable);
int NumControls = targettable.Length;
DropDownList ddl = new DropDownList();
DataTable dt = new DataTable();
dt.Load(dr2);
ddl.DataValueField = "id";
ddl.DataTextField = "text";
ddl.DataSource = dt;
ddl.DataBind();
for (int counter = 0; counter < NumberOfControls; counter++)
{
ddl.ID = "DropDownListID " + (counter + 1).ToString();
btnSubmit.Style.Add("top", "auto");
btnSubmit.Style.Add("left", "auto");
btnSubmit.Style.Add("position", "absolute");
if (counter < 7)
{
ddl.Style["top"] = 100 * counter + 80 + "px";
ddl.Style["left"] = 250 + "px";
int bSubmitPosition = NumberOfControls * 100 + 80;
btnSubmit.Style.Add("top", System.Convert.ToString(bSubmitPosition) + "px");
}
else if (counter >= 7)
{
ddl.Style["top"] = 100 * counter - 620 + "px";
ddl.Style["left"] = 550 + "px";
int bSubmitPosition = NumberOfControls * 100 - 620;
btnSubmit.Style.Add("top", System.Convert.ToString(bSubmitPosition) + "px");
}
ddl.SelectedIndexChanged += new EventHandler(SelectedIndexChanged);
ddl_ht.Add(counter, ddl.SelectedValue);
pnlDisplayData.Controls.Add(ddl);
pnlDisplayData.Controls.Add(new LiteralControl("<br><br><br>"));
pnlDisplayData.Visible = true;
pnlDisplayData.FindControl(ddl.ID);
// dr.Close();
}
}
private void CreateLabels()
{
for (int counter = 0; counter < NumberOfControls; counter++)
{
Label lbl = new Label();
lbl.ID = "Label" + counter.ToString();
lbl.Text = headers[counter];
lbl.Style["position"] = "absolute";
if (counter < 7)
{
lbl.Style["top"] = 100 * counter + 50 + "px";
lbl.Style["left"] = 250 + "px";
}
else if (counter >= 7)
{
lbl.Style["top"] = (100 * counter) - 650 + "px";
lbl.Style["left"] = 550 + "px";
}
pnlDisplayData.Controls.Add(lbl);
pnlDisplayData.Controls.Add(new LiteralControl("<br><br><br>"));
}
}
Where ADONET_methods.DisplayTableColumns(targettable) is:
public static SqlDataReader DisplayTableColumns(string tt)
{
SqlDataReader dr = null;
string TableName = tt;
string connString = "Server=(local);Database=AdventureWorks;Integrated Security = SSPI";
string errorMsg;
SqlConnection conn2 = new SqlConnection(connString);
SqlCommand cmd = new SqlCommand("getColumnNames"); //conn2.CreateCommand();
try
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.Connection = conn2;
SqlParameter parm = new SqlParameter("#TableName", SqlDbType.VarChar);
parm.Value = "Person." + TableName.Trim();
parm.Direction = ParameterDirection.Input;
cmd.Parameters.Add(parm);
conn2.Open();
dr = cmd.ExecuteReader();
}
catch (Exception ex)
{
errorMsg = ex.Message;
}
return dr;
}
The CreateLabels method above correctly shows me the labels. But the CreateDropDownLists method just shows me one dropdownlist with nothing in it. In other words, it is not selectable.
So how can I verify that the datareader is returning all 4 columns and inspect their values? I was able to find from datareader "COLUMN_NAME" but I don't know what properties to search to verify the column names.

It doesn't look like you're actually binding the dropdown anywhere in this code. You need to do something like this:
ddl.DataTextField = "COLUMN_NAME";
ddl.DataValueField = "COLUMN_NAME";
ddl.DataSource = dr.ExecuteReader();
ddl.DataBind();
Alternately, you can do this all in your page markup by using a SqlDataSource control.
<asp:SqlDataSource
id="SqlDataSource1"
runat="server"
DataSourceMode="DataSet"
ConnectionString="myConnString"
SelectCommand="myStoredProcedure"
>
</asp:SqlDataSource>
<asp:MyDropDownList id="ddl" runat="server" DataSource="SqlDataSource1"
DataTextField="COLUMN_NAME" DataValueField="COLUMN_NAME" />

In the ADONET_methods.DisplayTableColumns(targettable) method before returning dr, check if you get some value for dr.GetValue() using a breakpoint
string temp;
while(dr.Read())
{
temp = dr.GetValue(); //use a breakpoint here
}
Also instead of using dataReader and loading it to dataTable, you can better use a dataAdapter to fill the dataTable directly
public static DataTable DisplayTableColumns(string tt)
{
Datatable dt = new DataTable();
SqlDataAdapter da = new SqlDataAdapter();
try
{
da.Fill(dt);
}
catch (Exception ex)
{
errorMsg = ex.Message;
}
string temp;
foreach(DataRow row in dt.Rows)
{
foreach(DataColumn column in dt.Columns)
{
temp = (row[column]); // use your breakpoint here
}
}
return dt;
}