Is It possible to create a Linq for retrieving longest string values of DataReader columns ? Column data should be converted to string and then return longest string.
What I know so far is how to get same thing with DataTable which is Enumerable (as I asked here), example:
string maxString = dt
.AsEnumerable()
.Select(row => row[mycolumn].ToString())
.OrderByDescending(st => st.Length)
.FirstOrDefault();
I've tried combining upper solution for DataReader like this:
var enumerable = reader
.Cast<IDataRecord>();
string max_string = enumerable
.Select(record => record.GetName(col).ToString())
.Aggregate((s, a) => a.Length > s.Length ? a : s);
Unfortunally this doesn't work, I get
Sequence contains no elements in at
System.Linq.Enumerable.Aggregate
error. Thanks for help in advance :)
EDIT: I'm looking for a solution without loading data into Datatable etc., just directly from DataReader object. I'm trying to avoid "Out of memory exception", because data is large.
Latest attempt, suggested by Power Mouse (It returns correct value, but from column 1 only):
for (int col = 0; col < reader.FieldCount; col++)
{
string col_name = reader.GetName(col).ToString();
var enumerable = reader.Cast<IDataRecord>();
string max_string = enumerable.Where(x => enumerable.Max(y => y[col_name].ToString()
.Length) == x[col_name].ToString().Length)
.FirstOrDefault()?[col_name].ToString();
Console.WriteLine("max string of column is : " + max_string);
}
so according to your original request: you need to find the longest text in specific column when you utilizing a DataReader on the fly.
please review example
string storedString = String.Empty;
SqlConnection connection = new SqlConnection(this.Connection.ConnectionString);
using (connection)
{
string SQLcommand = "select * FROM (VALUES(1, 'xxx' ), (2, 'really long string xxxxxx'), (3, 'short string'), (4, 'another string')) t (id, fName)";
SqlCommand command = new SqlCommand(SQLcommand, connection);
connection.Open();
SqlDataReader reader = command.ExecuteReader();
while (reader.Read())
{
storedString = reader.Cast<IDataRecord>()
.Where(w=> w.GetOrdinal("fName").ToString().Length == reader.Cast<IDataRecord>()
.Max(m => m.GetOrdinal("fName")
.ToString().Length))
.Select(s=> s.GetString(1))
.FirstOrDefault();
}
}
Console.WriteLine($"The longest string: {storedString}. charcount= {storedString.Length}");
the result would be :
The longest string: really long string xxxxxx. charcount= 25
as you explained that you need to check multiple columns:
string storedNameString = String.Empty;
string storedName2String = String.Empty;
SqlConnection connection = new SqlConnection(this.Connection.ConnectionString);
using (connection)
{
string SQLcommand = "select * FROM (VALUES(1, 'xxx', 'dddddd' ), (2, 'really long string xxxxxx','dfghdt'), (3, 'short string', 'anothercolumn long string'), (4, 'another string', 'test')) t (id, fName, fName2)";
SqlCommand command = new SqlCommand(SQLcommand, connection);
connection.Open();
SqlDataReader reader = command.ExecuteReader();
while (reader.Read())
{
string fName = reader.GetString(reader.GetOrdinal("fName")).ToString();
if(fName.Length >= storedNameString.Length)
storedNameString = fName;
string fName2 = reader.GetString(reader.GetOrdinal("fName2")).ToString();
if (fName2.Length >= storedName2String.Length)
storedName2String = fName2;
}
}
Console.WriteLine($"The longest string: {storedNameString}. charcount= {storedNameString.Length}");
Console.WriteLine($"The longest string: {storedName2String}. charcount= {storedName2String.Length}");
I solved my problem, unfortunally without LINQ. Problem is that with DataReader you cannot just simply loop through rows & columns as you can with DataTable once stored in memory, but you must perfom somekind of same logic while reader.Read() method is running.
So, best thing I could came up with is to store column indexes and their string values into Dictionary while .Read() method is running.
Doing that, you must be careful about string blank spaces & null values. Here is my solution, which runs good for me:
Dictionary<int, string> col_values = new Dictionary<int, string>();
using (OracleDataReader reader = cmd.ExecuteReader())
{
for (int i = 0; i < reader.FieldCount; i++)
{
//First add indexes to Dictionary
// I add column names here - didn't test for string.Empty !!
col_values.Add(i, string.Empty);
}
//Then read row by row and modify Dictionary - If text is larger than string.Empty
//Dictionary must be .ToArray(), or else you'll have an error for modifying collection
while (reader.Read())
{
foreach (var item in col_values.ToArray())
{
string rdr_text = reader[item.Key].ToString().Trim()??string.Empty;
if (item.Value.Length<rdr_text.Length)
{
col_values[item.Key] = rdr_text;
}
}
}
foreach (var item in col_values)
{
//...And here we have all longest strings stored, for each column...Job done
}
}
For my purpuses this Iteration reads around 2.3 mio rows with 12 columns in 4 minutes. It's not fast, but at least It works. If anyone has better/faster idea please provide answer.
Related
as the title say's I am attempting to get to edit a list item inside of a foreach loop. I know it is not possible, so I am asking for solutions to work around this.
As this is taking data directly from a MySQL server, it need's to be done inside of this loop and with the correct indexes.
I attempted to use a second list and change the data outside of the foreach loop, but that did not account for the index positions of the items.
foreach (string friend in friendsOnline)
{
query = "SELECT `username` FROM `accounts` WHERE email = '" + friend + "'";
cmd = new MySqlCommand(query, conn);
rdr = cmd.ExecuteReader();
while (rdr.Read())
{
for (int i = 0; i<friendsOnline.Count; i++)
{
if (friendsOnline[i].Contains(friend))
{
friendsOnline[i] = rdr[0].ToString();
}
}
}
}
As I can see, you have emails in the friendsOnline:
string[] friendsOnline = new[] {
"Me#Shire.com",
"Dark.Lord#Mordor.gov",
};
and you want to change these emails into names:
string[] friendsOnline = new[] {
"Frodo Baggins",
"Sauron the Black",
};
First of all, I suggest executing query just once: SQL is too slow to be run in a loop; instead you can build a dictionary which eMail corresponds to which indexes:
using System.Linq;
...
// eMail - indexes correspondence
Dictionary<string, int[]> dict = friendsOnline
.Select((name, index) => (name : name, index : index))
.GroupBy(pair => pair.name, pair => pair.index, StringComparer.OrdinalIgnoreCase)
.ToDictionary(group => group.Key, group => group.ToArray());
// We take all users (no WHERE), but once
// We read both username and email
string sql =
#"SELECT `username`,
`email`
FROM `accounts`";
//DONE: using - we should Dispose IDisposable Command and Reader
using cmd = new MySqlCommand(sql, conn);
using rdr = cmd.ExecuteReader();
while (rdr.Read()) {
string name = Convert.ToString(rdr[0]);
string mail = Convert.ToString(rdr[1]);
// If we have correspondent indexes, we change friendsOnline
if (dict.TryGetValue(mail, out var indexes))
foreach (var index in indexes)
friendsOnline[index] = name;
}
I have a function that inserts SQL data based on a spreadsheet. I have had the issue of certain rows triggering the exception due to being truncated, and I am trying to figure out the rows that are causing the issue. (I have to query 3 different tables so I am using a function passing in SQL/command parameters/values instead of writing the same function 3 times)
The function works to insert the SQL data, except for the few rows that throws the ex message:
String or Binary data would be truncated. The statement has been
terminated
My question is how do I print out the row number that causes the above message to troubleshoot the data in the excel sheet. The size of the sheet is 100+ thousand rows, so I don't want to go through it row by row.
The function I have:
public static void insert_data(string[] cols, string[] vals, string sql)
{
int exception_count = 0;
List<string> rows = new List<string>();
string connectionString = "Server = ; Database = ; User Id = ; Password = ";
using (SqlConnection connection = new SqlConnection(connectionString))
using (SqlCommand command = new SqlCommand(sql, connection))
{
connection.Open();
for(int x = 0; x<cols.Length; x++)
command.Parameters.AddWithValue(cols[x], vals[x]);
try
{
command.ExecuteNonQuery();
}
catch (Exception ex)
{
exception_count =+ 1;
Console.WriteLine(ex.Message);
//rows.Add(--rows number--);
}
}
}
Like I can see, the rows come from array, so first use a lambda expression to find the rows that have more than the length that you want, or viceverse find rows than have less or equal length, depends of you.
public void test2()
{
//ensure that there is no empty rows in the array... of will thrown an exception
string[] vals = new string[7];
int myMaxColumnDatabaseLenght = 7;
vals[0] = "length7";
vals[1] = "length7";
vals[2] = "length7";
vals[3] = "length7";
vals[4] = "length_8";
vals[5] = "length__9";
vals[6] = "length__10";
Debug.WriteLine(vals.Count());
vals = vals.Where(x => x.Length <= myMaxColumnDatabaseLenght).ToArray();
Debug.WriteLine(vals.Count());
}
If you want make this dynamic, maybe you need query the length definition of the specific columns in SQL, if you want this part I will try to find for add information about that... .
Update: as the original question was essentially answered, I've marked this as complete.
I have a C# project in which I'd like to query a database. The SQL query will SELECT from a table, and in the WHERE clause I want to filter the results from a pre-defined list of values specified in C#.
List<string> productNames = new List<string >() { "A", "B", "C" };
foreach (name in productNames)
{
string query = #"
SELECT *
FROM products
WHERE name IN (#name);";
// Execute query
MySqlCommand cmd = new MySqlCommand(query, dbConn);
cmd.Parameters.AddWithValue("name", name);
MySqlDataReader row = cmd.ExecuteReader();
while (row.Read())
{
// Process result
// ...
}
}
However I'm getting an error:
There is already an open DataReader associated with this Connection
which must be closed first
Is it not possible then to use a for loop to add parameters this way to a SELECT statement?
You need to dispose your object to not get the exception. However you don't need to iterate over values and run a query for every value in the list. Try the following code. It makes a parameter for every value and adds it to command to use in "IN (...)" clause.
Also "using" keywords handles disposing objects.
List<string> productsIds = new List<string>() { "23", "46", "76", "88" };
string query = #"
SELECT *
FROM products
WHERE id IN ({0});";
// Execute query
using (MySqlCommand cmd = new MySqlCommand(query, dbConn))
{
int index = 0;
string sqlWhere = string.Empty;
foreach (string id in productsIds)
{
string parameterName = "#productId" + index++;
sqlWhere += string.IsNullOrWhiteSpace(sqlWhere) ? parameterName : ", " + parameterName;
cmd.Parameters.AddWithValue(parameterName, id);
}
query = string.Format(query, sqlWhere);
cmd.CommandText = query;
using (MySqlDataReader row = cmd.ExecuteReader())
{
while (row.Read())
{
// Process result
// ...
}
}
}
You are doing few things wrong. Let me point out them:
Everything is fine in the first iteration of the loop, when you are in second iteration the First Reader associated with the command remains opened and that result in current error.
You are using Where IN(..) you you can specify the values within IN as comma separated so there is no need to iterate through parameters.
You can use String.Join method to construct this values with a comma separator.
Now take a look into the following code:
List<int> productsIds = new List<int>() { 23, 46, 76, 88 };
string idInParameter = String.Join(",", productsIds);
// Create id as comma separated string
string query = "SELECT * FROM products WHERE id IN (#productId);";
MySqlCommand cmd = new MySqlCommand(query, dbConn);
cmd.Parameters.AddWithValue("#productId", idInParameter);
MySqlDataReader row = cmd.ExecuteReader();
while (row.Read())
{
// Process result
// ...
}
Please note if the id field in the table is not integers then you have to modify the construction of idInParameter as like the following:
idInParameter = String.Join(",", productsIds.Select(x => "'" + x.ToString() + "'").ToArray());
Pass the comma separated productIds string instead of looping. Read more about IN here.
string productIdsCommaSeparated = string.Join(",", productsIds.Select(n => n.ToString()).ToArray())
string query = #"
SELECT *
FROM products
WHERE id IN (#productId);";
// Execute query
MySqlCommand cmd = new MySqlCommand(query, dbConn);
cmd.Parameters.AddWithValue("productId", productIdsCommaSeparated );
MySqlDataReader row = cmd.ExecuteReader();
while (row.Read())
{
// Process result
// ...
}
The error you are getting is because you do not close the MySqlDataReader. You must close it to get rid of error before you call ExecuteReader in next iteration but this is not proper way in this case.
From what I tried and tested seems best solution (especially for text column types) is to create a list of individual parameters and add it as a range the to the query and parameters.
e.g:
List<MySqlParameter> listParams = new List<MySqlParameter>();
for (int i = 0; i < listOfValues.Length; i++)
{
listParams.Add(new MySqlParameter(string.Format("#values{0}", i),
MySqlDbType.VarChar) { Value = listOfValues[i] });
}
string sqlCommand = string.Format("SELECT data FROM table WHERE column IN ({0})",
string.Join(",", listParams.Select(x => x.ParameterName)));
......
using (MySqlCommand command = new MySqlCommand(sqlCommand, connection)
{
............
command.Parameters.AddRange(listParams.ToArray());
............
}
I am trying to populate a group of labels in a C# windows form with some values that are in a certain attribute (PlayerName) in a database that I have in access.
Currently the only code I have is:
OleDbConnection connection = new OleDbConnection(CONNECTION STRING HERE);
OleDbCommand command = new OleDbCommand();
command.Connection = connection;
command.CommandText = "SELECT PlayerName FROM [TotalPlayerName] WHERE Team = 1 AND SportID = " + Form1.IDNumber;
I need a list or array that holds these values so I can use them to populate the labels, but I am unaware of how to do this.
You need to call ExecuteReader to obtain a data reader and then loop through the rows of the result set like this:
List<string> result = new List<string>();
using (var reader = command.ExecuteReader())
{
while (reader.Read())
{
result.Add(reader.GetString(0));
}
}
Before you do this, don't forget to open the connection like this:
connection.Open();
There are a couple of things here..... for sake of best practice well its more standard practice... as I like to say!
Use USING as this cleans up after connection.. see here for great examples in a "using" block is a SqlConnection closed on return or exception?
using (OdbcDataReader DbReader = DbCommand.ExecuteReader())
{
int fCount = DbReader.FieldCount;
while (DbReader.Read())
{
Label1 = DbReader.GetString(0);
Label2 = DbReader.GetString(1);
Label3 = DbReader.GetString(2);
Label4 = DbReader.GetString(3);
for (int i = 0; i < fCount; i++)
{
String col = DbReader.GetString(i);
Console.Write(col + ":");
}
Console.WriteLine();
}
}
NB your SQL only return 1 field /String at the moment
while reading the data fill the list like
List<string> players = new List<string>();
OleDbDataReader rdr = command.ExecuteReader();
While(rdr.Read())
{
players.Add(rdr["PlayerName"].ToString());
}
You need to create a OleDbReader object to read the response from the query. You will also need to create a List to store the data Something like this:
List<string> playerNameList = new List<string>();
using (OleDbReader r = command.ExecuteReader())
{
while(reader.Read())
{
playerNameList.Add(reader.GetString(0));
}
}
One option might be using OleDbDataAdapter to fill a DataTable those values that returns your query;
var dt = new DataTable();
using(var da = new OleDbDataAdapter(command))
{
da.Fill(dt);
}
And since your query return one column, you can use AsEnumerable to that datatable to get them as a string like;
List<string> list = dt.AsEnumerable()
.Select(r => r.Field<string>("PlayerName"))
.ToList();
You can read: Queries in LINQ to DataSet
By the way, you should always use parameterized queries. This kind of string concatenations are open for SQL Injection attacks.
Also use using statement to dispose your connection and command automatically as I did for OleDbDataAdapter in my example.
how to split into a string array and pass them to command parameters or hiddenfield, just need to split the string "S0010M,AZI002M,3,12/26/2013 12:00:00 AM,VDIQ20"
to pass with parameters like
cmd.Parameters.AddWithValue("#DealerCode", "S0010M");
cmd.Parameters.AddWithValue("#Code", "AZI002M");
cmd.Parameters.AddWithValue("#Qty", 33);
cmd.Parameters.AddWithValue("#ExpireDate", "12/26/2015");
cmd.Parameters.AddWithValue("#BatchNumber", "VDIQ20");
i have big problem about this .. please can you help me to fix this , beaus still learning the subject..
after click on Return button , take the data from gridview, it can be more than one rows.
protected void btnReturn_Click(object sender, EventArgs e)
{
int rowIndex = 0;
StringCollection SetDEL_Stores = new StringCollection();
if (ViewState["CurrentData"] != null)
{
DataTable dtCurrentTable = (DataTable)ViewState["CurrentData"];
DataRow drCurrentRow = null;
if (dtCurrentTable.Rows.Count > 0)
{
for (int i = 1; i <= dtCurrentTable.Rows.Count; i++)
{
var DealerCode = HFDealerCode.Value;
var ItemIdentityCode = (Label)GridViewSalesReturn.Rows[rowIndex].Cells[2].FindControl("ItemIdentityCode");
var Qty = (Label)GridViewSalesReturn.Rows[rowIndex].Cells[8].FindControl("Quantity");
var ExpireDate = (Label)GridViewSalesReturn.Rows[rowIndex].Cells[6].FindControl("ExpireDate");
var BatchNumber = (Label)GridViewSalesReturn.Rows[rowIndex].Cells[7].FindControl("BatchNumber");
CultureInfo ci = new CultureInfo("en-GB");
SetDEL_Stores.Add(DealerCode + "," + ItemIdentityCode.Text + "," + decimal.Parse(Qty.Text) + "," + DateTime.ParseExact(ExpireDate.Text, "dd/MM/yyyy", CultureInfo.InvariantCulture) + "," + BatchNumber.Text);
rowIndex++;
}
InsertDEL_Stores(SetDEL_Stores);
}
}
}
//in InsertDEL_Stores(SetDEL_Stores); event , taking the stringline separated with "," ,,
private void InsertDEL_Stores(StringCollection SC_PurLinr)
{
String strConnString = ConfigurationManager.ConnectionStrings["CBConnectionString"].ConnectionString;
DataSet ds = new DataSet();
SqlConnection con = new SqlConnection(strConnString);
SqlCommand cmd = new SqlCommand("sp_DEL_Stores_IU", con);
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.AddWithValue("#DealerCode", SC_PurLinr[0]);
cmd.Parameters.AddWithValue("#Code", SC_PurLinr[1]);
cmd.Parameters.AddWithValue("#Qty", SC_PurLinr[2]);
cmd.Parameters.AddWithValue("#ExpireDate", SC_PurLinr[3]);
cmd.Parameters.AddWithValue("#BatchNumber", SC_PurLinr[4]);
con.Open();
cmd.ExecuteNonQuery();
con.Close();
}
It is not clear why you need a string collection first. If you want to keep the contents of the single rows in the GridView then start defining a class for your items where every single field is typed correctly (string for strings, numeric for numerics and datetime for dates) Copying the content of the grid in a string collection is just a waste of time and memory because every time you need to use the values stored in the string collection you need to find the correct string and split it to the individual fields.
I could just offer a pseudocode here because I haven't the possibility to test it.
(As an example I have named this class MyItem, but you could call it as you wish)
public class MyItem
{
public string DealerCode;
public string ItemCode;
public int Quantity;
public Datetime ExpireDate;
public string BatchNumber;
}
Then in your loop
// To keep the content of the grid keyed on the BatchNumber field
Dictionary<string, MyItem> items = new Dictionary<string, MyItem>();
for (int rowIndex = 0; i < dtCurrentTable.Rows.Count; i++)
{
MyItem itm = new MyItem();
itm.DealerCode = HFDealerCode.Value.ToString();
itm.ItemCode = GetGridValue(rowIndex, 2, "ItemIdentityCode");
itm.Quantity = Convert.ToDecimal(GetGridValue(rowIndex, 8, "Quantity");
itm.ExpireDate = Convert.ToDateTime(GetGridValue(rowIndex, 6, "ExpireDate");
itm.BatchNumber = GetGridValue(rowIndex, 7, "BatchNumber");
// Add the item to the dictionary for future reuses, however if you just want to store
// the item in the database this line is not needed
items.Add(itm.BatchNumber, itm);
// notice that the storing is executed inside the loop that extracts the values
// so every row is updated/inserted in the database
InsertDEL_Stores(itm);
}
GetGridValue is a method that you should write taking the parameters passed and returning a string with the value searched on the current row of your gridview. This could be simple as
string GetGridValue(int rowIndex, int cellIndex, string controlName)
{
Control c = GridViewSalesReturn.Rows[rowIndex].Cells[cellIndex].FindControl(controlName);
return (c != null ? c.Value.ToString() : "");
}
but you need to test it for its correctness.
However, after that you have an istance of MyItem class that you could store in the dictionary for future reuses or just pass it to the database working procedure
private void InsertDEL_Stores(MyItem itm)
{
String strConnString = ConfigurationManager.ConnectionStrings["CBConnectionString"].ConnectionString;
using(SqlConnection con = new SqlConnection(strConnString))
using(SqlCommand cmd = new SqlCommand("sp_DEL_Stores_IU", con))
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.AddWithValue("#DealerCode", itm.DealerCode);
cmd.Parameters.AddWithValue("#Code", itm.ItemCode);
cmd.Parameters.AddWithValue("#Qty", itm.Quantity);
cmd.Parameters.AddWithValue("#ExpireDate", itm.ExpireDate);
cmd.Parameters.AddWithValue("#BatchNumber", itm.BatchNumber);
con.Open();
cmd.ExecuteNonQuery();
}
}
I am aware that this code could raise more questions than the one that you try to resolve, neverless I think that this is more OOP than a simple string split
To split a string using commas as the separator character do the following
String[] values = str.split(",");
Then you can access the array in the following way
values[0];
But since your question is a bit confusing I suggest you read well the comments by other contributors what best suits your needs, how you are passing those values to the command parameters. Certainly, dictionaries and lists are more efficient than String collections