We Keep Coding

How to find the number of NULL/NOT NULL values from each column in a table in SQL Server [C#]?

I have the following code to get all the columns from a SQL table in C#:
public List<string> GetColumns(string tableName)
{
var columns = new List<string>();
using (var conn = new SqlConnection(_sqlServerConnectionString.SqlServerConnectionString))
{
conn.Open();
var selectQuery = $"SELECT COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = '{tableName}'";
using (var cmd = new SqlCommand(selectQuery, conn))
{
using (var reader = cmd.ExecuteReader(CommandBehavior.CloseConnection))
{
columns.Add(reader["COLUMN_NAME"].ToString());
}
}
conn.Close();
}
return columns;
}
In C#, how do I find out the number of NULL, NOT NULL and TOTAL values from each of these columns?
For example:
COLUMN 1, COLUMN 2
X, A
Y, NULLL
Y, NULL
Z, B
NULL, C
NULL, NULL
So for COLUMN1
TOTAL # of values: 6
NULL: 2
NOT NULL: 4
So for COLUMN2
TOTAL # of values: 6
NULL: 3
NOT NULL: 3
Total should be same for all columns.

NULL values are excluded from all aggregate functions except for count(), so
. . .
To determine the number of null/non-null values in a column,
select sum( case when t.col_1 is null then 1 else 0 end ) as null_count,
sum( case when t.col_1 is null then 0 else 1 end ) as not_null_count,
count(*) as total
from someTable t

Get DataTable Value from Adjacent Cell without ForEach loop?

I am working my way around this problem: Create multiple results from single row joining either 2 or 3 tables based off conditional of table1 results? and while I wish I could take Strawberry's advice, I can't, so I am trying now to do more in C# rather than through the DB, but also trying to be smart about how much CPU I utilize.
For scale, table1 may have 50,000 records which have those 20 codetype fields which have to be evaluated before they can be matched to table2, which has about 2,000 rows, or table3, which could have 200,000 rows. To avoid hammering the DB, I am going to store what is possible in memory, limit results by date as much as possible, but I want to avoid 2,000 foreach loops per 20 codetype matches.
To start off, I am getting the results I need from table2 and loading them into a C# DataTable stored as the variable named descriptionLookup:
id, description
13 Item 13 Description
15 Item 15 Description
17 Item 17 Description
18 Item 18 Description
21 Item 21 Description
28 Item 28 Description
45 Item 45 Description
And table3 as lookupTable:
id, table2id
1 15
33 17
21 28
doing a simple (not showing all surrounding code, just relevant):
var rawData = new DataTable();
using (OdbcCommand com = new OdbcCommand("SELECT id, description from table2", conn))
{
using (OdbcDataReader reader = com.ExecuteReader())
{
rawData.Load(reader);
conn.Close();
return rawData;
}
}
I then have that assigned to a variable that called the function. Now I have to deal with table1:
codeid1,codeid2,codeid3,...codeid20 ... codetype1,codetype2,codetype3,.....codetype20
18 13 1 33 0 0 1 1
13 21 45 0 0 1 0 0
Using a foreach row, I need to evaluate each codetype column for a 1 or a 0. When codetype=1 I need to grab the associated codeid, and then do a lookup from the data I am holding in memory as descriptionLookup to see what the table2id is that matches the id in lookuptable and then use that to lookup the description of the associated field in table2.
If codetype is 0, I just need to match codeid with the associated description field in table2.
I am looking at how to lay this out, and all I can think of is:
DataTable descriptionLookup= DB.ExecuteQuery("SELECT id, description from table2");
DataTable lookupTable= DB.ExecuteQuery("SELECT id, table2id from table3");
DataTable mainData= DB.ExecuteQuery("SELECT * from from table1");
foreach (var row in mainData)
{
var manDataId = row.GetColumn("id");
var subroutine = new Dictionary<string, string>();
for (var index = 1; index < 20; index++)
{
string description;
if (row.GetColumn("codetype" + index) == "1")
{
int idLookup = row.GetColumn(["codeid" +index]);
foreach (var row2 in lookupTable)
{
if (row3.GetColumn("id") == idLookup)
{
descriptionId = row3.GetColumn("table2id");
foreach (var row2 in descriptionLookup)
{
if (row.GetColumn("id") == descriptionId)
{
description = row2.GetColumn("description").ToString();
}
}
}
}
}elseif (row.GetColumn("codetype" + index) == "0")
{
descriptionId = row.GetColumn(["codeid" +index]);
foreach (var row2 in descriptionLookup)
{
if (row.GetColumn("id") == descriptionId)
{
description = row2.GetColumn("description").ToString();
}
}
}
subroutine.Add(manDataId.ToString(), description.ToString());
}
ArrayData.Add(subroutine);
}
I haven't tried running the above code, so there is probably a problem or two in there, but it gets the point across of looping through thousands of records using foreach (var row3 in idLookup). The alternative seems to be making a DB query, but that seems more intensive than just looping through what is in memory, but it seems like there is a better way that I am missing on how to get the id or table2id without using a foreach.
To make this the longest looking question in history :) Here is my SQL that I have so far:
SELECT table1.id, table1.field1, table1.field2,
table2.description, fee.amt as fee FROM table2
INNER JOIN table1
ON table2.id = table1.codeid1
OR table2.id = table1.codeid2
OR table2.id = table1.codeid3
OR table2.id = table1.codeid4
OR table2.id = table1.codeid5
OR table2.id = table1.codeid6
OR table2.id = table1.codeid7
OR table2.id = table1.codeid8
OR table2.id = table1.codeid9
OR table2.id = table1.codeid10
OR table2.id = table1.codeid11
OR table2.id = table1.codeid12
OR table2.id = table1.codeid13
OR table2.id = table1.codeid14
OR table2.id = table1.codeid15
OR table2.id = table1.codeid16
OR table2.id = table1.codeid17
OR table2.id = table1.codeid18
OR table2.id = table1.codeid19
OR table2.id = table1.codeid20
INNER JOIN fee ON table2.id = fee.id
WHERE table1.codetype1 = 0
AND table1.codetype2 = 0
AND table1.codetype3 = 0
AND table1.codetype4 = 0
AND table1.codetype5 = 0
AND table1.codetype6 = 0
AND table1.codetype7 = 0
AND table1.codetype8 = 0
AND table1.codetype9 = 0
AND table1.codetype10 = 0
AND table1.codetype11 = 0
AND table1.codetype12 = 0
AND table1.codetype13 = 0
AND table1.codetype14 = 0
AND table1.codetype15 = 0
AND table1.codetype16 = 0
AND table1.codetype17 = 0
AND table1.codetype18 = 0
AND table1.codetype19 = 0
AND table1.codetype20 = 0
This works great as long as there isn't any of the codetype that have a 1, otherwise that record will be skipped. There will likely not be a single row where all codeid / codetype are filled out, nor will there ever be a case where codetype will be 1 across the board to match the inverse of this query.

Join DataTables to get new DataTable via LINQ

I've a datatable named which contains data with a column of ID, firstname, lastname. Another datatable contains columns of code, userID1, userID2, userID3, work.
Now i want a new datatable which should contain the column of both the datatable with proper data.
New datatable should contain data as: ID, userfullname1, userfullname2, userfullname3, work.
Here we get the value of userfullname1 by firstname, lastname of datatable1 & userID1 of datatable2. Similarly we get the value of userfullname2 by firstname, lastname of datatable1 & userID2 of datatable2 & so on.
The value of ID in Datatable1 is same as userID1, userID2, userID3 in Datatable2.
Finally, i want to obtain a new datatable with code, userfullname1, userfullname2, userfullname3, work. But users IDs are in datatable1. So, i want to bind the names of Datatable1 to the all 3 userids of Datatable2 via their IDs whichare present in both the tables.
Datatable1 :
iD name
1 b
2 d
3 f
4 s
....
Datatable2 :
Code userid1 userid2 userid3 work
1f 1 3 6 gg
2g 1 4 7 gg
3b 3 4 7 gg
4v 4 3 8 gg
New Datatable :
Code username1 username2 username3 work
1f a b c gg
2g d f r gg
3b c h g gg
4v d s h gg
How can i join & get the new datatable ?

It sounds like you have a table with User Ids and you want to join your profile table to get those names. You can do this with sub-queries like:
var myTable = UserIds.Select(u => new
{
User1FullName = u.UserProfiles.FirstOrDefault(p => p.UserId == u.userId1).Select(p => p.FirstName + " " + p.LastName),
User2FullName = u.UserProfiles.FirstOrDefault(p => p.UserId == u.userId2).Select(p => p.FirstName + " " + p.LastName)
// etc...
});
You can do it with joins like:
var myTable = (from u in UserIds
join p1 in UserProfiles on u.UserId1 equals p1.UserId
join p2 in UserProfiles on u.UserId2 equals p2.UserId
// etc...
select new
{
User1FullName = p1.FirstName + " " + p1.LastName,
User2FullName = p2.FirstName + " " + p2.LastName,
// etc...
});

You can use LINQ to achieve what you want:
DataTable tblResult = new DataTable();
tblResult.Columns.Add("ID");
tblResult.Columns.Add("userfullname1");
tblResult.Columns.Add("userfullname2");
tblResult.Columns.Add("userfullname3");
tblResult.Columns.Add("Work");
var query = from r1 in datatable1.AsEnumerable()
from r2 in datatable2.AsEnumerable()
let id = r1.Field<int>("ID")
let userID1 = r2.Field<int>("userID1")
let userID2 = r2.Field<int>("userID2")
let userID3 = r2.Field<int>("userID3")
where id == userID1 && id == userID2 && id == userID3
select new { r1, r2, id, userID1, userID2, userID3 };
foreach (var x in query)
{
DataRow row = tblResult.Rows.Add();
string firstName = x.r1.Field<string>("firstname");
string lastName = x.r1.Field<string>("lastname");
string userfullname1 = string.Format("{0} {1} {2}", firstName, lastName, x.userID1);
string userfullname2 = string.Format("{0} {1} {2}", firstName, lastName, x.userID2);
string userfullname3 = string.Format("{0} {1} {2}", firstName, lastName, x.userID3);
row.SetField("ID", x.id);
row.SetField("userfullname1", userfullname1);
row.SetField("userfullname2", userfullname2);
row.SetField("userfullname3", userfullname3);
row.SetField("Work", x.r2.Field<string>("Work"));
}

If you just want your resulting table to have name and id, you should do it like this:
DataTable dtResult = new DataTable();
dtResult.Columns.Add("ID", typeof(string));
dtResult.Columns.Add("name", typeof(string));
var result = from datatable1 in table1.AsEnumerable()
join datatable2 in table2.AsEnumerable()
on datatable1.Field<string>("ID") equals datatable2.Field<string>("userID")
select dtResult.LoadDataRow(new object[]
{
datatable1.Field<string>("ID"),
string.Format("{0} {1}" ,datatable1.Field<string>("fname") ,datatable1.Field<string>("lname")),
}, false);
result.CopyToDataTable();

Find duplicate row in a datatable by id - add the duplicate column values as concatented string to a new column then remove dulplicate column

I have the following DataTable
RequestId userID ProductCode ProductValue
1 "10004" 70 85.50
2 "10004" 67 944.00
3 "10333" 30 97.00
4 "23344" 70 89.00
What I would like to achieve is the following - concatenating the ProductCode and ProductValue into a comma delimited string into a new column and any duplicate rows would be added to this comma delimited string. Then remove the duplicate row (order by the request id)
RequestId userID ProductCode ProductValue NewColumn
1 "10004" 70 85.50 "70,85.50,67,944.00"
3 "10333" 30 97.00 "30,97.00"
4 "23344" 70 89.00 "70,89.00"
Is this possible using Linq - or would I have to loop through the table and interrogate the fields as I go?

var newTable = oldTable.Clone();
newTable.Columns.Add("NewColumn");
newTable = oldTable.AsEnumerable()
.GroupBy(row => row.Field<string>("userID"))
.Select(g => {
var newRow = newTable.NewRow();
var firstRow = g.First();
for(int i = 0; i < 4; i++) newRow[i] = firstRow[i];
newRow["NewColumn"] = string.Join(", ",
g.Select(row=>row.Field<string>("ProductCode")
+ ", " + row.Field<decimal>("ProductValue")));
return newRow;
}).CopyToDataTable();
NOTE: I suppose the ProductCode is string, and the double quotes around userId and NewColumn values are not part of actual values.

DataTable Linq join many columns

I have a trouble with Linq Join. I want to join 2 tables, which they have same structure with n-columns. My problem is i don't know the names of those columns, so how can i rewrite those in select new?
Table 1: Here I have some parameters in ID, Name and LastName. Comment, Attribute and the rest are null
ID Name LastName Comment Attribute ...
"what" "ABC" ...
"hello" "SDE" ...
3 lola
1 de
4 miki
... ... ... ...
Table 2: Here is the same like Table 1 but it has some parameters in Comment, Attribute and the Rest.
ID Name LastName Comment Attribute ...
"what" "ABC" ...
"hello" "SDE" ...
1 de "hi"
4 miki "OKK"
3 lola "yo" "LL"
Result: I would like to have joined Table like this
ID Name LastName Comment Attribute ...
"what" "ABC" ...
"hello" "SDE" ...
3 lola "yo" "LL"
1 de "hi"
4 miki "OKK"
... ... ... ... ... ...
My Code would be:
var Result= from tb1 in table1.AsEnumerable()
join tb2 in tabl2.AsEnumerable()
on new
{
Name = tb1.Field<String>("Name"),
LastName = tb1.Field<String>("LastName"),
} equals new
{
Name=tb2.Field<String>("Name"),
LastName=tb2.Field<String>("LastName"),
}
into grp1
from tb3 in grp1.DefaultIfEmpty()
select new
{
ID = tb1.Field<String>("ID"),
Name = tb1.Field<String>("Name") ,
LastName = tb1.Field<String>("LastName"),
Comment = tb3!= null ? tb3.Field<String>("Comment") : null,
Attribute= tb3!= null ? tb3.Field<String>("Attribute") : null,
...
// Here should be next Columns Name but don't know how to put there
};
I tried with this code but my compiler just hanged out, dont know why
for (int i = 2; i < table1.Rows.Count; i++)
{
foreach (DataRow dr in table2.Rows)
{
if ((table1.Rows[i]["Name"].ToString() == dr["Name"].ToString())&&table1.Rows[i]["LastName"].ToString() == dr["LastName"].ToString())
{
table1.Rows.RemoveAt(i);
table1.ImportRow(dr);
}
}
}
dataGridView1.DataSource = table1;

For each row1 in table1, if there is a matching row2 in table2, use row2. Otherwise, use row1.
var newTable = table1.Clone();
foreach (DataRow row1 in table1.Rows) // To skip the first 2, use table1.Rows.Cast<DataRow>().Skip(2)
{
var row = table2.Rows.Cast<DataRow>().FirstOrDefault(row2 =>
row1["Name"].ToString() == row2["Name"].ToString() &&
row1["LastName"].ToString() == row2["LastName"].ToString()) ?? row1;
newTable.ImportRow(row);
}
dataGridView1.DataSource = newTable;

How about joining as you did and then copying the three known fields from table1's rows to table2's rows?
var copiedTable2 = table2.Copy(); // Copy table2 if you don't want it to be modified
var items = from tb1 in table1.AsEnumerable()
join tb2 in copiedTable2.AsEnumerable()
on new
{
Name = tb1.Field<String>("Name"),
LastName = tb1.Field<String>("LastName"),
} equals new
{
Name=tb2.Field<String>("Name"),
LastName=tb2.Field<String>("LastName"),
}
into grp1
from tb3 in grp1.DefaultIfEmpty()
select new
{
ID = tb1.Field<String>("ID"),
Name = tb1.Field<String>("Name") ,
LastName = tb1.Field<String>("LastName"),
Row = tb3 ?? table2.NewRow();
};
foreach(var item in items)
{
item.Row.SetField<String>("ID", item.ID);
item.Row.SetField<String>("Name", item.Name);
item.Row.SetField<String>("LastName", item.LastName);
}
var rows = items.Select(x => x.Row);
// How to set the rows as a DataGridView's DataSource
var result = table2.Clone();
foreach(var row in rows)
{
result.Rows.Add(row.ItemArray);
}
dataGridView.DataSource = result;

Try this, no need of using loops
var resultTable = from tb1 in table1.AsEnumerable()
join tb2 in tabl2.AsEnumerable()
on tb1["Name"].ToString() equals tb2["Name"].ToString()
where tb1["LastName"].ToString() == tb2["LastName"].ToString()
select r;
DataTable resultTable =result.CopyToDataTable();