We Keep Coding

Return ID of newly inserted row on a PostgreSQL database using C# and Npgsql?

I'm building a WinForms project in C# using a PostgreSQL database and the Npgsql framework.
For inserting a record, I need to return the ID of the new record. This SO question says to add SELECT SCOPE_IDENTITY() to the query string passed to cmd. So my query string looks like this:
string insertString = "INSERT INTO sometable (company_name, category, old_value, old_desc, new_value, new_desc, reference1, reference2) VALUES (#comp, #cat, #oldValue, #oldDesc, #newValue, #newDesc, #ref1, #ref2); SELECT SCOPE_IDENTITY();";
and then get the ID with something like
int modified = cmd.ExecuteNonQuery();
But that's likely SQL Server-specific. If I use that method, I get an exception at the above line saying, "fuction scope_identity() does not exist".
I wasn't able to find anything that seemed to address this on the Npgsql documentation.
Per the linked SO question and Denis' suggestions I've tried adding both
RETURNING id;
and
CURRVAL(pg_get_serial_sequence('my_tbl_name','id_col_name'))
to the query string, replacing SELECT SCOPE_IDENTITY(); with those statements in the code above. In both cases they work as intended in DBeaver on an insert, but in my C# code in my WinForm project, modified was set to "1".
NOTE: I re-titled the question and added more information about what I've done.

Add "returning idcolumn" to the end of the sql query, then run the command with the ExecuteScalar() method instead of ExecuteNonQuery(). It should return with an int.
string insert = "insert into table1 (col1) values (something) returning idcol";
int id = cmd.ExecuteScalar();

All the comments above were almost nearly spot on and got me to a solution but didn't exactly wrap it in a bow -- so I thought i'd post my implementation that works (with silly fake example tables of course).
private int? InsertNameIntoNamesTable(string name)
{
int? id = null;
using (var dbcon = new NpgsqlConnection(_connectionString))
{
dbcon.Open();
StringBuilder sb = new StringBuilder();
var sql = $#"
insert into names_table
(name)
values
({name})
returning id;
";
sb.Append(sql);
using (var cmd = new NpgsqlCommand(sql, dbcon))
{
id = (int)cmd.ExecuteScalar();
}
dbcon.Close();
}
return id;
}

Is it possible to return string values from a query when using Entity-Framework?

So far I know two ways to query the database. One of it is by using MySQL's Libraries; By declaring "MySqlCommand", "MySqlDataReader", "MySqlConnection" and utilizing the inbuilt methods.
And another is through using the Entity Framework (Which is my preferred option).
I've however run into a bit of a problem with this later method. I personally blame my lack of knowledge. It would seem that when conveying data to the end user, Entity-Framework favours the ObservableCollection<> or List<>.
For me, this means that in the View, I'd be using a Datagrid or List control. Normally this would be fine.
But then, what happens if I don't want a datagrid control in the mix? What if I want the query's result conveyed in a textboxes? How can I do the below without sacrificing the usage of the entity framework? Is it even possible?
string config = "server=localhost; userid = root; database = databaseName";
MySqlConnection con = new MySqlConnection(config);
MySqlDataReader reader = null;
// Run the select query
string query = "SELECT * FROM students WHERE id = " +id;
MySqlCommand command = new MySqlCommand(query, con);
con.Open();
reader = command.ExecuteReader();
while (reader.Read())
{
// Put the results in the appropriate strings so I can databind them
string studentName = (string)reader["studentName"];
string studentNum = (string)reader["studentNum"];
.....
}
con.Close();
How could I do this utilising the entity framework?

EF equivalents for this SQL query:
// note, that this leads to SQL injections,
// use parametrized queries instead!
"SELECT * FROM students WHERE id = " +id
will be:
var student = context.Students.Find(id) // this will lookup local data as well
or:
var student = context.Students.Single(_ => _.Id == id)
Both will return single Student instance, not a sequence/collection, and you can bind your UI to its properties or access them as usual:
student.StudentName

Try to something like that:
public static Studens GetStudent(int id)
{
return context.Studens.SingleOrDefault(x=>x.Id == id);
}

handling large amounts of data to include in an oracle select statement

Recent bug report states that a method being called is crashing the service causing it to restart. After troubleshooting, the cause was found to be an obnoxious Oracle SQL call with thousands of strings passed. There is a collection of strings being passed to a method from an external service which often is more than 10,000 records. The original code used a where clause on the passed collection using the LIKE keyword, which I think is really, really bad.
public IList<ContainerState> GetContainerStates(IList<string> containerNumbers)
{
string sql =
String.Format(#"Select CTNR_NO, CNTR_STATE FROM CONTAINERS WHERE CTRN_SEQ = 0 AND ({0})",
string.Join("OR", containerNumbers
.Select(item => string.Concat(" cntr_no LIKE '", item.SliceLeft(10), "%' ")))
);
return DataBase.SelectQuery(sql, MapRecordToContainerState, new { }).ToList();
}
Clarification of in house methods used which may be confusing:
DataBase.SelectQuery is an internal library method using generics which gets passed the sql string, a function to map the records to .NET objects, and the parameters being passed and returns an IEnumerable of Objects of type retuned by the Mapping function.
SliceLeft is an extension method from another internal helper library that just returns the first part of a string up to the number of characters specified by the parameter.
The reason that the LIKE statement was apparently used, is that the strings being passed and the strings in the database only are guaranteed to match the first 10 characters. Example ("XXXX000000-1" in the strings being passed should match a database record like "XXXX000000-8").
I believed that the IN clause using the SUBSTR would be more efficent than using multiple LIKE clauses and replaced the code with:
public IList<ContainerRecord> GetContainerStates(IList<string> containerNumbers)
{
string sql =
String.Format(#"Select CTNR_NO, CNTR_STATE FROM CONTAINERS WHERE CTRN_SEQ = 0 AND ({0})",
string.Format("SUBSTR(CNTR_NO, 1, 10) IN ({0}) ",
string.Join(",", containerNumbers.Select(item => string.Format("\'{0}\'", item.SliceLeft(10) ) ) )
)
);
return DataBase.SelectQuery(sql, MapRecordToContainerState, new { }).ToList();
}
This helped slightly, and there were fewer issues in my tests, but when there are huge amounts of records passed, there is still an exception thrown and core dumps occur, as the SQL is longer than the server can parse during these times. The DBA suggests saving all the strings being passed to a temporary table, and then joining against that temp table.
Given that advice, I changed the function to:
public IList<ContainerRecord> GetContainerStates(IList<string> containerNumbers)
{
string sql =
#"
CREATE TABLE T1(cntr_num VARCHAR2(10));
DECLARE GLOBAL TEMPORARY TABLE SESSION.T1 NOT LOGGED;
INSERT INTO SESSION.T1 VALUES (:containerNumbers);
SELECT
DISTINCT cntr_no,
'_IT' cntr_state
FROM
tb_master
WHERE
cntr_seq = 0
AND cntr_state IN ({0})
AND adjustment <> :adjustment
AND SUBSTR(CTNR_NO, 1, 10) IN (SELECT CNTR_NUM FROM SESSION.T1);
";
var parameters = new
{
#containerNumbers = containerNumbers.Select( item => item.SliceLeft(10)).ToList()
};
return DataBase.SelectQuery(sql, MapRecordToContainerState, parameters).ToList();
}
Now I'm getting a "ORA-00900: invalid SQL statement". This is really frustrating, how can I properly write a SQL Statement that will put this list of strings into a temporary table and then use it in a SELECT Statement to return the list I need?

There are couple possible places could cause this error, it seams that the "DECLARE GLOBAL TEMPORARY" is a JAVA API, I don't think .net has this function. Please try "Create global temporary table" instead. And, I don't know whether your internal API could handle multiple SQLs in one select sql. As far as I know, ODP.net Command class can only execute one sql per call. Moreover, "create table" is a DDL, it therefore has its own transaction. I can't see any reason we should put them in the same sql to execute. Following is a sample code for ODP.net,
using (OracleConnection conn = new OracleConnection(BD_CONN_STRING))
{
conn.Open();
using (OracleCommand cmd = new OracleCommand("create global temporary table t1(id number(9))", conn))
{
// actually this should execute once only
cmd.ExecuteNonQuery();
}
using (OracleCommand cmd = new OracleCommand("insert into t1 values (1)", conn)) {
cmd.ExecuteNonQuery();
}
// customer table is a permenant table
using (OracleCommand cmd = new OracleCommand("select c.id from customer c, t1 tmp1 where c.id=tmp1.id", conn)) {
cmd.ExecuteNonQuery();
}
}

Convert C# SQL Loop to Linq

I have a list Called ListTypes that holds 10 types of products. Below the store procedure loops and gets every record with the product that is looping and it stores it in the list ListIds. This is killing my sql box since I have over 200 users executing this constantly all day.
I know is not a good architecture to loop a sql statement, but this the only way I made it work. Any ideas how I can make this without looping? Maybe a Linq statement, I never used Linq with this magnitude. Thank you.
protected void GetIds(string Type, string Sub)
{
LinkedIds.Clear();
using (SqlConnection cs = new SqlConnection(connstr))
{
for (int x = 0; x < ListTypes.Count; x++)
{
cs.Open();
SqlCommand select = new SqlCommand("spUI_LinkedIds", cs);
select.CommandType = System.Data.CommandType.StoredProcedure;
select.Parameters.AddWithValue("#Type", Type);
select.Parameters.AddWithValue("#Sub", Sub);
select.Parameters.AddWithValue("#TransId", ListTypes[x]);
SqlDataReader dr = select.ExecuteReader();
while (dr.Read())
{
ListIds.Add(Convert.ToInt32(dr["LinkedId"]));
}
cs.Close();
}
}
}

Not a full answer, but this wouldn't fit in a comment. You can at least update your existing code to be more efficient like this:
protected List<int> GetIds(string Type, string Sub, IEnumerable<int> types)
{
var result = new List<int>();
using (SqlConnection cs = new SqlConnection(connstr))
using (SqlCommand select = new SqlCommand("spUI_LinkedIds", cs))
{
select.CommandType = System.Data.CommandType.StoredProcedure;
//Don't use AddWithValue! Be explicit about your DB types
// I had to guess here. Replace with the actual types from your database
select.Parameters.Add("#Type", SqlDBType.VarChar, 10).Value = Type;
select.Parameters.Add("#Sub", SqlDbType.VarChar, 10).Value = Sub;
var TransID = select.Parameters.Add("#TransId", SqlDbType.Int);
cs.Open();
foreach(int type in types)
{
TransID.Value = type;
SqlDataReader dr = select.ExecuteReader();
while (dr.Read())
{
result.Add((int)dr["LinkedId"]);
}
}
}
return result;
}
Note that this way you only open and close the connection once. Normally in ADO.Net it's better to use a new connection and re-open it for each query. The exception is in a tight loop like this. Also, the only thing that changes inside the loop this way is the one parameter value. Finally, it's better to design methods that don't rely on other class state. This method no longer needs to know about the ListTypes and ListIds class variables, which makes it possible to (among other things) do better unit testing on the method.
Again, this isn't a full answer; it's just an incremental improvement. What you really need to do is write another stored procedure that accepts a table valued parameter, and build on the query from your existing stored procedure to JOIN with the table valued parameter, so that all of this will fit into a single SQL statement. But until you share your stored procedure code, this is about as much help as I can give you.

Besides the improvements others wrote.
You could insert your ID's into a temp table and then make one
SELECT * from WhatEverTable WHERE transid in (select transid from #tempTable)
On a MSSQL this works really fast.
When you're not using a MSSQL it could be possible that one great SQL-Select with joins is faster than a SELECT IN. You have to test these cases by your own on your DBMS.

According to your comment:
The idea is lets say I have a table and I have to get all records from the table that has this 10 types of products. How can I get all of this products? But this number is dynamic.
So... why use a stored procedure at all? Why not query the table?
//If [Type] and [Sub] arguments are external inputs - as in, they come from a user request or something - they should be sanitized. (remove or escape '\' and apostrophe signs)
//create connection
string queryTmpl = "SELECT LinkedId FROM [yourTable] WHERE [TYPE] = '{0}' AND [SUB] = '{1}' AND [TRANSID] IN ({2})";
string query = string.Format(queryTmpl, Type, Sub, string.Join(", ", ListTypes);
SqlCommand select = new SqlCommand(query, cs);
//and so forth
To use Linq-to-SQL you would need to map the table to a class. This would make the query simpler to perform.

how to compare elements in a string with the database table values

In my project i have to give a string input through a text field, and i have to fill a database table with these values. I should first check the values of a specific table column, and add the input string only if it is not there in the table already.
I tried to convert the table values to a string array, but it wasn,t possible.
If anyone have an idea about this, your reply will be really valuable.
Thankx in advance.

Since you say your strings in the database table must be unique, just put a unique index on that field and let the database handle the problem.
CREATE UNIQUE INDEX UIX_YourTableName_YourFieldName
ON dbo.YourTableName(YourFieldName)
Whenever you will try to insert another row with the same string, SQL Server (or any other decent RDBMS) will throw an exception and not insert the value. Problem solved.
If you need to handle the error on the front-end GUI already, you'll need to load the existing entries from your database, using whatever technology you're familiar with, e.g. in ADO.NET (C#, SQL Server) you could do something like:
public List<string> FindExistingValues()
{
List<string> results = new List<string>();
string getStringsCmd = "SELECT (YourFieldName) FROM dbo.YourTableName";
using(SqlConnection _con = new SqlConnection("your connection string here"))
using(SqlCommand _cmd = new SqlCommand(getStringsCmd, _con)
{
_con.Open();
using(SqlDataReader rdr = _con.ExecuteReader())
{
while(rdr.Read())
{
results.Add(rdr.GetString(0));
}
rdr.Close();
}
_con.Close();
}
return results;
}
You would get back a List<string> from that method and then you could check in your UI whether a given string already exists in the list:
List<string> existing = FindExistingValues();
if(!existing.Contains(yournewstring))
{
// store the new value to the database
}
Or third option: you could write a stored procedure that will handle the storing of your new string. Inside it, first check to see whether the string already exists in the database
IF NOT EXISTS(SELECT * FROM dbo.YourTableName WHERE YourFieldName = '(your new string)')
INSERT INTO dbo.YourTableName(YourFieldName) VALUES(your-new-string-here)
and if not, insert it - you'll just need to find a strategy how to deal with the cases where the new string being passed in did indeed already exist (ignore it, or report back an error of some sorts).
Lots of options - up to you which one works best in your scenario!