Problem
This is partially me being my own worst enemy. I have unit tests that verify my ability to write and then retrieve all different base data types to/from a SQLite database. Among my tests I verify several different values for each data type including (but not limited to) <datatype>.MinValue and <datatype>.MaxValue.
When I write a decimal.MaxValue to the database, then try to retrieve it, I get an Overflow Exception (thanks to rounding within the database itself).
Note: I have stripped my actual classes to the bare-bones and placed them inside a test method so I could show everything more easily.
private static SQLiteConnection connection;
[TestMethod()]
public void WriteDecimal()
{
using (var cmd = new SQLiteCommand(connection))
{
cmd.CommandText = $"INSERT INTO foo(name, value) VALUES('bar', {decimal.MaxValue})";
cmd.ExecuteNonQuery();
cmd.CommandText = "SELECT * FROM foo;";
using (SQLiteDataReader rdr = cmd.ExecuteReader())
{
while (rdr.Read())
{
Console.WriteLine($"{rdr.GetInt32(0)} {rdr.GetString(1)} {rdr.GetValue(2)}");
}
}
}
}
#region Setup/Cleanup
[ClassInitialize()]
public static void Setup(TestContext context)
{
FileInfo dbFile = new FileInfo(Path.Combine(Environment.GetEnvironmentVariable("temp"), #"\Sqlite\myDb.db"));
dbFile.Directory.Create();
dbFile.Delete();
string connectionString = $"Data Source={dbFile?.FullName ?? ":memory:"}";
connection = new SQLiteConnection(connectionString);
connection.Open();
using (var cmd = new SQLiteCommand(connection))
{
cmd.CommandText = #"CREATE TABLE foo(id INTEGER PRIMARY KEY, name TEXT, value Number)";
cmd.ExecuteNonQuery();
};
}
[ClassCleanup()]
public static void Cleanup()
{
connection.Close();
}
#endregion
Output:
Message:
Test method WriteDecimal threw exception:
System.OverflowException: Value was either too large or too small for a Decimal.
Stack Trace:
Number.ThrowOverflowException(TypeCode type)
DecCalc.VarDecFromR8(Double input, DecCalc& result)
IConvertible.ToDecimal(IFormatProvider provider)
Convert.ChangeType(Object value, Type conversionType, IFormatProvider provider)
SQLite3.GetValue(SQLiteStatement stmt, SQLiteConnectionFlags flags, Int32 index, SQLiteType typ)
SQLiteDataReader.GetValue(Int32 i)
DatabaseDirect.WriteDecimal() line 54
Workaround
I found a workaround (I just don't like it). Essentially, I let it fail, then go back and try to grab it as a Double; then convert it to what I need; because it overflowed I know it has to either be the max value or the min value:
using (SQLiteDataReader rdr = cmd.ExecuteReader())
{
while (rdr.Read())
{
decimal newVal;
try
{
newVal = (decimal)rdr.GetValue(2);
}
catch (OverflowException)
{
double val = rdr.GetDouble(2);
Type t = rdr.GetFieldType(2);
newVal = val > 0 ? decimal.MaxValue : decimal.MinValue;
}
Console.WriteLine($"{rdr.GetInt32(0)} {rdr.GetString(1)} {newVal}");
}
}
Bigger Issue (as I see it)
This isn't the only place I encounter this issue. It also happens with decimal.MinValue and ulong.MaxValue. I'm not exactly a fan of my solution simply because I just assume that if there's an overflow I need the max/min value. I'd also like to generalize it so it doesn't hard-code the min/max values I may need. Again, I found a solution; but again, it is ugly (a function that passes in the type to convert the value to and then do a switch on it...yucky).
You might not be able to do this but when I got a decimal overflow error, I tried changing my sqlite column type from decimal to real and that eliminated the error.
Related
I have written the following in an attempt to execute a sql query and store the result in an Array:
public static ArrayList DbQueryToArry()
{
string SqlCString = myConnString;
SqlConnection connection = null;
ArrayList valuesList = new ArrayList();
connection = new SqlConnection(SqlCString);
connection.Open();
SqlCommand command = new SqlCommand("Select CLIENTNO, ACCOUNT_Purpose from audit.ACCOUNTS_AUDIT", connection);
SqlDataReader reader = command.ExecuteReader();
while (reader.Read())
{
valuesList.Add(Convert.ToInt32(reader[0].ToString()));
}
return valuesList;
}
But running the following
var myArray = DbQueryToArry();
Console.WriteLine(myArray.ToString());
Does not return the query result..
You will need to join them manually with string.Join or something similar:
Concatenates the elements of a specified array or the members of a
collection, using the specified separator between each element or
member.
Console.WriteLine(string.Join(",",myArray.ToArray()));
The reason why your version doesnt work is because, Console.Writeline has a bunch of overloads for different types, however it falls back to WriteLine(Object) when it can't find a specific resolution match.
The source code to WriteLine(Object value) is as follows (which can be found here).
public virtual void WriteLine(Object value) {
if (value==null) {
WriteLine();
}
else {
// Call WriteLine(value.ToString), not Write(Object), WriteLine().
// This makes calls to WriteLine(Object) atomic.
IFormattable f = value as IFormattable;
if (f != null)
WriteLine(f.ToString(null, FormatProvider));
else
WriteLine(value.ToString());
}
}
Notice how it calls value.ToString() ?
Object.ToString Method
Returns a string that represents the current object.
Remarks
Object.ToString is the major formatting method in the .NET Framework.
It converts an object to its string representation so that it is
suitable for display.
An ArrayList has no overload to ToString() that would be able to anticipate what you want to show, so it relies on the default.
The default Object.ToString() method
The default implementation of the ToString method returns the fully
qualified name of the type of the Object
Which brings me to my next point, don't use ArrayList, Use a generic array int[] or List<int> you will find it much more fun and rewarding (fun level 100)
I have a method which returns large number of rows from a database. (Please see following)
public static ACollection GetListFromDatabase(string customername)
{
DBFactory factory = DBFactory.Instance;
ACollection custcol = new ACollection();
//This is a collection class extended from System.Collections.CollectionBase
System.Data.IDataReader reader = null;
try
{
reader = factory.GPDb.ExecuteReader("spGetCustomerInfo", customernumber);
while (reader.Read())
{
ACollection cust = ACollection.ListFromReader(reader); // returns info and assign it to ACollection object.
custcol.InnerList.Add(cust);
}
}
catch (Exception e)
{
String error = e.Message;
}
finally
{
if (reader != null)
reader.Close();
}
return custcol;
}
When I run this method, I realized that count of custcol.InnerList is 32767 where it supposed to be around 34000. Then I saw that it gets into exception.
The error message says that "Value was either too large or too small for an Int16."
I believe, the capacity of this arraylist gets assigned as int16 in somehow. Can somebody help me to increase the capacity ?
Thanks
Edit:
here is the full stack trace
at System.Convert.ToInt16(Int64 value)
at System.Int64.System.IConvertible.ToInt16(IFormatProvider provider)
at System.Convert.ToInt16(Object value)
at Quest___Shared.CustomerCrossReference.ListFromReader(IDataReader reader) in C:\vsproject\CustomerCrossReference.cs:line 105
at Quest___Shared.ACollection.GetListFromDatabase(String customernumber) in C:\vsproject\ACollection.cs:line 88
Change the type of the variable that is giving you problems to Int32 or Int64, then you will be able to insert bigger or smaller numbers.
In our datalayer, I want to discourage the use of string concatenation and make people use parameters instead.
But as far as I know, there is no way to see if a parameter is a concatenated string.
I have no code example to show of detection, since I know of none.
var result = db.executeQuery("SELECT * FROM table WHERE id = " + id);
This is the kind of code I'd like to get rid of, either to replace with something like:
db.executeQuery($"SELECT * FROM table WHERE id = {id}");
or
db.executeQuery("SELECT * FROM table WHERE id = {0}", id);
Edit:
The command executeQuery is in our datalayer and handles parameters as SqlParameters, with types and values.
So in this case a SqlParameter called #id with type int would be created.
Regarding the FormattableString:
public T ExecuteObject<T>(FormattableString sql)
{
return executeSingleRow(sql.Format, sql.GetArguments()).ToType<T>();
}
Regarding the ExecuteQuery:
public int executeNonQuery(string sql, params object[] parameters)
{
var traceI = Traceadd(sql, parameters);
if (!open())
throw new Exception("Error executing query!", lastException);
try
{
command = Connection.CreateCommand();
command.CommandText = sql;
sql.SQLFormat(ref command, parameters);
var res = command.ExecuteNonQuery();
command.Parameters.Clear();
if (traceI != null)
traceI.Stop();
return res;
}
catch (Exception ex)
{
if (traceI != null)
traceI.Stop();
throw new DBException(command.CommandText, command.Parameters, ex);
}
}
If your executeQuery method only has a parameter of FormattableString, then you should be fine already - there's no conversion from string to FormattableString. For example:
using System;
class Program
{
static void Main(string[] args)
{
int id = 10;
ExecuteQuery("SELECT * FROM table WHERE id = " + id);
}
static void ExecuteQuery(FormattableString query)
{
}
}
That gives an error:
Test.cs(8,22): error CS1503: Argument 1: cannot convert from 'string' to 'System.FormattableString'
You just need to make sure that you don't have an overload of your method accepting string. The result of string concatenation is never a FormattableString. Indeed, I would strongly advise that you avoid ever overloading a method to accept FormattableString and string... there's no point in doing so if you're not going to change the behaviour, and if you are going to change the behaviour, that could be really confusing.
I'd personally consider changing to use an extension method on FormattableString though - something like:
public static SqlCommand ToCommand(
this FormattableString query,
SqlConnection connection)
{
// ...
}
That way you can separate the command creation from the execution... which means (aside from anything else) that you can then call ExecuteReader or ExecuteNonQuery without having any extra code yourself.
Could I get some help explaining this answer below and how it works with the delegate. Its the answer from here: C# abstraction and database layer
...if you are stuck on the idea of using a DataReader, you could pass a delegate to the helper, which gets invoked inside of the using statements:
public string GetMySpecId(string dataId)
{
return _dbHelper.ExecuteQuery(
dr =>
{
if(dr.Read())
{
return dr[0].ToString();
}
// do whatever makes sense here.
},
#"select ""specId"" from ""MyTable"" where ""dataId"" = :dataId",
new SqlParameter("dataId", dataId));
return result.Rows[0][0].ToString();
}
You could also use a lightweight tool like Dapper to simplify some of the syntax and take care of mapping to your data types. (You'd still need to deal with opening a connection and such.)
Declaring the ExecuteQuery Method from above should look something like this:
public DataTable ExecuteQuery(Func<DataReader, DataTable> delegateMethod, string sqlQuery, SqlParameter param)
{
using (SqlConnection conn = new SqlConnection(this.MyConnectionString))
{
conn.Open();
// Declare the parameter in the query string
using (SqlCommand command = new SqlCommand(sqlQuery, conn))
{
// Now add the parameter to the parameter collection of the command specifying its type.
command.Parameters.Add(param);
command.Prepare();
// Now, add a value to it and later execute the command as usual.
command.Parameters[0].Value = dataId;
using (SqlDataReader dr = command.ExecuteReader())
{
return delegateMethod(dr);
}
}
}
}
That should be right, you may have to swap the DataReader and the DataTable in the Func, I can't remember which comes first the param types or the return type.
Here's another example of using the Func delegate, there's also the Action Delegate if you don't need a return type.
Func Delegate Reading
Normal Delegate Reading
I have a stored procedure. One of its input parameters is expecting a char(8). I try to convert a string "AAA" to this particular parameter type, which is a DBType.AnsiStringFixedLength.
object v = Convert.ChangeType("AAA", param.DbType.GetTypeCode());
// param is AnsiStringFixedLength
However, all I get is an exception: Input string was not in a correct format.
And the stack trace says: at System.Number.StringToNumber(String str, NumberStyles options, NumberBuffer& number, NumberFormatInfo info, Boolean parseDecimal) [...]
Why is System.Convert trying to convert a string into a number, even though the prodecure's parameter is expecting a char(8)? How do I solve this? I don't want to use one huge switch case mapping all SQL types to CLR types...
EDIT:
This is the code in question: (A generic method to call any MS SQL stored procedure)
using (SqlConnection conn = new SqlConnection(this.config.ConnectionString))
{
using (SqlCommand cmd = conn.CreateCommand())
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.CommandText = this.config.StoredProcedureName;
conn.Open();
SqlCommandBuilder.DeriveParameters(cmd);
foreach (SqlParameter param in cmd.Parameters)
{
if (param.Direction == ParameterDirection.Input ||
param.Direction == ParameterDirection.InputOutput)
{
try
{
string rawParam = param.ParameterName.Replace("#", "");
if (this.config.Parameters.ContainsKey(rawParam))
{
try
{
param.Value = Convert.ChangeType(this.config.Parameters[rawParam],
param.DbType.GetTypeCode());
}
catch(Exception oops)
{
throw new Exception(string.Format("Could not convert to '{0}'.", param.DbType), oops);
}
}
else
throw new ArgumentException("parameter's not available");
}
catch (Exception e)
{
throw;
}
}
}
cmd.ExecuteNonQuery();
}
}
The actual parameter values are provided by this.config.Parameters - all of them are strings. I iterate through SqlCommand's parameter list and set them accordingly. Converting the string values to the parameter's Sql type is necessary here, and as far as I can see, the Sql type is provided by param.DBType.
You seem to mix up some things here, or I don't get what you try to do. The DbType (an enumeration) inherits Enum and that implements IConvertible -> You can call GetTypeCode(). But - you are now calling Enum.GetTypeCode(), which returns the underlying type. If you didn't specify it (and DbType didn't) any Enum is backed by an int.
What are you trying to solve with the code anyway? Why would you want to change the type of a string if the parameter is a string (although with a fixed length)?
Looking at the question some more it seems even more odd. You have an object v (probably for value?) - what do you care about the type?
object v1 = "Foo";
object v1 = 42;
What is the difference for you? I guess you want to pass the values to something else, but - if you only reference the value as object you might still need to cast it.
Please update your question and explain what you really want to do, what you expect to gain.
Regarding the comment:
I'm using Convert.ChangeType(object
value, TypeCode typeCode), so it's not
really converting into an Enum/int. At
least that's what I thought...
See above: DbType.GetTypeCode() is not what you want. Try it, give me the benefit of the doubt: What do you expect to get from DbType.AnsiStringFixedLength.GetTypeCode()? What is the actual result, if you try it?
Now to your code: You try to set the SqlParameter.Value property to the "correct" type. Two things: According to the documentation you probably want to set the SqlParameter.SqlValue, which is the value using SQL types according to the docs. SqlParameter.Value, on the other hand, is the value using CLR types and allows to infer both DbType and SqlValue. Sidenote, implementation detail: The SqlParameter.SqlValue setter just calls the setter of SqlParameter.Value again...
I would expect that the ADO.NET stuff converts the value on its own, if at all possible. What error are you getting without jumping through this hoops?