I have .NET Core application which communicate with a Postgres database using entity framework.
There was a enum used in model which create a Postgres enum type in postgres.
Now I need such migration that will change the enum into flags, so I need to change enum integer value in C#.
I feel I should also change the underlaying enum values in Postgres, but I don't see any numbers there, I just see only enum names in type definition.
How the Postgres stores enum internally?
How can I migrated it if integer value of enum is changed in C#?
I am not aware how Postgres stores enum internally and prefer to not rely on it.
Assuming that your enum type is called _enumtype and that column is called _enumcolumn then you can alter the table structure (enum -> integer) and keep the data in it like this:
alter table _mytable
alter column _enumcolumn type integer
using array_position(enum_range(null::_enumtype), _enumcolumn) - 1;
The enum type words will be replaced with 0, 1, 2, .... respectively.
So you are free to make whatever changes and updates you need after that.
This probably does not answer your question directly, but useful for understanding under the hood. Postgres stores enums with both numeric and string components. You use the string component for all references both setting and testing, references the numeric component generates an exception. Internally Postgres uses the numeric component for sorting (perhaps other uses I am not aware of). This numeric is (fyi) stored as a floating point value. Adding value(s) to an emum is a simple Alter type. Removing them cannot be done directly - it evolves deleting and recreating along with managing all dependencies. You can see the component values, both numeric and string with the query (see Demo)
select * from pg_catalog.pg_enum;
That gives you all values for all enums, unfortunately getting a specific one is more complicated.
Related
I have a field in a sqlite database, we'll call it field1, on which I'm trying to iterate over each record (there's over a thousand records). The field type is string. The value of field1 in the first four rows are as follows:
DEPARTMENT
09:40:24
PARAM
350297
Here is some simple code I use to iterate over each row and display the value:
while (sqlite_datareader.Read())
{
strVal = sqlite_datareader.GetString(0);
Console.WriteLine(strVal);
}
The first 3 values display correctly. However, when it gets to the numerical entry 350297 it errors out with the following exception on the .getString() method
An unhandled exception of type 'System.InvalidCastException' occurred in System.Data.SQLite.dll
I've tried casting to a string, and a bunch of other stuff. But I can't get to the bottom of why this is happening. For now, I'm forced to use getValue, which is of type object, then convert back to a string. But I'd like to figure out why getString() isn't working here.
Any ideas?
EDIT: Here's how I currently deal with the problem:
object objVal; // This is declared before the loop starts...
objVal = sqlite_datareader.IsDBNull(i) ? "" : sqlite_datareader.GetValue(i);
if (objVal != "")
{
strVal = (string)objVal;
}
What the question should have included is
The table schema, preferrably the CREATE TABLE statement used to define the table.
The SQL statement used in opening the sqlite_datareader.
Any time you're dealing with data type issues from a database, it is prudent to include such information. Otherwise there is much unnecessary guessing and floundering (as apparent in the comments), when so very useful, crucial information is explicitly defined in the schema DDL. The underlying query for getting the data is perhaps less critical, but it could very well be part of the issue if there are CAST statements and/or other expressions that might be affecting the returned types. If I were debugging the issue on my own system, these are the first thing I would have checked!
The comments contain good discussion, but a best solution will come with understanding how sqlite handles data types straight from the official docs. The key takeaway is that sqlite defines type affinities on a column and then stores actual values according to a limited set of storage classes. A type affinity is a type to which data will attempt to be converted before storing. But (from the docs) ...
The important idea here is that the type is recommended, not required. Any column can still store any type of data.
But now consider...
A column with TEXT affinity stores all data using storage classes NULL, TEXT or BLOB. If numerical data is inserted into a column with TEXT affinity it is converted into text form before being stored.
So even though values of any storage class can be stored in any column, the default behavior should have been to convert any numeric values, like 350297, as a string before storing the value... if the column was properly declared as a TEXT type.
But if you read carefully enough, you'll eventually come to the following at the end of section 3.1.1. Affinity Name Examples:
And the declared type of "STRING" has an affinity of NUMERIC, not TEXT.
So if the question details are taken literally and field1 was defined like field1 STRING, then technically it has NUMERIC affinity and so a value like 350297 would have been stored as an integer, not a string. And the behavior described in the question is precisely what one would expect when retrieving data into strictly-typed data model like System.Data.SQLite.
It is very easy to cuss at such an unintuitive design decisions and I won't defend the behavior, but
at least the results of "STRING" type are clearly stated so that the column can be redefined to TEXT in order to fix the problem, and
"STRING" is actually not a standard SQL data type. SQL strings are instead defined with TEXT, NTEXT, CHAR, NCHAR, VARCHAR, NVARCHAR, etc.
The solution is either to use code as currently implemented: Get all values as objects and then convert to string values... which should be universally possible with .Net objects since they should all have ToString() method defined.
Or, redefine the column to have TEXT affinity like
CREATE TABLE myTable (
...
field1 TEXT,
...
)
Exactly how to redefine an existing column filled with data is another question altogether. However, at least when doing the conversion from the original to the new column, remember to use a CAST(field1 AS TEXT) to ensure the storage class is changed for the existing data. (I'm not certain whether type affinity is "enforced" when simply copying/inserting data from an existing table into another or if the original storage class is preserved by default. That's why I suggest the cast to force it to a text value.)
Before I dedicate the time to do this, I'm checking if anyone knows of a feature that will allow me to do something a bit more dynamic with Elastic using C#/NEST
We have a situation where we will be receiving data in a field called "Value". That value is a string but it could represent a number, boolean, string, and so on...
I would like to store that on Elastic like the following...
item.value (the value that our system received)
item.value.boolean (the boolean if it can be converted)
item.value.integer (the integer if it can be converted)
I'd like to put the values in as the value to allow the values to be indexed to allow reporting to take place.
I know that I could create multiple .net properties on the object with each datatype represented but I'd prefer an attribute type solution where I could just mark one property and it would create the multiple fields on Elastic.
Does anyone know of a pre-built solution? I know with some effort I could create my own code to write this out but I'd prefer to not spend a day or two writing that out.
I have a SQL table on a database which has a column in it with the Bit datatype. I'm trying to define a method in my C# application which takes two of the columns from the table and uses them as parameters.
Here is the code
public void Edit_NAA_ApplicationsFirm(int ApplicationId, string UniversityOffer, bit Firm)
{
//This line declares a variable in which we store the actual values from NAA_Applications based on the associating ID
NAA_Applications AppFirm = Get_Applicant_Application(ApplicationId);
//Here we tell the application that the values edited are equal to the values within the table
AppFirm.Firm = Firm;
//Any of these changes are then saved.
_context.SaveChanges();
}
The only issue is the the program keeps trying to convert bit to BitConverter. When I change it to bit it has issues accepting it as a datatype.
It should be worth noting I'm building the application in an ASP.Net Framework solution.
Could anyone tell me what it is I'm doing wrong? Am I just referring to the datatype wrong?
Looks like you're using Entity Framework.
It'll use a .NET boolean to represent a T-SQL bit. That would be a sensible way to do it for any other data access method as well. boolean true will convert to 1 in the bit field, and false to 0.
In fact it's even documented, more than once, that this is the correct .NET CLR type to use. See https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/sql/linq/sql-clr-type-mapping , https://msdn.microsoft.com/en-us/library/cc716729(v=vs.100).aspx and probably others.
So in your case bool Firm would be appropriate.
The correct datatype for bit in c# is boolean. so
public void Edit_NAA_ApplicationsFirm(int ApplicationId, string UniversityOffer, bool Firm)
if I understand corectly, try use bool Firm
in c#, value 1 from database is converted to "true", and "true" from code is converted to 1 in database
Hypothetically, I have two SQL tables: Table and AuditTable. In Table is a column, org_id, of type float with nulls allowed. Also, org_id is not a primary key. A column with the same name resides in AuditTable. I also have an EditTable class used to make changes to Table and AuditTable. The members of EditTable are set via a user interface. EditTable also contains a org_id member.
There is no good reason why Table.org_id was made a float; it will always contain an integer value. However, since Table was already existing, I can't change the type of Table.org_id. However, since I created AuditTable and EditTable I can set AuditTable.org_id and EditTable.org_id to any type.
When Visual Studio converts Table into a C# class, Table.org_id is made a Nullable<double>. Should I make AuditTable.org_id a float with nulls allowed and make EditTable.org_id a nullable double to match Table.org_id? Or should I make both AuditTable.org_id and EditTable.org_id ints and then do some casting? However, I was thinking about staying away from casting to be on the safe side and just make the types match the orginal Table.
Thanks for any suggestions.
Oh, it is a bad idea to store join keys as floating point numbers. I wish that SQL actually banned this practice. The issue is that 0.9999999999 might look like 1.00000000, but they don't match when joining (or in a where clause). Much better to have what-you-see-is-what-you-get for such conditions.
First, go to whoever you can and beg/bribe/flatter/encourage them to do:
alter table `table` modify org_id int;
If that doesn't work, you have a conundrum. It is much better for query performance to have join keys be of the same types, even types that I don't agree with. Also, that is a pretty important concept for databases. So, you cannot change that join key.
Instead, I think you should add a new key into your table, called something like org_id_int. This would have the correct type, a useful index -- everything except a pretty name. Use this for your joins. Use the other key for the joins to the existing table, until it gets fixed.
Hi currently have the problem, that I want to insert values in a postgres database where the table contains self defined types like.
CREATE TYPE TestEnum AS ENUM ('Value1','Value2');
When I try to add a parameter in C# I always get an error because of the wrong NpgsqlDbType. So my question is what NpgsqlDbType to use for such self defined Types.
var parameter = new NpgsqlParameter(":p1", NpgsqlDbType.????)
{
Value = "Value1",
Direction = ParameterDirection.Input
}
Thanks for your help. I'm really going mad because of this problem.
After all, I found a solution that solves the problem, although it's not the real solution.
I now add a NpgsqlDbType.Varchar parameter and add a CAST(:p1 as "TestEnum") to the SQL
e.g.
INSERT INTO tableName (Col) VALUES ( CAST(:p1 as "TestEnum") )
It works for me, although I do not think that this is a very nice solution due to the cast.
If someone will find a better solution in future, please drop me a line. ;)
According to PostgreSQL 8.4.4 Documentation: 8.7. Enumerated Types:
An enum value occupies four bytes on disk.
That suggests that they are internally stored as a 32-bit integer. It uses the system catalog pg_enum to map from integers to the names and back.
Unfortunately, the documentation for NpgsqlDbType does not elucidate the meaning of each of those enum values, but my guess would be that NpgsqlDbType.Integer is likely to refer to a 32-bit integer.
However, I must confess that I am guessing here. It is also possible that NpgsqlParameter expects a type that represents the data you send to the DB with the query, which is clearly a string, so if NpgsqlDbType.Integer doesn’t work, my next guess would be NpgsqlDbType.Varchar.