We Keep Coding

SQL Server recommendation for storing segmented gl account codes in database

I've been tasked with an enhancement to our order system that will require importing segmented GL account codes for assignment on individual line items of an order.
I need to support querying the codes by segment1, segment2, etc in order to load cascading dropdown boxes for assignment by the user. The GL codes will have one or more segments delimited by a character. An example of a code is "1010.1034001.99.01".
I've loaded several thousand codes into a table for testing where the entire string value exists in one column (delimited by a character). I've created two variations of functions that return rows where segment1 value is equal to some parameter. The query also supports further querying by providing additional parameters for other segment values.
I intend to support these queries from the table using Entify Framework 6, but used sql functions to get a feel for what the performance may be when the GL account codes are stored in one column. Performance was not as good as I had hoped.
Does anyone have recommendations on how best to store this data (there may be 200,000 codes). Do you feel that I can query using EF and expect performant results?
Would a hierarchy organization make more sense for this data? Our team was hopeful to store the delimited values on one column.
Thanks in advance.

If you would use a table with three columns you could store the values cascading, enabling you to make your queries a lot easier and probably faster. Why would your team hope to store it in one column, what advantage does that have?
if you have
ID
Code
ParentCodeId
where ID is a unique key and ParentCodeId is a nullable reference to that unique Id you can split your exaple code as follows:
ID Code Parent
1 1010 null
2 1034001 1
3 99 2
4 01 3
By applying some logic when importing your codes, you can check if a code already exists as a parent on the needed level so you don;t have to repeat them, and that way you coul dget all codes that start with 10100 by selecting on selectiong on parentID 1.

How to deal with deprecated values in (country-)code lists

Let's say we have a code list of all the countries including their country codes. The country code is primary key of the Countries table and it is used as a foreign key in many places in the database. In my application the countries are usually displayed as dropdowns on multiple forms.
Some of the countries, that used to exists in the past, don't exist any more, for example Serbia and Montenegro, which had the country code of SCG.
I have two objectives:
don't allow the user to use these old values (so these values should not be visible in dropdowns when inserting data)
the user should still be able to (readonly) open old stuff and in this case the deprecated values should be visible in dropdowns.
I see two options:
Rename deprecated values, for instance from 'CountryName' to '!!!!!CountryName'. This approach is the easiest to implement, but with obvious drawbacks.
Add IsActive column to Countries table and set it to false for all deprecated values and true for all other. On all the forms where the user can insert data, display only values which are active. On the readonly forms we can display all values (including deprecated ones) so the user will be able to display old data. But on some of my forms the user should be able to also edit data, which means that the deprecated values should be hidden from him. That means, that each dropbox should have some initialization logic like this: if the data displayed is readonly, then include deprecated values in dropbox and if the data is for edit also, then exclude them. But this is a lot of work and error prone too.
And other ideas?

I deal with this scenario a lot, and use the 'Active' flag to solve the problem, much as you described. When I populate a drop-down list with values, I only load 'active' data and include upto 1 deprecated value, but only if it is being used. (i.e. if I am looking at a person record, and that person has a deprecated country, then that country would be included in the Drop-downlist along with the active countries. I do this in read-only AND in edit modes, because in my cases, if a person record (for example) has a deprecated country listed, they can continue to use it, but once they change it to a non-deprecated country, and then save it, they can never switch back (your use case may vary).
So the key differences is, even in read-only mode I don't add all the deprecated countries to the DDL, just the deprecated country that applies to the record I am looking at, and even then, it is only if that record was already in use.
Here is an example of the logic I use when loading the drop down list:
protected void LoadSourceDropdownList(bool AddingNewRecord, int ExistingCode)
{
using (Entities db = new Entities())
{
if (AddingNewRecord) // when we are adding a new record, only show 'active' items in the drop-downlist.
ddlSource.DataSource = (from q in db.zLeadSources where (q.Active == true) select q);
else // for existing records, show all active items AND the current value.
ddlSource.DataSource = (from q in db.zLeadSources where ((q.Active == true) || (q.Code == ExistingCode)) select q);
ddlSource.DataValueField = "Code";
ddlSource.DataTextField = "Description";
ddlSource.DataBind();
ddlSource.Items.Insert(0, "--Select--");
ddlSource.Items[0].Value = "0";
}
}

If you are displaying the record as read-only, why bother loading the standing data at all?
Here's what I would do:
the record will contain the country code in any case, I would also propose returning the country description (which admittedly makes things less efficient), but when the user loads "old stuff", the business service recognises that this record will be read only, and you don't bother loading the country list (which would make things more efficient).
in my presentation service I will then generally do a check to see whether the list of countries is null. If not (r/w) load the data into the list box, if so (r/o) populate the list box from the data in the record - a single entry in the list equals read-only.

You can filter with CollectionViewSource or you could just create a Public Enumerable that filters the full list using LINQ.
CollectionViewSource Class
LINQ The FieldDef.DispSearch is the active condition. IEnumerable is a little better performance than List.
public IEnumerable<FieldDefApplied> FieldDefsAppliedSearch
{
get
{
return fieldDefsApplied.Where(df => df.FieldDef.DispSearch).OrderBy(df => df.FieldDef.DispName);
}
}

Why would you still want to display (for instance) customer-addresses with their OLD country-code?
If I understand correctly, you currently still have 'address'-records that still point to 'Serbia and Montenegro'. I think if you solve that problem, your current question would be none-existent.
The term "country" is perhaps a little misleading: not all the "countries" in ISO 3166 are actually independent. Rather, many of them are geographically separate territories that are legally portions or dependencies of other countries.
Also note that 'withdrawn country-codes' are reserved for 5 years, meaning that after 5 years they may be reused. So moving away from using the country-code itself as primary key would make sense to me, especially if for historical reasons you would need to back-track previous country-codes.
So why not make the 'withdrawn' field/table that points to the new country-id's. You can still check (in sql for instance, since you were already using a table) if this field is empty or not to get a true/false check if you need it.
The way I see it: "Country" codes may change, country's may merge and country's may divide.
If country's change or merge, you can update your address-records with a simple query.
If country's divide, you need a way to determine what address is part of what country.
You could use some automated system do do this (and write lengthly books about it).
OR
(when it is a forum like site), you could ask the users that still have a withdrawn country that points to multiple alternatives in their account to update their country-entry at login, where they can only choose from the list of new country's that are specified in the withdrawn field.
Think of this simplified country-table setup:
id cc cn withdrawn
1 DE Germany
2 CS Serbia and Montenegro 6,7
3 RH Southern Rhodesia 5
4 NL The Netherlands
5 ZW Zimbabwe
6 RS Serbia
7 ME Montenegro
In this example, address-records with country-id 3, get updated with a query to country-id 5, no user interaction (or other solution) needed.
But address-records that specify country-id 2 will be asked to select country-id 6 or 7 (of course in the text presented to the user you use the country-name) or are selected to perform your custom automated update routine on.
Also note: 'withdrawn' is a repeating group and as such you could/should make it into a separate table.
Implementing this idea (without downtime) in your scenario:
sql statement to build a new country-table with numerical id's as primary key.
sql statement to update address-records with new field 'country-id' and fill this field with the country-id from the new country-table that corresponds with country-code specified in that record's address-field.
(sql statement to) create the withdrawn table and populate the correct data with in it.
then rewrite your the sql statements that supply your forms with data
add the check and 'ask user to update country'-routine
let new forms go live
wait/see for unintended bugs
delete old country-table and (now unused) country-code column from the "address"-table
I am very curious what other experts think about this idea!!

Web application user defined content with database content inserted

I have been attempting to research the best way to do the following:
Save a user-defined text based expression to a database which outlines content to be displayed on a page.
example: "The employment status of this person is a [employment] employee, [hours] hrs a week."
Where [employment], [hours] correspond to a key/value in a database
example:
TABLE
ID KEY VALUE
1 employment casual
1 hours 10
2 employment parttime
2 hours 25
At runtime a page where this user-defined expression would be used for display has access to a persons id and would need to collect the relevant details from the database, parse and replace the [employment], [hours] placeholders with the retrieved data.
example: "The employment status of this person is a casual employee, 10 hrs a week.
The solution I am looking for needs to be dynamic enough to handle a user entering any static text in and around any number of placeholders.
EDIT to clarify my meaning:
There will be a settings area where the user can enter the expression they want
displayed on the screen, which could be any text containing any 'key' values from
the database. Different users will see their own defined expression displayed.
another example: "The employee works [hours] hrs, [employment]"
would display as: "The employee works 10 hrs, casual"
I'm either not doing very well with my searching or just not understanding the results very well. I have found a lot of information regarding CustomExpressionBuilder, Lambda expressions, Expression trees, etc. and I am getting confused whether any of these are the right track for what I need to do. Perhaps I'm just not there yet on my understanding being new to this!
If I can get this working I would also love to be able to add some simple ternary type functionality and/or mathematical functionality. It seems like there is library called NCalc that would make this quite simple, but I could not see an example of the text replacement requirement that I initially need, or immediately see a way to tweak it to work.
Thanks for any advice, and I am more than happy to research further but need a little push in the right direction.

From your question, it's a bit vague as to how far you've gotten with this problem, ie if you are already able to retrieve all of the values from the database or not.
If you are simply trying to build a string expression as you mentioned above, once you have the values above, I would probably use either String.Format or perhaps a string builder.
For example, if we presume that you have the values out of the database and they are contained within properties on a Person class (just as an example), you could write something to the effect of:
var message = String.Format("The employment status of this person is a {0} employee, {1} hrs a week.", person.Employment, person.Hours);
Like I said, it's a bit vague from your question exactly where your issue is, but if you're just looking to build that one particular expression once you have the values, the above would work. You can replace the person.Employment and person.Hours arguments with the appropriate values once you have those out of the database.

Designing a data-driven logic system

I'm developing a tax calculation system that applies various taxes based on a set of supplied criteria.
The information frequently changes, so I'm trying to create a way to store all these logic rules in the database.
As you can imagine, there is a lot of compound logic involved in applying taxes.
For example, a tax might only apply if A is true, B is less than 100, and C equals 7.
My current design is terrible.
I have a few database columns for very common criteria filtering, such as location and tax year.
For more complex logic, I have a column that holds JavaScript, and in code, I run an interpreter to filter the results. Performance and maintainability suck.
I'd like to improve this design by making the logic entirely data-driven, but I'm having trouble figuring out how to correctly represent this logic within a relational database. What is a good way to model this logic in the database?

I have worked on this similar issue for over a year now for a manufacturing cost generation application. Similarly, it takes in loads of product design data input and base on the design, and other inventory considerations such as quantity, bulk purchase options, part supplier, electrical ratings etc. The result is a list of direct materials, labour and costs.
I knew from the onset that what I need is some kind of query language instead of a computational one, and it has to be scripted, not compiled. But I have yet to find a perfect solution:
METHOD 1 - SQL
I created tables that represents my objects and columns that represents properties and then manually typed in the all the SQL SELECT statments required in an item_rules table. What I did was to first save the object into the database, then then I did
rules = SELECT * FROM item_rules
foreach(rules as _rule)
{
count = SELECT COUNT(*) FROM (_rule[select_statement]) as T1
if(count > 1) itemlist.add(_rule[item_that_satisfy_rule])
}
What it does is it takes each rule in the item_rules table and run it against my object that is now in the tables. e.g. SELECT * FROM my_object WHERE A=5 AND B>10. If I successfully pick it up, I get a positive count and then I know I should include the corresponding rule item to my items list.
METHOD 2 - NCALC
Instead of storing the queries in SQL format, I found the NCALC opensource expression parsing library. NCALC takes a string expression and option variable and computes a result. The string expressions can be stored in plain text on the filesystem.
METHOD 3 - EXCEL
EXCEL is actually a very good piece of software for doing data lookups. You can create the formulas in excel and then feed data from your application into excel and then let excel run the formulas to give you the results. Advantage is that many people knows how to use excel, so different people can maintain it.
But like I say, none of these are perfect for me. I am just sharing and hopefully we can get better recommedations.

If you are to go with Jake's approach, You can use Dynamic Sql too.

Allowing a user to create a custom query of a table

I am writing a program that generates a single large table of information. The table is made up of two types of columns, columns that contain quantities, and columns that contain properties. Quantities are numeric values that can be summed, properties are qualitative values that attribute the quantity values.
If the data is in a table in a database I can write a query that selects specific properties and quantities and sums the quantities that have the same value for the selected properties.
Example:
Table:
Quanity1 Quanity2 Quanity3 Property1 Property2 Property3
12 43 12 RED Long Rough
43 23 23 Blue Short Smooth
43 90 34 RED Fat Bumpy
Query:
SELECT sum(Quanity1), sum(Quanity2), Property1 FROM Table Group By Property1
Result:
Quanity1 Quanity2 Property1
43 23 Blue
55 133 Red
What I want to do is give the user a graphical interface to do this with out knowing how to write SQL queries, or any code for that matter. Such as a set of list boxes where they select the properties and quantities they want to view and a table is displayed that shows the selected fields with the quantities summed. I may also later want to add the ability for the user to perform other SQL query like actions such as filtering based on certain conditions. Also I know later I'll need to be able to generate nice looking reports based on these user Queries.
I'm very new to ADO and .NET in general. But I'm thinking the best way to do this is to export my data into a System.Data.DataTable and then create an interface for the user to create a System.Data.DataView by generating a string for it's RowFilter property. Although, it's not obvious to me how I can not only filter and sort a DataTable but generate another Table or view that only contains specific columns from the big master table.
Overall does this sound like the best option, or is there another method I should consider? Does anyone have any specific tips or suggestions on how I should implement this? I was also questioning if any of this would be made easier with LINQ.
Update
I appreciate the suggestion of using Access or other available tool, but it's really not an option. Access is way too complicated for users here to try to figure out, and much more then I actually need. I'd always leave Access as an option for advanced users. But I would still like to setup a basic querying feature where the user selects the columns they want and the software automatically creates the view/query that selects and sums the appropriate columns.
Aside from being to complex the other issue with Access is there are to many clicks between changing something in my data structure and seeing a change in a report. I don't want the user to have to change something, re-export to access, open another program, and then open the report to see the effect of their change.

Consider buying an off-the-shelf query tool rather than re-inventing the wheel. The cheapest one that could do this sort of thing is MS Access or MSQuery in Excel. More elaborately you could use Report Builder (if your database is based on SQL Server - it comes for free with this) or a third-party tool such as Business Objects or Brio.
If you can live without tight integration this is far easier than trying to build your own ad-hoc query tool.

I also strongly recommend off-the-shelf - especially early on. If it becomes apparent later on that the users really need you to write a custom solution, then by all means go for it. But this early on I don't think it will be worth the time and effort you will spend.