We Keep Coding

migrating an access multi valued field column to c#

I am attempting to use the Microsoft.ACE.OLEDB.12.0 driver to read data from an access database. came upon an odd situation. one of the columns in the access database shows as a comma delimited list of ids.
Wells
________
345,456,7
6,387
when I looked at the column definition in access I thought it would say string but it does not, it says number. so I guess it is storing an array of integers in a single column?
I'm having a tough time getting a data reader to pick this up.
using
var w = DB_Reader.GetValue(DB_Reader.GetOrdinal("Wells"));
results in the error
The provider could not determine the Object value. For example, the
row was just created, the default for the Object column was not
available, and the consumer had not yet set a new Object value.

Well, at the end of the day, you can think of the mutli-value column as in fact a child table.
So, if you looking to migrate a master and child table, then in YOUR database, you need a relational set of tables to re-create what Access is doing behind the scene.
So, lets take a multi-value example and query.
Say we have this sql query in Access:
SELECT ID, Person_Name, FavorateColors FROM tPerson;
But, "favorite colors" is one of those MV columns. (and I should point out with the HUGE movement towards no-sql databases - they also often work this way also - same for XML or JSON data for that matter. However, be it some XML, JSON or Access mutli-value features? Well, you need that child table if you going to adopt a relational data model to represent this data.
Ok, so we run the above query, and you get this output:
In fact, when I used the lookup wizard - I picked a child table called tblColors.
but, how can we explode the above query to dig out the data?
Change the above query to this:
SELECT ID, Person_Name, FavorateColors.Value FROM tPerson
Note how we added ".value" after the MV column name. Now, when you run the query, you get the SAME result as if you had two tables, and did a left join. The parent table rows will like any relational database simple repeat for each child table value, and you get this:
Note how now the PK value and the row is repeating for each child mv value.
So, you are quite much free to query as per above - you get what amounts to a left joined table, and of course the parent record repeats.
So, just like XML, JSON, or in fact a query or a table of data with repeating parent row, and child rows? Well, you quite much forced to write code to split out this data, or re-normalize the data. This of course is far more common when receiving say JSON/XML data, or in fact often say data from a Excel sheet.
So, you have to process out the child record data, and create a relation for that data.
And thus now our question becomes how can we import JSON/XML/Excel data that really should have used two relational database tables.
So, assuming we want to process this data? You process it the same as for any data you have that should have been two related tables in the first place.
it really depends if this is a one time import, or you have to do this all the time?
If it was a one time deal, then I would use Access, and use a make table query based on the above query. You would in fact have to pluck up the PK ID from the child table. In above there is a child table called colors - we just missing that "junction" table in between that Access automatic created. The hidden tables are not exposed, and thus I would simply use a make table query in access, and then add a FK column that is the PK value from the tblColors.

How to get SQL table property length attribute from SQL database tables

I wonder if this is even possible, and I can't seem to find an answer. I am using Entity Framework / C# for a database application. I want to ensure that my front-end entities automatically match their counterparts in SQL for attributes like "StringLength". So, inside SQL, in my tables, if I specify, for example, "nvarchar(50)", I want to make sure that the user doesn't type more than 50 characters. I don't want to have to code this each time, so I'm looking for a way to have the C# application read the StringLength property from SQL and then apply logic in the front-end. I can't seem to find a way of getting this information by entity type/DbContext from the SQL database itself.
I thought I'd struck gold with this thread here: How to get the maximum length of a string from an EDMX model in code? but this seems to read from the entity schema defined in the C# application, not SQL.
I have attached a .png of the info I am trying to get from SQL for clarification. Any advice would be appreciated:

use this query to return Maximum length of column Description in table PaymentType
SELECT
CHARACTER_MAXIMUM_LENGTH
FROM
INFORMATION_SCHEMA.COLUMNS
WHERE
TABLE_NAME = 'PaymentType' and COLUMN_NAME = 'Description'

Entity Framework - how can I optimize “Contains” statement?

In our current application we have some performance issues with some of our queries. Usually we have something like:
List<int> idList = some data here…;
var query = (from a in someTable where idList.Contains(a.Id));
while for simple queries this is acceptable, it becomes a bottleneck when we have more items in idList (in some queries we have about 700 id’s to check, for example).
Is there any way to use something other then Contains? We are thinking of using some temporary tables to first insert the Ids, and then to execute join instead of Contains, but it would seem EntityFramework does not support such operations (creating temporary tables in code) :(
What else can we try?

I Suggest using LINQ PAD it offers a "Transform to SQL" option which allows you to see your query in SQL syntax.
there is a chance that this is the optimal solution (if youre not into messy stuff).
might try holding the idList as a sorted array and have the contains method replaced with a binary search. (you can implement your own extension).

You can try this:
var query = someTable.Where(a => idList.Any(b => b.Id == a.Id));

If you don't mind having a physical table you could use a semi-temporary table. The basic idea is:
Create a physical table with a "query id" column
Generate a unique ID (not random, but unique)
Insert data into the table tagging the records with the query ID
Pass the query id to the main query, using it to join to the link table
Once the query is complete, delete the temporary records
At worst if something goes wrong you will have orphaned records in the link table (which is why you use a unique query ID).
It's not the cleanest solution but it will be faster than using Contains if you have a lot of values to check against.

When Entity Framework starts being a performance bottleneck, generally it's time to write actual SQL.
So what you could do for example is build a table-valued function that takes a table-valued parameter (your list of IDs) as parameter. The function would just return the result of your JOIN.
Table valued function feature requires EF5, so it might be not an option if you're really stuck with EF4.

The idea is to refactor your queries to get rid of idList.
For example you should return the list of orders of male users 18-25 year, from France. If you filter users table by age, sex and country to get idList of users you end up with 700+ id's. Instead you make Orders table join with Users and apply filters to the Users table. So you don't have 2 requests (one for ids and one for orders) and it works much faster cause it can use indexes while joining the table.
Makes sense?

Linq to entities with raw stored procedure returns duplicate copies of data

Interfacing to SQL Server 2008R2:
I have a linq expression:
IQueryable<xxx> xxxResult =
(from t in _context.xxxx.AsNoTracking().Include("yyy")
where t.zzz >= lowNumber
&& t.zzz <= highNumber
&& t.qqq == someValue
select t);
(It probably doesn't matter on the exact query, but it's there in case it does.)
Linq generated SQL that the SQL Server generated a terrible plan, and, since I can't add index/join hints, I created a stored procedure that wrapped the SQL that the above Linq expression generated.
I know I should be able to access the stored procedure through Entity Framework, but I'm using a previous project that used a very light code-first implementation (no .edmx file, for instance) and I'm kinda new to the whole EF thing and didn't know how to tie the new procedure into EF. I know it can be done, but I am trying to call the procedure directly.
I worked this out:
IQueryable<xxx> xxxResult =
_context.xxxx.SqlQuery("GetData #p0, #p1, #p2", someValue, lowNumber, highNumber)
.AsNoTracking().AsQueryable();
This seems to work, except for one problem. When iterating over the linq queryable, everything works swimmingly. But, when I use the stored procedure, I get duplicate records.
For instance, if I have an xxx record that includes 3 yyy records in a collection, I get a single xxx record from the linq expression and it, as expected, includes 3 yyy records in the internal collection.
The stored procedure, for the same dataset, iterating over the queryable returns three xxx records, EACH of which contain the same 3 yyy records.
Again, the stored procedure executes the exact same SQL that the linq expression generated.
Why is that? Any ideas?
(Again, new to EF, so please forgive errors in terminology.)

I believe that EF is seeing your results as duplicate based on the primary key you have defined. In EF5, this would be defined using the "Entity Key" property on the fields which uniquely define the entity (a multi-part primary key would have this set on multiple fields).
If your procedure returns a record that matches one that it already returned (based soley on the primary key fields) then it will return a reference to the previous record.
Your LINQ expression uses .AsNoTracking which should prevent this caching behavior.
I'm guessing that the .AsNoTracking() using the stored proc occurs after it has been cached and doesn't have the effect you are looking for.
Make sure that you have your primary keys set properly on your model.
Here's an article that describes the behavior with a view:
http://jepsonsblog.blogspot.in/2011/11/enitity-framework-duplicate-rows-in.html which should be similar to what you are seeing with the stored procedure.
It looks like in Code First, you would use the [Key] annotation to specify your unique keys: http://msdn.microsoft.com/en-us/data/jj591583.aspx

How to get the primary key from a table without making a second trip?

How would I get the primary key ID number from a Table without making a second trip to the database in LINQ To SQL?
Right now, I submit the data to a table, and make another trip to figure out what id was assigned to the new field (in an auto increment id field). I want to do this in LINQ To SQL and not in Raw SQL (I no longer use Raw SQL).
Also, second part of my question is: I am always careful to know the ID of a user that's online because I'd rather call their information in various tables using their ID as opposed to using a GUID or a username, which are all long strings. I do this because I think that SQL Server doing a numeric compare is much (?) more efficient than doing a username (string) or even a guid (very long string) compare. My questions is, am I more concerned than I should be? Is the difference worth always keeping the userid (int32) in say, session state?
#RedFilter provided some interesting/promising leads for the first question, because I am at this stage unable to try them, if anyone knows or can confirm these changes that he recommended in the comments section of his answer?

If you have a reference to the object, you can just use that reference and call the primary key after you call db.SubmitChanges(). The LINQ object will automatically update its (Identifier) primary key field to reflect the new one assigned to it via SQL Server.
Example (vb.net):
Dim db As New NorthwindDataContext
Dim prod As New Product
prod.ProductName = "cheese!"
db.Products.InsertOnSubmit(prod)
db.SubmitChanges()
MessageBox.Show(prod.ProductID)
You could probably include the above code in a function and return the ProductID (or equivalent primary key) and use it somewhere else.
EDIT: If you are not doing atomic updates, you could add each new product to a separate Collection and iterate through it after you call SubmitChanges. I wish LINQ provided a 'database sneak peek' like a dataset would.

Unless you are doing something out of the ordinary, you should not need to do anything extra to retrieve the primary key that is generated.
When you call SubmitChanges on your Linq-to-SQL datacontext, it automatically updates the primary key values for your objects.
Regarding your second question - there may be a small performance improvement by doing a scan on a numeric field as opposed to something like varchar() but you will see much better performance either way by ensuring that you have the correct columns in your database indexed. And, with SQL Server if you create a primary key using an identity column, it will by default have a clustered index over it.

Linq to SQL automatically sets the identity value of your class with the ID generated when you insert a new record. Just access the property. I don't know if it uses a separate query for this or not, having never used it, but it is not unusual for ORMs to require another query to get back the last inserted ID.
Two ways you can do this independent of Linq To SQL (that may work with it):
1) If you are using SQL Server 2005 or higher, you can use the OUTPUT clause:
Returns information from, or
expressions based on, each row
affected by an INSERT, UPDATE, or
DELETE statement. These results can be
returned to the processing application
for use in such things as confirmation
messages, archiving, and other such
application requirements.
Alternatively, results can be inserted
into a table or table variable.
2) Alternately, you can construct a batch INSERT statement like this:
insert into MyTable
(field1)
values
('xxx');
select scope_identity();
which works at least as far back as SQL Server 2000.

In T-SQL, you could use the OUTPUT clause, saying:
INSERT table (columns...)
OUTPUT inserted.ID
SELECT columns...
So if you can configure LINQ to use that construct for doing inserts, then you can probably get it back easily. But whether LINQ can get a value back from an insert, I'll let someone else answer that.

Calling a stored procedure from LINQ that returns the ID as an output parameter is probably the easiest approach.