We Keep Coding

C# script - SQL query best practice

I have an Excel file with a long list of usernames. Col A contains old user names Col. B has the new names. I want to rename users in a SQL table based on the excel file. My question is the following:
Is it ok to call SQL with a using statement multiple times within a loop where I iterate through the excel? Or is there a better way where I open a single connection and make all the SQL update queries with “one” go?

The answer to this is always that "it depends" and "can you justify it".
Right or wrong aside, your business case may include circumstances than mean multiple connections are an acceptable solution.
When iterating a data list, although not the greatest performance, it is generally acceptable to execute individual statements that would only affect a single record in a database.
You might do this to capture specific error information about each row, and in your business logic you will not re-process the rows that did succeed.
If you need to fail the whole batch when one row fails then you would need to ensure that you use a transaction scope so that you can roll back the entire set.
You would however generally NOT create multiple connections. a standard code pattern would be to create a connection outside of the loop and re-use the same connection for each transaction.
using (var conn = new SqlConnection(connectionString))
{
...
start a transaction
...
try
{
foreach(var record in dataRecords)
{
try
{
...
Execute your transactions
...
}
catch(SqlException sx)
{
...
Process the exception,
record relevant information based on the input parameters for this record
...
throw; // or throw a new exception with formatted info...
}
}
}
catch(Exception ex)
{
...
rollback the transaction
...
}
}
A set-based approach usually offers greater performance, but that requires a bit of plumbing to set up form a best practises point of view, this advice from Gordon Linoff works well too.
Multiple transactions, or multiple executions within the same transaction is acceptable, multiple connections however should be avoided.

You should load the Excel table into a table in the database with two columns (at least) called old_username and new_username.
Then you can run an update directly in the database. You haven't specified the database. But because of the C# tag, I'll provide SQL Server syntax for the update -- this syntax varies by database:
update u
set username = nc.new_username
from users u join -- the table you want to update
name_changes nc
on u.username = nc.old_username;
That is, it is generally better to get the data into the database and do all the work there.

Programmatically Set Attribute Table in new Shapefile

I have a program that loads a shapefile into memory, groups some of the features based on business logic, creates a shapefile out of each group of features, and then saves the files to a cloud location for use in other applications.
The one sticking point in this process has been the attribute table. I had wanted to be able to set custom attributes for the features in the new shapefiles. With the code below I am able to update the datatable with the desired information, but I can't make it persist once I save and dispose the shapefile object.
var table = shapefile.DataTable;
var i = 0;
foreach(var branchObject in branches)
{
shapefile.AddFeature(branchObject.Feature);
var row = table.Rows[i];
row.BeginEdit();
row["BranchName"] = branchObject.Name;
row.EndEdit();
i++;
}
table.AcceptChanges();
This gives me a properly-populated DataTable in the shapefile, but when I open the shapefile in MapWindow5, the only field in the attribute table is the auto-generated Id.
I'm clearly missing some sort of "Save Changes" step that I thought was encompassed in "AcceptChanges()" or "Being/EndEdit()"...what else needs to be called on the table to make it update?
I have a feeling this was covered in one of the tutorials that I can't find since Codeplex sunsetted, but as it is Google hasn't been very helpful.

As it turns out, my DataTable and DataRows were fine. One has to explicitly tell the shapefile to update it's attribute table after changes are made.
shapefile.Filename = $"{filePathWithName}.shp";
shapefile.UpdateAttributes();
These two lines of code just before saving the shapefile, and I can now see the attribute table of my dreams in MapWindow5.
Note:
Calling
shapefile.UpdateAttributes();
without first setting the shapefile.Filename property will throw an exception.
Updating the attributes evidently requires saving to the .dbf file of the shapefile package, and it can't do that without knowing where that .dbf file is supposed to go. This called for some refactoring for me as the output shapefile didn't exist outside memory till the end of the process.

linq to update or insert on a csv file

I successfully made a c# class that uses Jet to execute a SELECT string on a csv file. However, I really need an UPDATE and/or INSERT statement and Jet apparently won't allow it. I've been looking into using LINQ, but can't seem to find any examples of an UPDATE clause for LINQ either. Anyone know about this? or perhaps a different class than LINQ that could accomplish this?
Basically, I want to read a csv file into memory and query on it (select columns, or distinct, etc), which is fine with Jet, but I also want to update rows and modify the text file.
basically:
UPDATE table
SET col3 = 42
WHERE col1='mouse', col2='dolphins';
and have that take effect data read from csv.
also, I can't figure out how to access columns by name with LINQ. any advice?
so far, a constructor for my class seems to parse the file ok (I can see it in the watch and immediate windows), but I don't know how to move on from here:
string Method = this.ToString();
this.strFilePath = filePath;
this.strDelim = delim;
Logger.Debug("Starting", Method);
try
{
fileLines = File.ReadAllLines(filePath);
}
catch (Exception Ex)
{
Logger.Error(Ex, Method);
}
this.fileTable = fileLines.Select(l => l.Split(delim));
Ignore the 'Logger', this is an in-house class that writes things to a log for our own purposes.

What you're asking can't easily be done, due to the way text files are organized.
In general, you can't arbitrarily update a text file like you can a file of records. Consider, for example, the following text file.
line1,43,27,Jim
line2,29,32,Keith
Now, you want to change the 43 on line 1 to 4300. That involves adding two characters to the file. So you end up having to move everything after 43 down two spaces and then insert the 00. But moving everything down two spaces requires extending the file and moving the entire text.
Text files are typically used for sequential access: reading and appending. Insertion or deletion affects the entire file beyond the point of modification. Unless you're going to re-write the entire file on every change, you simply do not want to use a text file for holding data that changes frequently.

C# WPF Database Exercise How to update database column

Doing a learning exercise.
Trying to update the database date column for all entries to a new specific value or go through each entry and change date to a new specific value.
I tried using DataContext & Linq but it keeps telling me missing reference which its not so I have reverted to this.
var results = webDataSet.GreyList;
foreach (var elements in results)
{
elements.Date = 55;
}
webDataSet.AcceptChanges();
greyWebTableAdapter.Update(webDataSet.GreyList);
Even If i put Update in a try catch it says it is successful but it will never update the database.
Plus I'd like to thank the people who have nothing to say yet down vote my questions, its people like you who really bring the community together.

To anyone else who took advice to add .AcceptChanges(); by using this it sets the dataSets modified row value from true to false; so when you update the method looks for modified true row's however they are all false so nothing gets updated. Also the other factor to not seeing the database change is because Visual studio 2015 creates a copy of the database from your local folder and duplicates it in your bin folder, so looking for changes in your local folders database is a waste of time.

Finding all the documents in a Lotus Notes database

I'm working on a c# application that extracts metadata from all the documents in a Lotus Notes database (.nsf file, no Domino server) thusly:
NotesDocumentCollection documents = _notesDatabase.AllDocuments;
if (documents.Count>0)
{
NotesDocument document = documents.GetFirstDocument();
while (document!=null)
{
//processing
}
}
This works well, except we also need to record all the views that a document appears in. We iterate over all the views like this:
foreach (var viewName in _notesDatabase.Views)
{
NotesView view = _notesDatabase.GetView(viewName);
if (view != null)
{
if (view.AllEntries.Count > 0)
{
folderCount = view.AllEntries.Count;
NotesDocument document = view.GetFirstDocument();
while (document!=null)
{
//record the document/view cross reference
document = view.GetNextDocument(document);
}
}
Marshal.ReleaseComObject(view);
view = null;
}
}
Here are my problems and questions:
We fairly regularly encounter documents in a view that were not found in NotesDatabase.AllDocuments collection. How is that possible? Is there a better way to get all the documents in a notes database?
Is there a way to find out all the views a document is in without looping through all the views and documents? This part of the process can be very slow, especially on large nsf files (35 GB!). I'd love to find a way to get just a list of view name and Document.UniversalID.
If there is not a more efficient way to find all the document + view information, is it possible to do this in parallel, with a separate thread/worker/whatever processing each view?
Thanks!

Answering questions in the same order:
I'm not sure how this is possible either unless perhaps there's a special type of document that doesn't get returned by that AllDocuments property. Maybe replication conflicts are excluded?
Unfortunately there's no better way. Views are really just a saved query into the database that return a list of matching documents. There's no list of views directly associated with a document.
You may be able to do this in parallel by processing each view on its own thread, but the bottleneck may be the Domino server that needs to refresh the views and thus it might not gain much.
One other note, the "AllEntries" in a view is different than all the documents in the view. Entries can include things like the category row, which is just a grouping and isn't backed by an actual document. In other words, the count of AllEntries might be more than the count of all documents.

Well, first of all, it's possible that documents are being created while your job runs. It takes time to cycle through AllDocuments, and then it takes time to cycle through all the views. Unless you are working on a copy or replica of the database that is is isolated from all other possible users, then you can easily run into a case where a document was created after you loaded AllDocuments but before you accessed one of the views.
Also, is it may be possible that some of the objects returned by the view.getXXXDocument() methods are deleted documents. You should probably be checking document.isValid() to avoid trying to process them.
I'm going to suggest using the NotesNoteCollection as a check on AllDocuments. If AllDocuments were returning the full set of documents, or if NotesNoteCollection does (after selecting documents and building the collection), then there is a way to do this that is going to be faster than iterating each view.
(1) Read all the selection formulas from the views, removing the word 'SELECT' and saving them in a list of pairs of {view name, formula}.
(2) Iterate through the documents (from the NotesNoteCollection or AllDocuments) and for each doc you can use foreach to iterate through the list of view/formula pairs. Use the NotesSession.Evaluate method on each formula, passing the current document in for the context. A return of True from any evaluated formula tells you the document is in the view corresponding to the formula.
It's still brute force, but it's got to be faster than iterating all views.