Database Reconnect After Failure - c#

A program that I am currently working on collects and exports data to a SQL database. If there is a failure in the database, I have coded so that it will write the data to a CSV file. It is going to be an application that is always running with a config file that the user can change info (Such as Database connection) while the process is running. I'm having the problem of once I get into a loop of writing to a CSV file, I cannot get back to putting the data in a database until the application is restarted. Here is the relevant code:
class MethodClass
{
public static void Method()
{
while (true)
{
ConfigurationManager.RefreshSection("appSettings");
//Normally Running Code that Generates the Data
//Tries to write data to database 3 times
if (dbWrite != 2 && dbWrite != 3)
{
dbWrite = 0;
}
while (dbWrite < 2)
{
try
{
db.TblData.InsertOnSubmit(newrecord);
db.SubmitChanges();
break;
}
catch
{
dbWrite++;
}
}
if (dbWrite == 2)
{
try
{
db.TblData.InsertOnSubmit(newrecord);
db.SubmitChanges();
}
catch (Exception e)
{
dbWrite++;
log.Warn("Database Write Error", e);
BackupMethod();
}
}
if (dbWrite == 3)
{
try
{
db.TblData.InsertOnSubmit(newrecord);
db.SubmitChanges();
dbWrite = 0;
}
catch
{
BackupMethod();
log.Info("Still not writing to Database");
}
}
}
}
Class ConfigVariables
{
public static string SqlUser = ConfigurationManager.appSettings["SqlUser"];
//Other Config Variables
}
For reference, I just checked and the issue seems to be with getting the info from my Config file while the program is running. If I change other values in the config file, they have no effect until the next time that the program is run. Just so people know, I am editing the App.exe.config file in the Debug/Release folder.
UPDATE: It seems that if I assign the variable after RefreshSection();, that it will work. However, I put the variables in a separate class for a reason. Is there a better way to do this?

I was able to fix the issue by creating a method in my ConfigVariables class that refreshed and reassigned the variables in a while loop on a separate thread. This allowed me to keep my variables organized in another class and not change anything with referencing them in the my code. Now I just have to do some performance testing; hopefully constantly doing this won't cause performance issues.

Related

C# Process.Start Causing AccessViolationException Randomly

I have been tackling this issue for 3 months now.
Error I am Getting in Native Debugging:
"Exception thrown at 0x5A222FC2 (comct123.dll) in FileReader.exe:
0xC0000005: Access violation reading location 0x0000000C."
Normal Debug:
'System.AccessVioliationException' in System.Windows.Forms.dll
My setup is really simple:
public static Form_Interface Interface;
public static void Initialize()
{
Application.SetCompatibleTextRenderingDefault(false);
Interface = new Form_Interface();
Interface.Filesdgv.DataSource = File.SortableBindingList;
Application.Run(Interface);
}
Seems simple enough, right? No.
So basically I have a simple Event that simply opens the file using Process.Start() and no matter what I do it will randomly crash with 'System.AccessVioliationException' in System.Windows.Forms.dll here:
private void Filesdgv_CellContentClick(object sender, DataGridViewCellEventArgs e)
{
Filesdgv.Invoke((Action)(() =>
{
try
{
int rowIndex = e.RowIndex;
if (rowIndex >= 0)
{
int columnIndex = e.ColumnIndex;
File file = (File)((DataGridView)sender).Rows[rowIndex].DataBoundItem;
switch (columnIndex)
{
case 0:
{
Process.Start(file.Location);
}
break;
}
}
}
catch
{
// This fking catch never works anyway.
}
}));
}
private void FileInterface_Load(object sender, EventArgs e)
{
foreach (string oCurrent in Directory.GetFiles(Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.DesktopDirectory), "Files To Check")))
if (oCurrent.EndsWith(".pdf", StringComparison.OrdinalIgnoreCase))
new File(oCurrent.Split('\\').Last(), oCurrent);
}
It doesn't matter if I am opening files/links or anything else, it still behaves in the same way.
The link and file location is a readonly field as well.
I have many other uses for reading row data and it never crashes, even if i spam click 10000 times, It will only crash randomly with Process.Start()
Things I tried:
Using BeginInvoke
Using Invoke
Not Using Invoke/BeginInvoke
Putting File Link into a string before reading it.
Using multiple Try Catch
Recoded on another machine... same results there aswell.
I tried using File.Open (either doesn't open the file or throws same error lmao)
Tried using [HandleProcessCorruptedStateExceptions], still won't catch the exception.
Dosen't matter if i click slow or fast still 1/30 chance it happens.
Tried Putting Task.Run(() => Process.Start()); you'd think that a thread will protect you from an exception? no still crashes...
File Class looks like this:
public class File
{
public static SortableBindingList<File> SortableBindingList = new SortableBindingList<File>(new List<File>());
public readonly string fileName;
public readonly string filePath;
public void AddRow()
{
Client.Interface.Invoke((Action)(() =>
{
lock (SortableBindingList)
if (!SortableBindingList.Contains(this))
SortableBindingList.Add(this);
}));
}
public string FileName
{
get
{
return fileName;
}
}
public string Location
{
get
{
return filePath;
}
}
public File(string fileName, string filePath)
{
this.fileName = fileName;
this.filePath = filePath;
AddRow();
}
}
Initalize() is called in static void Main(string[] args) btw.
There are no other threads running editing stuff or anything like that, the only thread running is the form thread. which waits for user input.
Solutions I am looking for:
Alternative Method to launch files/hyperlinks.
A way to avoid form crashing (try catch style)
Crashes even with static data!:
Other threads running although these were not started by me.
Task.Run(() =>
{
Thread.Sleep(100);
Process.Start("https://www.youtube.com");
});
This has fixed my issues, it seems that when trying to immediately run "process.start" during a click event, the GUI unfocusing + starting a new process the exact same moment causes an Exception. (Microsoft pls fix.)

Neo4j locks CSV file even after importing

EDIT It seems that locks (mostly/only?) stay locked when a transactional error occurs. I have to restart the database for it to work again, but it is not actively processing anything (no CPU/RAM/HDD activity).
Environment
I have an ASP.NET application that uses the Neo4jClient NuGet package to talk to a Neo4j database. I have N SimpleNode objects that need to be inserted where N can be anything from 100 to 50.000. There are other objects for the M edges that need to be inserted where M can be from 100 to 1.000.000.
Code
Inserting using normal inserts is too slow with 8.000 nodes taking about 80 seconds with the following code:
Client.Cypher
.Unwind(sublist, "node")
.Merge("(n:Node { label: node.label })")
.OnCreate()
.Set("n = node")
.ExecuteWithoutResults();
Therefore I used the import CSV function with the following code:
using (var sw = new StreamWriter(File.OpenWrite("temp.csv")))
{
sw.Write(SimpleNodeModel.Header + "\n");
foreach (var simpleNodeModel in nodes)
{
sw.Write(simpleNodeModel.ToCSVWithoutID() + "\n");
}
}
var f = new FileInfo("temp.csv");
Client.Cypher
.LoadCsv(new Uri("file://" + f.FullName), "csvNode", true)
.Merge("(n:Node {label:csvNode.label, source:csvNode.source})")
.ExecuteWithoutResults();
While still slow, it is an improvement.
Problem
The problem is that the CSV files are locked by the neo4j client (not C# or any of my own code it seems). I would like to overwrite the temp .CSV files so the disk does not fill up, or delete them after use. This is now impossible as the process locks them and I cannot use them. This also means that running this code twice crashes the program, as it cannot write to file the second time.
The nodes are inserted and do appear normally, so it is not the case that it is still working on them. After some unknown and widely varying amount of time, files do seem to unlock.
Question
How can I stop the neo4j client from locking the files long after use? Why does it lock them for so long? Another question: is there a better way of doing this in C#? I am aware of the java importer but I would like my tool to stay in the asp.net environment. It must be possible to insert 8.000 simple nodes within 2 seconds in C#?
SimpleNode class
public class SimpleNodeModel
{
public long id { get; set; }
public string label { get; set; }
public string source { get; set; } = "";
public override string ToString()
{
return $"label: {label}, source: {source}, id: {id}";
}
public SimpleNodeModel(string label, string source)
{
this.label = label;
this.source = source;
}
public SimpleNodeModel() { }
public static string Header => "label,source";
public string ToCSVWithoutID()
{
return $"{label},{source}";
}
}

How to make multiple database operations atomic/one transaction in C#?

I need to make the code below atomic/fail or succeed as a single unit. How could I go about achieving that?
void Processor(Input input)
{
var mapper = new Mapper(recordDetails);
int remainingRecords = GetCountForRemainingRecords(recordDetails);
try
{
while (remainingRecords > 0)
{
mapper.CreateRecords(dataset);
Validate(dataset);
//the Save(dataset) uses SqlBulkCopy maps tables, transaction, and saves it..
Save(dataset);
//I cannot perform the operation below on the dataset directly because dataset doesn't have the records that is in the database
//the method below eventually calls a stored proc that sends a list of users that was recently created
OutdateDuplicateUsers(dataset.userTable);
remainingRecords = MethodToGetUpdatedCount();
}
}
catch (Exception exception)
{
//exception handler..
}
}
Now if my OutdateDuplicateUsers throws an exception, I would still end up with the accounts that Save method persisted. I do not want that to happen.
I want both Save and OutdateDuplicateUsers method to be atomic. I read about this great article about TransactionScope and seemed it is exactly what I want. However, I could not get it to work. The implementation seems straight forward reading from the article, but I couldn't get it working myself.
What I tried:
void Processor(Input input)
{
var mapper = new Mapper(recordDetails);
int remainingRecords = GetCountForRemainingRecords(recordDetails);
try
{
while (remainingRecords > 0)
{
using (var scope = new TransactionScope())
{
try
{
mapper.CreateRecords(dataset);
Validate(dataset);
//the method Save(dataset) is using SqlBulkCopy; maps tables, uses transaction, and saves it..
Save(dataset);
//I cannot perform this opertaion on the dataset directly because dataset doesn't have the records that is in the database
//the method below eventually calls a stored proc that sends a list of users that was recently created
OutdateDuplicateUsers(dataset.userTable);
remainingRecords = MethodToGetUpdatedCount();
scope.Complete();
}
catch (Exception)
{
//not both at the same time. I tried using both, one at a time though.
TransactionScope.Dispose();
TransactionScope.Current.Rollback();
//exception handler
}
}
}
}
}
update:
The dataset is a strongly typed dataset and is schema only. The CreateRecords and Validate method populates the data based on the business logic. The 'mapper' takes in recordDetails which is, for instance, a list of Users (updated the snippet).
What I mean by doesn't work is that if OutdateDuplicateUser() method throws an exception and cannot complete the outdating operation, I could still see that the records have been persisted in the database from Save(dataset) method, which I am trying to prevent.

Trouble inserting into Azure db from Windows Store app

I'm currently working on a Windows Store app (for a school assignment), and I'm having trouble inserting data into my database which is stored in Azure. Whenever I attempt to insert data into the db, the MobileServiceInvalidOperationException gets thrown. My code is as follows:
In my model class
class Division
{
public string Id {get; set;}
[JsonProperty(PropertyName = "divisionTitle")]
public string DivisionTitle {get; set;}
}
And the relevant code in my MainPage.xaml.cs file
private MobileServiceCollection<Division, Division> divisionItems;
private IMobileServiceTable<Division> divisionTable = App.MobileService.GetTable<Division>();
private async void InsertDivision(Division divisionItem)
{
// This code inserts a new division Item into the database.
// When the operation completes and Mobile Services has
// assigned an Id, the item is added to the collection
try
{
await divisionTable.InsertAsync(divisionItem);
divisionItems.Add(divisionItem);
}
/////////////////////////////////////////////////////////
// The MessageDialog that pops up when this exception //
// gets thrown is: //
// //
// Internal Server Error (HTTP 500) //
////////////////////////////////////////////////////////
catch (MobileServiceInvalidOperationException e)
{
MessageDialog errormsg = new MessageDialog(e.Message,
string.Format("{0} (HTTP {1})",
e.Response.ReasonPhrase,
(int)e.Response.StatusCode));
var ignoreAsyncOpResult = errormsg.ShowAsync();
}
}
private void DivisionButtonSave_Click(object sender, RoutedEventArgs e)
{
var DivisionItem = new Division
{
DivisionTitle = DivisionInput.Text
};
InsertDivision(DivisionItem);
}
I also added a script in the management portal:
function insert(item, user, request) {
if (item.DivisionTitle.length > 15) {
request.respond(statusCodes.BAD_REQUEST, 'Division title must be under 15 characters');
}
else {
request.execute();
}
}
Before making the changes above, I was having no trouble communicating with Azure from within the app and wasn't having any problems inserting data. It's only after editing the script in Azure (the default insert method is simply the request.execute() statement), and since I added the InsertDivision method (I was previously entering data into the db directly from the event handler with the command await App.MobileService.GetTable<Division>().InsertAsync(DivisionItem);) that this problem has started to occur. I've tried a couple of different things and nothing has worked. After looking at my code does anything stick out? Thanks in advance to anyone who can help.
In the request sent to the service, the property DivisionTitle is sent with the first letter in lower case (since you defined it as such with the JsonProperty attribute):
{"divisionTitle":"the actual title"}
On your script, you're trying to access the property item.DivisionTitle (which doesn't exist, JavaScript is case-sensitive), and then access a property (length) of this undefined value. That will cause an error in your script. If you either change the script to use the actual JSON name (item.divisionTitle.length > 15) or change the JsonProperty declaration in the client to send the property with the first letter in upper case, it should work.
By the way, if you go to the "logs" tab in the portal, you should see some error which explains why you're getting the internal server error.

Lock text file during read and write or alternative

I have an application where I need to create files with a unique and sequential number as part of the file name. My first thought was to use (since this application does not have any other data storage) a text file that would contain a number and I would increment this number so then my application would always create a file with a unique id.
Then I thought that maybe at a time when there are more than one user submitting to this application at the same time, one process might be reading the txt file before it has been written by the previous process. So then I am looking for a way to read and write to a file (with try catch so then I can know when it's being used by another process and then wait and try to read from it a few other times) in the same 'process' without unlocking the file in between.
If what I am saying above sounds like a bad option, could you please give me an alternative to this? How would you then keep track of unique identification numbers for an application like my case?
Thanks.
If it's a single application then you can store the current number in your application settings. Load that number at startup. Then with each request you can safely increment it and use the result. Save the sequential number when the program shuts down. For example:
private int _fileNumber;
// at application startup
_fileNumber = LoadFileNumberFromSettings();
// to increment
public int GetNextFile()
{
return Interlocked.Increment(ref _fileNumber);
}
// at application shutdown
SaveFileNumberToSettings(_fileNumber);
Or, you might want to make sure that the file number is saved whenever it's incremented. If so, change your GetNextFile method:
private readonly object _fileLock = new object();
public int GetNextFile()
{
lock (_fileLock)
{
int result = ++_fileNumber;
SaveFileNumbertoSettings(_fileNumber);
return result;
}
}
Note also that it might be reasonable to use the registry for this, rather than a file.
Edit: As Alireza pointed in the comments, it is not a valid way to lock between multiple applications.
You can always lock the access to the file (so you won't need to rely on exceptions).
e.g:
// Create a lock in your class
private static object LockObject = new object();
// and then lock on this object when you access the file like this:
lock(LockObject)
{
... access to the file
}
Edit2: It seems that you can use Mutex to perform inter-application signalling.
private static System.Threading.Mutex m = new System.Threading.Mutex(false, "LockMutex");
void AccessMethod()
{
try
{
m.WaitOne();
// Access the file
}
finally
{
m.ReleaseMutex();
}
}
But it's not the best pattern to generate unique ids. Maybe a sequence in a database would be better ? If you don't have a database, you can use Guids or a local database (even Access would be better I think)
I would prefer a complex and universal solution with the global mutex. It uses a mutex with name prefixed with "Global\" which makes it system-wide i.e. one mutex instance is shared across all processes. if your program runs in friendly environment or you can specify strict permissions limited to a user account you can trust then it works well.
Keep in mind that this solution is not transactional and is not protected against thread-abortion/process-termination.
Not transactional means that if your process/thread is caught in the middle of storage file modification and is terminated/aborted then the storage file will be left in unknown state. For instance it can be left empty. You can protect yourself against loss of data (loss of last used index) by writing the new value first, saving the file and only then removing the previous value. Reading procedure should expect a file with multiple numbers and should take the greatest.
Not protected against thread-abortion means that if a thread which obtained the mutex is aborted unexpectedly and/or you do not have proper exception handling then the mutex could stay locked for the life of the process that created that thread. In order to make solution abort-protected you will have to implement timeouts on obtaining the lock i.e. replace the following line which waits forever
blnResult = iLock.Mutex.WaitOne();
with something with timeout.
Summing this up I try to say that if you are looking for a really robust solution you will come to utilizing some kind of a transactional database or write a kind of such a database yourself :)
Here is the working code without timeout handling (I do not need it in my solution). It is robust enough to begin with.
using System;
using System.IO;
using System.Security.AccessControl;
using System.Security.Principal;
using System.Threading;
namespace ConsoleApplication31
{
class Program
{
//You only need one instance of that Mutex for each application domain (commonly each process).
private static SMutex mclsIOLock;
static void Main(string[] args)
{
//Initialize the mutex. Here you need to know the path to the file you use to store application data.
string strEnumStorageFilePath = Path.Combine(
Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData),
"MyAppEnumStorage.txt");
mclsIOLock = IOMutexGet(strEnumStorageFilePath);
}
//Template for the main processing routine.
public static void RequestProcess()
{
//This flag is used to protect against unwanted lock releases in case of recursive routines.
bool blnLockIsSet = false;
try
{
//Obtain the lock.
blnLockIsSet = IOLockSet(mclsIOLock);
//Read file data, update file data. Do not put much of long-running code here.
//Other processes may be waiting for the lock release.
}
finally
{
//Release the lock if it was obtained in this particular call stack frame.
IOLockRelease(mclsIOLock, blnLockIsSet);
}
//Put your long-running code here.
}
private static SMutex IOMutexGet(string iMutexNameBase)
{
SMutex clsResult = null;
clsResult = new SMutex();
string strSystemObjectName = #"Global\" + iMutexNameBase.Replace('\\', '_');
//Give permissions to all authenticated users.
SecurityIdentifier clsAuthenticatedUsers = new SecurityIdentifier(WellKnownSidType.AuthenticatedUserSid, null);
MutexSecurity clsMutexSecurity = new MutexSecurity();
MutexAccessRule clsMutexAccessRule = new MutexAccessRule(
clsAuthenticatedUsers,
MutexRights.FullControl,
AccessControlType.Allow);
clsMutexSecurity.AddAccessRule(clsMutexAccessRule);
//Create the mutex or open an existing one.
bool blnCreatedNew;
clsResult.Mutex = new Mutex(
false,
strSystemObjectName,
out blnCreatedNew,
clsMutexSecurity);
clsResult.IsMutexHeldByCurrentAppDomain = false;
return clsResult;
}
//Release IO lock.
private static void IOLockRelease(
SMutex iLock,
bool? iLockIsSetInCurrentStackFrame = null)
{
if (iLock != null)
{
lock (iLock)
{
if (iLock.IsMutexHeldByCurrentAppDomain &&
(!iLockIsSetInCurrentStackFrame.HasValue ||
iLockIsSetInCurrentStackFrame.Value))
{
iLock.MutexOwnerThread = null;
iLock.IsMutexHeldByCurrentAppDomain = false;
iLock.Mutex.ReleaseMutex();
}
}
}
}
//Set the IO lock.
private static bool IOLockSet(SMutex iLock)
{
bool blnResult = false;
try
{
if (iLock != null)
{
if (iLock.MutexOwnerThread != Thread.CurrentThread)
{
blnResult = iLock.Mutex.WaitOne();
iLock.IsMutexHeldByCurrentAppDomain = blnResult;
if (blnResult)
{
iLock.MutexOwnerThread = Thread.CurrentThread;
}
else
{
throw new ApplicationException("Failed to obtain the IO lock.");
}
}
}
}
catch (AbandonedMutexException iMutexAbandonedException)
{
blnResult = true;
iLock.IsMutexHeldByCurrentAppDomain = true;
iLock.MutexOwnerThread = Thread.CurrentThread;
}
return blnResult;
}
}
internal class SMutex
{
public Mutex Mutex;
public bool IsMutexHeldByCurrentAppDomain;
public Thread MutexOwnerThread;
}
}

Categories