I have a code the gets data from my server. Every data will be inserted in my local database (SQLite Database) one by one. Every once in a whole I am getting these two errors. My codes below is where the exception are always appearing.
SQLite.SQLiteException: Busy
SQLite.SQLiteException: database is locked
What is/are the cause(s) why I always get these exceptions?
Update:
I added await conn.CloseAsync(); in every end of the code is that correct?
here is my first code, it will inquire to the server if there were updates and if there are updates it will insert or replace(update) the data in my local database:
var db = DependencyService.Get<ISQLiteDB>();
var conn = db.GetConnection();
string apifile = "sync-retailer-outlet-server-update-api.php";
var lastchecked = Preferences.Get("retaileroutletchangelastcheck", String.Empty, "private_prefs");
int count = 0;
var uri = new Uri(string.Format("http://" + domain + "/TBSApp/app_api/" + apifile + "?Host=" + host + "&Database=" + database + "&ContactID=" + contact + "&LastChecked=" + lastchecked, string.Empty));
try
{
SyncStatus("Getting retailer outlet data from server");
var response = await client.GetAsync(uri);
if (response.IsSuccessStatusCode)
{
var content = await response.Content.ReadAsStringAsync();
if (!string.IsNullOrEmpty(content))
{
var dataresult = JsonConvert.DeserializeObject<List<RetailerGroupData>>(content, settings);
var datacount = dataresult.Count;
for (int i = 0; i < datacount; i++)
{
SyncStatus("Saving retailer outlet server update to local database (" + (count + 1) + " out of " + dataresult.Count + ")");
var item = dataresult[i];
var retailerCode = item.RetailerCode;
var insertdata = new RetailerGroupTable
{
RetailerCode = retailerCode
};
await conn.InsertOrReplaceAsync(insertdata);
count++;
}
}
}
}
catch (Exception ex)
{
Crashes.TrackError(ex);
}
await conn.CloseAsync();
Here is the other code that creates the local database table everytime the StartPage is loaded.
public async void CreateTableAsync()
{
try
{
var db = DependencyService.Get<ISQLiteDB>();
var conn = db.GetConnection();
if (conn != null)
{
try
{
await conn.CreateTableAsync<UserTable>();
await conn.CreateTableAsync<ContactsTable>();
await conn.CreateTableAsync<ActivityTable>();
await conn.CreateTableAsync<CAFTable>();
await conn.CreateTableAsync<RetailerGroupTable>();
await conn.CreateTableAsync<UserEmailTable>();
await conn.CreateTableAsync<UserLogsTable>();
await conn.CreateTableAsync<SubscriptionTable>();
await conn.CreateTableAsync<ProvinceTable>();
await conn.CreateTableAsync<TownTable>();
}
catch (Exception ex)
{
Console.Write("Creating table error " + ex.Message);
}
}
}
catch (Exception ex)
{
Crashes.TrackError(ex);
await DisplayAlert("Application Error", "Error:\n\n" + ex.Message.ToString() + "\n\n Please contact your administrator", "Ok");
}
}
await conn.CloseAsync();
using (SQLiteConnection connection = db.GetConnection())
{
// Do whatever you want to do with your active connection
}
Note that you cannot reuse that connection outside the scope of the using block and have to get a new connection the same way the next time you want to access your database.
According to your description and code, I assume you use async-style inserts and are on different threads and thus an insert is timing out waiting for the lock of a different insert to complete. You can use synchronous inserts to avoid this condition.
Related
I created a service in windows Worker service that connects to the database at regular intervals and sends the first newest row in the table to the API
But how to make the service send the same row only once and wait until the next new one appears.
ID cannot have the same value, is it possible to create a condition when dbResult[0] (id) of the already sent row != dbResult[0] can be sent ?
My part of code:
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
try
{
string connectionString = "User=SYSDBA;" +
"Password=masterkey;" +
"Database=test.DTB;" +
"DataSource=localhost;" +
"Port:3050";
FbConnection mConnection = new FbConnection(connectionString);
mConnection.Open();
FbTransaction mTransaction = mConnection.BeginTransaction();
string SQLCommandText = "select first 1 * from TABLE where NAME = 449 order by DATE desc ";
FbCommand mCommand = new FbCommand(SQLCommandText, mConnection, mTransaction);
FbDataReader mReader = mCommand.ExecuteReader();
if (mReader.Read())
{
var values = new object[mReader.FieldCount];
{
mReader.GetValues(values);
var dbResult = values.Distinct().ToArray();
var dbResults = (string.Join("|", dbResult));
var result = new
{
ID = dbResult[0],
NAME = dbResult[1],
DATE = dbResult[2],
};
var jsonString = Newtonsoft.Json.JsonConvert.SerializeObject(result);
}
}
}
catch (Exception ex)
{
Log.Fatal(ex, "Problem reading the database.");
}
await Task.Delay(10000, stoppingToken);
}
}
}
UPDATE after get nice clue from Barr J
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
try
{
string connectionString = "User=SYSDBA;" +
"Password=masterkey;" +
"Database=test.DTB;" +
"DataSource=localhost;" +
"Port:3050";
FbConnection mConnection = new FbConnection(connectionString);
mConnection.Open();
FbTransaction mTransaction = mConnection.BeginTransaction();
string SQLCommandText = "select NAME, DATE from TABLE where NAME = 449 and ISSENT = 0";
FbCommand mCommand = new FbCommand(SQLCommandText, mConnection, mTransaction);
FbDataReader mReader = mCommand.ExecuteReader();
while (mReader.Read())
{
var values = new object[mReader.FieldCount];
{
mReader.GetValues(values);
var dbResult = values.Distinct().ToArray();
var dbResults = (string.Join("|", dbResult));
var result = new
{
ID = dbResult[0],
NAME = dbResult[1],
DATE = dbResult[2],
};
var jsonString = Newtonsoft.Json.JsonConvert.SerializeObject(result);
using FbConnection UpdateConnection = new(connectionString);
UpdateConnection.Open();
FbCommand writeCommand = new("update TABLE set ISSENT = #isSentValue where ID= #idValue", UpdateConnection);
writeCommand.Parameters.Add("#isSentValue", 1);
writeCommand.Parameters.Add("#idValue", dbResult[0]);
writeCommand.ExecuteNonQuery();
}
}
}
catch (Exception ex)
{
Log.Fatal(ex, "Problem reading the database.");
}
await Task.Delay(10000, stoppingToken);
}
}
}
Because you are using a worker and running async operation, I would not recommend to mix conditions that will complicate the already cumbersome debugging process.
I would add a column to your table as bit and use it as a flag column called IsSent
and then your query would look like:
select...... where name = X and IsSent = Y ...."
this way you will get only rows that are not sent.
you can otherwise query the row in the code and check if the ID was sent.
cleaner, better, easier.
Make your code maintainable.
I would suggest to implement MemCache or simple memory caching on your worker service, by caching your new data. Either the whole or just the key identifier field value. Then, you can pass that value the next time you query the Db where the Id > CachedId or Id != CachedId order by desc.
At any point, when you retrieve the data, you cache it/upsert your cache with that Id.
These are some examples/references for your knowledge
https://qawithexperts.com/article/c-sharp/in-memory-cache-c-explanation-with-example/302
I have CSV file. I read that CSV file and that csv file data converts to json then that json will upload in Iot hub.
That CSV file may contains 1400 to 5000 records. After inserting all records I will show the message like Data inserted successfully
Here when one single row got inserted that message showing popup. It's did not wait for all records getting insert.
private async Task DataSendstoAzureIotHub(string jsonserv, string deviceID)
{
DeviceClient s_deviceClient;
Console.WriteLine("upload time start" + DateTime.Now);
try
{
if (registryManager == null)
{
registryManager = RegistryManager.CreateFromConnectionString(s_connectionString01);
}
else
{
Device device = await registryManager.GetDeviceAsync(deviceID);
if (device != null)
{
string s_connectionStringNew = "*";
s_deviceClient = DeviceClient.CreateFromConnectionString(s_connectionStringNew, Microsoft.Azure.Devices.Client.TransportType.Amqp);
s_deviceClient.OpenAsync();
string messageString = "";
messageString = jsonserv;
var message = new Message(Encoding.UTF8.GetBytes(messageString));
message.Properties.Add("southnorth_msgid", "com.intel.wsn.sensorData");
Task task = s_deviceClient.SendEventAsync(message);
await task;
s_deviceClient.CloseAsync();
await Task.Delay(10);
}
else
{
device = await registryManager.AddDeviceAsync(new Device(deviceID));
string s_connectionStringNew = "*";
s_deviceClient = DeviceClient.CreateFromConnectionString(s_connectionStringNew, Microsoft.Azure.Devices.Client.TransportType.Amqp);
string messageString = "";
messageString = jsonserv;
var message = new Message(Encoding.UTF8.GetBytes(messageString));
message.Properties.Add("southnorth_msgid", "com.intel.wsn.sensorData");
Task task = s_deviceClient.SendEventAsync(message);
await Task.Delay(1000 * 10);
}
}
Console.WriteLine("File Upload time end " + DateTime.Now);
}
catch (Exception ex)
{
Log.Error(ex);
}
}
I had this piece of code that initially worked fine. However, After adding it to a class where I store my methods that are reused, it keeps failing. The exception that is caught states that the CancellationTokenSource has been Disposed. Can someone point me in the right direction?
I have tried creating a new client and Adding CancellationToken.None to the PutAsync() method from HTTPClient Class but it still fails with the CancellationTokenSource Disposed exception.
public async void AddProduct(Product product)
{
string storeId = "";
try
{
var storeData = JObject.Parse(Connect.Json).SelectToken("store").ToString();
var stores = JsonConvert.DeserializeObject<List<Store>>(storeData);
var store = stores[0];
storeId = store.Id;
store.Products.Add(product);
ProdInfo info = new Info();
foreach(Product p in store.Products)
{
info.AddedProducts = + p.Id;
}
var content = JsonConvert.SerializeObject(info);
using (Connect.Client)
using (var response = await Connect.Client.PutAsync(_url + "/stores/" + storeId, new StringContent(content)))
{
var cont = response.Content;
string result = await cont.ReadAsStringAsync();
if ((int)response.StatusCode == 200)
{
this.JobResult = result;
//this.JobResult = "Store has been successfully updated";
}
else
{
this.JobResult = result;
//this.JobResult = "Store was not updated!";
}
}
}
catch (Exception ex)
{
//this.JobResult = "Store has not been updated due to an error.";
this.JobResult = ex.Message;
}
}
I was able to solve this by simple removing 'using(Connect.Client)' from all of my methods. As #sellotape stated, They were disposing of the HttpClient before I was able to use it again. Thank you all for your contributions.
In my C# program I am using Entity Framework to synchronize a local SQL Server database with QuickBooks data. Getting the data from QuickBooks does not seem to have any issues. However I am running into a stumbling block when doing batch commits of entities.
Currently I am building up the DataContext with a configurable number of entities and then committing the entities in batch. So far the batch has not failed, but what if it does? My idea to combat this would be to iterate over the batch and submit each entity one at a time and then log the one(s) that is/are causing the commit failure.
However I do not see a way to do this with the data context since it appears to be an all or nothing matter when using SaveChanges(). Is there a way to handle what I am trying to accomplish, or should I be going about dealing with the failures in a completely different way?
Here is the code that I currently have, in case you want to take a look at it:
int itemsCount = 0;
int itemsSynced = 0;
int itemsFailed = 0;
ArrayList exceptions = new ArrayList();
int batchSliceCount = Properties.Settings.Default.SyncBatchSize; //Getting the max batch size from the settings
int index = 1; //Index used for keeping track of current batch size on data context
List<Customer> currentBatch = new List<Customer>(); // List to hold curent batch
db = new DataContext(DatabaseHelper.GetLocalDatabaseConnectionString());
foreach (var customer in QBResponse.customers)
{
itemsCount++;
try
{
string debugMsg = "Saving Customer with the Following Details....." + Environment.NewLine;
debugMsg += "ListId: " + customer.CustomerListId + Environment.NewLine;
debugMsg += "FullName: " + customer.FullName + Environment.NewLine;
int progressPercentage = (itemsCount * 100) / opResponse.retCount;
UpdateStatus(Enums.LogLevel.Debug, debugMsg, progressPercentage);
var dbCustomer = db.Customers.FirstOrDefault(x => x.CustomerListId == customer.CustomerListId);
if (dbCustomer == null)
{
// customer.CopyPropertiesFrom(customer, db);
Customer newCustomer = new Customer();
newCustomer.CopyCustomer(customer, db);
newCustomer.AddBy = Enums.OperationUser.SyncOps;
newCustomer.AddDateTime = DateTime.Now;
newCustomer.EditedBy = Enums.OperationUser.SyncOps;
newCustomer.EditedDateTime = DateTime.Now;
newCustomer.SyncStatus = true;
db.Customers.Add(newCustomer);
currentBatch.Add(newCustomer);
}
else
{
//dbCustomer.CopyPropertiesFrom(customer, db);
dbCustomer.CopyCustomer(customer, db);
dbCustomer.EditedBy = Enums.OperationUser.SyncOps;
dbCustomer.EditedDateTime = DateTime.Now;
dbCustomer.SyncStatus = true;
currentBatch.Add(dbCustomer);
}
try
{
if (index % batchSliceCount == 0 || index == opResponse.customers.Count()) //Time to submit the batch
{
UpdateStatus(Enums.LogLevel.Information, "Saving Batch of " + batchSliceCount + "Customers to Local Database");
db.SaveChanges();
itemsSynced += currentBatch.Count();
currentBatch = new List<Customer>();
db.Dispose();
db = new DataContext(DatabaseHelper.GetLocalDatabaseConnectionString());
}
}
catch (Exception ex)
{
string errorMsg = "Error occured submitting batch. Itterating and submitting one at a time. " + Environment.NewLine;
errorMsg += "Error Was: " + ex.GetBaseException().Message + Environment.NewLine + "Stack Trace: " + ex.GetBaseException().StackTrace;
UpdateStatus(Enums.LogLevel.Debug, errorMsg, progressPercentage);
//What to do here? Is there a way to properly iterate over the context and submit a change one at a time?
}
}
catch (Exception ex)
{
//Log exception and restart the data context
db.Dispose();
db = new DataContext(DatabaseHelper.GetLocalDatabaseConnectionString());
}
Thread.Sleep(Properties.Settings.Default.SynchronizationSleepTimer);
index++;
}
it depends on the exceptions you want to recover from...
If you are just looking for a way to retry if the connection was interrupted you could use a custom Execution Strategy based on DbExecutionStrategy that retries if specific errors occur as demonstrated in this CodeProject article.
I'm working on a Xamarin.Forms application in which I'm downloading some data from the server and showing them on the screen. This data is downloaded every 5-10 seconds in order to keep the application updated. My code for the data download looks like this:
public async Task<List<string>> RefreshProgressPageAsync()
{
var uri = new Uri(string.Format("http://someurladdress1"));
List<string> returnValue = new List<string>();
try
{
var response = await client.GetAsync(uri);
string allResult = string.Empty;
if (response.IsSuccessStatusCode)
{
var result = await response.Content.ReadAsStringAsync();
string[] valuePartsArray = result.Split(',');
string id = valuePartsArray[0].Substring(valuePartsArray[0].IndexOf(':') + 1);
string name = valuePartsArray[1].Substring(valuePartsArray[1].IndexOf(':') + 1);
string value = valuePartsArray[2].Substring(valuePartsArray[2].IndexOf(':') + 1);
returnValue.Add(string.Format("{0}|{1}|{2}", id, name, value));
}
uri = new Uri(string.Format("http://someurladdress2"));
response = await client.GetAsync(uri);
if (response.IsSuccessStatusCode)
{
var result = await response.Content.ReadAsStringAsync();
string[] valuePartsArray = result.Split(',');
string id = valuePartsArray[0].Substring(valuePartsArray[0].IndexOf(':') + 1);
string name = valuePartsArray[1].Substring(valuePartsArray[1].IndexOf(':') + 1);
string value = valuePartsArray[2].Substring(valuePartsArray[2].IndexOf(':') + 1);
returnValue.Add(string.Format("{0}|{1}|{2}", id, name, value));
}
return returnValue;
}
catch (Exception ex)
{
Debug.WriteLine(#" ERROR {0}", ex.Message);
return new List<string>();
}
}
Client is the HttpClient. There are two calls for the client because I want to download two different set of data in one RefreshProgressPageAsync call.
Now my problem with this is when second await client.GetAsync(uri) happens, I get a null point exception somewhere in the GetAsynch call, which is not caught by the try-catch block. I do have a workaround by not parsing the string and instead send it as is, but my question is what could cause this NullPointException?