I need to load data from SQLite to memory, and when the table contains more than 40k entries this process lasts for a few minutes. Data in db is encrypted so I decrypt it to load in memory.
Basically, I am:
loading all the data from the table
while it is reading I decrypt the info and add it to a dictionary
The code:
internal static void LoadInfo(Dictionary<int, InfoMemory> dic)
{
using (SQLiteConnection con = new SQLiteConnection(conString))
{
using (SQLiteCommand cmd = con.CreateCommand())
{
cmd.CommandText = String.Format("SELECT ID, Version, Code FROM Employee;");
int ID, version, code;
try
{
con.Open();
using (SQLiteDataReader rdr = cmd.ExecuteReader())
{
while (rdr.Read())
{
try { ID = Convert.ToInt32(Decrypt(rdr[0].ToString())); }
catch { ID = -1; }
try { version = Convert.ToInt32(Decrypt(rdr[1].ToString())); }
catch { version = -1; }
try { code = Convert.ToInt32(Decrypt(rdr[2].ToString())); }
catch { code = -1; }
if (ID != -1)
{
if (!dic.ContainsKey(ID)) dic.Add(ID, new InfoMemory(version, code));
}
}
rdr.Close();
}
}
catch (Exception ex) { Log.Error(ex.Message); }
finally { con.Close(); GC.Collect(); }
}
}
}
How can I make this process faster?
One way I try is by loading encrypted data to memory and decrypt when needed, but if I do that I consume more memory. Since I am working in mobile devices I want to maintain the memory consumption as low as possible.
One thing you can do is; instead of using try/catch:
int id, version, code;
if(!int.TryParse(rdr[0], out id))
{
id = -1;
}
if(!int.TryParse(rdr[1], out version))
{
version = -1;
}
if(!int.TryParse(rdr[2], out code))
{
code = -1;
}
Related
I am creating a simple inventory system using c#.
When I am generating the invoice number, the form is loaded but it doesn't show anything.
It is an auto-incremented invoice number; order is completed incrementally by 1.
For example, starting at E-0000001, after order we expect E-0000002. I don't understand why it is blank.
No error displayed. I tried to debug the code but I couldn't find what's wrong.
public void invoiceno()
{
try
{
string c;
sql = "SELECT MAX(invoid) FROM sales";
cmd = new SqlCommand(sql, con);
var maxInvId = cmd.ExecuteScalar() as string;
if (maxInvId == null)
{
label4.Text = "E-000001";
}
else
{
int intVal = int.Parse(maxInvId.Substring(2, 6));
intVal++;
label4.Text = String.Format("E-{0:000000}", intVal);
}
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
Console.Write(ex.StackTrace);
}
}
Let's extract a method - NextInvoiceId - we
Open connection
Execute query
Obtain next invoice number
Code:
private int NextInvoiceNumber() {
//TODO: put the right connection string here
using(var conn = new SqlConnection(ConnectionStringHere)) {
conn.Open();
string sql =
#"SELECT MAX(invoid)
FROM sales";
using (var cmd = new SqlCommand(sql, conn)) {
var raw = cmd.ExecuteScalar() as string;
return raw == null
? 1 // no invoces, we start from 1
: int.Parse(raw.Trim('e', 'E', '-')) + 1;
}
}
}
Then we can easily call it:
public void invoiceno() {
label4.Text = $"E-{NextInvoiceNumber():d6}";
}
Edit: You should not swallow exceptions:
try
{
...
}
// Don't do this!
catch (Exception ex) // All the exceptions will be caught and...
{
// printed on the Console...
// Which is invisible to you, since you develop Win Forms (WPF) application
Console.WriteLine(ex.ToString());
Console.Write(ex.StackTrace);
}
let system die and inform you that something got wrong
I want to display data from my remote sql-server in my android app. I am using web-service. I am able to connect and to insert but not to display with JSON.
This is the code from web-service
public DataTable RequestDetails(string request_name)
{
DataTable requestDetails = new DataTable();
requestDetails.Columns.Add(new DataColumn("Request ID", typeof(String)));
requestDetails.Columns.Add(new DataColumn("Date", typeof(String)));
if(dbConnection.State.ToString() == "Closed")
{
dbConnection.Open();
}
string query = "select ID_Requests,request_date from Requests where request_by='" + request_name + "'";
SqlCommand command = new SqlCommand(query, dbConnection);
SqlDataReader reader = command.ExecuteReader();
if (reader.HasRows)
{
while (reader.Read())
{
requestDetails.Rows.Add(reader["Request ID"], reader["Date"]);
}
}
reader.Close();
dbConnection.Close();
return requestDetails;
}
This is the android code:
protected class AsyncLoadDeptDetails extends
AsyncTask<DeptTable, JSONObject, ArrayList<DeptTable>> {
ArrayList<DeptTable> deptTable = null;
#Override
protected ArrayList<DeptTable> doInBackground(DeptTable... params) {
// TODO Auto-generated method stub
RestAPI api = new RestAPI();
try {
JSONObject jsonObj = api.RequestDetails(params[1].getName());
JSONParser parser = new JSONParser();
deptTable = parser.parseDepartment(jsonObj);
} catch (Exception e) {
// TODO Auto-generated catch block
Log.d("AsyncLoadDeptDetails", e.getMessage());
}
return deptTable;
}
#Override
protected void onPostExecute(ArrayList<DeptTable> result) {
// TODO Auto-generated method stub
for (int i = 0; i < result.size(); i++) {
data.add(result.get(i).getNo() + " " + result.get(i).getName());
}
adapter.notifyDataSetChanged();
Toast.makeText(context, "Loading Completed", Toast.LENGTH_SHORT).show();
}
}
And the JSONParser code:
public ArrayList<DeptTable> parseDepartment(JSONObject object)
{
ArrayList<DeptTable> arrayList=new ArrayList<DeptTable>();
try {
JSONArray jsonArray=object.getJSONArray("Value");
JSONObject jsonObj=null;
for(int i=0;i<jsonArray.length();i++)
{
jsonObj=jsonArray.getJSONObject(i);
arrayList.add(new DeptTable(jsonObj.getInt("Request ID"), jsonObj.getString("Date")));
}
} catch (JSONException e) {
// TODO Auto-generated catch block
Log.d("JSONParser => parseDepartment", e.getMessage());
}
return arrayList;
}
I would use Fiddler or something similar and have a look at the data being returned. I've always had trouble returning a .NET Datatable from a service, so if it were me I would convert the datatable to JSON before sending. This has been discussed on several posts before, but here's a great answer:
https://stackoverflow.com/a/17398078/3299157
I am creating a (spatial database file)sdf file from the data fetched from Geodatabase file. I am not storing the data anywhere still memory and cpu usage of program is very high.I am using the file having huge database how can I increase performance? I have also tried to use finalize and dispose method.
static void GetFeatureData(Geodatabase geodatabase, string tablename, string sdffilepath)
{
string query = "select * from " + tablename;
int no = 1;
foreach (Row row in geodatabase.ExecuteSQL(query))
{
IConnection con = OpenFDOSDFConnection(sdffilepath);
IInsert insertCommand = (IInsert)con.CreateCommand(OSGeo.FDO.Commands.CommandType.CommandType_Insert);
insertCommand.SetFeatureClassName(tablename);
for (int nFieldNumber = 0; nFieldNumber < row.FieldInformation.Count; nFieldNumber++)
{
string fieldName = row.FieldInformation.GetFieldName(nFieldNumber);
switch (row.FieldInformation.GetFieldType(nFieldNumber))
{
case FieldType.SmallInteger:
if (row.IsNull(fieldName))
{
insertCommand.PropertyValues.Add(new PropertyValue(fieldName, null));
}
else
{
insertCommand.PropertyValues.Add(new PropertyValue(fieldName, new Int32Value(row.GetShort(fieldName))));
}
break;
//All other datatypes
case FieldType.Geometry:
if (!row.IsNull(fieldName))
{
switch (row.GetGeometry().geometryType.ToString())
{
case "Point": insertCommand.PropertyValues.Add(new PropertyValue("Geometry", geometryValue));
break;
//All other Geometry cases
}
}
}
insertCommand.Execute();
insertCommand.Dispose();
con.Dispose();
Console.WriteLine(no++);
}
}
}
I have a database that may be on the network drive.
There are two things that I want to achieve:
When the first user connects to it in read-only mode (he doesn't
have a read-write access to the location, or the database is
read-only), other users must use the read-only connection also (even
if they have RW access).
When the first user connects to it in RW mode, others can not
connect to the database at all.
I'm using SQLite, and the concurrency should not be the problem, as the database should never be used by more than 10 people at the same time.
UPDATE: This is a sample that I'm trying to make work, so I could implement it in the program itself. Almost everything can be changed.
UPDATE: Now when I finally understood what #CL. was telling me, I made it work and this is the updated code.
using System.Diagnostics;
using System.Linq;
using System.IO;
using DbSample.Domain;
using DbSample.Infrastructure;
using NHibernate.Linq;
using NHibernate.Util;
namespace DbSample.Console
{
class Program
{
static void Main(string[] args)
{
IDatabaseContext databaseContext = null;
databaseContext = new SqliteDatabaseContext(args[1]);
var connection = LockDB(args[1]);
if (connection == null) return;
var sessionFactory = databaseContext.CreateSessionFactory();
if (sessionFactory != null)
{
int insertCount = 0;
while (true)
{
try
{
using (var session = sessionFactory.OpenSession(connection))
{
string result;
session.FlushMode = NHibernate.FlushMode.Never;
var command = session.Connection.CreateCommand();
command.CommandText = "PRAGMA locking_mode=EXCLUSIVE";
command.ExecuteNonQuery();
using (var transaction = session.BeginTransaction(ReadCommited))
{
bool update = false;
bool delete = false;
bool read = false;
bool readall = false;
int op = 0;
System.Console.Write("\nMenu of the day:\n1: update\n2: delete\n3: read\n4: read all\n0: EXIT\n\nYour choice: ");
op = System.Convert.ToInt32(System.Console.ReadLine());
if (op == 1)
update = true;
else if (op == 2)
delete = true;
else if (op == 3)
read = true;
else if (op == 4)
readall = true;
else if (op == 0)
break;
else System.Console.WriteLine("Are you retarded? Can't you read?");
if (delete)
{
System.Console.Write("Enter the ID of the object to delete: ");
var objectToRemove = session.Get<MyObject>(System.Convert.ToInt32(System.Console.ReadLine()));
if (!(objectToRemove == null))
{
session.Delete(objectToRemove);
System.Console.WriteLine("Deleted {0}, ID: {1}", objectToRemove.MyName, objectToRemove.Id);
deleteCount++;
}
else
System.Console.WriteLine("\nObject not present in the database!\n");
}
else if (update)
{
System.Console.Write("How many objects to add/update? ");
int number = System.Convert.ToInt32(System.Console.ReadLine());
number += insertCount;
for (; insertCount < number; insertCount++)
{
var myObject = session.Get<MyObject>(insertCount + 1);
if (myObject == null)
{
myObject = new MyObject
{
MtName = "Object" + insertCount,
IdLegacy = 0,
};
session.Save(myObject);
System.Console.WriteLine("Added {0}, ID: {1}", myObject.MyName, myObject.Id);
}
else
{
session.Update(myObject);
System.Console.WriteLine("Updated {0}, ID: {1}", myObject.MyName, myObject.Id);
}
}
}
else if (read)
{
System.Console.Write("Enter the ID of the object to read: ");
var objectToRead = session.Get<MyObject>(System.Convert.ToInt32(System.Console.ReadLine()));
if (!(objectToRead == null))
System.Console.WriteLine("Got {0}, ID: {1}", objectToRead.MyName, objectToRead.Id);
else
System.Console.WriteLine("\nObject not present in the database!\n");
}
else if (readall)
{
System.Console.Write("How many objects to read? ");
int number = System.Convert.ToInt32(System.Console.ReadLine());
for (int i = 0; i < number; i++)
{
var objectToRead = session.Get<MyObject>(i + 1);
if (!(objectToRead == null))
System.Console.WriteLine("Got {0}, ID: {1}", objectToRead.MyName, objectToRead.Id);
else
System.Console.WriteLine("\nObject not present in the database! ID: {0}\n", i + 1);
}
}
update = false;
delete = false;
read = false;
readall = false;
transaction.Commit();
}
}
}
catch (System.Exception e)
{
throw e;
}
}
sessionFactory.Close();
}
}
private static SQLiteConnection LockDbNew(string database)
{
var fi = new FileInfo(database);
if (!fi.Exists)
return null;
var builder = new SQLiteConnectionStringBuilder { DefaultTimeout = 1, DataSource = fi.FullName, Version = 3 };
var connectionStr = builder.ToString();
var connection = new SQLiteConnection(connectionStr) { DefaultTimeout = 1 };
var cmd = new SQLiteCommand(connection);
connection.Open();
// try to get an exclusive lock on the database
try
{
cmd.CommandText = "PRAGMA locking_mode = EXCLUSIVE; BEGIN EXCLUSIVE; COMMIT;";
cmd.ExecuteNonQuery();
}
// if we can't get the exclusive lock, it could mean 3 things
// 1: someone else has locked the database
// 2: we don't have a write acces to the database location
// 3: database itself is a read-only file
// So, we try to connect as read-only
catch (Exception)
{
// we try to set the SHARED lock
try
{
// first we clear the locks
cmd.CommandText = "PRAGMA locking_mode = NORMAL";
cmd.ExecuteNonQuery();
cmd.CommandText = "SELECT COUNT(*) FROM MyObject";
cmd.ExecuteNonQuery();
// then set the SHARED lock on the database
cmd.CommandText = "PRAGMA locking_mode = EXCLUSIVE";
cmd.ExecuteNonQuery();
cmd.CommandText = "SELECT COUNT(*) FROM MyObject";
cmd.ExecuteNonQuery();
readOnly = true;
}
catch (Exception)
{
// if we can't set EXCLUSIVE nor SHARED lock, someone else has opened the DB in read-write mode and we can't connect at all
connection.Close();
return null;
}
}
return connection;
}
}
}
Set PRAGMA locking_mode=EXCLUSIVE to prevent SQLite from releasing its locks after a transaction ends.
I don't know if it can be done within db but in application;
You can set a global variable (not sure if it's a web or desktop app) to check if anyone connected and he has a write access or not.
After that you can check the other client's state.
I am attempting to read a MySQL database from my C# project using the MySQL drivers for .net off the MySQL site.
Though I did a bit of research on this (including this), I am still flummoxed why this is happening. I later ran a spike and I still get the same error. (Prior to running this I populated the database with some default values.) Here's the spike code in toto.
class Program {
static void Main (string[] args) {
Console.WriteLine (GetUserAge ("john")); // o/p's -1
}
static int GetUserAge (string username) {
string sql = "select age from users where name=#username";
int val = -1;
try {
using (MySqlConnection cnn = GetConnectionForReading ()) {
cnn.Open ();
MySqlCommand myCommand = new MySqlCommand (sql, cnn);
myCommand.Parameters.AddWithValue ("#username", username);
using (MySqlDataReader reader = myCommand.ExecuteReader ()) {
DataTable dt = new DataTable ();
dt.Load (reader);
if (reader.Read ()) {
val = reader.GetInt32 (0);
}
}
}
} catch (Exception ex) {
Console.WriteLine (ex.Message);
} finally {
}
return val;
}
private static MySqlConnection GetConnectionForReading () {
string conStr = "Data Source=localhost;Database=MyTestDB;User ID=testuser;Password=password";
return new MySqlConnection (conStr);
}
}
The code above gives me the exception: "Invalid attempt to Read when reader is closed."
Later I modified the if-condition like so:
if (reader.HasRows && reader.Read ()) {
val = reader.GetInt32 (0);
}
And now the o/p is -1. (The data's in there in the table.) If for some reason the result set had zero rows, the reader should not have got into the if-block in the first place. I mean, the whole point of the Read() method is to check if there are any rows in the result set in the first place.
At my wit's end here... just cannot figure out where I'm going wrong.
Thank you for your help! :)
I think using DataTable.Load will "consume" the reader and, at the very least, position it at the end. It may even account for the closed connection (but I'm just guessing here). What if you remove that line? I don't think it makes any sense here.