We Keep Coding

Is filling different DataTables of the same DataSet a thread-safe operation?

I have many methods filling different DataTables in the same DataSet:
public override void FillMethod1(MyDataSet ds)
{
try
{
using (SqlDataAdapter adapter = new SqlDataAdapter(query, connectionObj))
{
using (SqlCommandBuilder adapterSCB = new SqlCommandBuilder(adapter))
{
adapter.Fill(ds.MyTable);
}
}
}
catch (Exception e)
{
//Exception handling
}
}
public override void FillMethod2(MyDataSet ds)
{
try
{
using (SqlDataAdapter adapter = new SqlDataAdapter(query, connectionObj))
{
using (SqlCommandBuilder adapterSCB = new SqlCommandBuilder(adapter))
{
adapter.Fill(ds.MyTable2);
}
}
}
catch (Exception e)
{
//Exception handling
}
}
[....]
using(MyDataSet ds = new MyDataSet())
{
FillMethod1(ds);
FillMethod2(ds);
[...]
}
And I'd like to parallelize these operation using Task:
using(MyDataSet ds = new MyDataSet())
{
Task fill1 = Task.Run(() => FillMethod1(ds));
Task fill2 = Task.Run(() => FillMethod2(ds));
[...]
Task.WaitAll(fill1, fill2, [...]);
}
After some research I found that DataSet is not thread-safe, but is it safe when working on different DataTable?

I created a class handling parallel fills on the same DataSet. Each fill has its own DataSet and it will be merged into the original one at the end of the execution.
To handle data table relations, it is possible to create a sequential list of fills that will be run in parallel with other fills. The merge method will wait for all operations to finish and merge results into the original DataSet (passed in the constructor)
public class FillParallelizer<TDataSet> : IDisposable
where TDataSet : DataSet, new()
{
private bool _seqFillStarted = false;
private IList<QueryObject> _queryObjList = new List<QueryObject>();
private Queue<FillOperation> _seqOperationBufferQueue = new Queue<FillOperation>();
public DataSet MasterDS { get; }
public FillParallelizer(DataSet masterDS)
{
MasterDS = masterDS ?? throw new ArgumentNullException(nameof(masterDS));
}
public FillParallelizer<TDataSet> ParallelFill(Action<TDataSet> methodAction)
{
if (methodAction == null) throw new ArgumentNullException(nameof(methodAction));
QueryObject queryObj = new QueryObject(methodAction);
_queryObjList.Add(queryObj);
return this;
}
public void Merge()
{
var taskList = _queryObjList.Select(x => x.Task).ToArray();
Task.WaitAll(taskList);
foreach (var item in _queryObjList)
{
MasterDS.Merge(item.TempDS);
}
}
public FillParallelizer<TDataSet> SequentialFill(Action<TDataSet> action)
{
if(!_seqFillStarted)
{
_seqFillStarted = true;
}
if (!_seqFillStarted) throw new ArgumentException("Sequential fill not started");
_seqOperationBufferQueue.Enqueue(new FillOperation(action));
return this;
}
public FillParallelizer<TDataSet> RunSequentialFill()
{
_seqFillStarted = false;
QueryObject queryObj = new QueryObject(new Queue<FillOperation>(_seqOperationBufferQueue));
_seqOperationBufferQueue.Clear();
_queryObjList.Add(queryObj);
return this;
}
public void Dispose()
{
if (_queryObjList == null || !_queryObjList.Any())
{
return;
}
foreach (var item in _queryObjList) item.Dispose();
}
class FillOperation
{
public Action<TDataSet> Action { get; }
public FillOperation(Action<TDataSet> action)
{
Action = action ?? throw new ArgumentNullException(nameof(action));
}
public void Run(TDataSet tempDS)
{
if (tempDS == null) throw new ArgumentNullException(nameof(tempDS));
Action(tempDS);
}
}
class QueryObject : IDisposable
{
public TDataSet TempDS { get; }
public Queue<FillOperation> FillOperationList { get; }
public Task Task { get; }
public QueryObject(Queue<FillOperation> fillOperationList)
{
TempDS = new TDataSet();
FillOperationList = fillOperationList ?? throw new ArgumentNullException(nameof(fillOperationList));
Task =
Task
.Run
(() =>
{
foreach (FillOperation op in FillOperationList) op.Run(TempDS);
}
);
}
public QueryObject(Action<TDataSet> action)
: this(new Queue<FillOperation>(new FillOperation[] { new FillOperation(action) }))
{
}
public void Dispose()
{
Task.Dispose();
}
}
}
An example of usage:
using(MyDataSet myDS = new MyDataSet())
using (FillParallelizer<MyDataSet> parallelizer = new FillParallelizer<MyDataSet>(myDS))
{
parallelizer
.ParallelFill(ds => MyDAO.FillTable1(ds))
.ParallelFill(ds => MyDAO.FillTable2(ds))
.SequentialFill(ds => MyDAO.FillTable3(ds))
.SequentialFill(ds => MyDAO.FillTable4(ds))
.SequentialFill(ds => MyDAO.FillTable5(ds))
.RunSequentialFill()
.ParallelFill(ds => MyDAO.FillTable6(ds))
.ParallelFill(ds => MyDAO.FillTable7(ds))
.Merge();
//All fills completed
}
Fill methods 3, 4, 5 are done sequentially but in parallel with other fills, to handle relations between tables

Index out of range with less than 6 rows of data

Small brief of what i need and what i currently have
I connect to a database and get my data from it and i get ( Name , LongNumber) and basically i have an event (action) that fires when the event occurs (this action gives me a LongNumber aswell).
I need to compare the LongNumbers between the one i got from the event (action) and the one from my database, and if they are similar i take the name and use it.
for example Hello, Alex ( alex is taken from the database)
Issue
I get Index out of range, and using my logic all texts will change what i'm trying to achieve is to change the text to the name of the person that got the Long number the same as the longNumber from the event
Code: Gettings Data from the database
using UnityEngine;
using System.Collections;
using MySql.Data.MySqlClient;
using System;
using System.Linq;
using System.Collections.Generic;
public class MySqlTestScript : MonoBehaviour
{
private static MySqlTestScript _instnace;
public static MySqlTestScript sharedInstance()
{
return _instnace;
}
string row = "";
public string host = "*****";
public string database = "*******";
public string usrename= "*******";
public string password = "******";
public List<Data> userData = new List<Data>();
Data data;
void Awake()
{
_instnace = this;
}
// Use this for initialization
public void Start()
{
GetDataFromDatabase();
}
public string GetDataFromDatabase()
{
string myConnectionString = "Server="+host+";Database="+database+";Uid="+usrename+ ";Pwd="+password+";";
MySqlConnection connection = new MySqlConnection(myConnectionString);
MySqlCommand command = connection.CreateCommand();
command.CommandText = "SELECT * FROM Users";
MySqlDataReader Reader;
try
{
connection.Open();
Reader = command.ExecuteReader();
while (Reader.Read())
{
for (int i = 0; i < Reader.FieldCount; i++)
{
//rfid_tags.Add (Reader.GetString("UserName"));
//rfid_tags.Add(Reader.GetString("RFID_Tag"));
data = new Data(Reader.GetString("UserName"), Reader.GetString("RFID_Tag"));
userData.Add(data);
// ws.DomainUrl = reader.GetString("DomainUrl");
// rfid_tags.Add(Reader.GetValue(i).ToString() + ",");
// row += Reader.GetValue(i).ToString() + ", ";
}
Debug.Log(row);
}
}
catch (Exception x)
{
Debug.Log(x.Message);
return x.Message;
}
connection.Close();
return row;
}
}
public class Data {
public string username { get; set; }
public string rfid { get; set; }
public Data(string _name, string _rfid)
{
username = _name;
rfid = _rfid;
}
public void SetUserName(string _userName) { username = _userName; }
public void SetRFID(string _rfid) { rfid = _rfid; }
}
Code for Comparing the values from the event(action) and showing the text
void OnTagsReported(ImpinjReader sender, TagReport report)
{
Debug.Log("OnTagsReported");
// This event handler is called asynchronously
// when tag reports are available.
// Loop through each tag in the report
// and print the data.
foreach (Tag tag in report)
{
Debug.Log(tag.Epc);
// Debug.Log(MySqlTestScript.sharedInstance().rfid_tags[0]);
Debug.Log("STEP ONE");
for (int i = 0; i < MySqlTestScript.sharedInstance().userData.Count; i++)
{
Debug.Log("STEP TWO");
if (tag.Epc.ToString().Trim() == MySqlTestScript.sharedInstance().userData[i].rfid)
{
Debug.Log("STEP THREE");
// TODO References the Name
Loom.QueueOnMainThread(() => {
namesTxt[i].text = MySqlTestScript.sharedInstance().userData[i].username;
});
}
}
As you can see in the Event script it's so unflexible and it gives index out of range if my database has less than 6 rows. I need to make it more friendly and generic
Any help would be appreciated and i hope my question is clear Thank you :)!

This should make more sense:
Database:
using UnityEngine;
using System.Collections;
using MySql.Data.MySqlClient;
using System;
using System.Linq;
using System.Collections.Generic;
public class MySqlTestScript : MonoBehaviour
{
private static MySqlTestScript _instnace;
public static MySqlTestScript sharedInstance()
{
return _instnace;
}
string row = "";
public string host = "*****";
public string database = "*******";
public string usrename= "*******";
public string password = "******";
public List<AppUser> users = new List<AppUser>();
void Awake()
{
_instnace = this;
}
// Use this for initialization
public void Start()
{
GetDataFromDatabase();
}
public string GetDataFromDatabase()
{
string myConnectionString = "Server=" + host + ";Database=" + database + ";Uid=" + usrename + ";Pwd=" + password + ";";
MySqlConnection connection = new MySqlConnection(myConnectionString);
MySqlCommand command = connection.CreateCommand();
command.CommandText = "SELECT * FROM users";
MySqlDataReader Reader;
try
{
connection.Open();
Reader = command.ExecuteReader();
while (Reader.Read())
{
users.Add(new AppUser() {
username = Reader.GetString("UserName").Trim(),
rfid = Reader.GetString("longNumbers ").Trim()
});
}
}
catch (Exception x)
{
Debug.Log(x.Message);
return x.Message;
}
connection.Close();
return row;
}
}
Data object:
public Class AppUser
{
public string rfid { get; set; }
public string username { get; set; }
}
Event/data comparison:
void OnTagsReported(ImpinjReader sender, TagReport report)
{
Debug.Log("OnTagsReported");
// This event handler is called asynchronously
// when tag reports are available.
// Loop through each tag in the report
// and print the data.
foreach (Tag tag in report)
{
Debug.Log(tag.Epc);
List<AppUser> appUsers = MySqlTestScript.sharedInstance().users;
int numUsers = appUsers.Count;
Debug.Log(numUsers);
for (int i = 0; i < numUsers; i++)
{
if (tag.Epc.ToString().Trim() == appUsers[i].rfid)
{
// TODO References the Name
Loom.QueueOnMainThread(() => {
if (i < namesTxt.Count) namesTxt[i].Text = appUsers[i].username; //assumes textnames is a "List" of textboxes and is already populated with instances of text1, text2 text3 etc. The if is just to guard against there being more rows in the DB than textboxes
});
}
}
}
}

One way to get around this is to wrap each of your cases in another if statement to check if the database has that many rows
if (MySqlTestScript.sharedInstance().longNumbers.Length > 1) {
if (tag.Epc.ToString().Trim() == MySqlTestScript.sharedInstance().longNumbers[0].ToString()) {
if(MySqlTestScript.sharedInstance().userNames[0] != null)
txt1.text = "Hello "+ MySqlTestScript.sharedInstance().userNames[0];
}
}
}
else {
txt1.text = "";
}
This will avoid your index out of range exception. In each wrapping case you will need to increment the comparison to be 1, 2, 3, etc.
It is not the prettiest solution but will avoid the exception. Another option would be to wrap in a try-catch block. Again not the prettiest but would accomplish the task.

Migrate data using FluentMigrator

I use FluentMigrator in my project.
And now I Add new column in table and how I can update data in this column by SQL query?
public override void Up()
{
Alter.Table("Images").AddColumn("Item_id").AsInt32().Nullable();
//do something like "Update Images img set img.Item_id=(Select i.Id
//from Items i where i.Image=img.Id)"
}
public override void Down()
{
Delete.Column("Item_id").FromTable("Images");
}

You can either use Insert.IntoTable:
Insert.IntoTable("Users").Row(new { Username = "CURRENT_USER" });
or Execute.WithConnectionto insert data using FluentMigrator

Hi I believe You can make it simpler for Your case, but I had a more complex issue. I added a new column of varbinary type, where I needed to keep a complex object and I needed to prepare it in c# code. So I couldn't just execute UPDATE combined with SELECT command.
I solved that by adding a static class where I execute SQL SELECT command, then I use retrieved data during migration to prepare UPDATE commands.
The important clue is to use ConnectionString which is accessible directly from FluentMigrator.Migration class and execute own SQL commands to get data, then process it in C# and then simply use Update.Table method of FluentMigrator.
Here is my code (simplified):
DatabaseHelper class:
/// <summary>
/// Helper class for executing SQL on a Database defined by given ConnectionString
/// </summary>
public static class DatabaseHelper
{
/// <summary>
/// Run SQL SELECT on a Database defined by given ConnectionString
/// </summary>
/// <param name="connectionString">Connection string</param>
/// <param name="sql">Full SQL SELECT command to be executed</param>
/// <returns>List of rows containing all columns as string[] array</returns>
public static string[][] SelectList(string connectionString, string sql)
{
using (SqlConnection sqlConnection = new SqlConnection($"{connectionString}"))
{
try
{
sqlConnection.Open();
SqlCommand command = new SqlCommand(sql, sqlConnection);
SqlDataReader reader = command.ExecuteReader();
if (!reader.HasRows)
return null;
List<string[]> rowsList = new List<string[]>();
//save all rows (including all columns) from the response to the list
while (reader.Read())
{
//every row has n columns
var row = new string[reader.FieldCount];
//fill every column
for (int i = 0; i < reader.FieldCount; i++)
{
row[i] = reader[i].ToString();
}
//add row to the list
rowsList.Add(row);
}
reader.Close();
sqlConnection.Close();
return rowsList.ToArray();
}
catch (Exception e)
{
if (sqlConnection.State == ConnectionState.Open)
sqlConnection.Close();
Console.WriteLine(e);
throw;
}
}
}
}
Migration class:
public class Migration123 : Migration
{
public override void Up()
{
Create.Column("NEW_COLUMN").OnTable("TABLE_NAME").AsCustom("NVARCHAR(255)").Nullable()
//migrate data from OLD_COLUMN to NEW_COLUMN
MigrateData();
...
}
private void MigrateData()
{
string[][] rowsArray = DatabaseHelper.SelectList(ConnectionString,"SELECT [ID],[OLD_COLUMN] FROM [TABLE_NAME]");
//if nothing to migrate then exit
if (rowsArray == null)
return;
foreach (var row in rowsArray)
{
//prepare a value which will be inserted into a new column, basing on old columns value
var someNewValueForNewColumn = row[1] + " (modified)";
//insert a value into a new column
Update.Table("TABLE_NAME").Set(new
{
NEW_COLUMN = someNewValueForNewColumn
}).Where(new
{
ID = row[0]
});
}
}
public override void Down()
{
...
}
}
Hope it helps somebody!

public class <MigrationClassName> : Migration
{
private readonly ScriptResourceManager _sourceManager;
public <MigrationClassName>(
ScriptResourceManager sourceManager)
{
_sourceManager = sourceManager;
}
public override void Up()
{
MigrateData();
}
private void MigrateData()
{
var script = _sourceManager
.Read("<FileName>.sql");
Execute.Sql(script);
}
}
using System.IO;
public class ScriptResourceManager
{
public string Read(string name)
{
var assembly = typeof(ScriptResourceManager).Assembly;
var resourcesBasePath = typeof(ScriptResourceManager).Namespace;
var resourcePath = $"{resourcesBasePath}.{name}";
using var stream = assembly.GetManifestResourceStream(resourcePath);
using var reader = new StreamReader(stream);
return reader.ReadToEnd();
}
}
SQL FILE
UPDATE table_name
SET column1 = value1, column2 = value2, ...
WHERE condition;

Random errors with my connection in c# with sql server

I have created this class for work with my DB connection.
Applications works fine but one or two times in a day i get this errors.
Invalid operation, the connection is completed. when running query: (different queries)
or other times I received this error.
Value Timeout expired. The timeout period expired before completion of the operation or the server is not responding. This error has occurred while attempting to connect to the server Principal.
and rare errors:
Error at the transport level when receiving results from the server. (provider: Session Provider, error:? 19 - You can not use the connection nf sica) when running query
Thread aborted. when executing query:
ExecuteNonQuery requires an open and available Connection. The current state of the connection is closed. when running query
My db Class(resumed):
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data.SqlClient;
using System.Data;
using System.Net.Mail;
using System.Web;
using System.IO;
namespace BBDD
{
public class clsBBDD
{
private string strConexion;
private int intTransaciones;
private SqlTransaction Transaccion;
private SqlConnection Conexion;
public clsBBDD(string ParamConexion)
{
strConexion = ParamConexion;
intTransaciones = 0;
}
public int Execute(string Sql)
{
return ExecutePrivado(Sql);
}
public int Execute(string Sql, bool Force)
{
return ExecutePrivado(Sql, Force);
}
private int ExecutePrivado(string Sql,bool Force = false)
{
int result = 0;
SqlCommand sentencia;
try
{
if (!Force && !Sql.ToUpper().Contains("WHERE") && !Sql.ToUpper().Contains("INSERT"))
{
throw new Exception("Sentencia Update o Delete sin WHERE");
}
if (intTransaciones > 0)
{
sentencia = new SqlCommand(Sql, Conexion, Transaccion);
}
else
{
abrirConexion();
sentencia = new SqlCommand(Sql, Conexion);
}
sentencia.CommandTimeout = 0;
result = sentencia.ExecuteNonQuery();
cerrarConexion();
}
catch (Exception e)
{
SendMailError(Sql, e);
result = 0;
}
return result;
}
public bool AbrirTrans()
{
try
{
intTransaciones += 1;
if (intTransaciones == 1)
{
abrirConexion();
Transaccion = Conexion.BeginTransaction();
}
}
catch (Exception e)
{
SendMailError("Error al abrir transacción", e);
return false;
}
return true;
}
public bool CerrarTrans(bool CommitTrans)
{
try
{
intTransaciones -= 1;
if (intTransaciones == 0)
{
if (CommitTrans)
{
Transaccion.Commit();
}
else
{
Transaccion.Rollback();
}
cerrarConexion();
}
}
catch (Exception e)
{
SendMailError("Error al cerrar transacción", e);
return false;
}
return true;
}
public object GetValue(string Sql)
{
object resultado = null;
try
{
SqlCommand sentencia = getCommand(Sql);
SqlDataReader reader = sentencia.ExecuteReader();
while (reader.Read())
{
resultado = reader[0];
}
reader.Close();
cerrarConexion();
}
catch (Exception e)
{
SendMailError(Sql, e);
resultado = null;
}
return resultado;
}
public DataRow GetValuesInRow(string sql)
{
DataRowCollection dsr;
DataRow result;
try
{
dsr = GetDSr(sql);
if (dsr.Count > 0)
{
result = dsr[0];
}
else
{
result = null;
}
}
catch (Exception e)
{
SendMailError(sql,e);
result = null;
}
return result;
}
public DataSet GetDS(string Sql)
{
DataSet result = new DataSet();
SqlDataAdapter adapter;
try
{
SqlCommand command = getCommand(Sql);
adapter = new SqlDataAdapter(command);
adapter.Fill(result);
cerrarConexion();
}
catch (Exception e)
{
SendMailError(Sql, e);
result = null;
}
return result;
}
public DataRowCollection GetDSr(string sql)
{
DataRowCollection result;
try
{
DataSet ds = GetDS(sql);
result = ds.Tables[0].Rows;
}
catch (Exception e)
{
SendMailError(sql, e);
result = null;
}
return result;
}
private void abrirConexion()
{
Conexion = new SqlConnection(strConexion);
if (Conexion.State == ConnectionState.Closed)
{
Conexion.Open();
}
}
private void cerrarConexion(bool Force = false)
{
try
{
if (intTransaciones == 0 && Conexion != null)
{
Conexion.Close();
}
}
catch
{
}
}
private SqlCommand getCommand(string Sql)
{
SqlCommand result;
if (intTransaciones == 0)
{
abrirConexion();
}
result = Conexion.CreateCommand();
if (intTransaciones > 0)
{
result.Transaction = Transaccion;
}
result.CommandText = Sql;
result.CommandTimeout = 0;
return result;
}
}
}
How I Initialize class:
public partial class frmBase : System.Web.UI.Page
{
protected clsBBDD BD;
protected void Page_Init(object sender, EventArgs e)
{
BD = new clsBBDD(ConfigurationManager.ConnectionStrings["ConnectionString"].ConnectionString);
clsBase.BD = BD;
}
}
All my webforms heritage from frmBase and all clases heritatge from clsBase
For use BD I call like this:
string sql;
DataRowCollection DSTrabajos;
sql = "UPDATE tabletest SET validacion = '" + ValidacionText + "', ModificadoPor = '" + Username + "' WHERE referencia = " + ReferenciaID;
BD.Execute(sql);
sql = "SELECT orderID FROM TrabajosINBOX where referencia = " + ReferenciaID;
DSTrabajos = BD.GetDSr(sql);
foreach (DataRow r in DSTrabajos)
{
//more code inside
}
My connection string params
Data Source=ServerIP;Initial Catalog=BBDDNAME;User ID=USER;Password=***********;Max Pool Size=500;MultipleActiveResultSets=true

Refresh Application Automatically When Data changed in SQL Server

I use SQL Server and I have 3 Application servers. When a table in my database have changed I need to those application servers refresh there local cached data. I use a trigger to known change and send a message via Service broker queue. Then I create a stored procedure and assign it to activate stored procedure of my queue, In this stored procedure I receive message, but I don't know How should I call refresh method in my application.

I had similar issue but this code resolved the issue :
public class QueryNotification
{
public DataSet DataToWatch { get; set; }
public SqlConnection Connection { get; set; }
public SqlCommand Command { get; set; }
public string GetSQL()
{
return "SELECT * From YourTable";
}
public string GetConnection()
{
return ConfigurationManager.ConnectionStrings["ConnectionString"].ConnectionString;
}
public bool CanRequestNotifications()
{
try
{
var perm = new SqlClientPermission(PermissionState.Unrestricted);
perm.Demand();
return true;
}
catch
{
return false;
}
}
public void GetData()
{
DataToWatch.Clear();
Command.Notification = null;
var dependency = new SqlDependency(Command);
dependency.OnChange += dependency_OnChange;
using (var adapter = new SqlDataAdapter(Command))
{
adapter.Fill(DataToWatch, "YourTableName");
}
}
private void dependency_OnChange(object sender, SqlNotificationEventArgs e)
{
var i = (ISynchronizeInvoke)sender;
if (i.InvokeRequired)
{
var tempDelegate = new OnChangeEventHandler(dependency_OnChange);
object[] args = { sender, e };
i.BeginInvoke(tempDelegate, args);
return;
}
var dependency = (SqlDependency)sender;
dependency.OnChange -= dependency_OnChange;
GetData();
}
}
Update:
Check for permission:
public bool CanRequestNotifications()
{
try
{
var perm = new SqlClientPermission(PermissionState.Unrestricted);
perm.Demand();
return true;
}
catch
{
return false;
}
}
For Instance in your window load:
if (!_queryNotification.CanRequestNotifications())
{
MessageBox.Show("ERROR:Cannot Connect To Database");
}
SqlDependency.Stop(_queryNotification.GetConnection());
SqlDependency.Start(_queryNotification.GetConnection());
if (_queryNotification.Connection == null)
{
_queryNotification.Connection = new SqlConnection(_queryNotification.GetConnection());
}
if (_queryNotification.Command == null)
{
_queryNotification.Command = new SqlCommand(_queryNotification.GetSQL(),
_queryNotification.Connection);
}
if (_queryNotification.DataToWatch == null)
{
_queryNotification.DataToWatch = new DataSet();
}
GetData();

You should look at using the SqlDependency class.
More information at: http://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqldependency(v=vs.110).aspx

I can suggest you to try solving the problem using TCP. Each app listens to a port and when another app updates the db it sends a message to the other apps saying they need to refresh.
Hope that was a good idea.