We Keep Coding

Item enqueue in concurrent queue is not adding

I am working on an application in which I am getting orders from an third party app. The application is written on windows form so I am using service stack to add routes in my application.
I have three classes. One contains endpoint
public class Service : ServiceStack.Service
{
Repository _repository;
public OrderService()
{
_repository = Repository.GetInstance();
}
[Authenticate]
public void Post(Order order)
{
if (order != null)
{
_repository.AddItem(order);
}
}
}
The second class is processing the orders and this class is a singleton class.
public sealed class Repository
{
private static object _myLock = new object();
private static Repository _mySingleton = null;
private ConcurrentQueue<Order> _queue;
public static bool orderCheck = true;
private Repository() {
_queue = new ConcurrentQueue<Order>();
}
public void AddItem(Order order)
{
_queue.Enqueue(order);
}
public static Repository GetInstance()
{
if (_mySingleton == null)
{
lock (_myLock)
{
if (_mySingleton == null)
{
_mySingleton = new Repository();
}
}
}
return _mySingleton;
}
public void CreateOrder()
{
while (orderCheck)
{
Order order = null;
_queue.TryDequeue(out order);
if (order != null)
{
try
{
// performing business logic with order
}
catch (Exception ex)
{
throw new Exception(ex.Message);
}
}
else
{
Thread.Sleep(10000);
}
}
}
}
The third class creates a new thread when the application is started:
new Thread(delegate ()
{
var repo = Repository.GetInstance();
repo.CreateOrder();
}).Start();
The problem is that the endpoint added the order information in the queue, but when I try to dequeue in the Repository class then it's not available on the tryDequeue method.
I put the getHashCode of ConcurrentQueue and I found the hashcode showing differently in while loop and in AddItem method.

How do I properly use of UnitOfWorks, Repository and Dispose

I have taken over the coding of a software and the way it is currently done, I am not sure that the Dispose() method in the UnitOfWorks is getting called. I have noticed that my SQL service tends to stop after a while (days) saying that all connections are being used up.
I would like to know if the below code is properly done and if not, how I can go about rectifying so that it calls the dispose() method.
UnitOfWorks.cs
public class UnitOfWorks
{
private KaprukaEntities context;
private GenericRepository<Transaction> transactionRepository;
private bool disposed;
public GenericRepository<Transaction> TransactionRepository
{
get
{
if (this.transactionRepository == null)
this.transactionRepository = new GenericRepository<Transaction>(this.context);
return this.transactionRepository;
}
}
public UnitOfWorks(KaprukaEntities entities)
{
this.context = entities;
}
public void Save()
{
this.context.SaveChanges();
}
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}
protected virtual void Dispose(bool disposing)
{
if (!this.disposed && disposing)
this.context.Dispose();
this.disposed = true;
}
public interface IDisposable
{
void Dispose();
}
}
The UnitOfWorks is getting called by a service class.
TransactionService.cs
public class TransactionService
{
private UnitOfWorks _unitOfWork;
private KaprukaEntities entities;
public TransactionService(UnitOfWorks unitOfWork)
{
this._unitOfWork = unitOfWork;
this.entities = new KaprukaEntities();
}
public bool Add(Transaction entity)
{
this._unitOfWork.TransactionRepository.Add(entity);
this._unitOfWork.Save();
var er = this.entities.GetValidationErrors();
return true;
}
public bool Update(Transaction entity)
{
this._unitOfWork.TransactionRepository.Update(entity);
this._unitOfWork.Save();
return true;
}
public bool Delete(Transaction entity)
{
this._unitOfWork.TransactionRepository.Delete(entity);
this._unitOfWork.Save();
return true;
}
public IEnumerable<Transaction> GetAll()
{
return this._unitOfWork.TransactionRepository.GetAll();
}
public IEnumerable<Transaction> GetAll(Expression<Func<Transaction, bool>> filter, Func<IQueryable<Transaction>, IOrderedQueryable<Transaction>> orderBy, string includeProperties)
{
return this._unitOfWork.TransactionRepository.Get(filter, orderBy, includeProperties);
}
}
The way this is getting called is as follows:
There is a file named updateTransaction.aspx and in its code behind, the code calls the service as such:
updateTransaction.aspx
TransactionService TransServ = new TransactionService(new UnitOfWorks(new KaprukaEntities()));
var TransDetails = TransServ.GetAll(x => x.TransactionID == intTID, null, "").SingleOrDefault();
I can't see any code to close or dispose anything.

Your transactionservice constructor creates an extra db context, when one is already in the unit of work class:
this.entities = new KaprukaEntities();

How can I make a command synchronous when updating an object's state?

I am developing a game engine. An angularjs front-end talks to an endpoint in a .NET WebAPI project running on IIS. This web api is a facade that just runs commands against the domain logic of the game which is in a separate project. I have a unit of work that is currently using entity framework.
So, angularjs code calls an endpoint in a webapi website on IIS which creates a command to run against the game. For example: open bank account.
This command would load the game's instance from the data store, check the completion and then execute the command to open the bank account. There are a lot of events that happen in the execute commands that change game instance data.
I have a lot of these commands that all have the same base class but only one is ever supposed to be called at once.
public abstract class GameInstanceCommandHandler<T>
: CommandHandlerBase<T, GameInstanceIDCommandResult>
where T : GameInstanceCommand
{
protected GameInstance GameInstance { get; private set; }
protected GameInstanceCommandHandler() { }
public GameInstanceCommandHandler(IUnitOfWork uow)
: base(uow)
{
}
public override GameInstanceIDCommandResult Execute(T command)
{
Check.Null(command, "command");
GameInstance = UnitOfWork.GetGameInstance(command.GameInstanceID);
if (GameInstance == null)
return GetResult(false, "Could not find game instance.");
if (GameInstance.IsComplete)
{
var completeResult = GetResult(GameInstance.ID);
completeResult.Messages.Add("Game is complete, you cannot update the game state.");
completeResult.Success = false;
completeResult.IsGameComplete = true;
return completeResult;
}
UnitOfWork.LoadCollection(GameInstance, p => p.AppInstances);
var result = ExecuteCommand(command);
UnitOfWork.Save();
return result;
}
protected GameInstanceIDCommandResult GetResult(bool success, string message)
{
return new GameInstanceIDCommandResult(success, message);
}
protected GameInstanceIDCommandResult GetResult(Guid id)
{
return new GameInstanceIDCommandResult(id);
}
protected void LoadGameAndGameApps()
{
UnitOfWork.LoadReference(GameInstance, p => p.Game);
UnitOfWork.LoadCollection(GameInstance.Game, p => p.GameApps);
}
protected abstract GameInstanceIDCommandResult ExecuteCommand(T command);
}
All base classes override the abstract ExecuteCommand. Everything runs fine, I can run my engine using a console project, or in IIS or where ever, it's fine.
The issue is when multiple commands want to change the game instance state at the same time. If I called a command to calculate the interest on a bankaccount in the game, currently 5 calls to the same command will create 5 calculations.
I want to make sure that only one command is allows to execute at a time for a given game instance. As this is a separate library and isn't just built for running in an IIS process I know that this concern should be handled in this file. I thought about updating the code to match below:
public abstract class GameInstanceCommandHandler<T>
: CommandHandlerBase<T, GameInstanceIDCommandResult>
where T : GameInstanceCommand
{
private static readonly object #lock = new object();
private static volatile List<Guid> GameInstanceIDs = new List<Guid>();
protected GameInstance GameInstance { get; private set; }
protected GameInstanceCommandHandler() { }
public GameInstanceCommandHandler(IUnitOfWork uow)
: base(uow)
{
}
public override GameInstanceIDCommandResult Execute(T command)
{
Check.Null(command, "command");
lock(#lock)
{
// if game id is updating then return
if(GameInstanceIDs.Any(p => p == command.GameInstanceID))
return GetResult(false, "The game is already being updated.");
// (lock for update)
GameInstanceIDs.Add(command.GameInstanceID);
}
try
{
GameInstance = UnitOfWork.GetGameInstance(command.GameInstanceID);
if (GameInstance == null)
return GetResult(false, "Could not find game instance.");
if (GameInstance.IsComplete)
{
var completeResult = GetResult(GameInstance.ID);
completeResult.Messages.Add("Game is complete, you cannot update the game state.");
completeResult.Success = false;
completeResult.IsGameComplete = true;
return completeResult;
}
UnitOfWork.LoadCollection(GameInstance, p => p.AppInstances);
var result = ExecuteCommand(command);
UnitOfWork.Save();
return result;
}
catch (Exception ex)
{ }
finally
{
lock (#lock)
{
GameInstanceIDs.Remove(command.GameInstanceID);
}
}
return GetResult(false, "There was an error.");
}
protected GameInstanceIDCommandResult GetResult(bool success, string message)
{
return new GameInstanceIDCommandResult(success, message);
}
protected GameInstanceIDCommandResult GetResult(Guid id)
{
return new GameInstanceIDCommandResult(id);
}
protected void LoadGameAndGameApps()
{
UnitOfWork.LoadReference(GameInstance, p => p.Game);
UnitOfWork.LoadCollection(GameInstance.Game, p => p.GameApps);
}
protected abstract GameInstanceIDCommandResult ExecuteCommand(T command);
}
This fixes my issue completely but there are a number of issues.
Will this cause me headaches when I am running it in IIS?
If I want to load balance this library on many app servers then this won't work.
If there are a million game instances then the list will grow huge and performance will suffer.
The other fix is this by moving the lock to the database:
public abstract class GameInstanceCommandHandler<T>
: CommandHandlerBase<T, GameInstanceIDCommandResult>
where T : GameInstanceCommand
{
private static readonly object #lock = new object();
protected GameInstance GameInstance { get; private set; }
protected GameInstanceCommandHandler() { }
public GameInstanceCommandHandler(IUnitOfWork uow)
: base(uow)
{
}
public override GameInstanceIDCommandResult Execute(T command)
{
Check.Null(command, "command");
lock(#lock)
{
GameInstance = UnitOfWork.GetGameInstance(command.GameInstanceID);
if (GameInstance == null)
return GetResult(false, "Could not find game instance.");
if(GameInstance.IsLocked)
return GetResult(false, "Game is locked by another command.");
// Lock the game in the database or datastore
GameInstance.Lock();
UnitOfWork.Save();
// Unlock only local copy
GameInstance.UnLock();
}
try
{
if (GameInstance.IsComplete)
{
var completeResult = GetResult(GameInstance.ID);
completeResult.Messages.Add("Game is complete, you cannot update the game state.");
completeResult.Success = false;
completeResult.IsGameComplete = true;
return completeResult;
}
UnitOfWork.LoadCollection(GameInstance, p => p.AppInstances);
var result = ExecuteCommand(command);
// this will unlock the gameinstance on the save
UnitOfWork.Save();
return result;
}
catch (Exception ex)
{ }
return GetResult(false, "There was an error.");
}
protected GameInstanceIDCommandResult GetResult(bool success, string message)
{
return new GameInstanceIDCommandResult(success, message);
}
protected GameInstanceIDCommandResult GetResult(Guid id)
{
return new GameInstanceIDCommandResult(id);
}
protected void LoadGameAndGameApps()
{
UnitOfWork.LoadReference(GameInstance, p => p.Game);
UnitOfWork.LoadCollection(GameInstance.Game, p => p.GameApps);
}
protected abstract GameInstanceIDCommandResult ExecuteCommand(T command);
}
Perhaps I am thinking about this the wrong way. Any help would be great.

I have decided that the best way currently is to have a queue of IDs. I try to immediately enter the game instance id and if the insert is OK then continue. After doing work remove the id from the list. If the id insert fails then it already exists.
This fix is pretty much the same as the lock but I am relying on a unique key value store. I am probably going to create an interface and class that handles this so if the database fails I can store the failures in a file or someplace else to make sure to clear out the ids later.
I am definitely up for more suggestions to this as it really has a bad code smell.
public abstract class GameInstanceCommandHandler<T>
: CommandHandlerBase<T, GameInstanceIDCommandResult>
where T : GameInstanceCommand
{
protected GameInstance GameInstance { get; private set; }
protected GameInstanceCommandHandler() { }
public GameInstanceCommandHandler(IUnitOfWork uow)
: base(uow)
{
}
public override GameInstanceIDCommandResult Execute(T command)
{
Check.Null(command, "command");
try
{
AddCommandInstance(command.GameInstanceID);
}
catch(Exception ex)
{
return GetResult(false, "Only one command can be ran on an instance at a time.");
}
GameInstance = UnitOfWork.GameInstances.FirstOrDefault(p => p.ID == command.GameInstanceID);
if (GameInstance == null)
{
RemoveCommandInstance(command.GameInstanceID);
return GetResult(false, "Could not find game instance.");
}
if (GameInstance.IsComplete)
{
var completeResult = GetResult(GameInstance.ID);
completeResult.Messages.Add("Game is complete, you cannot update the game state.");
completeResult.Success = false;
completeResult.IsGameComplete = true;
RemoveCommandInstance(command.GameInstanceID);
return completeResult;
}
UnitOfWork.LoadCollection(GameInstance, p => p.AppInstances);
GameInstanceIDCommandResult result = null;
try
{
result = ExecuteCommand(command);
UnitOfWork.Save();
}
catch(Exception ex)
{
result = GetResult(false, ex.Message);
}
finally
{
RemoveCommandInstance(command.GameInstanceID);
}
return result;
}
private void AddCommandInstance(Guid gameInstanceID)
{
UnitOfWork.Add(new CommandInstance() { ID = gameInstanceID });
UnitOfWork.Save();
}
private void RemoveCommandInstance(Guid gameInstanceID)
{
UnitOfWork.Remove(UnitOfWork.CommandInstances.First(p => p.ID == gameInstanceID));
UnitOfWork.Save();
}
protected GameInstanceIDCommandResult GetResult(bool success, string message)
{
return new GameInstanceIDCommandResult(success, message);
}
protected GameInstanceIDCommandResult GetResult(Guid id)
{
return new GameInstanceIDCommandResult(id);
}
protected void LoadGameAndGameApps()
{
UnitOfWork.LoadReference(GameInstance, p => p.Game);
UnitOfWork.LoadCollection(GameInstance.Game, p => p.GameApps);
}
protected abstract GameInstanceIDCommandResult ExecuteCommand(T command);
}

Repository Pattern and Local Caching

I have the following interfaces/class:
public interface IUnitOfWork : IDisposable
{
event EventHandler<EventArgs> Saved;
DbSet<T> Set<T>() where T : class;
DbEntityEntry<T> Entry<T>(T entity) where T : class;
void Commit();
}
And an implementation of a repository:
public class CachedSqlRepository<T, TKey, TContext> : ICacheRepository<T, TKey, TContext>
where T : class
where TContext : DbContext, IDisposable, new()
{
//A list of the Navigation Properties to include
private readonly Expression<Func<T, object>>[] _NavigationProperties;
public CachedSqlRepository(params Expression<Func<T, object>>[] navigationProperties)
{
_NavigationProperties = navigationProperties;
using (TContext dbContext = new TContext()) //Fetch the List of Entities
{
RefreshCache(dbContext);
}
}
/// <summary>
/// The Collection of Items in the database
/// Note this is a Cache, but should replicate whats in the DB
/// </summary>
public IList<T> Items { get; private set; }
public bool Any(Func<T, bool> predicate)
{
return Items.Any(predicate);
}
public void RefreshCache(DbContext context)
{
switch (_NavigationProperties.Length)
{
case 0:
Items = context.Set<T>().ToList();
break;
case 1:
Items = context.Set<T>().Include(_NavigationProperties[0]).ToList();
break;
//more here
}
}
/// <summary>
/// Refresh the internal cache
/// </summary>
public void RefreshCache()
{
using (TContext dbContext = new TContext())
{
RefreshCache(dbContext);
}
}
public IEnumerable<T> FilterBy(Func<T, bool> predicate)
{
return Items.Where(predicate);
}
public T Add(T entity)
{
T newEntity;
using (TContext dbContext = new TContext())
{
newEntity = dbContext.Set<T>().Add(entity);
if (dbContext.SaveChanges() == 1) //1 change was made
Items.Add(newEntity);
}
return newEntity;
}
public void Delete(TKey id)
{
using (TContext dbContext = new TContext())
{
var attachedEntry = dbContext.Set<T>().Find(id);
if (attachedEntry == null) return; //it doesnt exist anyway!
dbContext.Set<T>().Remove(attachedEntry);
dbContext.SaveChanges();
RefreshCache(dbContext);
}
}
public void Update(T entity, TKey id)
{
if (entity == null) throw new ArgumentException("Cannot update a null entity.");
using (TContext dbContext = new TContext())
{
var entry = dbContext.Entry(entity);
if (entry.State != EntityState.Detached) return;
T attachedEntity = dbContext.Set<T>().Find(id);
if (attachedEntity != null)
{
var attachedEntry = dbContext.Entry(attachedEntity);
attachedEntry.CurrentValues.SetValues(entity);
}
else
{
entry.State = EntityState.Modified; // This should attach entity
}
dbContext.SaveChanges();
RefreshCache(dbContext);
}
}
#region Transaction Methods
public IUnitOfWork StartTransaction()
{
return new EFUnitOfWork(new TContext());
}
public T TransactionAdd(T entity, IUnitOfWork context)
{
context.Saved += OnSave;
return context.Set<T>().Add(entity);
}
public void TransactionDelete(TKey id, IUnitOfWork context)
{
var attachedEntry = context.Set<T>().Find(id);
if (attachedEntry == null) return; //it doesnt exist anyway
context.Saved += OnSave;
context.Set<T>().Remove(attachedEntry);
}
public void TransactionUpdate(T entity, TKey id, IUnitOfWork context)
{
if (entity == null) throw new ArgumentException("Cannot update a null entity.");
var entry = context.Entry(entity);
if (entry.State != EntityState.Detached) return;
T attachedEntity = context.Set<T>().Find(id);
if (attachedEntity != null)
{
var attachedEntry = context.Entry(attachedEntity);
attachedEntry.CurrentValues.SetValues(entity);
}
else
{
entry.State = EntityState.Modified; // This should attach entity
}
context.Saved += OnSave;
}
private void OnSave(object sender, EventArgs e)
{
RefreshCache();
}
#endregion
}
It is adapted from various patterns on the net. I don't expect this to be useful for tables with hundreds of thousands of rows, but for lookup tables etc - I'm not always hitting the DB.
It works, but some things aren't super clean, for example where I refresh the cache - sometimes I have to pull all the data again (currently a work in progress).
Is this sound design? Or am I reinventing the wheel here?

An interesting question +1. In my view, context content caching is one that is best done properly or left well alone. And use DB caching.
Why:
Parallel WPs all have a cache
Each WP may have threads, the context is not thread safe
Should each thread have a cache?
Is your cache session persistent?
No: you reload each request
Yes: you use global caching on ASP.NET, EnterpriseLibary cache or similar?
Are you managing the cache correctly?
How do you deal with concurrency and changes
Have you considered best practice around Context lifetime? Some experts suggest short lifetime only
Is DB located on LAN near WebServer?
Have you compared response times when using DB buffer access?
Having looked into this topic in various environments, not just EF/.NET/SQL Server, I have come to the conclusion that unless the DB server has become or is tending to be the CPU bottleneck and can't be easily scaled, it is a very reasonable approach to supply memory to DB and let it cache 100sMB
before building or trying to cache entries.
I would rather throw GBs or RAM at SQL Server before coding myself in app knots on WebServer.
When every microsecond counts, or your DB is separated on a network span with latency/throughput issues and your data is non volatile and needs no cache expiry/concurrency management. Then go ahead and implement caching.
Consider memory use, cache construction time and memory persistency model carefully.
Look at some tools made for caching for ideas and as potential solutions. e.g. Enterprise Caching Block.
Good Luck.

EF 4.1 / Repository / UnitOfWork / MySQL : There is already an open DataReader associated with this Connection which must be closed first

I read all the similar titles questions, but i didn't find the answer to my problem, so I open a new question:
I have two mysql tables:
tb1 (int_id, code, description, ..., my_id);
tb2 (int_id, code, description, ..., tb1_id);
I create a generic repository to manage DbContext and GetAll, GetById, GetByLambda, Insert, Update and Delete methode.
namespace Model.DataAccessLayer
{
public class Repository<T> : IRepository<T> where T : EntityObject, IEntity
{
protected DbContext dbContext = null;
public virtual DbContext DbContext
{
get { return dbContext; }
set { dbContext = value; }
}
public ObjectContext ObjectContext
{
get { return ((IObjectContextAdapter)DbContext).ObjectContext; }
}
public void Dispose()
{
ObjectContext.Dispose();
System.GC.SuppressFinalize(this);
}
public virtual IQueryable<T> GetAll()
{
return ObjectContext.CreateObjectSet<T>();
}
public virtual IEnumerable<T> GetByLambda(Func<T, bool> p)
{
return GetAll().Where(p);
}
public virtual void Save()
{
DbContext.SaveChanges();
}
...
}
}
and the inherited class which I used:
namespace Model.DataAccessLayer
{
public class RepositoryP<T> : Repository<T> where T : EntityObject, IEntity
{
public myEntities MyContext
{
get { return (myEntities)ObjectContext; }
//set { ObjectContext = value; }
}
public override DbContext DbContext
{
get
{
if (dbContext == null)
{
dbContext = new DbContext("myEntities");
}
return dbContext;
}
set
{
base.DbContext = value;
}
}
}
}
It works great when using only one table.
When I try to use my two tables and the foreign key relation between them, it doesn't work.
For example I try to get all records from table tb2 where tb1.my_id=5 with the following join tb1.int_id = tb2.tb1_id.
List<tb2> lj = new Tb2DAO().GetByLambda(l => l.tb1.my_id == 5).ToList();
(Tb2DAO inherited from my generic repository class.)
I have the following MySQL error:
"MySql.Data.MySqlClient.MySqlException: There is already an open DataReader associated with this Connection which must be closed first."
I think this comes from my DbContext which is not common to my two tables entities.
So I tried to implement the UnitOfWork Pattern to solve this problem, like this:
namespace Model.DataAccessLayer
{
public class UnitOfWork : IDisposable
{
private DbContext dbContextUnit = null; //= new DbContext();
private Tb1DAO tb1DAO;
private Tb2DAO tb2DAO;
public Tb1DAO tb1
{
get
{
if (this.tb1DAO == null)
{
if (dbContextUnit != null)
{
this.tb1DAO = new Tb1DAO { DbContext = dbContextUnit };
}
else
{
this.tb1DAO = new Tb1DAO();
dbContextUnit = this.tb1DAO.DbContext;
}
}
return tb1DAO;
}
}
public Tb2DAO tb2
{
get
{
if (this.tb2DAO == null)
{
if (dbContextUnit != null)
{
this.tb2DAO = new Tb2DAO { DbContext = dbContextUnit };
}
else
{
this.tb2DAO = new Tb2DAO();
dbContextUnit = this.tb2DAO.DbContext;
}
}
return tb2DAO;
}
}
public void Save()
{
dbContextUnit.SaveChanges();
}
private bool disposed = false;
protected virtual void Dispose(bool disposing)
{
if (!this.disposed)
{
if (disposing)
{
dbContextUnit.Dispose();
}
}
this.disposed = true;
}
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}
}
}
And now in my code I tried to use the Unit of Work like this :
UnitOfWork unitOfWork = new UnitOfWork();
List<tb2> tb2List = unitOfWork.tb2.GetByLambda(l => l.index_job.job_id == job_id).ToList();
But I have always the same error message :
"MySql.Data.MySqlClient.MySqlException: There is already an open DataReader associated with this Connection which must be closed first."
is there something I am doing wrong ? Please can you help me ? This notion of repository and unit Of work are new for me
and I am confused else I read lot of think about it may be not the right one...
I also to try to add MultipleActiveResultSets=true to my connection but it is not recognized.
many thank to all & regards,
wst

Your GetByLambda() method is calling GetAll() which creates a new context using ObjectContext.CreateObjectSet<T>(). So you now have more than one context open. I would advise using the standard EF associations and the repository pattern, then you can avoid this entire mess. Here is a link that may help you get started - http://blogs.msdn.com/b/adonet/archive/2011/03/15/ef-4-1-code-first-walkthrough.aspx.

I'm not sure about this method
public virtual IQueryable<T> GetAll()
{
return ObjectContext.CreateObjectSet<T>();
}
It's create new object set about which EF context doesn't know.
You can try to pass Func<ObjectContext, ObjectResult<T>> from derived class to base. (It should return ObjectResult which are got from EF context. F.e. context => context.Entities).