We Keep Coding

Threads monitoring a Queue<Actions>

I doing a small project to map a network (routers only) using SNMP. In order to speed things up, I´m trying to have a pool of threads responsible for doing the jobs I need, apart from the first job which is done by the main thread.
At this time I have two jobs, one takes a parameter the other doesn´t:
UpdateDeviceInfo(NetworkDevice nd)
UpdateLinks() *not defined yet
What I´m trying to achieve is to have those working threads waiting for a job to
appear on a Queue<Action> and wait while it is empty. The main thread will add the first job and then wait for all workers, which might add more jobs, to finish before starting adding the second job and wake up the sleeping threads.
My problem/questions are:
How to define the Queue<Actions> so that I can insert the methods and the parameters if any. If not possible I could make all functions accept the same parameter.
How to launch the working threads indefinitely. I not sure where should I create the for(;;).
This is my code so far:
public enum DatabaseState
{
Empty = 0,
Learning = 1,
Updating = 2,
Stable = 3,
Exiting = 4
};
public class NetworkDB
{
public Dictionary<string, NetworkDevice> database;
private Queue<Action<NetworkDevice>> jobs;
private string _community;
private string _ipaddress;
private Object _statelock = new Object();
private DatabaseState _state = DatabaseState.Empty;
private readonly int workers = 4;
private Object _threadswaitinglock = new Object();
private int _threadswaiting = 0;
public Dictionary<string, NetworkDevice> Database { get => database; set => database = value; }
public NetworkDB(string community, string ipaddress)
{
_community = community;
_ipaddress = ipaddress;
database = new Dictionary<string, NetworkDevice>();
jobs = new Queue<Action<NetworkDevice>>();
}
public void Start()
{
NetworkDevice nd = SNMP.GetDeviceInfo(new IpAddress(_ipaddress), _community);
if (nd.Status > NetworkDeviceStatus.Unknown)
{
database.Add(nd.Id, nd);
_state = DatabaseState.Learning;
nd.Update(this); // The first job is done by the main thread
for (int i = 0; i < workers; i++)
{
Thread t = new Thread(JobRemove);
t.Start();
}
lock (_statelock)
{
if (_state == DatabaseState.Learning)
{
Monitor.Wait(_statelock);
}
}
lock (_statelock)
{
if (_state == DatabaseState.Updating)
{
Monitor.Wait(_statelock);
}
}
foreach (KeyValuePair<string, NetworkDevice> n in database)
{
using (System.IO.StreamWriter file = new System.IO.StreamWriter(n.Value.Name + ".txt")
{
file.WriteLine(n);
}
}
}
}
public void JobInsert(Action<NetworkDevice> func, NetworkDevice nd)
{
lock (jobs)
{
jobs.Enqueue(item);
if (jobs.Count == 1)
{
// wake up any blocked dequeue
Monitor.Pulse(jobs);
}
}
}
public void JobRemove()
{
Action<NetworkDevice> item;
lock (jobs)
{
while (jobs.Count == 0)
{
lock (_threadswaitinglock)
{
_threadswaiting += 1;
if (_threadswaiting == workers)
Monitor.Pulse(_statelock);
}
Monitor.Wait(jobs);
}
lock (_threadswaitinglock)
{
_threadswaiting -= 1;
}
item = jobs.Dequeue();
item.Invoke();
}
}
public bool NetworkDeviceExists(NetworkDevice nd)
{
try
{
Monitor.Enter(database);
if (database.ContainsKey(nd.Id))
{
return true;
}
else
{
database.Add(nd.Id, nd);
Action<NetworkDevice> action = new Action<NetworkDevice>(UpdateDeviceInfo);
jobs.Enqueue(action);
return false;
}
}
finally
{
Monitor.Exit(database);
}
}
//Job1 - Learning -> Update device info
public void UpdateDeviceInfo(NetworkDevice nd)
{
nd.Update(this);
try
{
Monitor.Enter(database);
nd.Status = NetworkDeviceStatus.Self;
}
finally
{
Monitor.Exit(database);
}
}
//Job2 - Updating -> After Learning, create links between neighbours
private void UpdateLinks()
{
}
}

Your best bet seems like using a BlockingCollection instead of the Queue class. They behave effectively the same in terms of FIFO, but a BlockingCollection will let each of your threads block until an item can be taken by calling GetConsumingEnumerable or Take. Here is a complete example.
http://mikehadlow.blogspot.com/2012/11/using-blockingcollection-to-communicate.html?m=1
As for including the parameters, it seems like you could use closure to enclose the NetworkDevice itself and then just enqueue Action instead of Action<>

Parallel Tasks Calling Static function

I Have issue with parllel Tasks
My Code
namespace ITDevices
{
using System.Linq;
using System.Collections.Generic;
using System.Threading.Tasks;
using System;
/*Device Modal*/
public class Device
{
public string IP { get; set; }
public string Name { get; set; }
public string MAC { get; set; }
}
/*Entry Class*/
class Program
{
static async void Main(string[] args)
{
List<Task<Device>> Tasks = new List<Task<Device>>();
for(int i=2;i==0;i--)
{
Tasks.Add(Task.Factory.StartNew<Device>(
()=> {
Device free = Helper.GetFreeDevice();
return free;
}
));
}
await Task.WhenAll(Tasks.ToArray());
foreach(Task<Device> item in Tasks)
{
Console.WriteLine(item.Result.IP);
}
Console.ReadLine();
}
}
/*Devices Helper*/
static class Helper
{
public static List<Device> UsedDevices = new List<Device>();
public static Device GetFreeDevice()
{
List<Device> OnlineDevices = new List<Device>()
{
new Device { IP="192.168.1.15",Name="PerryLabtop",MAC="AC:DS:F2:CC:2D:7A"},
new Device { IP="192.168.1.20",Name="MAYA-PC",MAC="7D:E9:2C:FF:E7:2D"},
new Device { IP="192.168.1.2",Name="server",MAC="D8:C2:A4:DC:E5:3A"}
};
Device FreeDevice = OnlineDevices.Where(x => !UsedDevices.Contains(x)).SingleOrDefault();
if (FreeDevice != null)
UsedDevices.Add(FreeDevice);
return FreeDevice;
}
}
}
//Output
//192.168.1.15
//192.168.1.15
But expected output must be
//192.168.1.15
//192.168.1.20
When debugging the project
all tasks execute GetFreeDevice() function at same time line by line
I need to make tasks wait while current GetFreeDevice() function execution done .. or any thing helpful
THANKS ALL

Several problems must be solved to make it work properly:
You probably reversed condition in for loop, because int i = 2; i == 0; i-- will do nothing. Replace i == 0 with i != 0
async Main method doesn't make sense (see for example this blog) and actually doesn't even compile in Visual Studio. To fix this, you can for example wait for tasks to complete synchronously (use .Wait() instead of await)
To prevent multiple threads to run GetFreeDevice() method simultaneously, simply place lock around code that uses shared objects - which is whole method body in your case.
Because you create new list of OnlineDevices each time GetFreeDevice() method is called, UsedDevices.Contains(x) will not work as expected. By default, objects are compared by their reference. So .Contains(x) will compare Device objects in UsedDevices list (which were put there in one of previous calls) with newly created Device objects, which will never be equal (references of these object will be different despite IP, Name and MAC being the same). To fix this, you could either override Equals() and GetHashCode() methods on Device class, or (as i did) create just one static list of Device objects.
You must replace SingleOrDefault() with FirstOrDefault(). With SingleOrDefault(), program would throw exception if there is more than one unused device, while FirstOrDefault() will take first unused device even if there is more than one.
Full source code with all proposed fixes:
namespace ITDevices
{
/*Device Modal*/
public class Device
{
public string IP { get; set; }
public string Name { get; set; }
public string MAC { get; set; }
}
/*Entry Class*/
class Program
{
static void Main(string[] args)
{
List<Task<Device>> Tasks = new List<Task<Device>>();
for (int i = 2; i != 0; i--)
{
Tasks.Add(Task.Factory.StartNew<Device>(
() => {
Device free = Helper.GetFreeDevice();
return free;
}
));
}
Task.WhenAll(Tasks.ToArray()).Wait();
foreach (Task<Device> item in Tasks)
{
Console.WriteLine(item.Result.IP);
}
Console.ReadLine();
}
}
/*Devices Helper*/
static class Helper
{
public static List<Device> UsedDevices = new List<Device>();
static List<Device> OnlineDevices = new List<Device>()
{
new Device { IP="192.168.1.15",Name="PerryLabtop",MAC="AC:DS:F2:CC:2D:7A"},
new Device { IP="192.168.1.20",Name="MAYA-PC",MAC="7D:E9:2C:FF:E7:2D"},
new Device { IP="192.168.1.2",Name="server",MAC="D8:C2:A4:DC:E5:3A"}
};
static Object LockObject = new Object();
public static Device GetFreeDevice()
{
lock (LockObject)
{
Device FreeDevice = OnlineDevices.Where(x => !UsedDevices.Contains(x)).FirstOrDefault();
if (FreeDevice != null)
UsedDevices.Add(FreeDevice);
return FreeDevice;
}
}
}
}

try:
public static Device GetFreeDevice()
{
List<Device> OnlineDevices = new List<Device>()
{
new Device { IP="192.168.1.15",Name="PerryLabtop",MAC="AC:DS:F2:CC:2D:7A"},
new Device { IP="192.168.1.20",Name="MAYA-PC",MAC="7D:E9:2C:FF:E7:2D"},
new Device { IP="192.168.1.2",Name="server",MAC="D8:C2:A4:DC:E5:3A"}
};
Device FreeDevice = OnlineDevices.Where(x => !UsedDevices.Contains(x)).SingleOrDefault();
if (FreeDevice != null)
lock (UsedDevices)
UsedDevices.Add(FreeDevice);
return FreeDevice;
}
------------------------ UPDATE
try:
public static Device GetFreeDevice()
{
List<Device> OnlineDevices = new List<Device>()
{
new Device { IP="192.168.1.15",Name="PerryLabtop",MAC="AC:DS:F2:CC:2D:7A"},
new Device { IP="192.168.1.20",Name="MAYA-PC",MAC="7D:E9:2C:FF:E7:2D"},
new Device { IP="192.168.1.2",Name="server",MAC="D8:C2:A4:DC:E5:3A"}
};
lock (UsedDevices)
{
Device FreeDevice = OnlineDevices.Where(x => !UsedDevices.Contains(x)).SingleOrDefault();
if (FreeDevice != null)
UsedDevices.Add(FreeDevice);
}
return FreeDevice;
}

Saving subscription state for resuming it later

Update - solved
The final solution differs a bit from Brandon's suggestion but his answer brought me on the right track.
class State
{
public int Offset { get; set; }
public HashSet<string> UniqueImageUrls = new HashSet<string>();
}
public IObservable<TPicture> GetPictures(ref object _state)
{
var localState = (State) _state ?? new State();
_state = localState;
return Observable.Defer(()=>
{
return Observable.Defer(() => Observable.Return(GetPage(localState.Offset)))
.SubscribeOn(TaskPoolScheduler.Default)
.Do(x=> localState.Offset += 20)
.Repeat()
.TakeWhile(x=> x.Count > 0)
.SelectMany(x=> x)
.Where(x=> !localState.UniqueImageUrls.Contains(x.ImageUrl))
.Do(x=> localState.UniqueImageUrls.Add(x.ImageUrl));
});
}
IList<TPicture> GetPage(int offset)
{
...
return result;
}
Original Question
I'm currently struggling with the following problem. The PictureProvider implementation shown below is working with an offset variable used for paging results of a backend service providing the actual data. What I would like to implement is an elegant solution making the current offset available to the consumer of the observable to allow for resuming the observable sequence at a later time at the correct offset. Resuming is already accounted for by the intialState argument to GetPictures().
Recommendations for improving the code in a more RX like fashion would be welcome as well. I'm actually not so sure if the Task.Run() stuff is appropriate here.
public class PictureProvider :
IPictureProvider<Picture>
{
#region IPictureProvider implementation
public IObservable<Picture> GetPictures(object initialState)
{
return Observable.Create<Picture>((IObserver<Picture> observer) =>
{
var state = new ProducerState(initialState);
ProducePictures(observer, state);
return state;
});
}
#endregion
void ProducePictures(IObserver<Picture> observer, ProducerState state)
{
Task.Run(() =>
{
try
{
while(!state.Terminate.WaitOne(0))
{
var page = GetPage(state.Offset);
if(page.Count == 0)
{
observer.OnCompleted();
break;
}
else
{
foreach(var picture in page)
observer.OnNext(picture);
state.Offset += page.Count;
}
}
}
catch (Exception ex)
{
observer.OnError(ex);
}
state.TerminateAck.Set();
});
}
IList<Picture> GetPage(int offset)
{
var result = new List<Picture>();
... boring web service call here
return result;
}
public class ProducerState :
IDisposable
{
public ProducerState(object initialState)
{
Terminate = new ManualResetEvent(false);
TerminateAck = new ManualResetEvent(false);
if(initialState != null)
Offset = (int) initialState;
}
public ManualResetEvent Terminate { get; private set; }
public ManualResetEvent TerminateAck { get; private set; }
public int Offset { get; set; }
#region IDisposable implementation
public void Dispose()
{
Terminate.Set();
TerminateAck.WaitOne();
Terminate.Dispose();
TerminateAck.Dispose();
}
#endregion
}
}

I suggest refactoring your interface to yield the state as part of the data. Now the client has what they need to resubscribe where they left off.
Also, once you start using Rx, you should find that using synchronization primitives like ManualResetEvent are rarely necessary. If you refactor your code so that retrieving each page is its own Task, then you can eliminate all of that synchronization code.
Also, if you are calling a "boring web service" in GetPage, then just make it async. This gets rid of the need to call Task.Run among other benefits.
Here is a refactored version, using .NET 4.5 async/await syntax. It could also be done without async/await. I also added a GetPageAsync method that uses Observable.Run just in case you really cannot convert your webservice call to be asynchronous
/// <summary>A set of pictures</summary>
public struct PictureSet
{
public int Offset { get; private set; }
public IList<Picture> Pictures { get; private set; }
/// <summary>Clients will use this property if they want to pick up where they left off</summary>
public int NextOffset { get { return Offset + Pictures.Count; } }
public PictureSet(int offset, IList<Picture> pictures)
:this() { Offset = offset; Pictures = pictures; }
}
public class PictureProvider : IPictureProvider<PictureSet>
{
public IObservable<PictureSet> GetPictures(int offset = 0)
{
// use Defer() so we can capture a copy of offset
// for each observer that subscribes (so multiple
// observers do not update each other's offset
return Observable.Defer<PictureSet>(() =>
{
var localOffset = offset;
// Use Defer so we re-execute GetPageAsync()
// each time through the loop.
// Update localOffset after each GetPageAsync()
// completes so that the next call to GetPageAsync()
// uses the next offset
return Observable.Defer(() => GetPageAsync(localOffset))
.Select(pictures =>
{
var s = new PictureSet(localOffset, pictures);
localOffset += pictures.Count;
})
.Repeat()
.TakeWhile(pictureSet => pictureSet.Pictures.Count > 0);
});
}
private async Task<IList<Picture>> GetPageAsync(int offset)
{
var data = await BoringWebServiceCallAsync(offset);
result = data.Pictures.ToList();
}
// this version uses Observable.Run() (which just uses Task.Run under the hood)
// in case you cannot convert your
// web service call to be asynchronous
private IObservable<IList<Picture>> GetPageAsync(int offset)
{
return Observable.Run(() =>
{
var result = new List<Picture>();
... boring web service call here
return result;
});
}
}
Clients just need to add a SelectMany call to get their IObservable<Picture>. They can choose to store the pictureSet.NextOffset if they wish.
pictureProvider
.GetPictures()
.SelectMany(pictureSet => pictureSet.Pictures)
.Subscribe(picture => whatever);

Instead of thinking about how to save the subscription state, I would think about how to replay the state of the inputs (i.e. I'd try to create a serializable ReplaySubject that, on resume, would just resubscribe and catch back up to the current state).

Where's the best place for validation... constructor or leave to client to call?

This is really a generic (and probably a more subjective too) question. I have some classes where I use an interface to define a standard approach to validating the object state. When I did this, I got to scratching my head... is it best to
1.) allow the constructor (or initializing method) to silently filter out the errant information automatically or...
2.) allow the client to instantiate the object however and let the client also call the interface's IsValid property or Validate() method before moving forward?
Basically one approach is silent but could be misleading in that the client may not be aware that certain pieces of information were filtered away due to it not meeting the validation criteria. The other approach then would be more straight forward, but also adds a step or two? What's typical here?
Okay, after a long day of trying to keep up with some other things, I finally did come up with an example. Please for me for it as it's not ideal and by no means something wonderful, but hopefully should serve well enough to get the point across. My current project is just too complicated to put something simple out for this, so I made something up... and trust me... totally made up.
Alright, the objects in the example are this:
Client: representing client-side code (Console App btw)
IValidationInfo: This is the actual interface I'm using in my current project. It allows me to create a validation framework for the "back-end" objects not necessarily intended for the Client to use since the business logic could be complicated enough. This also allowed me to separate validation code and call as-needed for the business logic.
OrderManager: This is an object the client-side code can use to manage their orders. It's client-friendly so-to-speak.
OrderSpecification: This is an object the client-side code can use to request an order. But if the business logic doesn't work out, an exception can be raised (or if necessary the order not added and exceptions ignored...) In my real-world example I actually have an object that's not quite so black-and-white as to which side of this fence it goes... thus my original question when I realized I could push validation request (calling IsValid or Validate()) to the cilent.
CustomerDescription: represents customers to which I've classified (pretending to have been read from a DB.
Product: Represents a particular product which is classified also.
OrderDescription: Represents the official order request.The business rule is that the Customer cannot order anything to which they've not been classified (I know.. that's not very real-world, but it gave me something to work with...)
Ok... I just realized I can't attach a file here, so here's the code. I apologize for it's lengthy appearance. That was the best I could do to create a client-friendly front-end and business logic back-end using my Validation interface:
public class Client
{
static OrderManager orderMgr = new OrderManager();
static void Main(string[] args)
{
//Request a new order
//Note: Only the OrderManager and OrderSpecification are used by the Client as to keep the
// Client from having to know and understand the framework beyond that point.
OrderSpecification orderSpec = new OrderSpecification("Customer1", new Product(IndustryCategory.FoodServices, "Vending Items"));
orderMgr.SubmitOrderRequest(orderSpec);
Console.WriteLine("The OrderManager has {0} items for {1} customers.", orderMgr.ProductCount, orderMgr.CustomerCount);
//Now add a second item proving that the business logic to add for an existing customer works
Console.WriteLine("Adding another valid item for the same customer.");
orderSpec = new OrderSpecification("Customer1", new Product(IndustryCategory.FoodServices, "Sodas"));
orderMgr.SubmitOrderRequest(orderSpec);
Console.WriteLine("The OrderManager now has {0} items for {1} customers.", orderMgr.ProductCount, orderMgr.CustomerCount);
Console.WriteLine("Adding a new valid order for a new customer.");
orderSpec = new OrderSpecification("Customer2", new Product(IndustryCategory.Residential, "Magazines"));
orderMgr.SubmitOrderRequest(orderSpec);
Console.WriteLine("The OrderManager now has {0} items for {1} customers.", orderMgr.ProductCount, orderMgr.CustomerCount);
Console.WriteLine("Adding a invalid one will not work because the customer is not set up to receive these kinds of items. Should get an exception with message...");
try
{
orderSpec = new OrderSpecification("Customer3", new Product(IndustryCategory.Residential, "Magazines"));
orderMgr.SubmitOrderRequest(orderSpec);
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
Console.ReadLine();
}
}
public interface IValidationInfo
{
string[] ValidationItems { get; }
bool IsValid { get; }
void Validate();
List<string> GetValidationErrors();
string GetValidationError(string itemName);
}
public class OrderManager
{
private List<OrderDescription> _orders = new List<OrderDescription>();
public List<OrderDescription> Orders
{
get { return new List<OrderDescription>(_orders); }
private set { _orders = value; }
}
public int ProductCount
{
get
{
int itemCount = 0;
this.Orders.ForEach(o => itemCount += o.Products.Count);
return itemCount;
}
}
public int CustomerCount
{
get
{
//since there's only one customer per order, just return the number of orders
return this.Orders.Count;
}
}
public void SubmitOrderRequest(OrderSpecification orderSpec)
{
if (orderSpec.IsValid)
{
List<OrderDescription> orders = this.Orders;
//Since the particular customer may already have an order, we might as well add to an existing
OrderDescription existingOrder = orders.FirstOrDefault(o => string.Compare(orderSpec.Order.Customer.Name, o.Customer.Name, true) == 0) as OrderDescription;
if (existingOrder != null)
{
List<Product> existingProducts = orderSpec.Order.Products;
orderSpec.Order.Products.ForEach(p => existingOrder.AddProduct(p));
}
else
{
orders.Add(orderSpec.Order);
}
this.Orders = orders;
}
else
orderSpec.Validate(); //Let the OrderSpecification pass the business logic validation down the chain
}
}
public enum IndustryCategory
{
Residential,
Textile,
FoodServices,
Something
}
public class OrderSpecification : IValidationInfo
{
public OrderDescription Order { get; private set; }
public OrderSpecification(string customerName, Product product)
{
//Should use a method in the class to search and retrieve Customer... pretending here
CustomerDescription customer = null;
switch (customerName)
{
case "Customer1":
customer = new CustomerDescription() { Name = customerName, Category = IndustryCategory.FoodServices };
break;
case "Customer2":
customer = new CustomerDescription() { Name = customerName, Category = IndustryCategory.Residential };
break;
case "Customer3":
customer = new CustomerDescription() { Name = customerName, Category = IndustryCategory.Textile };
break;
}
//Create an OrderDescription to potentially represent the order... valid or not since this is
//a specification being used to request the order
this.Order = new OrderDescription(new List<Product>() { product }, customer);
}
#region IValidationInfo Members
private readonly string[] _validationItems =
{
"OrderDescription"
};
public string[] ValidationItems
{
get { return _validationItems; }
}
public bool IsValid
{
get
{
List<string> validationErrors = GetValidationErrors();
if (validationErrors != null && validationErrors.Count > 0)
return false;
else
return true;
}
}
public void Validate()
{
List<string> errorMessages = GetValidationErrors();
if (errorMessages != null && errorMessages.Count > 0)
{
StringBuilder errorMessageReported = new StringBuilder();
errorMessages.ForEach(em => errorMessageReported.AppendLine(em));
throw new Exception(errorMessageReported.ToString());
}
}
public List<string> GetValidationErrors()
{
List<string> errorMessages = new List<string>();
foreach (string item in this.ValidationItems)
{
string errorMessage = GetValidationError(item);
if (!string.IsNullOrEmpty(errorMessage))
errorMessages.Add(errorMessage);
}
return errorMessages;
}
public string GetValidationError(string itemName)
{
switch (itemName)
{
case "OrderDescription":
return ValidateOrderDescription();
default:
return "Invalid item name.";
}
}
#endregion
private string ValidateOrderDescription()
{
string errorMessage = string.Empty;
if (this.Order == null)
errorMessage = "Order was not instantiated.";
else
{
if (!this.Order.IsValid)
{
List<string> orderErrors = this.Order.GetValidationErrors();
orderErrors.ForEach(ce => errorMessage += "\n" + ce);
}
}
return errorMessage;
}
}
public class CustomerDescription : IValidationInfo
{
public string Name { get; set; }
public string Street { get; set; }
public string City { get; set; }
public string State { get; set; }
public int ZipCode { get; set; }
public IndustryCategory Category { get; set; }
#region IValidationInfo Members
private readonly string[] _validationItems =
{
"Name",
"Street",
"City",
"State",
"ZipCode",
"Category"
};
public string[] ValidationItems
{
get { return _validationItems; }
}
public bool IsValid
{
get
{
List<string> validationErrors = GetValidationErrors();
if (validationErrors != null && validationErrors.Count > 0)
return false;
else
return true;
}
}
public void Validate()
{
List<string> errorMessages = GetValidationErrors();
if (errorMessages != null && errorMessages.Count > 0)
{
StringBuilder errorMessageReported = new StringBuilder();
errorMessages.ForEach(em => errorMessageReported.AppendLine(em));
throw new Exception(errorMessageReported.ToString());
}
}
public List<string> GetValidationErrors()
{
List<string> errorMessages = new List<string>();
foreach (string item in this.ValidationItems)
{
string errorMessage = GetValidationError(item);
if (!string.IsNullOrEmpty(errorMessage))
errorMessages.Add(errorMessage);
}
return errorMessages;
}
public string GetValidationError(string itemName)
{
//Validation methods should be called here... pretending nothings wrong for sake of discussion & simplicity
switch (itemName)
{
case "Name":
return string.Empty;
case "Street":
return string.Empty;
case "City":
return string.Empty;
case "State":
return string.Empty;
case "ZipCode":
return string.Empty;
case "Category":
return string.Empty;
default:
return "Invalid item name.";
}
}
#endregion
}
public class Product
{
public IndustryCategory Category { get; private set; }
public string Description { get; private set; }
public Product(IndustryCategory category, string description)
{
this.Category = category;
this.Description = description;
}
}
public class OrderDescription : IValidationInfo
{
public CustomerDescription Customer { get; private set; }
private List<Product> _products = new List<Product>();
public List<Product> Products
{
get { return new List<Product>(_products); }
private set { _products = value; }
}
public OrderDescription(List<Product> products, CustomerDescription customer)
{
this.Products = products;
this.Customer = customer;
}
public void PlaceOrder()
{
//If order valid, place
if (this.IsValid)
{
//Do stuff to place order
}
else
Validate(); //cause the exceptions to be raised with the validate because business rules were broken
}
public void AddProduct(Product product)
{
List<Product> productsToEvaluate = this.Products;
//some special read, validation, quantity check, pre-existing, etc here
// doing other stuff...
productsToEvaluate.Add(product);
this.Products = productsToEvaluate;
}
#region IValidationInfo Members
private readonly string[] _validationItems =
{
"Customer",
"Products"
};
public string[] ValidationItems
{
get { return _validationItems; }
}
public bool IsValid
{
get
{
List<string> validationErrors = GetValidationErrors();
if (validationErrors != null && validationErrors.Count > 0)
return false;
else
return true;
}
}
public void Validate()
{
List<string> errorMessages = GetValidationErrors();
if (errorMessages != null && errorMessages.Count > 0)
{
StringBuilder errorMessageReported = new StringBuilder();
errorMessages.ForEach(em => errorMessageReported.AppendLine(em));
throw new Exception(errorMessageReported.ToString());
}
}
public List<string> GetValidationErrors()
{
List<string> errorMessages = new List<string>();
foreach (string item in this.ValidationItems)
{
string errorMessage = GetValidationError(item);
if (!string.IsNullOrEmpty(errorMessage))
errorMessages.Add(errorMessage);
}
return errorMessages;
}
public string GetValidationError(string itemName)
{
switch (itemName)
{
case "Customer":
return ValidateCustomer();
case "Products":
return ValidateProducts();
default:
return "Invalid item name.";
}
}
#endregion
#region Validation Methods
private string ValidateCustomer()
{
string errorMessage = string.Empty;
if (this.Customer == null)
errorMessage = "CustomerDescription is missing a valid value.";
else
{
if (!this.Customer.IsValid)
{
List<string> customerErrors = this.Customer.GetValidationErrors();
customerErrors.ForEach(ce => errorMessage += "\n" + ce);
}
}
return errorMessage;
}
private string ValidateProducts()
{
string errorMessage = string.Empty;
if (this.Products == null || this.Products.Count <= 0)
errorMessage = "Invalid Order. Missing Products.";
else
{
foreach (Product product in this.Products)
{
if (product.Category != Customer.Category)
{
errorMessage += string.Format("\nThe Product, {0}, category does not match the required Customer category for {1}", product.Description, Customer.Name);
}
}
}
return errorMessage;
}
#endregion
}

Any reason you wouldn't want the constructor to noisily throw an exception if the information is valid? It's best to avoid ever creating an object in an invalid state, in my experience.

It's completely depends on the client. There's a trade-off as you already mentioned. By default approach number 1 is my favorite. Creating smart classes with good encapsulation and hiding details from client. The level of smartness depends who is going to use the object. If client is business aware you can reveal details according to the level of this awareness. This is a dichotomy and should not be treated as black or white.

Well if I correctly understood, there are basically two question - whether you should fail right away or later and whether you should omit/assume certain information.
1) I always prefer failing as soon as possible - good example is failing at compile time vs failing at run time - you always want to fail at compile time. So if something is wrong with the state of some object, as Jon said - throw exception right away as loudly as you can and deal with it - do not introduce additional complexity down the road as you'll be heading for if/elseif/elseif/elseif/else mumbo jumbo.
2) When it comes to user input, if you are in position to simply filter out errors automatically - just do it. For example, I almost never ask users for country - if I really need it, I automatically detect it from IP and display it in the form. It's way easier if user just needs to confirm/change the data - and I don't need to deal with null situation.
Now, in case we are talking about the data generated by code during some processing - for me situation is drastically different - I always want to know an much as possible (for easier debugging down the road) and ideally you never should destroy any piece of information.
To wrap up, in your case I would recommend that you keep IsValid as simple yes/no (not yes/no/maybe/kindaok/etc). If you can fix some problems automatically - do it, but consider that they keep object in IsValid yes. For everything else, you throw exception and go to IsValid=no.

Message queue in an IRC bot

I am currently writing an IRC bot. I'd like to avoid excess flood, so I decided to create a message queue that would send the next message every X milliseconds, but my attempt failed. Line 43:
unset.Add((string)de.Key);
throws an OutOfMemory exception. I have absolutely no idea what I'm doing wrong.
Perhaps I should also explain the general idea behind such (possibly complicated) way of queuing.
Firstly, the main Hashtable queueht stores ConcurrentQueue<string> types, where targets for the messages serve as keys. I would like the bot to iterate through the hashtable, sending one message from each queue (and removing the key if the queue is emptied). I couldn't think of a suitable method to work on the hashtable itself, so I decided to create another queue, ConcurrentQueue<string> queue, which would store keys and their order of use when emptying the queue.
Assuming a hypothetical situation with several hundred items in a queue (which might be possible), any new request would be delayed by Lord knows how long (built-in delay between messages plus latency), so I have the method Add() rebuild queue. I create a deep copy of queueht (or so I hope) and generate a new queue based on this disposable copy, getting rid of it in the process.
I assume my train of thought and/or code to be horribly wrong, since I have nearly no experience with threading, collections more complicated than simple arrays and OOP habits/conventions whatsoever. I would really appreciate the solution to my problem with an explanation. Thanks in advance!
EDIT: Posting the entire class.
class SendQueue
{
Hashtable queueht;
ConcurrentQueue<string> queue;
Timer tim;
IRCBot host;
public SendQueue(IRCBot host)
{
this.host = host;
this.tim = new Timer();
this.tim.Elapsed += new ElapsedEventHandler(this.SendNewMsg);
this.queueht = new Hashtable();
this.queue = new ConcurrentQueue<string>();
}
public void Add(string target, string msg)
{
try
{
this.queueht.Add(target, new ConcurrentQueue<string>());
}
finally
{
((ConcurrentQueue<string>)this.queueht[target]).Enqueue(msg);
}
Hashtable ht = new Hashtable(queueht);
List<string> unset = new List<string>();
while (ht.Count > 0)
{
foreach (DictionaryEntry de in ht)
{
ConcurrentQueue<string> cq = (ConcurrentQueue<string>)de.Value;
string res;
if (cq.TryDequeue(out res))
this.queue.Enqueue((string)de.Key);
else
unset.Add((string)de.Key);
}
}
if (unset.Count > 0)
foreach (string item in unset)
ht.Remove(item);
}
private void SendNewMsg(object sender, ElapsedEventArgs e)
{
string target;
if (queue.TryDequeue(out target))
{
string message;
if (((ConcurrentQueue<string>)queueht[target]).TryDequeue(out message))
this.host.Say(target, message);
}
}
}
EDIT2: I am aware that while (ht.Count > 0) will be executed indefinitely. It's just a part leftover from previous version which looked like that:
while (ht.Count > 0)
{
foreach (DictionaryEntry de in ht)
{
ConcurrentQueue<string> cq = (ConcurrentQueue<string>)de.Value;
string res;
if (cq.TryDequeue(out res))
this.queue.Enqueue((string)de.Key);
else
ht.Remove((string)de.Key);
}
}
But the collection cannot be modified when it's evaluated (and I found that out the hard way), so it's no longer like that. I just forgot to change the condition for while.
I took liberty of trying TheThing's solution. While it seems to fulfil its purpose, it doesn't send any messages... Here's its final form:
class User
{
public User(string username)
{
this.Username = username;
this.RequestQueue = new Queue<string>();
}
public User(string username, string message)
: this(username)
{
this.RequestQueue.Enqueue(message);
}
public string Username { get; set; }
public Queue<string> RequestQueue { get; private set; }
}
class SendQueue
{
Timer tim;
IRCBot host;
public bool shouldRun = false;
public Dictionary<string, User> Users; //Dictionary of users currently being processed
public ConcurrentQueue<User> UserQueue; //List of order for which users should be processed
public SendQueue(IRCBot launcher)
{
this.Users = new Dictionary<string, User>();
this.UserQueue = new ConcurrentQueue<User>();
this.tim = new Timer(WorkerTick, null, Timeout.Infinite, 450);
this.host = launcher;
}
public void Add(string username, string request)
{
lock (this.UserQueue) //For threadsafety
{
if (this.Users.ContainsKey(username))
{
//The user is in the user list. That means he has previously sent request that are awaiting to be processed.
//As such, we can safely add his new message at the end of HIS request list.
this.Users[username].RequestQueue.Enqueue(request); //Add users new message at the end of the list
return;
}
//User is not in the user list. Means it's his first request. Create him in the user list and add his message
var user = new User(username, request);
this.Users.Add(username, user); //Create the user and his message
this.UserQueue.Enqueue(user); //Add the user to the last of the precessing users.
}
}
public void WorkerTick(object sender)
{
if (shouldRun)
{
//This tick runs every 400ms and processes next message to be sent.
lock (this.UserQueue) //For threadsafety
{
User user;
if (this.UserQueue.TryDequeue(out user)) //Pop the next user to be processed.
{
string message = user.RequestQueue.Dequeue(); //Pop his request
this.host.Say(user.Username, message);
if (user.RequestQueue.Count > 0) //If user has more messages waiting to be processed
{
this.UserQueue.Enqueue(user); //Add him at the end of the userqueue
}
else
{
this.Users.Remove(user.Username); //User has no more messages, we can safely remove him from the user list
}
}
}
}
}
}
I tried switching to ConcurrentQueue, which should work as well (though in a more thread-safe way, not that I know anything about thread safety). I also tried switching to System.Threading.Timer, but that doesn't help either. I've run out of ideas long ago.
EDIT: Being a complete and utter idiot, I didn't set the time for Timer to start. Changing the bool part to a Start() method that changes the timer's dueTime and interval made it work. Problem solved.

From what I can best understand, you want to be able to queue users in order and each of their request.
Meaning, if one user request like 1000 request, others can still send theirs and the bot serves 1 request from each user in a FIFO manner.
If so, then what you need is a manner, similar to this functionality:
class User
{
public User(string username)
{
this.Username = username;
this.RequestQueue = new Queue<string>();
}
public User(string username, string message)
: this(username)
{
this.RequestQueue.Enqueue(message);
}
public string Username { get; set; }
public Queue<string> RequestQueue { get; private set; }
}
///......................
public class MyClass
{
public MyClass()
{
this.Users = new Dictionary<string, User>();
this.UserQueue = new Queue<User>();
}
public Dictionary<string, User> Users; //Dictionary of users currently being processed
public Queue<User> UserQueue; //List of order for which users should be processed
public void OnMessageRecievedFromIrcChannel(string username, string request)
{
lock (this.UserQueue) //For threadsafety
{
if (this.Users.ContainsKey(username))
{
//The user is in the user list. That means he has previously sent request that are awaiting to be processed.
//As such, we can safely add his new message at the end of HIS request list.
this.Users[username].RequestQueue.Enqueue(request); //Add users new message at the end of the list
return;
}
//User is not in the user list. Means it's his first request. Create him in the user list and add his message
var user = new User(username, request);
this.Users.Add(username, user); //Create the user and his message
this.UserQueue.Enqueue(user); //Add the user to the last of the precessing users.
}
}
//**********************************
public void WorkerTick()
{
//This tick runs every 400ms and processes next message to be sent.
lock (this.UserQueue) //For threadsafety
{
var user = this.UserQueue.Dequeue(); //Pop the next user to be processed.
var message = user.RequestQueue.Dequeue(); //Pop his request
/////PROCESSING MESSAGE GOES HERE
if (user.RequestQueue.Count > 0) //If user has more messages waiting to be processed
{
this.UserQueue.Enqueue(user); //Add him at the end of the userqueue
}
else
{
this.Users.Remove(user.Username); //User has no more messages, we can safely remove him from the user list
}
}
}
}
Basically, we have a queue of users. We pop the next user, process his first request and add him to the end of the user list if he has more request waiting to be processed.
Hope this clears some functionality. For the record, the code above is more of a pseudocode than a functional code xD

From what I can see, you never escape from the while since you never remove items from the temporary hashtable ht until outside of it. Thus, the count will always be > 0.

Try this:
class User
{
public User(string username)
{
this.Username = username;
this.RequestQueue = new Queue<string>();
}
private static readonly TimeSpan _minPostThreshold = new TimeSpan(0,0,5); //five seconds
public void PostMessage(string message)
{
var lastMsgTime = _lastMessageTime;
_lastMessageTime = DateTime.Now;
if (lastMsgTime != default(DateTime))
{
if ((_lastMessageTime - lastMsgTime) < _minPostThreshold)
{
return;
}
}
_requestQueue.Enqueue(message);
}
public string NextMessage
{
get
{
if (!HasMessages)
{
return null;
}
return _requestQueue.Dequeue();
}
}
public bool HasMessages
{
get{return _requestQueue.Count > 0;}
}
public string Username { get; set; }
private Queue<string> _requestQueue { get; private set; }
private DateTime _lastMessageTime;
}
class SendQueue
{
Timer tim;
IRCBot host;
public bool shouldRun = false;
public Dictionary<string, User> Users; //Dictionary of users currently being processed
private Queue<User> _postQueue = new Queue<User>();
public SendQueue(IRCBot launcher)
{
this.Users = new Dictionary<string, User>();
this.tim = new Timer(WorkerTick, null, Timeout.Infinite, 450);
this.host = launcher;
}
public void Add(string username, string request)
{
User targetUser;
lock (Users) //For threadsafety
{
if (!Users.TryGetValue(username, out targetUser))
{
//User is not in the user list. Means it's his first request. Create him in the user list and add his message
targetUser = new User(username);
Users.Add(username, targetUser); //Create the user and his message
}
targetUser.PostMessage(request);
}
lock(_postQueue)
{
_postQueue.Enqueue(targetUser);
}
}
public void WorkerTick(object sender)
{
if (shouldRun)
{
User nextUser = null;
lock(_postQueue)
{
if (_postQueue.Count > 0)
{
nextUser = _PostQueue.Dequeue();
}
}
if (nextUser != null)
{
host.Say(nextUser.Username, nextUser.NextMessage);
}
}
}
}
UPDATE: changed after better understanding requirements.
This provides both per user flood control and overall throttling. It is also much simpler.
Note this was written on the fly and hasn't even been compiled, and there are probably some threading issues around User instances that need to be considered, but it should work.