I have created a simple form which is loading a csv file with few columns like email, name, city and I'm trying to send them daily updates via 2 smtp servers I have. I thought to use a backgroundworker because it's progress capabilities(for the progressbar I have) but I read also on this website that there are other alternatives like task, thread or threadpool.
I also read that sending emails must be done with an async method. I don't know if it's possible or if it's optimized for performance, but I'm trying to do the following:
for each smtp I wanna use a thread to read, let's say, 10 lines from the csv file,
split the line by fields and send the info to another thread which will send the message to each email in that small list. I wanna do this to save some precious time for each smtp auth procedure.
Because connecting to the smtp for each email requires the whole socket procedure everytime, like HELO, AUTH, DATA. If I'm gonna send a list of email, I can keep the socket open after auth and change only the data I send. 10 lines is just a variable which can be custom changed.
So, 2 tasks for each smtp, means 4 in total or can be increased based on the number of smtps I will use.
Can I also use additional threads for each task or backgroundworker?
I'm kinda confused because I'm new to c# and I haven't found any example about this. It's been more than a week since I'm trying to understand how backgroundworker/threads work but still unsuccessfully. Any help would be appreciated and any idea better than mine to improve the performance Thanks!
First off you need to understand the difference between the different types of threads.
A threadpool is just a collection of threads available for use. The threads used by the pool are managed by the pool & returned there after use. The idea is you get performance benefits because the cost of creating the thread only happens once.
A background worker is a thread that runs in the background. The only difference between a background and foreground thread, is a background thread will not prevent a process from terminating.
Sending emails async is more SHOULD rather than MUST. The smtp component has no concept of seperating the connect form the send.
I think what you want to do is:
initalise a number of threads, say 10.
Read a line from the csv.
Pass this line to a thread from above. If no threads are available, wait till one is.
In the thread parse the line and send the mail. Sending doesn't need to be async with this method but you can if you want. When you're done sending, return the thread to the pool ready for the next line
You can manage your threads as a pool - there are many examples around if you google for thread pools - or you can do it by hand using manualresetevents to handle the wait. The advantage of doing it with a pool is you can change the number of workers available easily without affecting your code.
Related
I have my main Winforms application.
There are 6 Threads working in parrael + main thread, atleast that is what it ment to be .
I have created one Thread that is an Tcp Server, listening on specific port.
listenerThread = new Thread(new ThreadStart(AsynchronousSocketListener.StartListening));
listenerThread.Start();
I have also 5 different Threads that are doing different kind of stuff (for example updating database, counting averages/sums, being TCP Clients etc.)
My question is:
Is it possible that my TCP Server (which is working on one of 6 threads) wont read a message when one of those 5 different threads thread will take the computing power of CPU, and the TCP Server's Thread thread will have to wait ?
Another question: If that could happend, how could i avoid that ?
This is a summary of my comments above
"Is it possible that my TCP Server (which is working on one of 6 threads) wont read a message when one of those 5 different threads thread will take the computing power of CPU, and the TCP Server's Thread thread will have to wait ?"
Received data is buffered to an extent however if your code does not respond in an appropriate time then it could result in dropped data.
The same could be said in a single-core system and another app, say Prime95 is busy not playing nice and calculating prime numbers right next to yours.
Another question: If that could happend, how could i avoid that ?
When handling I/O and I'll focus on TCP here is to perform the minimal amount of processing in your data received handler irrespective of whether that handler is of the form IAsyncResult or async/await.
A good general flow is:
start an asynchronous read
read result of asynchronous read
place result in a queue
loop back to #1
Meanwhile you process the read results from #2 in a different mechanism whether that be a timer; GUI app-idle-loop; or a different thread so long as the thread processing the results has nothing to do with the flow above.
The reason being is that in a scenario involving reasonably high data transfer rates, if you were to read a block of data and then proceed to immediately update that Telerik datagrid UI showing 1000s of rows, there is a high chance that the next read operation will result in dropped data because you didn't respond in a sufficient amount of time.
I have following scenario:
C# application (.net 4.0/4.5), with 5-6 different threads. Every thread has a different task, which is launched every x seconds (ranging from 5 to 300).
Each task has following steps:
Fetch items from Sql Server
Convert items in Json
Send data to webserver
Wait for reply from server.
Since this tasks can fail at some point (internet problems, timeout, etc) what is best solution in .NET world?
I thought about following solutions:
Spawn new thread every x seconds (if there is not another thread of this type in execution)
Spawn one thread for each type of task and loop steps every x seconds (to understand the way to manage exceptions)
Which would be more secure and robust? Application will run on unattended systems, so it should be able to remain in execution regardless of any possible exception.
Threads are pretty expensive to create. The first option isn't a great one. If the cycle of "do stuff" is pretty brief (between the pauses), you might consider using the ThreadPool or the TPL. If the threads are mostly busy, or the work takes any appreciable time, then dedicated workers are more appropriate.
As for exceptions: don't let exceptions escape workers. You must catch them. If all that means is that you give up and retry in a few seconds, that is probably fine.
You could have modeled the whole thing using a producer consumer pattern approach. You have a producer who puts the new task description in the queue and you can have multiple consumers (4 or 5 threads) who process from the queue. The number of consumers or the processing thread could vary depending on the load, length of the queue.
Each task involves reading from DB, converting the format, sending to web server and then process the response from web server. I assume each task would do all these steps.
In case of exceptions for an item in the queue, you could potentially mark the queue item as failed and schedule it for a retry later.
Is there a way in C# to send a message to another thread based on the thread's thread id or name?
Basically for a project in school, my professor wants us to do a producer/consumer deal, but passing objects serialized to a string(such as xml) from producer to consumer. Once a string is pulled from a buffer in the consumer thread, each of those strings is decoded(including the threadid) and processed and the original producer is notified via callback. So how do I send an event to the original producer thread with just the thread id?
You can write a class which has a Dictionary< string, thread > member containing all your threads. When you create a thread add it to the dictionary so you can return it by name (key) later from anywhere in the class. This way you can also share resources among your threads, but be sure to lock any shared resources to prevent concurrency issues.
Imagine you run a company, and you could hire as many employees as you liked, but each employee was really single-minded, so you could only give them one order ever. You couldn't get much done with them, right? So if you were a smart manager, what you'd do is say "Your order is 'wait by your inbox until you get a letter telling you what to do, do the work, and then repeat'". Then you could put work items into the worker's inboxes as you needed work done.
The problem then is what happens if you give an employee a long-running, low priority task (let's say, "drive to Topeka to pick up peanut butter for the company picnic"). The employee will happily go off and do that. But then the building catches fire, and you need to know that if you issue the order "grab the fire extinguisher and put the fire out!" someone is going to do that quickly. You can solve that problem by having multiple employees share a single inbox- that way, there is a higher probability that someone will be ready to execute the order to douse the flames, and not be off driving through Kansas.
Guess what? Threads are those difficult employees.
You don't "pass messages to a thread". What you can do is set up a thread or group of threads to observe a common, shared data structure such as a blocking queue (BlockingCollection in .NET, for example), and then put messages (like your strings) into that queue for processing by the consumer threads (which need to listen on the queue for work).
For bidirectional communication, you would need two queues (one for the message, and one for the response). The reason is that your "main" thread is also a bad employee- it only can process responses one at a time, and while it is processing a response from one worker, another worker might come back with another response. You'd want to build a request/response coordination protocol so that the original requestor knows which request a response is associated with- usually requests have an ID, and responses reference the request's ID so that the original requestor knows which request each response is for.
Finally you need proper thread synchronization (locking) on the queues if that isn't built in to the Producer/Consumer queue that you are working with. Imagine if you were putting a message into a worker's inbox, and that worker was so eager to read the message that he grabbed it from your hand and tore it in half. What you need is the ability to prevent more than one thread from accessing the queue at a time.
When using threads you do not try to send messages between them. Threads can use a shared memory to synchronize themselves - This is called synchronized objects. In order to manage threads for a consumer/producer system you can use a queue (a data structure) and not a message system. (see example here: C# producer/consumer).
Another possible solution (which I would not recommend) is : You can use GetThreadId to return the ID of a given native thread. Then all you need to find is the thread handle and pass it to that function. GetCurrentThreadId returns the ID of the current thread. There you can access it's name property.
A message is simply a method call, and to make a method call you first need an instance object which expose some methods to be called, thus sending a message to a thread means finding active object which lives in that thread and calling it's specific method.
Finding each thread's main worker's object could be handled through the threads coordinator, so if an object in a specific thread wants to send a message to another object (in other thread), it would first send it's request to threads coordinator and the coordinator sends the message/request to it's destination.
What i have now is a real-time API get bunch of messages from network and feed into pubsub manager class. there might be up to 1000 msg/sec or more at times. there are 2 different threads each connected to its own pubsub. subscribers are WPF windows. manager keeps list of windows and their DispatcherSynchornisationContext.
A thread calls the manager through interface method.
Manager publishes through Post:
foreach (var sub in Subscribers[subName])
{
sub.Context.Post(sub.WpfWindow.MyDelegate, data);
}
can this be optimised.
P.S. Please dont ask why do I think it is slow and all.. I dont have limits. Any solution is infinitely slow. I have to do my best to make it as fast as possible. I am asking for help to assess - can it be done faster? Thank you.
EDIT: found this: http://msdn.microsoft.com/en-us/library/aa969767.aspx
The argument with a queue stays. WHat I do is put stuff into the queue, the queue triggers a task that then invokes into the messaging thread and pulls X items of data (1000, or how many there are). The one thing that killed me was permanent single item invokation (which is slow), but doing it batchy works nicely. I can keep up wiitz nearly zero cpu load on a very busy ES data feed in crazy times for time and sales.
I have a special set of componetns for that that I will open source one of the next week and there is an ActionQueue (taking a delegate to call when items need processing). This is now a Task (was a queued thread work item before). I took time to process up 1000 messages per invocation - but if you do a price grid you may need more.
Note: use WPF hints to enable gpu caching of rendered bitmaps.
In addition:
Run every window on it's own thread / message pump
HEAVILY use async queues. The publisher should never block, every window has it's own target queue that is async.
You want processing as decoupled as possible. Brutally decoupled.
Here is my suggestion for you:
I would use a ConcurrentQueue (comes with the namespace System.Collections.Concurrent;) The background workers feed their messages in that queue. The UI Thread takes a timer and draws (let's say every 500 msec) a bunch of messages out of that queue and shows them to the user. Another possible way is, that the UI thread only will do that on demand of the user. The ConcurrentQueue is designed to be used from different thread and concurrently (as the name says ;-) )
I've got an architecture that involves browsers polling via ajax every 3 seconds for updates and I'd like to change that to long-polling.
I'd like to have 1, 2.. {n} clients long-polling, waiting for updates and have something happen on the server to signal the waiting clients to return. My first thought was to use an EventWaitHandle, and I can do this easily if I just want to support 1 client. I'd just have an AutoResetEvent WaitHandle that would WaitOne to block the client, maybe with a timeout, maybe not. Either way, an AutoResetEvent will only allow me to support 1 client (since it only wakes 1 waiting thread) and I want n clients.
I'm pretty sure I need to use a ManualResetEvent WaitHandle, but I'm not sure when to call Reset after I Set it (when waking the threads). Should I simply Thread.Sleep some arbitrary amount in between the Set and Reset?
In psuedo code, the waking logic would be
get ManualResetEventWaitHandle
call Set
ensure all waiting clients have woken, while preventing new requests from blowing through
call Reset now that all waiting clients have received their updates
Its that 3rd line that i'm having a hard time with. Currently I am tossing around the idea of having a LastTxID that the client / server maintain and potentially using 2 wait handles. However, before I went crazy with this implementation I wanted to get feedback here to see how they would implement the waking logic.
Edit: assume I've got the problems associated with having max concurrent users figured out, either by tweaking IIS or hosting via WCF or some other solution. I only want to focus on the waking logic.
One idea, in pseudo code
Maintain a thread-safe list of connection id's, possibly use the session id
Give every connection its own AutoResetEventWaitHandle
In a thread safe manner, loop through those wait handles and set them when there is an update
on session end, in a thread safe manner, remove that connection / session id from the list
I'd love to get some feedback on this
con's to this approach
have to maintain a list of connections
have to generate {n} wait handles for {n} clients