We Keep Coding

S22.Imap System.IO.IOException: The stream could not be read, during client.GetMessages(uids)

I have an application that processes invoices from vendors by using a Gmail inbox and S22.Imap to read emails and pdf attachments. It has been used for several years now without any issue, but lately, I have been seeing "The stream could not be read" errors when trying to get all messages from the inbox to be processed using the command:
IEnumerable<uint> uids = client.Search(SearchCondition.Unseen());
IEnumerable<MailMessage> messages = client.GetMessages(uids);
This exception seems to only occur when a certain vendor sends a large number of emails at once. I have been looking through the S22.Imap documentation, but have not found anything of help.
Note: I did see this way of getting emails, but the original method I used above is the actual example for "Downloading unseen mail messages" in the documentation:
// Fetch the messages and process each one
foreach (uint uid in uids)
{
MailMessage message = client.GetMessage(uid);
ProcessMessage(message);
}
My problem here is that I have not been able to reproduce the exception locally, and I don't really understand how it relates to the client.GetMessages(uids) command.
Is there a maximum number of messages that can be processed this way, or is there a better way to read and process emails?
If client.GetMessages(uids) is the best way to get those emails, is there a good way that I could catch this exception and continue when it does happen?
I would appreciate any advice. Thanks,

Not sure if this will ever even help anyone since it is so particular to this one situation with how we are using s22 to process emails, but I figured I would post the answer anyhow.
The main issue that was happening was that all the emails were being marked 'unread' when using the client.GetMessages(uids) syntax. This marked all emails in the inbox as 'read' (which I counted unread emails to make sure that every email was processed). Changing this method to go through the collected uid one per time prevented this problem. The error still happens once in a while, but it does not mess anything up for the user. The message that failed to get processed just gets processed the next time the app runs. So basically, I just changed how I retrieved the unread emails with S22.Imap and dealt with the errors by catching them and moving on.
//get the message one at a time from uids. catching any rare errors with a message
try
{
MailMessage message = client.GetMessage(uid);
ProcessMessage(message);
}
catch(Exception e)
{
System.Threading.Thread.Sleep(1000);
SendErrorEmail("An exception has been caught: " + e.ToString() +
Environment.NewLine + "UID: " + uid.ToString());
}

Send bulk email with MailKit in C#

We are using MailKit to successfully send emails by creating a client, making a connection and sending a mail. Very standard stuff which works great as we receive a message in our application, we do something and send on an email.
However we want to be able to process 10's of thousands of emails as quickly as possible.
Our destination email server may be unavailable and therefore we could quickly backup messages therefore producing a backlog.
With MailKit, what is the best and quickwest way to process the mails so that they get sent as quickly as possible. For example at the moment each mail may be processed one after the other and if they take a second each to process it could take a long time to send 40000 mails.
We have been using a parallel foreach to spin up a number of threads but this has limitations. Any suggestions or recommendations would be appreciated.
Code sample added: CORRECTION, NEW CODE SAMPLE ADDED. This is much faster but I cannot get it to work creating a new connection each time. Exchange throws errors 'sender already specified'. This is currently sending around 6 mails per second on average.
var rangePartitioner = Partitioner.Create(0, inpList.Count, 15);
var po = new ParallelOptions { MaxDegreeOfParallelism = 30 };
Parallel.ForEach(rangePartitioner, (range, loopState) =>
{
using (var client = new SmtpClient(new SlabProtocolLogger()))
{
client.Connect(_appSettings.RelayAddress, _appSettings.RelayPort);
client.AuthenticationMechanisms.Remove("XOAUTH2");
for (int i = range.Item1; i < range.Item2; i++)
{
var message = _outboundQueueRepository.Read(inpList[i]).Load();
client.Send(message.Body, message.Metadata.Sender, message.Metadata.Recipients.Select(r => (MailboxAddress)r));
_outboundQueueRepository.Remove(inpList[i]);
};
}
});

Correct me if I'm wrong, but it looks to me like the way this works is that the Parallel.Foreach is creating some number of threads. Each thread is then creating an SMTP connection and then looping to send a batch of messages.
This seems pretty reasonable to me.
The only advice I can give you that might optimize this more is if many of these messages have the exact same content and the same From address and that the only difference is who the recipients are, you could vastly reduce the number of messages you need to send.
For example, if you are currently doing something like sending out the following 3 messages:
Message #1:
From: no-reply#company.com
To: Joe The Plumber <joe#plumbing-masters.com>
Subject: We've got a new sale! 50% off everything in stock!
some message text goes here.
Message #2
From: no-reply#company.com
To: Sara the Chef <sara#amazing-chefs.com>
Subject: We've got a new sale! 50% off everything in stock!
some message text goes here.
Message #3:
From: no-reply#company.com
To: Ben the Cobbler <ben#cobblers-r-us.com>
Subject: We've got a new sale! 50% off everything in stock!
some message text goes here.
Your code might create 3 threads, sending 1 of the messages in each of those threads.
But what if, instead, you created the following single message:
From: no-reply#company.com
To: undisclosed-recipients:;
Subject: We've got a new sale! 50% off everything in stock!
some message text goes here.
and then used the following code to send to all 3 customers at the same MimeMessage?
var sender = new MailboxAddress (null, "no-reply#company.com");
var recipients = new List<MailboxAddress> ();
recipients.Add (new MailboxAddress ("Joe the Plumber", "joe#plumbing-masters.com"));
recipients.Add (new MailboxAddress ("Sara the Chef", "sara#amazing-chefs.com"));
recipients.Add (new MailboxAddress ("Ben the Cobbler", "ben#cobblers-r-us.com"));
client.Send (message, sender, recipients);
All 3 of your customers will receive the same email and you didn't have to send 3 messages, you only needed to send 1 message.
You may already understand this concept so this might not help you at all - I merely mention it because I've noticed that this is not immediately obvious to everyone. Some people think they need to send 1 message per recipient and so end up in a situation where they try to optimize sending 1000's of messages when really they only ever needed to send 1.

So we found a wider ranging fix which improved performance massively in addition to the improvements we found with the Parrallel ForEach loop. Not related to MailKit but I thought I would share anyway. The way that our calling code was creating our inputList was to use DirectoryInfo.GetDirectories to enumerate over all the first in the directory. In some cases our code took 2 seconds to execute over the directory with 40k files in it. We changed this to EnumerateDirectories and it effecitvely freed up the mail sending code to send many many emails.
So to confirm, the Parallel loop worked great, underlying performance issue elsewhere was the real bottleneck.

How to Check SMTP Server is up and running

I am currently checking SMTP Server is up and running before sending mail using below code :
public static bool ValidateSMTPServer()
{
bool valid = false;
System.Net.Sockets.TcpClient smtpTest = new System.Net.Sockets.TcpClient();
try
{
smtpTest.Connect(ConfigurationManager.AppSettings["SMTPServerName"], Convert.ToInt32(ConfigurationManager.AppSettings["PortNumber"]));
return true;
}
catch
{
return valid;
}
finally
{
smtpTest.Close();
}
}
But the problem is it takes around 25 seconds to respond back if server is not running..which decrease my page performance. I do this on Click of button before sending mails.
Is there any better approach than this ? or i can do some changes to existing code.
And in case of server is not running it goes to catch part and i make my flag return false which means smtp is down..is there any way i can get the status code that server is not running...SMTPException is also i tried..if i use the same program breaks.
Any help is appreciated.

You can check in the background - and store that in a static variable. Do it in a loop and you avoid the wait.
The setup could be changed using a local SMTP Server to take them from a drop folder. Lots of pseudo negatives but at least you can send when the server is down / overloaded / starting. I personally prefer that. I used that way for ages - sadly MS slowly is retiring their SMTP service. It is great if a server can just drop emails in a folder and then they get sent over time.
In a larger setup the SMTPO servre should not be down because it is a cluster behind a load balancer ;)

How do you loop in C# until a new email has arrived?

using System;
using Limilabs.Mail;
using Limilabs.Client.POP3;
class Program
{
static void Main(string[] args)
{
using (System.IO.StreamWriter file = new System.IO.StreamWriter(#"C:\Users\*******\Desktop\WriteLines.txt", true)){
using (Pop3 pop3 = new Pop3())
{
Console.WriteLine("Connecting...");
pop3.ConnectSSL("pop3.live.com"); // or ConnectSSL for SSL
pop3.Login("****#live.com", "****");
// Receive all messages and display the subject
MailBuilder builder = new MailBuilder();
foreach (string uid in pop3.GetAll())
{
IMail email = builder.CreateFromEml(
pop3.GetMessageByUID(uid));
file.WriteLine("Header: " + email.Subject);
file.WriteLine("Message: " + email.Text);
}
pop3.Close();
}
}
}
}
So I have this problem where I want to have this program running 24/7; it is still incomplete but this is just to show basic C# retrieving emails and such. How can I have a loop in C# that only gets halted or forced to anything only when a new email arrives in the inbox? I would like the program only to actually do something upon getting an email sent to in in realtime. I know I can go and print out my emails one by one all at once but I want it to be running and only when a new message is received do I want i to do anything. For example idk, if I was to send a message like clean desktop, it would stop and be parsed and if a valid command sequence specified by another program I am going to make, then it will carry out that command and then keep on looping endlessly waiting for another new message. It'll basically be a server running 24/7 that only responds to new emails. That is what I am going for. All help is greatly appreciated.

Pop3 is a polling protocol. It does not allow to inform you that a new mail has arrived. You would have to e.g. the Exchange protocol that supports this afaik. Thus, for POP3 mailboxes, mail programs usually poll for new emails every e.g. 15min. I'd suspect that your POP3-library does not support this directly. Thus, you would have to use a Timer class and poll for new emails.
Documentation for the Timer class can be found in the MSDN library:
http://msdn.microsoft.com/en-us/library/system.timers.timer%28v=vs.110%29.aspx

You would probably want to use a windows service that polls your email inbox.
A similar question was asked about how to do this here. One of the answers provides a good layout in how to do this.
In addition, if you are trying have your website perform this functionality (because you don't have access to install a windows service) you could use Quartz.net highlighted in this question.

Why can SmtpClient.SendAsync only be called once?

I'm trying to write a notification service (for completely legit non-spam purposes) in .NET using SmtpClient. Initially I just looped through each message and sent it, however this is slow and I would like to improve the speed. So, I switched to using 'SendAsync', but now get the following error on the second call:
An asynchronous call is already in progress.
I read this to mean that MS crippled System.Net.Mail to prevent mass-mailers. Is this correct? If so, is there a better way to do this in .NET, and still be able to log the results of each email(which is important to our client). If not, why can SendAsync only be called once?

According to the documentation:
After calling SendAsync, you must wait
for the e-mail transmission to
complete before attempting to send
another e-mail message using Send or
SendAsync.
So to send multiple mails at the same time you need multiple SmtpClient instances.

You might be able to use the following:
ThreadPool.QueueUserWorkItem(state => client.Send(msg));
This should allow your messages to be queued and sent as threads become available.

Obviously, this is not an attempt to stop mass mailers.
The reason is that the SmtpClient class is not thread safe. If you want to send multiple emails simultaneously, you have to spawn a few worker threads (there are a few ways to do that in the .NET Framework) and create separate instances of SmtpClient in each of them.

I think you misunderstand the XXXAsync class of methods. The purpose of these asynchronous calls is to allow the program to continue running, without the need of the method to finish processing and return first. You can then proceed with the result later by subscribe to something like XXXReceived event of the object.
To send more than one mail simultaneously, you may consider using more Threads.

You may only send one at a time per SMTP client. If you wish to make more than one send call, create more than one SMTP client.
HTH,
Colby Africa

As noticed by everyone else here, you can only send one email at a time, but the way to send another once the first has been sent is to handle the .SendCompleted event of the SmtpClient class, and then move on to the next email and send that.
If you want to send many emails simultaneously, then as the others have said, use multiple SmtpClient objects.

There is a reason to reuse the SmtpClient, it limits the # of connections to the SMTP server. I cannot instantiate a new class SmtpClient class for each thread the reports are building on or the SMTP server will balk with too many connections error. This is the solution I came up with when I couldn't find an answer here.
I ended up using an AutoResetEvent for keeping everything in sync. That way, I can keep calling my SendAsync in each thread, but wait for it to process the email and use the SendComplete event to reset it so the next one can continue.
I setup the Auto Reset Event.
AutoResetEvent _autoResetEvent = new AutoResetEvent(true);
I setup the shared SMTP Client when my class is instantiated.
_smtpServer = new SmtpClient(_mailServer);
_smtpServer.Port = Convert.ToInt32(_mailPort);
_smtpServer.UseDefaultCredentials = false;
_smtpServer.Credentials = new System.Net.NetworkCredential(_mailUser, _mailPassword);
_smtpServer.EnableSsl = true;
_smtpServer.SendCompleted += SmtpServer_SendCompleted;
Then when I call the send async, I wait for the event to clear, then send the next one.
_autoResetEvent.WaitOne();
_smtpServer.SendAsync(mail, mail);
mailWaiting++;
I use the SMTPClient SendComplete event to reset the AutoResetEvent so the next email will send.
private static void SmtpServer_SendCompleted(object sender, System.ComponentModel.AsyncCompletedEventArgs e)
{
MailMessage thisMesage = (MailMessage) e.UserState;
if (e.Error != null)
{
if (e.Error.InnerException != null)
{
writeMessage("ERROR: Sending Mail: " + thisMesage.Subject + " Msg: "
+ e.Error.Message + e.Error.InnerException.Message);
}
else
{
writeMessage("ERROR: Sending Mail: " + thisMesage.Subject + " Msg: " + e.Error.Message);
}
}
else
{
writeMessage("Success:" + thisMesage.Subject + " sent.");
}
if (_messagesPerConnection > 20)
{ /*Limit # of messages per connection,
After send then reset the SmtpClient before next thread release*/
_smtpServer = new SmtpClient(_mailServer);
_smtpServer.SendCompleted += SmtpServer_SendCompleted;
_smtpServer.Port = Convert.ToInt32(_mailPort);
_smtpServer.UseDefaultCredentials = false;
_smtpServer.Credentials = new NetworkCredential(_mailUser, _mailPassword);
_smtpServer.EnableSsl = true;
_messagesPerConnection = 0;
}
_autoResetEvent.Set();//Here is the event reset
mailWaiting--;
}