I am building a web site, and the client wants a newsletter "system" on it.
How do I send this kinds of mass (>1000) emails?
I read somewhere that using sendasync method of the smtpclient does the trick.
But it constantly gives me an "Email faliure" exception. And, I don't know what to do with that right there...
So, basically my question is, is it ok to send the emails using the SEND method of the smtpclient, but each mail in it's own thread.
for eg.
NewsletterEmail newsletterEmail = new NewsletterEmail(emailAdress[i], mailSubject, mailBody);
Thread t = new Thread(new ThreadStart(newsletterEmail.MakeAndSendEmail));
t.IsBackground = true;
t.Start();
i think , you should rethink about your startegey for sending bulk emails
Creating > 1000 threads is not a good idea , it may even crash your server or it may make your server respond very slow.
Tell your client about Constant Contact. They are going to handle this much better than you ever could. It's also cheaper than your time.
In the event that fails you have a couple options.
If they already have an email server, leverage that to do the email broadcasting. In other words, relay the mail through that server.
If you can't do that, go download a free email server. I've been using hMailserver. Set it up and relay through it.
If you can't do that, write your own SMTP processing engine. Don't attempt to send the emails directly from ASP.Net. Queue them up in a database and write a windows service to handle the mail broadcasting.
Sending emails can sometimes take several seconds per email. This could completely hose your website while it's trying to handle sending 1000 emails.
A number of mail servers are configured with grey listing, meaning that they require you to send the same email twice in order to prove you aren't a spammer.
Next, getting the DNS appropriately set up can be a PITA. Which is why I suggest constant contact. I have one client that took nearly 5 years to finally get their DNS configured; and yes, I gave them explicit instructions once a year on what to do. Reverse DNS is critical.
Another thing is that some recipient servers have a limit on the number of threads they will accept from you at once. Most mail servers are built to take this into consideration. If you cross that boundary, then the recipient servers will consider you a spammer and take appropriate action.
Another problem area is in sending to a bad address, over and over again. AOL and others will consider you a spammer just for this one thing.
Point is, you really don't want to write this yourself.
Your best bet would be to have a separate process send the emails. Either have it run on a schedule to check for emails that need to be sent (maybe store the emails in a table?), or, if you don't like the scheduled process idea, then you can have a console application that is started by your website.
Something else to keep in mind is that if you are sending too many emails in a short period of time, it becomes very easy to get black-listed and then none of the emails from your domain will make it to any servers that have you black-listed.
I've taken two approaches:
1) Lazy, sloppy approach of configuring the asp.net process timeout long enough to complete via Send
2) Create a console app that is spawned by the web app.
I built a similar system a year or two ago. There's so many things that can go wrong when you send an email programmatically. For this reason, do yourself a favor and seperate the messages from the process of sending them. This way, you can "teach" your system that "badEmail#goodDomian.com" should be ignored or other similar situations.
You can store the message, subject, or whatever level of seperation of data you desire in the database along with a flag for meta-data like "SentOn", "FailedOn", etc. I sent my message one at a time to allow for individual errors to be stored and/or handled. I used SmtpMail.Send(), but whatever method you choose should work as long as you built something smart and recoverable.
Have a look at the MailChimp API: http://www.mailchimp.com/api/gettingstarted/
Related
I've been tasked with creating some sort of service that will take any e-mail sent to an e-mail address and handle the contents of the e-mail (including binary attachments.)
I've got full access to the server (Windows Server 2008) that will run this service, and have decided to code the solution using the .NET framework (3.5).
Now I'm thinking about the different ways I can do this. Since I'm not very experienced in handling e-mails programmatically, the first solution that came into my head was to create an e-mail client that will periodically poll an existing e-mail server for incoming e-mail (using the POP3 protocol) and process them. But there are other ways to do it.
One could be to use IMAP to stay connected to the mail server and fetch e-mails as soon as they arrive (more responsive, but I believe the IMAP protocol is harder to implement.) The follow-up question here is: where can I find reliable e-mail libraries that support the POP3 or the IMAP protocol?
Another could be to somehow configure the e-mail server to directly pipe the e-mails sent to a specific address to my binary code (I've seen this done on Linux servers.) I have no idea how to go about this, though.
The last I can think of would be to create a dummy e-mail server on its own address that handles the e-mail directly as it arrives, but to me this seems like a bad idea.
Does anyone have experience in this area? I would imagine that having to write e-mail handlers for ticket support systems isn't that uncommon, all I need in addition to that is to handle the attachments.
I'd be grateful for any hints and tips.
As with alot of things - it depends. Ask yourself the following questions:
What are your latency requirements--do you need to process incoming messages as quickly as possible, or can processing be batched? If it can be batched, then how often would you have to process the "inbox"?
What are your throughput requirements? How many messages are we talking about per minute here? How big are the messages? This would affect the decision you make about polling interval if using a batch scenario;
What sort of e-mail system are you integrating with? If it's Exchange, what programmatic interfaces are available to access a mailbox? Until the most recent version of Exchange, interestingly enough, there were issues with accessing a mailbox on an Exchange server (The client-side CDO COM components needed to be used which is not ideal---and there were security limitations).
By far the simplest approach is to poll a mailbox using POP3. However, if you need to respond immediately to an incoming message, then this isn't going to cut it.
As far as possible avoid writing your own SMTP service--it's been done a thousand times before and you're just creating unnecessary work for yourself and exposing yourself to security threats. If you absolutely have to respond immediately to messages, then rather set up an instance of Sendmail or Postfix to spawn a process that you have written.
If you're going to go for the POP3 solution (it looks like you are), then have a read of related questions "Free POP3 .NET library?" and "Reading Email using POP3 in C#".
I've used webdav in the past with c# to access an exchange server periodically and process emails.
This has worked quite well, and I'd probably use that method again if I need to do it.
Some of the .net components from http://www.quiksoft.com/ might help with your requirement.
The app polls a POP3 mail server every x minutes(s) and works it's way through the messages in the queue, and deletes them when it's processed each msg.
The QuikSoft tools also provide ways to parse the emails to get the content from each msg.
Receiving the email is not the hardest part, parsing, extracting attachments is.
If any one is interested in commercial product take a look at Mail.dll.
It supports IDLE command you have mentioned for instant notifications.
Mail.dll includes POP3, IMAP clients and powerful MIME parser:
using(Imap imap = new Imap())
{
imap.Connect("imap.server.com");
imap.Login("user", "password");
imap.SelectInbox();
List<long> uidList = imap.SearchFlag(Flag.Unseen);
foreach (long uid in uidList)
{
IMail mail = new MailBuilder()
.CreateFromEml(imap.GetMessageByUID(uid));
Console.WriteLine(mail.Subject);
}
imap.Close(true);
}
Please note that this is commercial product that I've created.
You can download it at http://www.lesnikowski.com/mail
I have a client who would like a system developed which handles sending out alert emails to opted-in registered users. The alerts are based on geographic travel-related incidents and these are provided via a third party web service. The registered userbase is expected to be around 500,000 users and each user can subscribe to multiple alert categories.
I'm just trying to scope out what would be required technically to create this alerting functionality. In my mind we would create something like the following:
Poll the alert service once hourly
Store a "queue" of alerts in a temporary database table
Process the queue table as a scheduled task and send out the emails to the opted-in users
The part that I'm really not sure about is the physical sending of the emails. I was wondering if anyone could recommend any good options here. I guess we're looking to either create a custom component to send out the emails or use a dedicated third party service (like dotMailer maybe? Or a custom mass mail server?). I've read that rolling out your own solution runs the risk of having IP's blacklisted if you're not careful, and obviously we want to keep this all completely legitimate.
I should also mention that we will be looking to create the solution using .NET (C#) and we're limited to SQL Server Express 2008 edition.
Any thoughts much appreciated
Many thanks in advance
For the Poll , Queue and Send operations I'd create a windows service that calls the external service , operates on the data and then gathers the results to send out the emails updating the database as necessary with sent flags etc.
I handled a similar project recently and we found many ISPs or Hosting Providers got really twitchy when we mentioned mass emails. You should defintly check out the http://en.wikipedia.org/wiki/CAN-SPAM_Act_of_2003 CAN SPAM guidelines (or similar in your territory).
As long as you play by the rules and follow the guidelines you should be OK sending out from a local mail server however its important that you ensure that DNS Lookups or Reverse DNS lookups on the MX records all point back and forth to each other properly. Indeed this would be easier to out source to a third party mail provider or ISV but when we tried we were unable to find a good fit and ended up doing it ourselves.
Additionally you may want to glance at SPF records and other means to increase mass email delivery! For what its worth this can be a very tricky task to implement as SMTP (my least favourite protocol) is painful to try to debug and people get very upset if they receive multiples or unsolicited emails so ensure you have an Opt-in policy and appropriate checks to prevent duplicate delivery.
I am developing a site for which I would like to protect buyers by anonymizing their email addresses.Similar to craigslist's system, when a seller needs to contact a buyer they should be able to send an email to an anonymized address such as 1425415125#mysite.com which will then be routed to the user's email address.
My plan right now is to:
Set up a bucket (catch-all) inbox
Generate a random key for each buyer which will be the user specific ('1425415125' above) section of the email address
Monitor the bucket inbox and parse out this user specific section. Once I know the user, the email can be forwarded to the correct address
My questions are as follows:
Can you see any issues with the above solution
Are there any open source solutions to the existing problem
Are there any gotchas that one should be aware of when developing such a system?
Thanks in advance
JP
I did something related, though not quite the same. I setup a catch all inbox on my existing pop3 server (you probably have one already I'm guessing). I then used OpenPop.NET to read all new messages on a timer (say every 30 seconds). In my case I stopped at just processing the messagse but it's easy enough to generate a new message to the appropriate address and copy the body over, then send the new message out on your SMTP server.
One problem I see with your setup, and maybe it's just a misunderstanding on my part, is that while you are protecting the users original email address they will continue to be reachable at 1425415125#mysite.com basically forever. If I understand the way craigslist works, each posting has a different email address, and once the posting has been deleted/removed (or shortly after) the email address stops working. This makes it so that people can't just keep bugging you on that email address. The solution to this issue is easy, just make the email address coorespond to a post id or some other id rather then the users id in the database. The lookup will be just as quick but they will have a new email address each time.
You may wish to look at mail "piping" - the ability for someone to send an email to a mail server, which then gets thrown immediately to an executable, which then forwards your mail onto the recipient (by pulling the real email address from the database based on the incoming address from the piped message).
My personal recommendation would be to check out HMailServer, which has a COM API (the admin side is written in PHP, hence the requirement for legacy interop), is free and open-source, and is very well-documented. It doesn't have mail piping built-in, but is easily extensible given the API and support for scripts which run on server-side message events
HTH,
Benjamin
I think this solution will make sense and is in use in a lot of cases. The hardest part is actually receiving the messages. You can actually handle all of this within your web app if you need to. I wrote a blog post highlighting a couple of ways to receive email in your web app. It applies mainly to Rails but the concepts should be transferable.
The way you are lookimg to do it is the way I created a similar service. I would not recommend you writing your own smtp server. Use an existing mailserver and just use polling or some event based api.
The benefits of using a 3rd party mailserver is you can use existing backup and management tools on it.
Edit: I just noticed this has beed answered here with a better explanation. Pipe incoming email to a script on Windows IIS SMTP?
I do not see any problem with your setup, infact that is correct way to do because if your scheduled application fails, the emails will still be in the catch-all email box. Only once the email has been successfully delivered to somebody, the email should be deleted. You will be able to monitor and log the activity of your own application to monitor progress and failures.
I do not recommend Piping because, if for any reason piping goes successfully but your exe crashes, you will loose the email. Tracking will be difficult. Scheduling the jobs will not be possible.
If your application is independent of mail server, it is easy to manage it and replace your mail server whenever possible. It is easy to extend.
In this you will have to use some pop reader library, and schedule your application to run frequently.
In addition to email, you may consider a pull rather than push delivery mechanism, e.g: a message center web frontend or RSS feed. I say this because deliverability problems to various ISPs can be very difficult to troubleshoot and in my experience your users will never believe it's their ISP.
I'm having an issue sending large volumes of emails out from an ASP.Net application. I won't post the code, but instead explain what's going on. The code should send emails to 4000 recipients but seems to stall at 385/387.
The code creates the content for the email in a string.
It then selects a list of email address to send to.
Looping through the data via a datareader it picks out the email address and sends an email.
The email sending is done by a separate method which can handle failures and returns it's outcome.
As each record is sent I produce an XML node in an XML document to log each specific attempt to send.
The loop seems to end prematurely and the XML document is saved to disk.
Now I know the code works. I have run it locally using the same SMTP machine and it worked fine with 500 records. Granted there was less content, but I can't see how that would make any difference.
I don't think the page itself times out, but even if it did, I was sure .Net would continue processing the page, even if the user saw a page time out error.
Any suggestions appreciate because I'm pretty stumped.
You're sending lots of emails. During the span of a single request? IIS will kill a request if it takes longer than a certain (configurable) amount of time.
You need to use a separate process to do stuff like this. Whether that's a Timer you start from within global.asax, or a Thread which checks for a list of emails in a database/app_data directory, or a service you send a request to via WCF, or some combination of these.
The way I've handled this in the past is to queue the emails into a SQL Server table and then launch another thread to actually process/send the emails. Another aspx utility page can give me the status of the queue or restart the processing.
I also highly recommend that use an existing, legit, third-party mailing service for your SMTP server if you are sending mail out to the general public. Otherwise you run the risk of your ISP shutting off your mail access or (worse) your own server being blacklisted.
If the web server has a timeout setting, it will kill the page if it runs too long.
I recommend you check the value of HttpServerUtility.ScriptTimeout - if this is set then when a script has run for that length of time, it will be shut down.
Something you could do to help is go completely old-school - combine some Response.Writes with a few Response.Flush to send some data back to the client browser, and this tends to keep the script alive (certainly worked on an old ASP.NET 1.1 site we had).
Also, you need to take into account when this script is being run - the server may well also have been configured to perform an application reset (by default this is set to every 29 hours in IIS), if your server is set to something like 24 hours and this coincides with the time your script it run, you could be seeing that too - although the fact that the script's logging its response probably rules that out - unless your XML document is badly formed?
All that being said, I'd go with Will's answer of using a seperate process (not just a thread hosted by the site), or as Bryan said, go with a proper mailing service, which will help you with things like bounce backs, click tracking, reporting, open counts, etc, etc.
We are writing a feature to send a reminder email to customers in x number of days and just wondered if it was possible to delay the sending of the emails similar to how you can in Outlook (New Mail > Options button > Do not deliver before) in C#.
Does anyone know of a way of doing this?
If you want to submit the email to an SMTP server to hold for a number of days, the server would have to support this feature. I would recommend against doing it that way, however. If you had a problem with missing emails it could be hard to track down.
Here's what I would do:
Schedule the emails in your system by making a table entry.
Include a "SendDate" column in the table where you can set a date in the future
Write a Windows service that wakes up once every 15 minutes (for example) and sends the emails in that table where SendDate < currentDate
One possibility is to create a service that runs on a scheduled task processing 'pending' mail. You might store the pending mail in a SQL Database. (Dave details this)
This article looks promissing if you would like to avoid a 'real' service
http://www.codeproject.com/KB/aspnet/ASPNETService.aspx
No.
The way to accomplish this would be for your application to write the email to a queue (could be database, file, MSMQ, etc), and then write a separate process that, at a later date, reads mail from that queue and sends it.
As far as I know, the SMTP protocol doesn't have any kind of "delayed send" feature. Also, the feature you reference in Outlook actually just keeps it in your client's outbox until the designated date, so it's a client-side feature, not a server-side feature.
The SmtpClient is a pretty dumb client and doesn't support this -- sends are immediate whether they are synchronous or asynchronous. You may be able to do it using an Exchange integration. I'm not sure what APIs are exposed for Exchange, though. The simplest way would be to keep the message in a database table along with the time-to-send and have an off-line process that regularly scans this table and looks for messages whose time-to-send value has passed and are not marked as sent.