We Keep Coding

Prevent Exchange 2013 from converting message body text/plain to HTML

I'm using MailKit v2.8 to parse data out of emails in using IMAP to connect to a Microsoft Exchange 2013 account. The body of messages being sent to my Exchange inbox will be "text/plain" 100% of the time. This process works completely fine for new emails (and has been in production use for a couple months), however replies/forwards to those emails are being converted to HTML presumably by Exchange when fetching. The header of the reply email on the server still specifies that the message body is "text/plain." Outlook also displays the response in plain text, but for some reason when I try to fetch the TextPart of the message summary using MailKit, it is returning null.
MailKit email fetching code:
using var imap = new ImapClient {
ServerCertificateValidationCallback = (mySender, cert, chain, sslPolicyErrors) => { return true; },
CheckCertificateRevocation = false
};
try {
await imap.ConnectAsync(_config.ImapServer, _config.ImapPort, SecureSocketOptions.SslOnConnect);
imap.AuthenticationMechanisms.Remove("XOAUTH2");
await imap.AuthenticateAsync(_config.ImapUsername, _config.ImapPassword);
var inbox = imap.Inbox;
if (!string.IsNullOrWhiteSpace(_config.Inbox)) { // set inbox to subfolder for devenv
inbox = await imap.Inbox.GetSubfolderAsync(_config.Inbox);
}
await inbox.OpenAsync(FolderAccess.ReadWrite);
var uIds = await inbox.SearchAsync(SearchQuery.All);
var msgs = await inbox.FetchAsync(uIds, MessageSummaryItems.UniqueId | MessageSummaryItems.BodyStructure | MessageSummaryItems.Envelope);
foreach (var msg in msgs) {
var bodyPart = msg.TextBody; // <-- this returns null for the latter email, but contains a body for the former
var body = await inbox.GetBodyPartAsync(msg.UniqueId, bodyPart) as TextPart;
if (_config.SendingAddresses.Any(msg.Envelope.From.Mailboxes.Select(a => a.Address).Contains)) { // sent from valid address
// parse and process email body
} else {
// discard and expunge
}
}
} catch (Exception e) {
// log exception
}
For brevity, here's an example. This email contains a TextBody when fetched using MailKit:
Received: from [ExchangeHost] ([ExchangeIP]) by [ExchangeHost]
([ExchangeIP]) with Microsoft SMTP Server (TLS) id 15.0.1320.4; Thu, 6 Aug 2020
17:30:12 -0400
Received: from [SenderHost] ([SenderIP]) by
[ExchangeHost] ([ExchangeIP]) with Microsoft SMTP Server id 15.0.1320.4
via Frontend Transport; Thu, 6 Aug 2020 17:30:11 -0400
IronPort-SDR: [redacted]
X-IronPort-AV: [redacted]
X-AuditID: [redacted]
MIME-Version: 1.0
Message-ID: <[redacted]>
From: <[SenderAddr1]>
To: <[MyExchangeAddr]>
Date: Thu, 6 Aug 2020 14:30:03 -0700
Subject: [redacted]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Return-Path: [SenderAddr1]
X-MS-Exchange-Organization-AuthSource: [ExchangeHost]
X-MS-Exchange-Organization-AuthAs: Anonymous
X-GFI-SMTP-Submission: 1
X-GFI-SMTP-HelloDomain: [SenderHost]
X-GFI-SMTP-RemoteIP: [SenderIP]
X-MS-Exchange-Organization-Network-Message-Id: [redacted]
X-MS-Exchange-Organization-AVStamp-Enterprise: 1.0
This email header is a reply to the aforementioned email. When fetched in MailKit, it did not have a TextBody, but rather it had an HtmlBody:
Received: from [ExchangeHost] ([ExchangeIP]) by [ExchangeHost]
([ExchangeIP]) with Microsoft SMTP Server (TLS) id 15.0.1320.4 via Mailbox
Transport; Mon, 10 Aug 2020 11:27:16 -0400
Received: from [ExchangeHost] ([ExchangeIP]) by [ExchangeHost]
([ExchangeIP]) with Microsoft SMTP Server (TLS) id 15.0.1320.4; Mon, 10 Aug 2020
11:27:16 -0400
Received: from [SenderHost] ([SenderIP]) by
[ExchangeHost] ([ExhcangeIP]) with Microsoft SMTP Server id 15.0.1320.4
via Frontend Transport; Mon, 10 Aug 2020 11:27:15 -0400
IronPort-SDR: [redacted]
X-IronPort-AV: [redacted]
From: <[SenderAddr2]>
To: <[MyExchangeAddr]>, <[SenderAddr1]>
Subject: RE: [redacted]
Thread-topic: [redacted]
Thread-index: [redacted]
Date: Mon, 10 Aug 2020 15:27:07 +0000
Message-ID: <[redacted]>
References: <[redacted]>
In-Reply-To: <[redacted]>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-exchange-transport-fromentityheader: Hosted
x-tm-snts-smtp: [redacted]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Return-Path: [SenderAddr2]
X-GFI-SMTP-Submission: 1
X-GFI-SMTP-HelloDomain: [SenderHost]
X-GFI-SMTP-RemoteIP: [SenderIP]
X-MS-Exchange-Organization-Network-Message-Id: [redacted]
X-MS-Exchange-Organization-AVStamp-Enterprise: 1.0
X-Auto-Response-Suppress: DR, OOF, AutoReply
X-MS-Exchange-Organization-AuthSource: [ExchangeHost]
X-MS-Exchange-Organization-AuthAs: Anonymous
The HTMLBody of the latter email from MailKit:
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Exchange Server">
<!-- converted from text -->
<style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left: #800000 2px solid; } --></style>
</head>
<body>
<!-- the stuff that should be in plain text, formatted as HTML -->
</body>
</html>
In Outlook, that latter email is formatted in plain text, just like the Content-Type of its header specifies. Since the message is properly formatted in Outlook, my question is either:
is there something I need to do when using MailKit to prevent this conversion from happening?
(as I suspect is more likely) is there some option my sysadmin needs to set for the Exchange account to prevent this automatic conversion from occurring?
I've read the solutions here and in other topics, but none of them appear to be applicable today as any question asked on this topic is nearly a decade old.

IMAP (and therefor MailKit) has no way to specify not to do any conversions, it's just assumed that no conversions will take place because that's not something that other IMAP servers do.
Presumably the same option for Exchange 2003 exists in 2013 as well.

So, I discovered that the reason for Exchange acting up was because I was duplicating the emails I was receiving for testing purposes. Despite the duplicated emails having IDENTICAL headers, Exchange was converting only the copied ones to HTML when being fetched using MailKit. The originals are being fetched as expected, however. For future reference to anyone, do not copy/paste emails in your inbox if you are expecting it to retain its plain text!

how can .NET, on Azure VWindows Server 2012, decrypt SMS text message from an AT&T gophone

I wrote a cool .NET Windows application that communicates by SMS text messages with users, or by email. It treats SMS text messages just like email. When my Windows server 2012 receives a text message from a mobile device, or an email, both kinds arrives into C:\inetpub\mailroot\Drop\ and my app processes them.
It all has been working great with email from Gmail, Hotmail, etc., and from SMS from Verizon. But then my friend comes over and we test receiving from his AT&T gophone... Blooey! I get the email (see below) from his mobile device's SMS and all looks normal, except the actual text message payload is scrambled.
How do I descramble? Why is the text scrambled, in the first place?
X-SENDER: ###########MMS.ATT.NET
X-RECEIVER: xxxxxxxxxx#xxxxxxx.COM
RECEIVED: FROM BTHCEG-MOMTA01-MMS.MYCINGULAR.NET ([]) BY WITH MICROSOFT SMTPSVC(8.5.9600.16384);
MON, 5 JUN 2017 17:42:02 -0700
RETURN-PATH: <############MMS.ATT.NET>
RECEIVED: FROM [] ([:14264] HELO=ALPNMS03)
BY BTHCEG-MOMTA01 (ENVELOPE-FROM <############MMS.ATT.NET>)
(ECELERITY 3.0.23.37692 R(37717)) WITH ESMTP
ID D2/65-04620-B5AF5395; MON, 05 JUN 2017 17:42:03 -0700
X-MMS-MESSAGE-TYPE: M-SEND-REQ
X-MMS-TRANSACTION-ID: 1496709721-5
X-MMS-MMS-VERSION: 1.2
TO: PRAY############COM
FROM: ############MMS.ATT.NET
DATE: MON, 5 JUN 2017 20:42:02 -0400 (EDT)
X-MMS-SENDER-VISIBILITY: SHOW
CONTENT-TYPE: MULTIPART/MIXED;
BOUNDARY="----=_PART_7984369_300459990.1496709722943"
MIME-VERSION: 1.0
MESSAGE-ID: <1096997833.194273661496709722943.JAVAMAIL.NEMS#ALPNMS03>
X-ORIGINALARRIVALTIME: 06 JUN 2017 00:42:02.0454 (UTC) FILETIME=[B6A1AF60:01D2DE5D]
------=_PART_7984369_300459990.1496709722943
CONTENT-TYPE: TEXT/PLAIN; CHARSET=UTF-8
CONTENT-DISPOSITION: ATTACHMENT; FILENAME=TEXT_0.TXT; CHARSET=US-ASCII
CONTENT-ID: 0
CONTENT-LOCATION: TEXT_0.TXT
CONTENT-TRANSFER-ENCODING: BASE64
..................BELOW IS THE SCRAMBLED TEXT MESSAGE: ............
TXKGCHJHEWVYIGLZOIBMB3IGSGVPZGKGDG8GAGF2ZSBWZWFJZSBPBIBOZXIGZMFTAWX5
......THE ACTUAL MESSAGE WAS: "My prayer is: for Heidi to have peace in her family"
------=_PART_7984369_300459990.1496709722943--
X-SENDER: ############MMS.ATT.NET
X-RECEIVER: PRAY############.ORG
RECEIVED: FROM BTHCEG-MOMTA02-MMS.MYCINGULAR.NET ([###########]) BY PRAYSHEP WITH MICROSOFT SMTPSVC(8.5.9600.16384);
MON, 5 JUN 2017 18:14:30 -0700
RETURN-PATH: <############MMS.ATT.NET>
RECEIVED: FROM [###########] ([###########:51516] HELO=ALPNMS03)
BY BTHCEG-MOMTA02 (ENVELOPE-FROM <############MMS.ATT.NET>)
(ECELERITY 3.0.23.37692 R(37717)) WITH ESMTP
ID 29/CD-12903-7F106395; MON, 05 JUN 2017 18:14:31 -0700
X-MMS-MESSAGE-TYPE: M-SEND-REQ
X-MMS-TRANSACTION-ID: 1496711669-7
X-MMS-MMS-VERSION: 1.2
TO: PRAY############.ORG
FROM: ############MMS.ATT.NET
DATE: MON, 5 JUN 2017 21:14:30 -0400 (EDT)
X-MMS-SENDER-VISIBILITY: SHOW
CONTENT-TYPE: MULTIPART/MIXED;
BOUNDARY="----=_PART_7989525_395020720.1496711670941"
MIME-VERSION: 1.0
MESSAGE-ID: <1241288163.194397521496711670942.JAVAMAIL.NEMS#ALPNMS03>
X-ORIGINALARRIVALTIME: 06 JUN 2017 01:14:30.0518 (UTC) FILETIME=[3FC4A960:01D2DE62]
------=_PART_7989525_395020720.1496711670941
CONTENT-TYPE: TEXT/PLAIN; CHARSET=UTF-8
CONTENT-DISPOSITION: ATTACHMENT; FILENAME=TEXT_0.TXT; CHARSET=US-ASCII
CONTENT-ID: 0
CONTENT-LOCATION: TEXT_0.TXT
CONTENT-TRANSFER-ENCODING: BASE64
................. SCRAMBLED FROM TEXT MESSAGE SMS: ...........
QWFHYWFHYWFHYWFHYWFHYWFHYWFHYWFHYWFHYWFHYWFHYWFHYWFHYWFH
............ACTUAL WAS ALL "AAAAAAAAAAAAA..."
------=_PART_7989525_395020720.1496711670941--

So this HttpRequest is a Multi-part form.
Each file is split between the boundary:
BOUNDARY="----=_PART_7984369_300459990.1496709722943"
And then each file has its own set of information:
------=_PART_7984369_300459990.1496709722943
CONTENT-TYPE: TEXT/PLAIN; CHARSET=UTF-8
CONTENT-DISPOSITION: ATTACHMENT; FILENAME=TEXT_0.TXT; CHARSET=US-ASCII
CONTENT-ID: 0
CONTENT-LOCATION: TEXT_0.TXT
CONTENT-TRANSFER-ENCODING: BASE64
This boundary has the CONTENT-TRANSFER-ENCODING Header, meaning that all of its contents (TEXT_0.TXT) is encoded as BASE64.
I took a look at what you posted the message was and it should be case sensitive resulting in TXkgcHJheWVyIGlzOiBmb3IgSGVpZGkgdG8gaGF2ZSBwZWFjZSBpbiBoZXIgZmFtaWx5, so you may have some data corruption, or the way you are viewing the encoding is changing it.
I can't provide a code sample as I'm not sure if you are using Webforms or MVC, but here is an example with asp.net MVC Web Api 2

"Operation would change object type, which is not permitted." when sending email with EWS Managed API

I'm loading an existing email from MIME data and getting this error sometimes.
For instance:
EmailMessage ewsMsg = new EmailMessage(service); // service is ExchangeService instance
ewsMsg.MimeContent = new MimeContent("UTF-8", mimeBytes);
ewsMsg.Send(); // ServiceRemoteException here
Header section of mimeBytes which cause this exception to occur:
From: Microsoft Outlook
<MicrosoftExchange329e71ec88ae4615bbc36ab6ce41109e#contoso.com>
To: Arno Bost <ArnoB#contoso.com>
Subject: Undeliverable:
Thread-Index: AQHR3esBLcRFBp+/o0msZmG72Y4S8aAeRn0L
Date: Mon, 18 Jul 2016 18:32:08 +0400
Message-ID: <fa80f868-17fe-40d1-8501-f88c836bb2d8#SLC-DC01.contoso.com>
References: <3F10E8D8921E424C8D867E5F1F5A9F1F0311848916#SLC-DC01.contoso.com>
In-Reply-To:
<3F10E8D8921E424C8D867E5F1F5A9F1F0311848916#SLC-DC01.contoso.com>
Content-Language: en-US
X-MS-Exchange-Organization-AuthAs: Internal
X-MS-Exchange-Organization-AuthMechanism: 05
X-MS-Exchange-Organization-AuthSource: SLC-DC01.contoso.com
X-MS-Has-Attach:
X-Auto-Response-Suppress: All
X-MS-Exchange-Organization-SCL: -1
X-MS-TNEF-Correlator:
Content-Type: multipart/report;
boundary="_000_fa80f86817fe40d18501f88c836bb2d8SLCDC01contosocom_";
report-type=delivery-status
MIME-Version: 1.0
Other emails get sent successfully. Does it mean Exchange cannot send multipart/report emails this way?
I'm not sure this is caused by multipart/report content-type, maybe it's just a coincidence but currently I don't have other ideas as emails with other content-types (not delivery reports) do not seem to cause this exception.

System.Net.Mail creating invalid emails and eml files? Inserting extra dots in host names

It appears that .NET's SmtpClient is creating emails with an extra dot in host names if the dot was to appear at the beginning of a MIME encoded line (e.g. test.com sometimes shows up as test..com). Example code:
[TestMethod]
public void TestEmailIssue()
{
var mail = new System.Net.Mail.MailMessage();
var smtpClient = new System.Net.Mail.SmtpClient();
mail.To.Add("Test#test.com");
mail.Subject = "Test";
mail.From = new System.Net.Mail.MailAddress("test#test.com");
mail.Body = "Hello this is a short test of the issue:"
+" <a href='https://test.com/'>https://test.com/</a>: ";
smtpClient.PickupDirectoryLocation = "C:\\temp\\";
smtpClient.DeliveryMethod = System.Net.Mail.SmtpDeliveryMethod.SpecifiedPickupDirectory;
smtpClient.Send(mail);
}
This creates an .eml file that looks like this:
X-Sender: test#test.com
X-Receiver: Test#test.com
MIME-Version: 1.0
From: test#test.com
To: Test#test.com
Date: 6 Jul 2011 15:55:28 -0400
Subject: Test
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: quoted-printable
Hello this is a short test of the issue: https://test=
..com/'>https://test.com/:=20
When sending the file, or opening in Outlook (or any other program), the double dots show up (i.e. test..com). Note that if I remove the extra space (in "is a"), that test.com shows correctly since the dot no longer appears at the beginning of the line.
This causes a problem when trying to send website addresses, and we get calls from clients saying this they cannot click our links.
Has anyone else experienced this? How can we resolve this issue other than writing our own encoding?

This is actually per RFC 2821 (4.5.2 Transparency)
Before sending a line of mail text, the SMTP client checks the first character of the line. If it is a period, one additional period is inserted at the beginning of the line.
.Net is just storing the file in "ready to transmit" mode which means that it doesn't have to monkey with the email before sending, instead it can transmit it as is. Unfortunately this format isn't 100% the same as Outlook Express's EML format apparently. You should be able to set the encoding to UTF-8 (or something similar) and that will kick in Base-64 encoding for you.
mail.BodyEncoding = System.Text.Encoding.UTF8;

In .Net 2.0
X-Sender: test#test.com
X-Receiver: Test#test.com
MIME-Version: 1.0
From: test#test.com
To: Test#test.com
Date: 6 Jul 2011 21:29:04 +0100
Subject: Test
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: quoted-printable
Hello this is a short test of the issue: <a href=3D'https://test.com/'>https://test.com/</a>:=
It looks like it is wrapping the text at a certain character length per line. I vaguely remember there was an issue in .Net 2.0 where by default it doesn't do this which can cause problems with spam filters.
In fact increasing the size of the message gives the following in .Net 4.0:
X-Sender: test#test.com
X-Receiver: Test#test.com
MIME-Version: 1.0
From: test#test.com
To: Test#test.com
Date: 6 Jul 2011 21:34:21 +0100
Subject: Test
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: quoted-printable
Hello this is a short test of the sssssssssssssssssissue: <a hre=
f=3D'https://test.com/'>https://test.com/</a>:=20
Seems like a bug.
A workaround might be to change the BodyEncoding to something other than ASCII.

Looking at .NET 4 source code, maybe what you are experiencing has something to do with MailWriter.WriteAndFold method. Also in MailWriter class there is
static int writerDefaultLineLength = 76
variable, meaning character count per line. Maybe because you removed one extra space character, it started working.

Acquire Twitter request token failed

I followed the instruction at http://dev.twitter.com/pages/auth#request-token, and developed a c# class to do the OAuth authorization. I used the parameters on the page, and the output signature base string and signature match that on the page. So I think the algorithm part is correct. Then I replaced the parameters with the ones in my twitter application, but I failed to acquire the request token from Twitter service. And the response data is "Failed to validate oauth signature and token".
Here's the request I send (I used http, instead of https for debug):
POST http://api.twitter.com/oauth/request_token HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Authorization: OAuth oauth_callback="http%3A%2F%2Flocalhost%3A3005%2Fthe_dance%2Fprocess_callback%3Fservice_provider_id%3D11", oauth_consumer_key="GDdmIQH6jhtmLUypg82g", oauth_nonce="QP70eNmVz8jvdPevU3oJD2AfF7R7odC2XJcn4XlZJqk", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1272323042", oauth_version="1.0", oauth_signagure="IP%2FEEoc4tKdiobM%2FKH5cPK69cJM%3D"
Host: api.twitter.com
Proxy-Connection: Keep-Alive
And here's the response:
HTTP/1.1 401 Unauthorized
Connection: Keep-Alive
Connection: Proxy-Support
Content-Length: 44
Via: 1.1 APS-PRXY-09
Expires: Tue, 31 Mar 1981 05:00:00 GMT
Date: Fri, 08 Apr 2011 05:47:20 GMT
Content-Type: text/html; charset=utf-8
Server: hi
Proxy-Support: Session-Based-Authentication
Status: 401 Unauthorized
X-Transaction: 1302241640-40339-46793
Last-Modified: Fri, 08 Apr 2011 05:47:20 GMT
X-Runtime: 0.01519
Pragma: no-cache
X-Revision: DEV
Cache-Control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0
Set-Cookie: k=207.46.55.29.1302241640766556; path=/; expires=Fri, 15-Apr-11 05:47:20 GMT; domain=.twitter.com
Set-Cookie: guest_id=13022416407746962; path=/; expires=Sun, 08 May 2011 05:47:20 GMT
Set-Cookie: _twitter_sess=BAh7CDoPY3JlYXRlZF9hdGwrCEiBpjMvAToHaWQiJWMzMTViOGZiNDkzMDRi%250ANjNhMmQwYmVkZDBhNTc2NTc4IgpmbGFzaElDOidBY3Rpb25Db250cm9sbGVy%250AOjpGbGFzaDo6Rmxhc2hIYXNoewAGOgpAdXNlZHsA--177afd5c0f6fe30005ab9a9412e6f85ab03cbfa7; domain=.twitter.com; path=/; HttpOnly
Vary: Accept-Encoding
Failed to validate oauth signature and token
This is how I generate the normalized parameters:
string.Join("&", (from d in this.BuildParameterDict()
select string.Format("{0}={1}", OAuthEncoding.Encode(d.Key), OAuthEncoding.Encode(d.Value))))
The BuildParameterDict method will sorted build a list with: parameters from query string; parameters from body; parameters sepcific to 'oauth', except the 'oauth_signature'.
Then the signature base string is generated by:
StringBuilder sb = new StringBuilder();
sb.Append(OAuthEncoding.Encode(this._request.Method));
sb.Append('&');
sb.Append(OAuthEncoding.Encode(this.GetNormalUri()));
sb.Append('&');
sb.Append(OAuthEncoding.Encode(this.GetNormalParameters()));
This is the generated base string with parameters from the above page:
POST&https%3A%2F%2Fapi.twitter.com%2Foauth%2Frequest_token&oauth_callback%3Dhttp%253A%252F%252Flocalhost%253A3005%252Fthe_dance%252Fprocess_callback%253Fservice_provider_id%253D11%26oauth_consumer_key%3DGDdmIQH6jhtmLUypg82g%26oauth_nonce%3DQP70eNmVz8jvdPevU3oJD2AfF7R7odC2XJcn4XlZJqk%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1272323042%26oauth_version%3D1.0
which is identical to the string on that page.

Your oauth signature is listed as "oauth_signagure" in your request.
oAuth parameters has to be sorted before sending, but signature has to be at the end of the authorization request.(9.1.1 in http://oauth.net/core/1.0/#anchor14)
You may also need to specify a realm="/oauth/request_token". It's optional, but as I remember correctly Twitter wants this one for a request token.
If you can add your code we might find what's going on, as you might not be building your request and key for signature hashing correctly.