I currently have a process that is designed to iterate through an Inbox, parse the emails contained within, dump those to a JSON file, and delete the emails. They cannot go to the DeletedItems folder, and must be removed from the server immediately (so DeleteMode.HardDelete is the only option here).
There is an admin monitoring the Inbox to make sure the process is truly deleting the emails, and had alerted me this morning that the RecoverableItems folder is getting filled up. I had consulted this to see if I had done something wrong. Additionally, I had made sure to look over Delete Modes with the EWS API.
Now, the pertinent piece of code I have is the following:
Boolean moreItems = true;
int loopNumber = 0;
while (moreItems)
{
string filepath = "Outputs/";
string filename = "some_filename.json";
using (StreamWriter sw = new StreamWriter(filepath + filename))
{
int itemsParsed = 0;
while (moreItems && itemsParsed < 50000)
// I want to split up the files into 50k item files.
{
try
{
FindItemsResults<Item> itemSet = i.GetItemSet();
foreach (Item x in itemSet.Items)
{
//Console.WriteLine($"Getting mailitem {x.Id}");
try
{
x.Load(itemSetProperties);
EmailMessage email = em.BindEmail(service , x.Id);
metadata.Id = x.Id.ToString();
metadata.Subject = x.Subject.ToString();
metadata.Sender = email.From.Address.ToString();
metadata.Body = x.Body.ToString();
metadata.DateTimeReceived = x.DateTimeReceived;
metadata.DateTimeSent = x.DateTimeSent;
metadata.HasAttachments = email.HasAttachments;
serializer.Serialize(sw , metadata);
moreItems = itemSet.MoreAvailable;
itemsParsed += 1;
x.Delete(DeleteMode.HardDelete);
}
catch (ServiceRequestException sre)
{
Console.WriteLine("\nTimeout caught.\n");
//Connection keeps getting reset every once in a while
}
catch (XmlException xle)
{
x.Delete(DeleteMode.HardDelete);
// There are emails with incomplete xml in the body. These can be deleted
}
catch(Exception ex)
{
Console.WriteLine($"Caught new exception: \n{ex.Message}");
Environment.Exit(0);
// I want to catch new exceptions and figure out why they're happening
}
Console.WriteLine($"Parsed {itemsParsed} items");
}
}
catch (ServiceRequestException exterior_sre)
{
Console.WriteLine("Caught exterior exception");
// Still haven't figured this out yet, but it occurs once
// in a rare while, and continuing does not affect the overall process
}
}
}
loopNumber += 1;
Console.WriteLine($"Moving to loop number {loopNumber}");
}
The issue is that the items being deleted are still being sent to the RecoverableItems folder in the dumpster. In the interrim, I had asked that the admin see about altering the retention policy on RecoverableItems such that it would get cleared out sooner than the default.
My main question is why this is happening? The emails appear to be getting a SoftDelete, so I know that the delete call to EWS is occurring successfully, but the HardDelete seems to be getting ignored.
I am still relatively inexperienced with the EWS API and C# as a whole, so apologies in advance for any ignorance on my part.
Update
I have confirmed with the Admin that items are not landing in the "Purges" folder, where they should be landing. I also have checked with this piece of documentation, that outlines the need for setting up retention policies on the Recoverable Items folder.
I'll keep adding edits for this as I go.
I believe it is doing as it is designed with Hard Delete -
Hard delete
The item or folder is permanently deleted.
Hard-deleted
items are placed in the Purges folder of the dumpster. This is like
when the recycling truck empties your curbside recycle container. The
items cannot be accessed from an email client like Outlook or Outlook
Web App, and, unless there is a hold set on the mailbox, the items
will be permanently deleted after a set period of time.
You can read
more about item retention in the article Configure Deleted Item
Retention and Recoverable Items Quotas.
NOTE: Folders are not placed in the Purges folder when they are hard
deleted. Hard-deleted folders are removed from the mailbox.
https://learn.microsoft.com/en-us/exchange/client-developer/exchange-web-services/deleting-items-by-using-ews-in-exchange
Related
This is my first time asking a question on StackOverflow, so let me know if you need more clarification. I am attempting to send emails with an attachment to users of a system, however I am having difficulty cleaning up the attachments on the file system after the messages are sent.
Background
I am using SSIS to compile a listing of email recipients and message content from a SQL database, then using a Script Component to do the actual email sending with C#. I am more of a DBA/Project Manager by role, and know enough coding to do some small things, but it has been quite some years since I did any .Net coding on a daily basis.
Each time I run the routine, the attachment files (Excel files in this case) are created successfully in a dedicated directory, then the emails are generated using those attachments and placed in a pickup folder. As each email is created and sent, I want to delete the attachment from its directory, but I get this error:
The process cannot access the file 'D:\mailtest\{filename}.xls' because it is being used by another process.
The specific file that it errors on is different each time, and of the ~3000 emails that are to be generated, it fails at around the 1200-1500 emails mark.
Code
//Create message
using (MailMessage msg = new MailMessage())
{
msg.To.Add(new MailAddress(EmailRecipient));
if (Row.Cc.Length > 0)
{
msg.CC.Add(new MailAddress(Row.Cc));
}
if (Row.Bcc.Length > 0)
{
msg.Bcc.Add(new MailAddress(Row.Bcc));
}
msg.From = new MailAddress(EmailSender);
msg.Subject = MessageSubject;
msg.Body = MessageBody +
"\n" +
"\n" +
this.Variables.EmailFooter;
msg.IsBodyHtml = true;
msg.BodyEncoding = System.Text.Encoding.UTF8;
//Add attachment data
if (File.Exists(attachmentPath))
{
Attachment data = new Attachment(attachmentPath);
data.ContentDisposition.FileName = "Expiring Training.xls";
msg.Attachments.Add(data);
}
if (this.Variables.UsePickupDirectory) //Drops items into pickup directory
{
SmtpClient client = new SmtpClient(SMTPEndPoint, SMTPPort)
{
EnableSsl = false,
DeliveryMethod = SmtpDeliveryMethod.SpecifiedPickupDirectory,
PickupDirectoryLocation = this.Variables.PickupDirectory,
Credentials = new NetworkCredential(UserName, Password)
};
try
{
client.Send(msg);
}
catch (Exception e)
{
//Debug.WriteLine(e);
Row.ErrorMessage = e.Message;
}
finally
{
msg.Dispose(); //Release file
}
}
else //Send to SMTP Server
{
SmtpClient client = new SmtpClient(SMTPEndPoint, SMTPPort)
{
EnableSsl = true,
DeliveryMethod = SmtpDeliveryMethod.Network,
Credentials = new NetworkCredential(UserName, Password)
};
try
{
client.Send(msg);
}
catch (Exception e)
{
//Debug.WriteLine(e);
Row.ErrorMessage = e.Message;
}
finally
{
//Add sleep to prevent sending more than 10 emails per second
System.Threading.Thread.Sleep(100);
msg.Dispose(); //Release file
}
}
}
//Remove attachment file
if (File.Exists(attachmentPath))
{
File.Delete(attachmentPath);
}
Things I've Tried
Initially, I did not have the using block. I saw this suggestion on a similar SO question, but adding it here seems to make no difference.
I've added the Dispose() to the finally block of each path (only the pickup directory is used in this case), which I understand should release all locks on the files used as attachments.
Without either of those things, it fails on the first file encountered, which leads me to believe it is working, but only for a while, and then fails suddenly at a random point during the execution.
The file specified in the error is not showing in process explorer when I search for it, so maybe it is released quickly after the error, so that I cannot search in time?
I tried moving the "Delete" functionality to a separate process entirely, running directly after all emails had been sent, but the same message would appear anyway.
I'd be thankful for any advice if anyone has a clue what could be happening.
New info
Adding some extra error handling did improve things, but didn't totally fix it. I've had several runs make it all the way through successfully, and whenever there was an error, it was only on 1 out of the ~3000 files. It did show the same error, though: "The process cannot access the file 'D:\mailtest{...}.xls' because it is being used by another process." That error was from the File.Delete line. It makes me wonder how just by adding more stuff it errors less. Is the sending and deleting happening so fast that it's stepping on its own toes? And throwing stuff in slows it down enough that it's able to keep up?
New Info 2
I took Jamal's advice and added a 500ms delay between send and delete, but only if the first delete attempt failed. So far no errors on 10 straight runs, whereas it was failing every single run in some way prior to adding it. However, the FireInformation message never appeared in the output window, leading me to think it never reached that block, so I'm not sure why adding it seems to work.
//Remove attachment file
if (File.Exists(attachmentPath))
{
try
{
File.Delete(attachmentPath);
}
catch
{
try
{
this.ComponentMetaData.FireInformation(0, "Delete failed", "Trying Delete again after 500ms", "", 0, fireAgain);
System.Threading.Thread.Sleep(500);
File.Delete(attachmentPath);
}
catch (Exception e)
{
Row.ErrorMessage = e.Message;
this.ComponentMetaData.FireInformation(e.HResult, e.Source, e.Message, e.HelpLink, 0, fireAgain);
}
}
}
I want to mimic the latest version of Outlook's search functionality via c# and a windows form. Specifically, I want to "Search All Mailboxes" for a given string. There are over 50 folders and 90,000 emails.
Currently, I am able to search any one folder using LINQ and get results. I wrote some code to iterate through all the folders and create one massive IEnumerable<MailItem> that I can query.
public IEnumerable<MailItem> SharedInbox
{
get
{
outlook.ActiveExplorer().Session.GetDefaultFolder(OlDefaultFolders.olFolderInbox);
Outlook.Recipient recip = Outlook.Application.Session.CreateRecipient("TOCCard#Capitalone.Com");
Microsoft.Office.Interop.Outlook.MAPIFolder folder =
outlook.GetNamespace("MAPI").GetSharedDefaultFolder(recip, OlDefaultFolders.olFolderInbox);
Folders subFolders = folder.Folders;
IEnumerable<MailItem> mItems = folder.Folders[1].Items.OfType<MailItem>();
if (recip.Resolve())
{
System.Diagnostics.Debug.WriteLine("Email Address Resolve Successful.\r\n");
try
{
foreach (MAPIFolder fold in subFolders)
{
System.Diagnostics.Debug.WriteLine("Try Folder: " + fold.Name + " \r\n");
try
{
mItems = mItems.Concat(fold.Items.OfType<MailItem>());
}
catch
{
System.Diagnostics.Debug.WriteLine("No items found:\r\n");
}
}
return mItems;
}
catch
{
return null;
}
}
else
{
System.Diagnostics.Debug.WriteLine("ELSE");
return null;
}
}
}
This eventually works, but you can probably imagine that this excruciatingly slow and as such is useless.
I am new to LINQ and I feel like there must be a faster way. Can this code be adjusted? I am not an Exchange admin and have no access to the exchange servers beyond my own inbox. Also I am not married to LINQ and would be happy to use other methods. I will appreciate your help.
NOTE: I just noticed that the above code is going through folder[1] twice. I can fix that, but it is not significantly affecting the time it takes.
No not use LINQ with OOM. Use Items.Find/FindNext or Items.Restrict.
You can also use Aplication.AdvancedSearch (keep in mind it is asynchronous).
So I'm having a problem with automating my code to check-in files to TFS, and it's been driving me up the wall! Here is my code:
string location = AppDomain.CurrentDomain.BaseDirectory;
TfsTeamProjectCollection baseUserTpcConnection = new TfsTeamProjectCollection(uriToTeamProjectCollection);
IIdentityManagementService ims = baseUserTpcConnection.GetService<IIdentityManagementService>();
TeamFoundationIdentity identity = ims.ReadIdentity(IdentitySearchFactor.AccountName, #"PROD1\JR", MembershipQuery.None, ReadIdentityOptions.None);
TfsTeamProjectCollection impersonatedTpcConnection = new TfsTeamProjectCollection(uriToTeamProjectCollection, identity.Descriptor);
VersionControlServer sourceControl = impersonatedTpcConnection.GetService<VersionControlServer>();
Workspace workspace = sourceControl.CreateWorkspace("MyTempWorkspace", sourceControl.AuthorizedUser);
String topDir = null;
try
{
Directory.CreateDirectory(location + "TFS");
String localDir = location + "TFS";
workspace.Map("$/Automation/", localDir);
workspace.Get();
destinationFile = Path.Combine(localDir, Name + ".xml");
string SeconddestinationFile = Path.Combine(localDir, Name + ".ial");
bool check = sourceControl.ServerItemExists(destinationFile, ItemType.Any);
PendingChange[] pendingChanges;
File.Move(sourceFile, destinationFile);
File.Copy(destinationFile, sourceFile, true);
File.Move(SecondsourceFile, SeconddestinationFile);
File.Copy(SeconddestinationFile, SecondsourceFile, true);
if (check == false)
{
workspace.PendAdd(localDir,true);
pendingChanges = workspace.GetPendingChanges();
workspace.CheckIn(pendingChanges, Comments);
}
else
{
workspace.PendEdit(destinationFile);
pendingChanges = workspace.GetPendingChanges();
workspace.CheckIn(pendingChanges, Comments);
}
and the problem is that whenever it's NEW files (PendEdit works correctly when the files already exist in TFS) that my code is attempting to check in, and it runs through this code:
if (check == false)
{
workspace.PendAdd(localDir,true);
pendingChanges = workspace.GetPendingChanges();
workspace.CheckIn(pendingChanges, Comments);
}
The files, instead of being in the included changes in pending changes, are instead in the excluded changes like so:
and when the line that actually does the check-in runs, I'll get a "The array must contain at least one element" error, and the only way to fix it is to manually add those detected changes, and promote them to included changes, and I simply can't for the life of me figure out how to do that programatically though C#. If anyone has any guidance on what direction I should take for this, I would really appreciate it! Thank you!
Edit: I've also discovered another way to solve this by reconciling the folder, which also promotes the detected changes, but again the problem is I can't seem to figure out how to program that to do it automatically.
I know that running the visual studio developer command prompt, redirecting to the folder that this mapping is in, and the running "tf reconcile /promote" is one way, but I can only automate that as far as the /promote part, because that brings up a toolbox that a user would have to input into, which defeats the purpose of the automation. I'm at a loss.
Next Edit in response to TToni:
Next Edit in response to TToni:
I'm not entirely sure if I did this CreateWorkspaceParameters correctly (see picture 1), but this time it gave the same error, but the files were not even in the excluded portions. They just didn't show up anywhere in the pending changes (see picture 2).
Check this blog:
The workspace has a method GetPendingChangesWithCandidates, which actually gets all the “Excluded” changes. Code snippet is as below:
private void PendChangesAndCheckIn(string pathToWorkspace)
{
//Get Version Control Server object
VersionControlServer vs = collection.GetService(typeof
(VersionControlServer)) as VersionControlServer;
Workspace ws = vs.TryGetWorkspace(pathToWorkspace);
//Do Delete and Copy Actions to local path
//Create a item spec from the server Path
PendingChange[] candidateChanges = null;
string serverPath = ws.GetServerItemForLocalItem(pathToWorkspace);
List<ItemSpec> its = new List<ItemSpec>();
its.Add(new ItemSpec(serverPath, RecursionType.Full));
//get all candidate changes and promote them to included changes
ws.GetPendingChangesWithCandidates(its.ToArray(), true,
out candidateChanges);
foreach (var change in candidateChanges)
{
if (change.IsAdd)
{
ws.PendAdd(change.LocalItem);
}
else if (change.IsDelete)
{
ws.PendDelete(change.LocalItem);
}
}
//Check In all pending changes
ws.CheckIn(ws.GetPendingChanges(), "This is a comment");
}
I'm trying to find out if a user added new music to the Music folder on the phone since app was last used.
I try to do this by checking the DateModified of the Music folder (which updates correctly on the computer when adding new music to the phone):
async void GetModifiedDate ()
{
BasicProperties props = await KnownFolders.MusicLibrary.GetBasicPropertiesAsync();
Debug.WriteLine("DATEMODIFIED: " + props.DateModified.ToString());
}
Unfortunately this returns:
DATEMODIFIED: 1/1/1601 1:00:00 AM +01:00
Am I doing something wrong or is there another quick way to check if user added new music?
KnownFolders.MusicLibrary is a virtual location. Therefore I think, may be a problem in getting its properties.
The other problem is that DateModified may be a bad idea, as it may stay the same when user adds a file. You cannot relay on it. (some information). You can check it - when I've tried to move files in folders, their DateModified hadn't changed.
So in this case, I'm afraid you will have to list your files in MusicLibrary and then decide, what to save for future comparison. The sum of filesizes can be a good idea, hence there is a little chance that two different music files would be the same size. It depends also if you want to be notified if the user had moved file from one folder to another (total size won't change). If you want to ensure more you can remember the whole list of Tuple<file.FolderRelativeId, fileSize> (for ecample).
As FileQueries are not yet available for Windows Phone, you will have to retrive files recursively the simple code can look like this:
// first - a method to retrieve files from folder recursively
private async Task RetriveFilesInFolder(List<StorageFile> list, StorageFolder parent)
{
foreach (var item in await parent.GetFilesAsync()) list.Add(item);
foreach (var item in await parent.GetFoldersAsync()) await RetriveFilesInFolder(list, item);
}
private async Task<List<StorageFile>> GetFilesInMusic()
{
StorageFolder folder = KnownFolders.MusicLibrary;
List<StorageFile> listOfFiles = new List<StorageFile>();
await RetriveFilesInFolder(listOfFiles, folder);
return listOfFiles;
}
Once you have a list of your files, you can decide what to remember for further comparison upon next app launch.
You can check the folder size instead:
ulong musicFolderSize = (ulong)ApplicationData.Current.LocalSettings.Values["musicFolderSize"];
BasicProperties props = await KnownFolders.MusicLibrary.GetBasicPropertiesAsync();
if (props.Size != musicFolderSize)
{
// ...
ApplicationData.Current.LocalSettings.Values["musicFolderSize"] = props.Size;
}
Adding onto Romansz's answer:
The only way to guarantee that you know of a file change would be to track the files on the device and then compare if there is a change between launches.
A lazier way to get all of the files would be to use KnownFolders.MusicLibrary.GetFilesAsync(CommonFileQuery.OrderByName); which is a deep query by default. It will grab all of the files in one query, but can be really slow on massive directories. You then have to page through all of the files as shown
//cache the virtual storage folder for the loop
StorageFolder musicLibrary = KnownFolders.MusicLibrary;
uint stepSize= 50;
uint startIndex = 0;
while (true)
{
IReadOnlyList<StorageFile> files = await musicLibrary.GetFilesAsync(CommonFileQuery.OrderByName,startIndex,stepSize);
foreach (var file in files)
{
//compare to see if file is in your list
}
if (files.Count < stepSize) break;
startIndex += stepSize;
}
I am working on a client that access an Exchange Web Service via a web reference. (not the Managed API). This is the first time I'm worked with EWS so I hope its just a simple mistake that I overlooked.
I have a method called MoveItem that is supposed to take email message and move it from the Inbox to destinationFolder. When I run this code, the item does disappear from the Inbox however it never shows up in the destination folder. I've spent a couple of days looking at examples online and I've also not been able to find anyone else that has had a similar issue. Can anyone tell me what I am doing wrong? Thanks in advance
Scott
static void MoveItem(ExchangeServiceBinding esb, BaseFolderType destinationFolder, MessageType msg)
{
ItemIdType[] items = new ItemIdType[1] { (ItemIdType)msg.ItemId };
BaseFolderIdType destFolder = destinationFolder.FolderId;
MoveItemType request = new MoveItemType();
request.ItemIds = items;
request.ToFolderId = new TargetFolderIdType();
request.ToFolderId.Item = destFolder;
try
{
MoveItemResponseType response = esb.MoveItem(request);
ArrayOfResponseMessagesType aormt = response.ResponseMessages;
ResponseMessageType[] rmta = aormt.Items;
foreach (ResponseMessageType rmt in rmta)
{
if (rmt.ResponseClass == ResponseClassType.Error)
{
throw new Exception("Item move failed.");
}
}
}
catch (Exception e)
{
Console.WriteLine(e.Message);
}
}
Definitely not an expert here, but this is the following code I used to grab an item and move it into a folder:
Folder rootfolder = Folder.Bind(service, WellKnownFolderName.MsgFolderRoot);
rootfolder.Load();
foreach (Folder folder in rootfolder.FindFolders(new FolderView(100)))
{
// Finds the emails in a certain folder, in this case the Junk Email
FindItemsResults<Item> findResults = service.FindItems(WellKnownFolderName.JunkEmail, new ItemView(10));
// Enter your destination folder name below this:
if (folder.DisplayName == "Example")
{
// Stores the Folder ID in a variable
var fid = folder.Id;
Console.WriteLine(fid);
foreach (Item item in findResults.Items)
{
// Load the email, move it to the specified folder
item.Load();
item.Move(fid);
}
}
}
One thing about the EWS is that when moving to a folder, the ID of the folder is key. In this example, I first find the folders in the inbox, then access the messages. Therefore, any code regarding the individual message would go after the folders have been found.
I have same problem. The moved message doesn't exists in folder (outlook.com) but can be retrieved via APIs with new assigned id.
Outlook.com can find it in search with an error while loading message body:
Error: Your request can't be completed right now. Please try again later.