We Keep Coding

File system issue in Azure app service - Unable to create Directories

We have a dotnet core application hosted in Azure app service (Windows machine) in our production environment. It consists of two components -
Email Service
Business Rules Engine
The Email service downloads all emails first to a folder Attachments in the same directory where the application is hosted (D:\home\wwwroot\). For each email, a separate directory (with a guid value) is created under the Attachments directory.
The Business Rules engine accesses that folder and uses the email and it's attachments. Once done, we clear out all contents from the Attachments directory.
The problem we're seeing is that after a certain number of emails are processed, all of a sudden our application is unable to create directories under the Attachments folder. The statement
Directory.CreateDirectory({path})
throws an error saying the specified path could not be found.
The only way we've been able to resolve this is to restart the app service and it again happily goes on it's way creating directories, processing emails until it fails again in a day or so 8-|
What we've tried -
Ours was a multithreaded app, so assuming that maybe one thread is holding a lock on the filesystem due to incorrect or incomplete disposing of resources, we changed it to single threaded processing
Where the directories were being created, we used DirectoryInfo, so tried using DirectoryInfo.Refresh() after every directory deletion, creation etc
Wherever FileStream was being used, we've added explicit .Dispose() statements to dispose of the FileStream
Called GC.Collect() at the end of each run of our service
I suspect this issue is due to the Azure environment but we've not been able to identify what is causing this issue. Has anybody had any such issues and if so how was it resolved?

I made some changes to my code based on what I read in these links here which gives a good summary of the storage system in Azure app service -
https://www.thebestcsharpprogrammerintheworld.com/2017/12/13/how-to-manually-create-a-directory-on-your-azure-app-service/
https://github.com/projectkudu/kudu/wiki/Understanding-the-Azure-App-Service-file-system
D:\local directory points to a folder that is accessible only to that instance of the service, instead of what I was using earlier which is shared among instances - D:\home.
So I changed the code to resolve the %Temp% environment variable, which resolved to D:\local\Temp and then used that location to store the downloaded Emails.
So far multiple testing runs have been executed without any exceptions related to the file system.

Yes, based on your issue description,it does look to be a sandbox restriction. To provide more more on this, the standard/native Azure Web Apps run in a secure environment called a sandbox. Each app runs inside its own sandbox, isolating its execution from other instances on the same machine as well as providing an additional degree of security and privacy which would otherwise not be available.
Azure App Service provides pre-defined application stacks on Windows like ASP.NET or Node.js, running on IIS. The preconfigured Windows environment locks down the operating system from administrative access, software installations, changes to the global assembly cache, and so on (see Operating system functionality on Azure App Service). If your application requires more access than the preconfigured environment allows, you can deploy a custom Windows container instead.
Symbolic link creation: While sandboxed applications can follow/open existing symbolic links, they cannot create symbolic links (or any other reparse point) anywhere.
Additionally, you can check if the files has read-only attribute, to check for this, go to Kudu Console (({yoursite}.scm.azurewebsites.net)) and run attrib somefile.txt, and check if it includes the R (read-only) attribute.

IIS Does not refresh changes on web references ASP.NET

I am working over a Virtual Machine using Microsoft Azure, and I installed an app into the IIS, however the when I want to replace the files I've transferred via FTP (From my Local computer to a VM folder) the IIS does not refresh the changes. These are the steps I've run:
Site -- Add new website
Fill up all the required fields
Start the app
but what I can see in the browser is old application, I tried to modified the code but the changes never displayed in the screen.
Note: it is a web service which I try to modified.
Note: I've tried, iisreset, stop the web site, re-start it, re-start the server, deleting the web site, re-create the web site but nothing works.
I am using 4.5.0 in my web app, in [Web.config].
Windows Server 2016.
IIS 10.
Is there something which I am doing incorrectly?

The issue was that there is something in VS2015 solution which does not allow to refresh the web service once the new files are updated via FTP. What I did was to install the https://learn.microsoft.com/en-us/visualstudio/debugger/remote-debugging-aspnet-on-a-remote-iis-7-5-computer and configure the Visual Studio to Deploy the web service to IIS in the Virtual Machine. This was the link which helped a lot.
https://learn.microsoft.com/en-us/aspnet/web-forms/overview/deployment/visual-studio-web-deployment/deploying-to-iis
RECOMMENDATION
DO NOT deploy a web site or service using FTP, it is better to set the ports in the VM and run the deployment from VS properly.

Check to be absolutely sure that you are overwriting the original files. I have seen some ftp clients show a very small status window that can make it easy to assume the files are transferring due to the flood of messages streaming by when in reality there are permissions issues preventing you from overwriting files. Expand the logging window for whatever client you're using so that you can confirm for certain that your files are actually transmitting. If they are, maybe you're dropping them in the wrong folder.

Is it a good idea to run an EXE from IIS from the code behind?

I have an ASP.NET Web API in which I am trying to download a ZIP file through a controller - say, DownloadZipController.
For generating this ZIP file, my controller method calls a factory method createandGetZip which launches an exe and waits for the EXE to finish its execution. When the EXE produces the ZIP it is passed to the factory method createandGetZip. This ZIP generation takes some time - around 2 minutes.
I have hosted this web api on IIS server with the AppPool identity as LOCALSYSTEM.
My question is: is this the best way to launch an EXE from my code hosted on IIS server? Or is there any other way to launch the EXE from the code without giving AppPool identity as LOCALSYSTEM as I have read that this may lead to a security breach.
Please advise me on the best possible way or just a way better than what I am currently using.

I believe this is fine in practice (I do this often with DoScan.exe, the Symantec virus scanner). Try to guard against the external process spinning forever, and limit the app pool security rights on the EXE, if that matters to you.
Also, of course, make sure you know what the EXE is doing.

What is the possibility of making asp.net web application update itself?

I have a web application that i would like it to check for updates, download and install them.
i know there are already some updater frameworks that works for windows applications, but is it possible for web applications ?
The first thing came to my mind when thinking of this is:
File permissions (i might not be able to replace all my application files due to file permissions)
Also touching the web.config or the bin folder will cause the application to restart.
I also thought about executing an exe from my web application that does the job, but i dont know if it could get shutdown because of a restart to the web application.
I would appreciate any ideas or solution to that case.
Thanks

Take a look at WebDeploy
It is meant to ease such tasks. Where you want to deploy a publish to a production server.
Rather than having your server check for updates and update itself, it would be simpler to just push updates to it when you have them.
Web Deploy allows you to efficiently synchronize sites, applications
or servers across your IIS 7.0 server farm by detecting differences
between the source and destination content and transferring only those
changes which need synchronization. The tool simplifies the
synchronization process by automatically determining the
configuration, content and certificates to be synchronized for a
specific site. In addition to the default behavior, you still have the
option to specify additional providers for the synchronization,
including databases, COM objects, GAC assemblies and registry
settings.
Administrative privileges are not required in order to deploy Web
applications.
Server administrators have granular control over the operations that can be performed and can delegate tasks to non-administrators.
This needs you to be running IIS7 though.

How to start process from ASP.NET Web Service (and allow to it to do everything what it wants to do)?

There is a Web Service installed on Amazon Server. Exposed WebMethod should start an executable. But, it seems that process (executable) started by WebMethod has not permissions to finish its job. If a WebMethod is called locally (using IE on Amazon VM) I can trace some events into log file placed on the path: C:\\LogFiles. But, if it is called from remote machine, there is no even log files. Locally, on my machine all works fine.
The question: is there any way or settings in IIS7 to allow to my WebService to create process that can do everything I want to do? In web.config file I added a line:
<identity impersonate="true" userName="USERNAME" password="password"/>
(userName and password are, of course, written correctly in the file).
Also, I tried to use impersonization as it is explained here, but there is no result. My process can't do its job, it cannot even trace actions into log file. Locally, on my machine, everything works fine.
Any idea how to change settings or whatever into IIS7?
EDIT: In addition to the main question: my WebService is not able even to create log files on the path: C:\\LogFiles - although it is able if it started locally, but remotely there is no simple log file that contains some string. How to allow it to create simple text files?

If all else fails, you may start such a process separately and make it wait for a signal. You can supply a signal in many ways — via an IP socket, via a mailslot, via a named pipe. The web service will post requests to the command pipe (or queue), and the 'executor' process will pop commands, execute them, and wait for more commands.

You should avoid trying to start external processes from ASP.NET - if anything, because your application will then be running under the context of the ASP.NET account. (Yes, you could use impersonation to launch into another account, but, lets not go there)
Instead, install a Windows Service which can receive a signal* to launch the executable you wish.
This has the advantage that you can customise what account the service runs under, without putting passwords inside your code.
(*) Signalling could be achieved through a number of means:
WCF Service Call (using a WCF Service being hosted by the Windows service)
Monitoring for a filesystem change to a known directory.

If you were using Linux, I would have given you the smartest solution ever, setting SUID flag, which is not possible in Windows.
The problem with impersonation is that it works as soon as you have some control over the server machine, more than having appropriate credentials.
You mentioned Amazon VM: I'm pretty certain that they won't allow, for security reasons, to perfrom impersonation. [Add] Or, better, they won't allow anybody to write in C:\
Option 1
Switch to Mono/Linux, set SUID bit using chmod from console and rock!!
Option 2
If you can run the executable other way than ASP.NET (ie. you have a Remote Desktop, SSH*) as privileged account (note: privileged doesn't mean Administrator) then you can redesign your application to have ASP.NET invoke services from your daemon process using WCF, Web Services or Remoting. But, in this case, you have to redesign your executable to be a stand-alone server.
[Add] None of the solution fix if your hosting provider doesn't allow you to write in paths such as C:\, but only allows you to write under your home directory.
*It works on Windows too!!!! And I mean the server!!!