We Keep Coding

Preferred architecture for running a separate process

I'm wondering what the preferred architecture would be for the following situation:
I'm required to have a .NET application that will perform batch upload of multiple data files concurrently to a SQL Server. This will be invoked from a WPF application which will allow the user to select the files and the destination tables, as well as reporting on the individual progress for each upload (including error messages). I have absolutely no problem writing the code for any of this. However, there is a requirement that the user is able to close the WPF application altogether and for the upload process to continue. Further, if the user restarts the WPF application from the same machine, it should be able to get a handle on the existing uploads and report on the status as if the program were never closed.
My question is what are the ways of achieving this and which would seem the most standard/suitable?
I've considered simply not actually closing the WPF application but hiding all the windows, but this seems a cheat. Would it be best to create a WCF service on the server where the upload is taking place and simply upload the file? I don't think I can do that and report progress % etc though. What about a locally-running Windows Service, can I achieve a similar effect? Should I be thinking of MemoryMappedFiles?
Appreciate all your thoughts.

Because you are talking about long-running task, I would use local Windows Service communicating with your WPF application through MSMQ. For example, each file to upload can be represented by one MSMQ message. Your WPF application will be putting messages into queue and Windows Service, periodically and without any impact if WPF is running or not, should take it from queue and process. This will provide simple and reliable channel of providing tasks (uploads).
To provide internal status of the Windows Service to its clients (your WPF application), I would host inside it a WCF endpoint with simple service that is telling, for example, about progress.

What .Net tools should I consider using to build an application to provide monitoring of our real time systems?

I want to build some sort of interface that will monitor our real time routing/switching system. I would like to give a lot of visual feedback to be able to monitor its status visually. Our system and clients are not co-located so they would need to connect via TCP/IP.
I would like to be able to service any number of monitoring clients (although this will probably only ever be about 4-6 clients). I thought of using SilverLight but there appears to be one or two tricks involved in getting SilverLight to connect back to an application running on a different port.
I have also thought of using HTML5 canvas and websockets. Another alternative is to just create the clients using normal Window Forms and perhaps WPF. But this means that to monitor the application the client will have to be downloaded before. I would prefer something that is as easily accessible as web app?
What are some of the more common application stacks to achieve this? What should I watch out for?
EDIT:
Just to add: This will be an internal tool only. But we have offices in a couple of locations.

any choice in this direction could be subjective and arguable, surely somebody could suggest any possible web framework or language...
I would consider, however because of your .NET and C# tags, ASP.NET MVC 3, so basically web based plugin-less ( NO Silverlight ) HTML 5 solution.
Consider that StackOverflow is done in same way (MVC, ASP.NET, SQL Server... ) and outperforms as we all know.
the way you grab the underlying events from TCP, so the way you capture and provide the data from TCP, it's another thing from the front end, I would probably write a Windows Service if the traffic is so high and you want to grab and store data anything regardless any active client connection.
There are plenty of real time charting controls out there also for MVC, MS Chart Control. DevExpress, ExtJS integrated ones...

"real time" and Browser is bothering me.
I would indeed go WPF or WinForms. Using the ClickOnce-Deployment you can make this a no-pain for the user and you can roll-out new versions just by redeploying them and having the user restart the application.
In my company this works really fine and we have no problems whatsoever. The only problem with this is, that the app.config is somewhat hard to find and keep current/valid (redeploy) but in your case this won't change per client (or so I guess).

I agree with #Davide - I would go for a WebService that will obtain all routing/switching data in realtime. You will have a web application and on the client side you will have JQuery/AJAX fetching realtime data from the WebService component.

I've seen cool demo's of Web Orb doing something similar to what you want. http://www.themidnightcoders.com/

If you are starting from scratch, it would be good to check out WCF (Windows Communication Foundation). It's great because it can expose your functionality in many ways, using nothing more than modifying a config file.
If you want a Windows client app, you can host it in a Windows Service, or simply include it as a side assembly. For web apps, you can choose between various formats (JSON, XML), channels (HTTP, TCP) and protocols (SOAP, ODP).
If I got it right, there will be a server-side application which will collect information from the devices and expose it to clients as a service. In that case, a WCF application might be hosted in a Windows Service or IIS on a server machine, and expose the data though one or more endpoints (HTTP, TCP).
I am not aware of problems in connecting a SilverLight app to a service, but I would rather go for a HTML5/JavaScript combo instead, for easier deploying and compatibility with a wider range of devices (no plugins needed). ASP.NET MVC should be the best choice for the web app.

Windows service app that can dynamically accept console commands

I just wanted some input on a project that I'm working on. Basically, I'm creating a service that monitors and processes new files in a directory specified by a configuration file and other parameters through the command line. It should also output text via the command line, i.e. when a user types in '-help' it will display it's usage. A goal is to also make it so that the user can change the configuration file on the fly, so the service will constantly monitor the config file for changes and adjust accordingly.
The challenge I'm seeing is trying to consolidate the ability to enter commands through the command line, display output, as if it were a console app, and have the service be able process these commands while it is running in the Service Control Manager. So far in my research, the solutions I've stumbled upon seem to show how to create a Windows Service App that can also run as a console app, but it either operates as one or the other, not both. Any suggestions or input is appreciated.
UPDATE:
Thank you all for your suggestions, I did some reading on various Windows supported IPC mechanisms. I boiled my options down to File Mapping, Named Pipes and RPC. I'm assuming for now that the Windows Service app and Helper Console app will be on the same computer and will not need to communicate over a network. I'll be looking at Named Pipes first.

The service needs to offer a possibility to communicate with it, but this can't be done directly, it has to be done by some sort of IPC (inter process communication), this could be .net Remoting, WCF, tcp/ip. You will have to write a helper program that parses the command line and uses this IPC to send the commands to the service, which can act appropiately on receiving it.

I would write a console app that sends requests to the service and displays its responses. If you wanted to be clever, you could include both parts in the same executable. I can't think of any way to make the service write to and read from the console while it's running as a service.

Windows Application in future to be controlled over browser

I am writing a Windows application using C#. I am planning on later to allow it to be controlled over the intranet using browser also. So in future we should be able to control it both using the local interface or over the intranet from the browser.
Is there any pre-defined architecture which will allow me to do this? What are the methods of achieving this? I am new to C#/.Net.
EDIT:
The windows application needs to access the communication ports extensively, and needs to be pretty stable and would probably run for some days together.
Thanks...

I can't tell you if a specific package exists that would ease the development. But, if I were to attempt it, after Googling and not finding something already available and meeting my needs, I would likely make my application a WCF host. Create service entry points to accept control messages remotely. You would also need some well-know location where to register your application so the remote system could find it. You should be sure to provide the user with a way of disabling the application remote control feature.
Your host interface will need to run on its own thread to remain performant. Since you are new to C#, and presumably windows forms application development, you will need to read up on how to properly talk to the GUI controls from a non-GUI thread.
Alternatively, you may want to implement your application as two distinct units, one with a GUI that does all the user interaction. It would form service requests to send to the host portion (with no GUI). Your app could then operate locally or be controlled remotely.

One solution I have used in a similar situation has three parts :-
1) Win32 (local) Service
Manages the COM ports and does whatever is necessary with the attached hardware
2) WinForms/Console Application
Runs on the local machine and communicates with the local service via named pipes or TCP.
3) Web Server + Web App
Runs on local or remote machine & communicates with local service.
The local user can shut the WinForms application down and log-off without affecting the service or remote users.

The newest version of Silverlight (the version that ships with Visual Studio 2010) allows what Microsoft terms the "Out Of Browser Experience" (OOB for short).
This allows the user to set up the Silverlight application as a desktop application as well as running through a browser.
Rudi Grobbler has just blogged about how he went about setting this up on his PC.

Make program startup through windows service c#

I have a program starting up through the registry, but I want it to start up more efficiently. So I asked some questions and I have found out that a windows service is the way forward. I have looked at how to make a windows service but I have not found any answers for what I need.
I am trying to create a checkbox on my application so that when it is checked it adds a service to start up my application when the person logs in, and when it is not checked it deletes the service.
Any help would be appreciated,
Thanks.

This is not the correct way. Services cannot find out when a user logs in since Vista. In your prior thread you got the advice to create two programs, although that wasn't explicitly stated. A service that runs after the machine boots and a user interface that allows the user to communicate with the service after she logs in.
You'll need an inter-process communication mechanism to get them to talk to each other. Named pipes, a socket, Remoting or WCF will work.
Read the MSDN Library docs on the ServiceBase class, that's what you'll need to create a service.