We Keep Coding

.NET Website Crashing

I work on a very large, high traffic ecommerce website. We're currently migrating our site from ColdFusion to .NET. We've recently run into an issue during this conversion that I was hoping for a little help with. Our current website is about 1/3 .net now and 2/3 ColdFusion.
One issue though, is that when we release our latest project, which is a project to convert the My Account section everything is fine for awhile, but anywhere between 3 to 24 hours the website just crashes. In order to get it back up, we need to restart IIS and sometimes ColdFusion. When I say crashes, I mean it just hangs, sits there and spins forever.
We have really good server monitoring, but when we look at the services memory nothing look unusual except for the number of connections to SQL. For some reason fairly quickly before the crash SQL shoots up in the number of connections, it goes from around 24 connections to around 100, just sits there and the site goes down until we restart services.
We currently use SQL Server 2005, Entity framework as our data access method and we're on IIS 7.5. Our web server is virtual but our database is physical.
We've had multiple people on our team go through all of the code in this new project to confirm that their were no connections that were being left open, as based on the connections issues that's sort of how is seems. We couldn't find any connections left open, not one.
This is an example of our current data access to entity:
/// <summary>
/// Get Products by their Primary Category ID. Default Category ID is 0: Top Level Categories.
/// </summary>
/// <param name="languageCode">Two character language code of Categories being searched. Defined in dbo.Languages, LanguageCode field.</param>
/// <param name="primaryCategoryId">int - Primary Category ID</param>
/// <returns>List<Product%gt;</returns>
public List<Products.Product> GetProducts(string languageCode, int primaryCategoryId = 0)
{
CatalogEntity context = null;
EntityConnection conn = null;
try
{
conn = this.GetConnection();
context = new CatalogEntity(conn);
List<I_Products> Products = context.GetProductsByPrimaryCatId(primaryCategoryId, languageCode).Distinct().ToList();
return Products.Select(Product => new Products.Product(Product)).Distinct().ToList();
}
catch (System.Exception ex)
{
string message = "Error occurred while calling GetProducts.";
throw new Exception.CatalogDataException(message, CodeLibrary.Core.Helpers.ProcessHelper.GetProcessName(this), ex);
}
finally
{
if (conn != null && conn.State == ConnectionState.Open) conn.Close();
if (context != null) context.Dispose();
conn.Dispose();
}
}
Again, this is just one example of one of our data access methods in C#. Don't see any issues with this do you? Again, we use this format across the board. We've confirmed this.
With the new .net project, we use .net membership provider. We use a CLR to encrypt users passwords with a hash so that we can use the same hash method in CF. Not sure if this is the issue but thought it was worth mentioning.
Any ideas?

There is a list of possibilities here. For example, when a call to SQL server fails to return data to CF, CF can hang onto the thread. It becomes a sort of "phantom thread." CF then creates new connections to the DB server and adds them to the connection pool - resulting in the many extra connections you are seeing. It's counted against the "simultaneous requests" setting in the CF admin. When there are enough of them "hanging" your requests queue and your server locks up even if it doesn't appear anything is going on. You can see this behavior by enabling metrics, by using the server monitor (if on the Enterprise version) or by using fusionreactor (an excellent and inexpensive 3rd party introspective monitor for your CF/Java server).
Of course that is what is happening. You have to find out why it's happening. Among the possibilities are:
Networking - sometimes autosynch on ports to your switches can interupt connections and result in hanging "phantom" threads. See this post on Hanging jrun and networking.
Database Locking - this can produce issues like this and may be occuring even if you think you are not seeing it. It's sometimes tricky to catch. One particular locking issue that can be troublesome is the "max degree of parallelism" which can result in fairly idle looking DB connections that are nonetheless hanging.
You will probably need to get a bit more information on the CF side of things to know exactly what is going on here.
Follow up ... I'm providing some possibilities from the CF side even though your question was from the .NET side. I'm assuming that CF could be in play since restarting CF sometimes fixes the issue.

UDID for windows 8

Is there any unique device ID (UDID) or any similar ID I can read out on Windows 8 that doesn't change with hardware changes, app-reinstallation etc.?
If No - what is the best way to generate it yourself?

No. Yes.
No, there is not such ID because (in theory) you can change ANY hardware component so you may get a completely different ID (that's why Microsoft suggest to calculate a score based on ASHWID).
Yes, there is such ID (but it may not be applicable in your case).
If you can't rely on hardware because it's easy to add memory, change disks, add another network card (for example turning on/off bluetooth or wi-fi) then you have to rely on a "software" ID.
In the registry there is an unique ID generated during Windows installation and it won't change until you reinstall Windows. You can find such ID in HKLM/Software/Microsoft/Cryptography, it's a string named MachineGuid.
If you can identify a component you're pretty sure that won't change (motherboard for example) you may use a simple WMI query to get its serial number but you should always provide a fallback because many many MBs returns a fake S/N (and virtual machines may returns always the same one). What's the proper solution...well it depends on what you have to do with that ID. Identify the user? Check for license? Encrypt data? Each of these has a different "best practice" for ID.
Get an unique ID for the device
If you have to identify a particular device (regardless to the user) you have many options, what I'd prefer to do is to generate an ID using only stable data (S/N from motherboard and BIOS, for example). This won't help you if he/she completely renew its hardware but it should be stable enough (but you have to define what is enough in your case). You may even use the S/N of the primary disk (with portable devices it's pretty stable and you may even use it in combination with other serial numbers to build your own ID). You can get this informations through WMI or (if you're targeting WinRT) through specific bytes of the ASHWID structure.
Encrypt data
In this case you have to think when data may be unrecoverable. If with a small hardware change your users won't be able to read their previous files well, they'll be unhappy. In this case I would suggest to use the MachineGuid, unless they reinstall the OS they wouldn't have to worry (but do them a favor and provide a way to read back that GUID somewhere). If you're sure you're targeting a portable device like a phone or a tablet then disk serial number (or CPU ID, if available, or MB or BIOS) may be appropriate too (because it's pretty uncommon they'll change).
Licensing
I would use a combination of many (stable) IDs. As for an unique identifier for the device you can't be sure nothing will change. In the past MAC address was vastly used for this but mobile devices changed these rules (because it's easy to turn off a NIC). You can still use them but you have to put extra care (and code) to manage that situation. Again a combination of multiple IDs (chosen carefully) can help you to minimize customers effort when they change their hw/sw setup. In this case a good compromise could be the OS serial number (not the MachineGuid). If they install a new OS then they have to update your license too (but I would use it combined with something else to be sure they won't use the same OS copy on multiple computers or virtual machines).
Note about virtual machines
If you have to target VMs too then things become more complicated. In theory an user can create multiple copies of the same VM with exactly the same hardare and software configuration. If this is an issue and if you can't address this properly (for example using a network check) I would suggest you don't support them at all (just quit if you detect a VM).

here is a code example in JavaScript that filters form ASHWID the hardware modules that are unlikely to be changed (CPU id, size of memory, serial number of the disk device and bios) and convert it to string. the code is based on a code from this thread
// get the hardware Profile id and convert it to byte Array
var hardwareToken = Windows.System.Profile.HardwareIdentification.getPackageSpecificToken(null);
var byteArray = Windows.Security.Cryptography.CryptographicBuffer.copyToByteArray(hardwareToken.id);
var deviceSerial = '',
offset = 0;
// we filter the hardware modules that are unlikely to be changed, and aggregate them to a string.
while (offset < hardwareToken.id.length)
{
// CPU ID of the processor || Size of the memory || Serial number of the disk device || BIOS
if ((byteArray[offset] == 1 || byteArray[offset] == 2 || byteArray[offset] == 3 || byteArray[offset] == 9) && byteArray[offset+1] == 0)
{
for (var i=0; i<4; i++)
{
deviceSerial += byteArray[offset+i].toString();
}
}
offset += 4;
}

Can processor id be used for AES crypto key?

I am using the code below to get processor id:
System.Management.ManagementClass theClass = new System.Management.ManagementClass("Win32_Processor");
System.Management.ManagementObjectCollection theCollectionOfResults = theClass.GetInstances();
foreach (System.Management.ManagementObject currentResult in theCollectionOfResults)
{
MessageBox.Show(currentResult["ProcessorID"].ToString());
}
Is processor ID a unique ID for every computer? I want to generate a random unique number for every computer to use as a AES crypto key. I don't want to use MAC address because i know that MAC address can be changed.

Processor ID is not guaranteed to be unique for every cpu or guaranteed to exist at all.
Hardware can be replaced - you can't change that fact.
Suggestions:
if you need to uniquely identify a machine - it is ok to use the processor Id or MAC address or both for uniqueness. (or any other hardware related number).
but you must keep in mind you id the hardware - not the user, so if the hardware has changed it is much like if someone you know took a face lift and now you don't recognize him.
If you need to identify the user (which I think is better) just let him choose a username and a password.
If you need to make sure that only one previously identified user is using your app, and your'e afraid he will be pirating your app.. the only way is having a server/central point.

It' s bad solution because CPU must support CPUID instructions and system may be contains more one CPU.

Help with Hardware ID

I'm using C# and .NET 2.0. My app needs some way to check the different users so I'll use hardware id, but I saw that I can use only:
mac address - easily changeable and not everybody have it // NO
processor id - using WMI it returns a value for the cpu model, it's not unique // NO
motherboard serial - not every motherboard have it // NO
the Windows's volume serial - I'm not sure if it will be changed on Windows reinstall and format of the volume
So is there something I didn't mention for hwid? I want something that everybody have and it won't be erased on windows reinstall. Else I'd have to use the windows's volume serial number.
EDIT: From the commments I think it's best to use HDD id. How to get it?
EDIT2: I just read that the SCSI drives don't have serial. Is that true?
FINAL EDIT: I'm already using the root drive serial on my app. It work's pretty well. Thanks all.

HDD Serial number: unique, unchangeable, and everyone has it.
not a perfect option but...

Well i'd go for more than one id. If you combine enough IDs they will get you enough uniqueness.
EDIT: you might also go for the place on the harddisk your program was installed too (Platter, Cylinder etc.)

Why do you want to use hardware Id? I'd go with some kind of forms-based (or AD based) security, myself.
Given that, however: The thing about HWID is that it identifies that particular computer configuration: it is designed to change if there are system changes. You mention that Windows Volume Serial could change on reinstall and reformat, but won't your software also have to be reinstalled at that point? HDD Serial could also change if, say, the user swaps HDDs for some reason.
If you have to use HWID, you'll probably need to select one (or more) of the available options to provide uniqueness and either code around, or inform users about, hardware changes requiring a re-install and/or reconfigure of your software.

The MAC address is not very reliable.
You should use something that cannot be changed such a CPU ID or HDD IDE ID.
Here is want I mean:
http://www.soft.tahionic.com/download-hdd_id/index.html
http://www.soft.tahionic.com/download-hdd_id/hardware%20ID%20programmer%27s%20DLL.html
And to answer your question, Yes, SCSI does not have a hardware ID.

Do you have to think about security or only reliability, i.e. does the user want to tamper your ID check?
If you need (high) security, buy one of the "software protection" products, this is far cheaper than to do it yourself.
No high security required? =>
Another possibility is to use the User or Machine SID. You have to take into account that your application is executed elvated => User is Administrator then.
Last but not least: Write a random value of some bytes into the registry (possibly at various places) under HKEY_CURRENT_USER and use these as an ID.

Can a C# program measure its own CPU usage somehow?

I am working on a background program that will be running for a long time, and I have a external logging program (SmartInspect) that I want to feed with some values periodically, to monitor it in realtime when debugging.
I know I can simply fire up multiple programs, like the Task Manager, or IARSN TaskInfo, but I'd like to keep everything in my own program for this, as I also wants to add some simple rules like if the program uses more than X% CPU, flag this in the log.
I have a background thread that periodically feeds some statistics to SmartInspect, like memory consumption, working set, etc.
Is it possible for this thread to get a reasonably accurate measure of how much of the computer's CPU resources it consumes? The main program is a single-threaded application (apart from the watchdog thread that logs statistics) so if a technique is limited to how much does a single thread use then that would be good too.
I found some entries related to something called rusage for Linux and C. Is there something similar I can use for this?
Edit: Ok, I tried the performance counter way, but it added quite a lot of GC-data each time called, so the graph for memory usage and garbage collection skyrocketed. I guess I'll just leave this part out for now.

You can also use System.Diagnostics.Process.TotalProcessorTime and System.Diagnostics.ProcessThread.TotalProcessorTime properties to calculate your processor usage as this article describes.

Have a look at System.Diagnostics.PerformanceCounter. If you run up perfmon.exe, you'll see the range of performance counters available to you (set the 'performance object' to 'Process'), one of which is '% Processor Time'.

You can through the System.Diagnostic.PerformanceCounter class. Here's an example of somebody monitoring CPU usage:
http://blogs.msdn.com/dotnetinterop/archive/2007/02/02/system-diagnostics-performancecounter-and-processor-time-on-multi-core-or-multi-cpu.aspx
Note that this does require elevated privileges. And there may be a performance hit using it.

It is good that you are logging to monitors like smartinspect. But windows itself gathers the data for each resource in this case your program (or process). WMI is the standard for Application monitoring. We can view the data captured by WMI. Many application management, health monitoring or applicaiton monitoring tools support WMI out of the box.
So I would not recommend you to log your CPU usage within the application to a log file.
If you think availablity and performance is critical then go for solutions like Microsoft Operations manager solution.
To get an idea about WMI and to get the list of process see below:
- Win32_PerfFormattedData_PerfProc_Process to get Cpu time, filter is processID
See this article
- You can get the processID from Win32_process class.
WMI Made Easy For C# by Kevin Matthew Goss
oConn.Username = "JohnDoe";
oConn.Password = "JohnsPass";
System.Management.ManagementScope oMs = new System.Management.ManagementScope("\\MachineX", oConn);
//get Fixed disk stats
System.Management.ObjectQuery oQuery = new System.Management.ObjectQuery("select FreeSpace,Size,Name from Win32_LogicalDisk where DriveType=3");
//Execute the query
ManagementObjectSearcher oSearcher = new ManagementObjectSearcher(oMs,oQuery);
//Get the results
ManagementObjectCollection oReturnCollection = oSearcher.Get();
//loop through found drives and write out info
foreach( ManagementObject oReturn in oReturnCollection )
{
// Disk name
Console.WriteLine("Name : " + oReturn["Name"].ToString());
// Free Space in bytes
Console.WriteLine("FreeSpace: " + oReturn["FreeSpace"].ToString());
// Size in bytes
Console.WriteLine("Size: " + oReturn["Size"].ToString());
}
You can monitor the process from Remote system as well.

This code project article describes how to use the high performance timer:
http://www.codeproject.com/KB/cs/highperformancetimercshar.aspx
You can use it to time the execution of your code.
Here you can find a number of open source C# profilers:
http://csharp-source.net/open-source/profile