I am currently working on a large-scale website, that is very dynamic, and so needs to store a large volume of information in memory on a near-permanent basis (things like configuration settings for the checkout, or the tree used to implement the menu structure).
This information is not session-specific, it is consistent for every thread using the website.
What is the best way to hold this data globally within ASP, so it can be accessed when needed, instead of re-loaded on each use?
Any AppSettings in web.config are automatically cached (i.e., they aren't read from the XML every time you need to use them).
You could also manually manipulate the cache yourself.
Edit: Better links...
Add items to the cache
Retrieve items from the cache
Caching Application Data
It's not precisely clear whether your information is session specific or not...if it is, then use the ASP Session object. Given your description of the scale, you probably want to look at storing the state in Sql Server:
http://support.microsoft.com/kb/317604
That's the 101 approach. If you're looking for something a little beefier, then check out memcached (that's pronounced Mem-Cache-Dee):
http://www.danga.com/memcached/
That's the system that apps like Facebook and Twitter use.
Good luck!
Using ASP.NET caching feature is a good option I think. In addition to John's answer, you can use Microsoft's Patterns & Practices team's Caching Application Block.
This is a good video exploring the different ways to can retain application state.
http://www.asp.net/learn/3.5-videos/video-11.aspx
It brushes on the Application object which is global for the whole application, for all users and shows you how to create a hit counter (obviously instead of storing an integer you could store objects). If you need to make changes, you do need to use a lock for concurrency, and I'm not sure how it handles LARGE amounts of data because I've never had to keep that much there.
I usually keep things like that in the Application object.
If the pages are dependent upon one another and they post to one another, you could use the page's request object. Probably not the answer you're looking for, but definitely one of the smallest in memory to use.
I have run into the same situation in the past and found an interface to be the most scalable solution. Application cache may be the answer today, but will it scale to meet your needs?
If you need to scale up, you may find cookies, or some type of temp database storage to be the trick. Simply add a new method to your interface, and set the interface to choose the "mode" from web.config.
Related
I'm developing a PC app in Visual Studio where I'm showing the status of hundreds of sensors that are connected via WiFi. The thing is that I need to hold on to the sensor data even after I close the app, so I'm considering some form of permanent storage. These are the options I've considered:
1) My Sensor object is relatively compact with only a few properties. I could serialize all the objects before closing the app and load them every time the app starts anew.
2) I could throw all the properties (which are mostly strings and doubles) into a simple text file and create a custom protocol for storage and retrieval.
3) I could integrate a database with my app. Someone told me this is the best way to go about it, but I'm a bit hesitant seeing as I'm not familiar with DBs.
Which method would yield the best results in terms of resource usage and speed? Or is there some other, better way to go about this?
First thing you need is to understand is your problem. For example, when the program is running do you need to have everything in memory at the same time or do you work with your sensors one at a time?
What is a "large amount of data"? For example, to me that will never be less than million (or billion in some cases).
Once you know that you shouldn't be scared of using something just because you are not familiar to it. Otherwise you are not looking for the best solution for your problem, you are just hacking around it in a way that you feel comfortable.
This being said, you have several ways of doing this. Like you said you can serialize data, using json to store and a few other alternatives but if we are talking about a "large amount of data that we want to persist" I would always call for the use of Databases (the name says a lot). If you don't need to have everything in memory at the same time then I believe that this is you best option.
I personally don't like them (again, personal choice) but one way of not learning SQL (a lot) while you still use your objects is to use an ORM like NHibernate (you will also need to learn how to use it so you don't get things a slower).
If you need to have everything loaded at the same time (most often that is not the case so be sure of this) you need to know what you want to keep and serialize it. If you want that data to be readable by another tool or organize in a given way consider a data format like XML or JSON.
Also, you can use mmap-file.
File is permanent, and keep data between program run.
So, you just keep your data structs in the mmap-ed area, and no more.
MSDN manual here:
https://msdn.microsoft.com/en-us/library/windows/desktop/aa366556%28v=vs.85%29.aspx
Since you need to load all the data once at the start of the program, the database case seems doubtful. The DB necessary when you need to load a bit of data many times.
So first two cases seem more preferred. I would advice to hide a specific solution behind an interface, then you'll can change it later.
Standard .NET serialization of sensors' array is more simple probably, and it will be easier to expand.
I am basically wanting to fetch an IDIctionary from a place which is global setting repository. We basically use IOC and I had put it there but for certain reasons , it has to be moved to web.config. I have gone through some articles and the nearest I got was this:
http://brijbhushan.net/2011/04/21/how-to-store-custom-objects-in-web-config/
Again this article talks about storing it programmatically. I just want something simple where a programmer can come at the design time and store or change my dictionary and that's it. Any idea on how to achieve this?
The web.config file is not really a good place to store and deserialize objects from. I would recommend using XAML files for this purpose. This is much more appropriate and done by MS themselves. You can get more information here http://blogs.microsoft.co.il/blogs/bursteg/archive/2009/05/18/xaml-in-net-4-0-serialization-and-deserialization-using-xamlservices.aspx.
This is the scenario: I have a list of about 5000 URLs which have already been published to various customers. Now, all of these URLs' location has changed on my server side. The server is still the same though. This is a ASP.NET website with .NET3.5/C#.
My requirement is : Though the customers use the older source URL they should be redirected to the new URL without any perceived change or intermediate redirection message etc.
I am trying to make sense of the whole scenario:
Where would I put the actual mapping of Old URL to New URL -- in a database or some config. file or is there a better option?
How would I actual implement a redirect:
Should I write a method with Server.Transfer ot Response.Redirect?
And is there a best practice to it like - placing the actual re-routing in HTTPModules..or is it Application_BeginRequest?
I am looking to achieve with a best-practice compliant methodology and very low performance degradation, if any.
If your application already uses a database then I'd use that. Make the old URL the primary key and lookups should be very fast. I'd personally wrap the whole thing in .NET classes that abstracts it and allow you to create a Dictionary<string,string> of all the URLs which can be loaded into memory from the DB and cached. This will be even faster.
Definitely DON'T use Server.Transfer. Instead you should do a 301 Permanently Moved redirect. This will let search engines know to use the new URL. If you were using NET 4.0 you could use the HttpResponse.RedirectPermanent method. However, in earlier versions you have to set the headers yourself - but this is trivial.
Keep the data in a database, but load into ASP.NET cache to reduce access time.
You definitely want to use HTTPModules. It's the accepted practice, and having recently tried to do it inside Global.asax, I can tell you that unless you want to do only the simplest kind of stuff (i.e. "~/mypage.aspx/3" <-> "~/mypage.aspx?param1=3) it's much more complicated and buggy than it seems.
In fact, I regret even trying to roll my own URL rewriting solution. It's just not worth it if you want something you can depend on. Scott Guthrie has a very good blog post on the subject, and he recommends UrlRewriter.net or UrlRewriting.net as a couple of free, open-source URL rewriting solutions.
Good luck.
I am constrained by the following, no way around it:
Read-Only Data: Microsoft Access
JET 4.0 OLDB
ASP.NET 2.0
Shared Host, very little control.
OR Mapper - LLBL Gen Pro
The app is a read-only tool that reads a lot of Microsoft Access Databases in the APP_Data folder. Works fine mostly.
Under load it starts failing accessing the Access MDBs.
What is the best strategy for accessing the Access MDBs to limit errors in accessing them? Right now I try, then Thread.Sleep(500) on an error then try again.
I think there may be ways to modify the isolation/concurrency/locking options when accessing the Access databases to eliminate overhead of managing locks. Perhaps try "Mode=Share Deny None;" in the connection string. I would not use this if you are modifying data in any way at anytime though as it's pretty much throwing out all the isolation/concurrency management that you get with a database. Use at your own risk.
How frequently does the data change? If it's read-only can you load the data from the databases into cache and read it from there instead of directly from the databases?
What kind of specific errors getting. I assume they are connection errors?
This is a horrible solution but if you truly are "lost on a desert island with only these tools" and the access databases are completely read-only, then create multiple copies of each of them and allow only a certain number of connection into any of them at a time. For example, if you have 2 access databases, MdbAA and MdbBB then create copies like:
MdbAA01
MdbAA02
MdbAA03
MdbBB01
MdbBB02
MdbBB03
Then when a request comes for MdbAA, see how many requests are currently accessing MdbAA01, if over the threshold, then try MdbAA02, etc. Do the same for any requests to the MdbBB file.
Like I said this is very bad solution but if you truly have no choice then it might work for you. But realistically it sounds like the app has outgrown Access (and the shared host) so it is time to upgrade the architecture.
Spend a bit of money and get some sql storage. How much time have you spent working on fitting a crutch to a broken sounding system?
If the project is worth doing then it's worth investing some cold hard cash.
If a business is trying to force you down this route, explain why the option you have is not viable. If you suggested the option in the first place, grow a pair and explain why you were wrong, but this is how you can fix it.
Sorry if that comes off as flippant.
I am developing my first multilingual C# site and everything is going ok except for one crucial aspect. I'm not 100% sure what the best option is for storing strings (typically single words) that will be translated by code from my code behind pages.
On the front end of the site I am going to use asp.net resource files for the wording on the pages. This part is fine. However, this site will make XML calls and the XML responses are only ever in english. I have been given an excel sheet with all the words that will be returned by the XML broken into the different languages but I'm not sure how best to store/access this information. There are roughly 80 words x 7 languages.
I am thinking about creating a dictionary object for each language that is created by my global.asax file at application run time and just keeping it stored in memory. The plus side for doing this is that the dictionary object will only have to be created once (until IIS restarts) and can be accessed by any user without needing to be rebuilt but the downside is that I have 7 dictionary objects constantly stored in memory. The server is a Win 2008 64bit with 4GB of RAM so should I even be concerned with memory taken up by using this method?
What do you guys think would be the best way to store/retrieve different language words that would be used by all users?
Thanks for your input.
Rich
From what you say, you are looking at 560 words which need to differ based on locale. This is a drop in the ocean. The resource file method which you have contemplated is fit for purpose and I would recommend using them. They integrate with controls so you will be making the most from them.
If it did trouble you, you could have them on a sliding cache, i.e. sliding cache of 20mins for example, But I do not see anything wrong with your choice in this solution.
OMO
Cheers,
Andrew
P.s. have a read through this, to see how you can find and bind values in different resource files to controls and literals and use programatically.
http://msdn.microsoft.com/en-us/magazine/cc163566.aspx
As long as you are aware of the impact of doing so then yes, storing this data in memory would be fine (as long as you have enough to do so). Once you know what is appropriate for the current user then tossing it into memory would be fine. You might look at something like MemCached Win32 or Velocity though to offload the storage to another app server. Use this even on your local application for the time being that way when it is time to push this to another server or grow your app you have a clear separation of concerns defined at your caching layer. And keep in mind that the more languages you support the more stuff you are storing in memory. Keep an eye on the amount of data being stored in memory on your lone app server as this could become overwhelming in time. Also, make sure that the keys you are using are specific to the language. Otherwise you might find that you are storing a menu in german for an english user.