We Keep Coding

best alternative to cookie,when size limitation kicks in

I have been working with a fairly huge database. I want to populate the webcontrols(dropdownlists) of the page during pageload event to give the webpage as much flexibility as possible. For example, I have a dropdownlist ,that user can select, which will be populated from the unique rows of a specific column of a datatable.
Now, i don't want to fire the oracle query each time page load happens because that will slow down the webpage significantly (about 1mins each time). So i started to think of cookies as solution. I soon found out that limitation of cookie size (4kb) is way too small for my purpose. I will need around 10-15kb of size if i really want to store the datarows locally! So I tried to search if there were any way to increase the cookie size limitation to accommodate my needs and I found the possible solution is localstorage.
Is there really a way to increase the cookie size limitation?
What is the simplest alternative? is it really localstorage? or are there anything else to look into?
details: I am using C#/ASP.NET + ORACLE

Here are all storage options for state information in ASP.Net -
Cache - a memory pool stored on the server and shared across users
Session - stored on the server and unique for each user
Cookies - stored on the client and passed with each HTTP request to the server
QueryString - passed as part of the complete URL string
Context.Items - HttpContext and lasts only the lifetime of that request
Profile - stored in a database and maintains information across multiple sessions
Now, i don't want to fire the oracle query each time page load happens
because that will slow down the webpage significantly (about 1mins
each time).
You have two options for your scenarion -
If data shared across users, use Cache
If data is unique for each users, use Session
Ideally, you do not want to store in ViewState it will make your page very heavy unless you configure to store ViewState in Sql Server (which is out of the scope of this question).
Update:
localStorage - Not all browsers can handle localStorage, so make sure you check it first.
<script type="text/javascript">
if(window.localStorage) {
window.localStorage.SetItem('keyName','valueToUse');
// OR
window.localStorage.keyName = 'valueToUse';
}
</script>
FYI: ASP.NET does not offer specific methods for handling local storage; you can only manipulate at client side via javascript.

You don't want to use cookies.
I had a similar requirement in an application I've been working on. I needed to let a user select a value from ~15,000 choices using an autocomplete input box. I did not want to hit the database each time the component needed to find a value. I decided to use session storage to store all possible values and then feed those values to the autocomplete component. It works great. You could also use local storage.
For my needs, session storage was a better choice because I didn't need to values to persist after the user closed the browser tab. Also, before I store the 15,000 items in session storage, I compress them, using JavaScript compression, to save space. I'm using lz-string for my compression.

You don't want to use cookies for this purpose. There's not only the size limitation, but that cookie is sent back and forth with every request to the server, which could potentially reduce responsiveness in your application depending on your server and user's bandwidth.
Another alternative is to use the Viewstate which stores the data on the page itself. This doesn't have the space limitation, but it does have the same problem as cookies with the increased request size.
You can also use Session which will store the data in memory on the server for each session. This reduces bandwidth requirements, but does take up additional memory space on the server itself, which may not be desirable. Tons of data in memory + tons of sessions could = insufficient memory.
So figure out what your particular tradeoffs are. Viewstate is the most similar to your current solution.
To access viewstate or session in your page, just do Viewstate['KEY'] or Session['KEY']
Another thing to note is that the Session data is persisted for the entire session and across page loads/redirects/etc. Viewstate is ONLY available for that page. If you go to another page, Viewstate is discarded.
If you data is the same for all users, then you could also use Cache, which stores the data at the Application level instead of session. This still takes up memory space on the server, but will not increase as sessions do (a mere 15k in your case!). You can also specify expiration for the cache (either sliding or absolute), which allows you to automatically free up the memory being used if that resource isn't being accessed for a certain amount of time. Just make sure you always check if the data is in the cache before using it as .NET will discard the information if it expires OR if it just needs to clean up memory for some other purpose.

does storing Data in ViewState slows down the page load?

i have stored my dataset in the View State(because i need to filter the data on different client clicks and show the data) but i feel like the page loading is taking a lot of time, even a checbox checked event(with AutoPostback) which does not have any code to execute is taking almost 2-3 seconds.
is this just because of the view state data, if so are there any alternatives for which i can achieve my tasks? and i need the data to be shown quicky on client events so i have been using the view state. any work around would help.

As #Tushar mentioned above, ViewState is not the place you want to be storing large amounts of data. It's really only meant to preserve the state of controls between round trips, and it can really lead to poor app performance.
Instead you should look into the following server managed options:
Application State - Used for storing data that is shared between all users. Uses server memory.
Session State - Used for storing data specific to a user's session. Also uses server memory. Data can be persisted through app restarts, as well throughout a web-garden or server-farm. More info from MSDN here: http://msdn.microsoft.com/en-us/library/z1hkazw7.aspx
The biggest cons of those methods are memory management, as both options consume server memory, and keep data until there is either a restart of some sorts, or until the session is dropped. Thus, these methods don't always scale well.
Also, here is an MSDN article discussing the various .net methods of state management, with pros and cons for each method :
A third option is to implement a caching strategy by either using the .NET caching libraries, building your own and/or using 3rd party caching servers/libraries. The benefit to using cache is that you have the data automatically expire after any given specified amount of time. However, complexities are introduced when working in a web-garden or server-farm environment.
The biggest thing to remember, is that any of the strategies mentioned above will require some planning and consideration in regards to managing/sharing the data.

If you're storing a large amount of data in ViewState, you'll notice performance issues. Although ViewState is really meant for a "this page only" and Session is meant for "this session", you'll reach a limit with ViewState size where the Session is ultimately much better for performance.
It's worth noting that you might be having some other type of issue, not just an issue with the ViewState (i.e. your database query may be taking a long time and could possibly be cached).

The ViewState makes the page slightly larger due to the extra data embedded in the page's HTML to hold the serialized ViewState. Whether that extra size will cause load problems depends on the connection speed, and on the size of the view state relative to the rest of the page.
The ViewState is sent back to the server with each HTTP request (so including your AutoPostback). Again, whether that causes a noticeable performance issue depends on the view state size and the connection speed.
On a broadband(ish) connection with the amount of ViewState data one would find in a typical page, you would not see 2-3 seconds additional processing time.
Diagnosing
Use the developer tools in your browser (in IE, press F12). You can monitor web requests including the exact header and body sent and received. You can also see the timing for each HTTP request. If the ViewState is not huge (not more than 1-2K perhaps) and your connection speed is not excessively slow, that is not your culprit.
Alternatives
You can hold state entirely server-side, or put any state items that are large entirely on the server. You can use Ajax requests to process page events that depend on that state.

Instead of loading data from a data-source multiple times, only do it one time. The other answers talk about accessing the data. I have run into instances where I load the data every time I do a post-back.
string myString;
public string MyString
{
get
{
// If there is already data in "myString", do not load it again!
if (this.ViewState["myData"] == null)
{
// Load data one time
this.ViewState["myData"] = "Hello";
}
return this.ViewState["myData"] as string;
}
}

How much ViewState slows down your page depends upon have much view state you have. I've inherited pages that generated over a megabyte of viewstate and seen the web server spend 10 seconds just processing the view state. If you don't want to rewrite your application and you need the large amount of view state, you need to investigate alternate strategies for saving / restoring view state. Saving ViewState to a database or even a plain file is much faster -- don't have to stream viewstate to/from client on each request.
Best strategy is to avoid viewstate in the first place though.
Just thought I should add, some controls are simply ViewState pigs, some grids are just terrible for viewstate consumption.

You can view the source of your page and get the ViewState value and use the online ViewState decoder at below url to check how much large are the values stored in your ViewState field for your pages:
http://ignatu.co.uk/ViewStateDecoder.aspx
If you find your viewstate is having large stored values then you should find alternatives for storing your Dataset.
Anyways, you should avoid putting the Dataset into your ViewState.

bad idea to store DataTable object in asp.net session

I understand that storing DataTable in session variable in asp.net is bad since it will use a lot of server's memory. What I don't understand is that then what do you do when:
User comes to a page where it requires to load a DataTable object (from SQL Server).
User clicks on radio button for simple event (Ex. some controls get disabled).
If you don't save the DataTable object in the session, you have to load it from the SQL server again upon postback on same page instead of just fetching it from the session?
Thanks for help.

DataTable's are pretty heavy objects and are not recommended to be stored in ViewState or Session for that matter. The scenario you describe is about caching data. So, why not use ASP.NET's cache?
ViewState, while it does not use as much memory on the server as Session or Cache, still requires serialization/deserialization on the server, requiring some temporary memory usage, in addition to providing your users a large payload of data on each request to/from the server (just take a peek at View Source in any browser and you'll see a very large hidden input with base-64 encoded data). If you don't use encryption, anyone can decode that data being delivered in each request, causing a potential security problem if any of that data is sensitive. ViewState is also meant for small amounts of data and is usually best to stick to the primary data types like ints and strings.
Session generally isn't a good idea either as it also requires serialization/deserialization, which adds additional overhead in addition to the strain on memory per user. Session has memory limits that decrease per user as you increase concurrent users. Session data does not "expire" until the actual session expires for each user, which by default is 30 minutes. Session is great for user-specific data, but is recommended to keep very small and again stick to the primary data types like ints and strings.
Cache does not serialize any data and is limited in size only due to the bitness of the OS. On Windows 2003 32-bit, you have 800 MB total application pool size to work with (1.2 or 1.3 GB if you use the /3GB switch). Under 64-bit, there's much more freedom and limitations are realistically only what you configure up to the amount of available system memory. A benefit of cache is that as memory pressure increases, cache can be expired to free memory for more important things. You also have control as to when items get expired when memory pressure isn't a factor (expiry's are not guaranteed). Take an additional step and you can put a cache dependency on data in the database, if using SQL Server, allowing the data itself to decide when to expire your cache, ensuring fresh data.
Lastly, the often forgotten about Application object can be used, but only for data that you know can be shared across users and does not need to change that often (hopefully not until an application restart).
Use Microsoft's documentation for ViewState, Session, Cache, and Application objects, to determine the wisest use of each for your particular scenario. A combination of using these correctly in addition to using AJAX (to avoid full page postbacks) and HTTP compression to reduce the payload delivered to the client can make for a very responsive site.
In more complex scenarios like Web farms and load balancing, there are additional issues to think about. Questions you will need to ask yourself will be things like: Should a new session be created if a user hits a different server than the originally requested one? Should cache work no matter what server a user hits? These questions will bring you to solutions that may change where you store data. InProc Session is more forgiving than using, say SQL Server, as a session server, as there are additional serialization restrictions.

Another way to store the DataTable, if you only want to use it at page level, is in ViewState. ViewState["dtbl"] = DataTable;
And you can access it from the ViewState Simply DataTable dtbl = (DataTable)ViewState["dtbl"];

Saving Values in Session or Cache - Difference? ASP.NET

I'd like to know when exactly I should use the Session and when exactly I should use the cache. Are there differences in performance? Can one of them handle a lot of data better? Should the Cache only be used for stuff that's associated with the Application whilst the Session should only be used for stuff that's associated with the current session/user?
Is it wiser to save values which I received from a DB in the Session or the Cache - is there a difference at all assuming I make the cache-keys unique? E.g. Cache["MyKey"+UserId.ToString()].
Also, in general, is using the Session/Cache a lot wiser than retrieving Data from a DB or a Webservice or is there a limit of data that'll be retrieved quicker?

Some differences between session and cache:
The session is per user while the cache is per application
You can store the session data out-of-process (SessionServer or SqlServer) e.g. when using a web farm
What you put into the session stays there until the session is terminated/abandoned or times-out
With the cache, you can specify that items are automatically removed after some time (absolute) or after they were not accessed for some time (sliding)
You can also use SqlCacheDependencies to have items removed from the cache when some data changes in a database
the ASP.NET runtime will also automatically remove items from the cache if the available memory gets low
As for performance:
As long as you use InProc session state, I guess there won't be any performance difference between session and cache.
As soon as you use external session state, performance will obviously be lower than the InProc-cache but I don't have any numbers (depends on network, SQL server power, etc).
BTW: there are also distributed caching solutions which allow having an external, shared cache (similar to the out-of-process session state).

What is the efficient way to store multiple values in Sql Server Session database?

I have 10 - 12 variables on my aspx page which needs to be stored in SQL server Session database(load balancer environment).
What is the the efficient way of storing and retrieving them
group every thing in a class and and store in session ( i am thinking this will reduce the number of database calls)
Storing each variable in session ( I am thinking this will make database calls for every session object i stored)
Can any of you please suggest me the efficient way to do it?

It probably doesn't matter in terms of performance, so use whatever produces the cleanest code.
Each separate session item does not require a separate database call. The entire session is loaded from the db at the start of the request; the entire** session is then written back to the db at the end of the request (assuming that the request isn't using a read-only session).
There will be minor performance differences due to how native types are serialised compared to custom objects (size of the serialised data and/or CPU cost to serialise). These differences will usually be minor compared to the cost of the round-trips to the database.
** I must admit that I can't actually remember whether the entire session is written back to the db or only those items that have been added/modified during the current request.