We Keep Coding

best alternative to cookie,when size limitation kicks in

I have been working with a fairly huge database. I want to populate the webcontrols(dropdownlists) of the page during pageload event to give the webpage as much flexibility as possible. For example, I have a dropdownlist ,that user can select, which will be populated from the unique rows of a specific column of a datatable.
Now, i don't want to fire the oracle query each time page load happens because that will slow down the webpage significantly (about 1mins each time). So i started to think of cookies as solution. I soon found out that limitation of cookie size (4kb) is way too small for my purpose. I will need around 10-15kb of size if i really want to store the datarows locally! So I tried to search if there were any way to increase the cookie size limitation to accommodate my needs and I found the possible solution is localstorage.
Is there really a way to increase the cookie size limitation?
What is the simplest alternative? is it really localstorage? or are there anything else to look into?
details: I am using C#/ASP.NET + ORACLE

Here are all storage options for state information in ASP.Net -
Cache - a memory pool stored on the server and shared across users
Session - stored on the server and unique for each user
Cookies - stored on the client and passed with each HTTP request to the server
QueryString - passed as part of the complete URL string
Context.Items - HttpContext and lasts only the lifetime of that request
Profile - stored in a database and maintains information across multiple sessions
Now, i don't want to fire the oracle query each time page load happens
because that will slow down the webpage significantly (about 1mins
each time).
You have two options for your scenarion -
If data shared across users, use Cache
If data is unique for each users, use Session
Ideally, you do not want to store in ViewState it will make your page very heavy unless you configure to store ViewState in Sql Server (which is out of the scope of this question).
Update:
localStorage - Not all browsers can handle localStorage, so make sure you check it first.
<script type="text/javascript">
if(window.localStorage) {
window.localStorage.SetItem('keyName','valueToUse');
// OR
window.localStorage.keyName = 'valueToUse';
}
</script>
FYI: ASP.NET does not offer specific methods for handling local storage; you can only manipulate at client side via javascript.

You don't want to use cookies.
I had a similar requirement in an application I've been working on. I needed to let a user select a value from ~15,000 choices using an autocomplete input box. I did not want to hit the database each time the component needed to find a value. I decided to use session storage to store all possible values and then feed those values to the autocomplete component. It works great. You could also use local storage.
For my needs, session storage was a better choice because I didn't need to values to persist after the user closed the browser tab. Also, before I store the 15,000 items in session storage, I compress them, using JavaScript compression, to save space. I'm using lz-string for my compression.

You don't want to use cookies for this purpose. There's not only the size limitation, but that cookie is sent back and forth with every request to the server, which could potentially reduce responsiveness in your application depending on your server and user's bandwidth.
Another alternative is to use the Viewstate which stores the data on the page itself. This doesn't have the space limitation, but it does have the same problem as cookies with the increased request size.
You can also use Session which will store the data in memory on the server for each session. This reduces bandwidth requirements, but does take up additional memory space on the server itself, which may not be desirable. Tons of data in memory + tons of sessions could = insufficient memory.
So figure out what your particular tradeoffs are. Viewstate is the most similar to your current solution.
To access viewstate or session in your page, just do Viewstate['KEY'] or Session['KEY']
Another thing to note is that the Session data is persisted for the entire session and across page loads/redirects/etc. Viewstate is ONLY available for that page. If you go to another page, Viewstate is discarded.
If you data is the same for all users, then you could also use Cache, which stores the data at the Application level instead of session. This still takes up memory space on the server, but will not increase as sessions do (a mere 15k in your case!). You can also specify expiration for the cache (either sliding or absolute), which allows you to automatically free up the memory being used if that resource isn't being accessed for a certain amount of time. Just make sure you always check if the data is in the cache before using it as .NET will discard the information if it expires OR if it just needs to clean up memory for some other purpose.

Are there limits for session variables in ASP.net?

I will be populating DataTable and other controls from a complex object.
Where should I store such an object?
At what size does session variables starts affecting the performance of page?

Data in the Session object is stored in memory on the server. Thus the storage limit is the memory available to the server. This data is not sent to the client at any stage, unless you explicitly do so. Instead the MVC code sends a cookie to the client browser once you have assigned any value to the Session object. The value of this cookie is then used to uniquely identify the session.
So...
The Session object is designed specifically so that you can store session-specific data on the server, so is a suitable place for you to put session-specific data structures like you describe.
Because the Session object is server-side only, using Session to store the results of computationally expensive operation that is invariant across multiple page refreshes will speed up page loads, since you can use the previous result instead of having to create it again. Unless you blow out the memory limits on the server, you're not going to see any performance degradation.

If it's a per-session object, the Session dictionary is a reasonable
place to store it
If you are using the InProcess session store, the size of the object never affects page performance (at least until all of the data causes the process to swap). Other session stores may have a slight impact, based on how long it takes to move the data from e.g. SQL to the local process. This will be fast until your object gets really big.

Is there a better solution than using session variables; should session variables really be used in Web Pages, at all?

I am using WebMatrix (C#) to design Intranet web applications for the organization I work for.
I have made one database driven site, and am currently working on another, and for some time, I have been having some trouble with the Session variables randomly becoming null and throwing errors. So much so, that one simple Session variable was switched with the use of the server cache memory instead (which I originally thought would be more volatile, but that remains to be seen, so far...)
One question is: Is there any practical use of the Session variable, at all? If it truly is as volatile as it has seemed to be, they appear to be good for just about nothing.\
I know they are technically cookies, so I know that their data shouldn't be relied upon, but therein lies the problem. I need to send data to other pages that I "can" rely upon. This leaves out both Session variables and cookies.
I generally stay away from query strings or url data, for their blatant "plain-text" display of any sensitive information (like Social Security Numbers), with which, even SSL won't help.
The server cache memory is also volatile and is not to be relied upon.
AppState variables aren't user specific.
That leaves hidden input fields... The problem here is that, sometimes after "Post" I do Response.Redirect, and such, so C# doesn't always render a page after post (it seems).
Maybe this is just a lack of knowledge on my part, but I kind of seem cornered no matter which way I go.
Do I really have to save all page information into a separate database with every page and retrieve it with a sql query on the other page just to get reliable and not "blatantly" displayed information from one page to another with Web Pages? Still even this method would be a problem using several different users, no?

ASP.NET has different Session State modes:
InProc, this is in-memory. Session State lives as long as the IIS Application Pool isn't recycled or the entire IIS is restarted.
SqlServer. This is Session State is stored in a SQL Server storage. This is great because sessions survives after an IIS Application Pool or entire IIS restart, but it's a bottleneck as it means that every access to the session requires a deserialization and/or serialization of the Session State objects and, after all, database connections and so on.
StateServer. Similar to SqlServer, but using a Session State Server provided by Microsoft. This mode isn't used at all, but it's an option... (I've no experience with it).
Custom. You can implement some interfaces/abstract classes and define your own Session State storage.
In the other hand, Session State is server-side, it has nothing to do with cookies. It's a way of simulating state in a stateless world ruled by HTTP. Since ASP.NET writes an HTTP cookie in the browser, it can link a browser session with an unique server session too.
About if Session State is useful or not.... In my case, I've decided to avoid "states" at all. I prefer to stay stateless as much as possible. I write down some HTTP cookies in the browser to identity settings, preferences or users, and I do things per-request.
I prefer that because a good caching mechanism can insanely optimize the requests' performance as in most of the cases you wouldn't be accessing data directly in the store but in some cache, which means a lot of speed!
By the way, Session State shouldn't be used to store large object but basic values. Session State isn't a cache. For example, Session State could be a good place to store something like current logged-in user or his/her role. Or their profile identifier. Who knows.
Any other data should be queried in a per-request basis.
And again: cache should be your friend in terms of optimizing your environment and don't accessing the database or whatever in each request, compromising the system performance.

bad idea to store DataTable object in asp.net session

I understand that storing DataTable in session variable in asp.net is bad since it will use a lot of server's memory. What I don't understand is that then what do you do when:
User comes to a page where it requires to load a DataTable object (from SQL Server).
User clicks on radio button for simple event (Ex. some controls get disabled).
If you don't save the DataTable object in the session, you have to load it from the SQL server again upon postback on same page instead of just fetching it from the session?
Thanks for help.

DataTable's are pretty heavy objects and are not recommended to be stored in ViewState or Session for that matter. The scenario you describe is about caching data. So, why not use ASP.NET's cache?
ViewState, while it does not use as much memory on the server as Session or Cache, still requires serialization/deserialization on the server, requiring some temporary memory usage, in addition to providing your users a large payload of data on each request to/from the server (just take a peek at View Source in any browser and you'll see a very large hidden input with base-64 encoded data). If you don't use encryption, anyone can decode that data being delivered in each request, causing a potential security problem if any of that data is sensitive. ViewState is also meant for small amounts of data and is usually best to stick to the primary data types like ints and strings.
Session generally isn't a good idea either as it also requires serialization/deserialization, which adds additional overhead in addition to the strain on memory per user. Session has memory limits that decrease per user as you increase concurrent users. Session data does not "expire" until the actual session expires for each user, which by default is 30 minutes. Session is great for user-specific data, but is recommended to keep very small and again stick to the primary data types like ints and strings.
Cache does not serialize any data and is limited in size only due to the bitness of the OS. On Windows 2003 32-bit, you have 800 MB total application pool size to work with (1.2 or 1.3 GB if you use the /3GB switch). Under 64-bit, there's much more freedom and limitations are realistically only what you configure up to the amount of available system memory. A benefit of cache is that as memory pressure increases, cache can be expired to free memory for more important things. You also have control as to when items get expired when memory pressure isn't a factor (expiry's are not guaranteed). Take an additional step and you can put a cache dependency on data in the database, if using SQL Server, allowing the data itself to decide when to expire your cache, ensuring fresh data.
Lastly, the often forgotten about Application object can be used, but only for data that you know can be shared across users and does not need to change that often (hopefully not until an application restart).
Use Microsoft's documentation for ViewState, Session, Cache, and Application objects, to determine the wisest use of each for your particular scenario. A combination of using these correctly in addition to using AJAX (to avoid full page postbacks) and HTTP compression to reduce the payload delivered to the client can make for a very responsive site.
In more complex scenarios like Web farms and load balancing, there are additional issues to think about. Questions you will need to ask yourself will be things like: Should a new session be created if a user hits a different server than the originally requested one? Should cache work no matter what server a user hits? These questions will bring you to solutions that may change where you store data. InProc Session is more forgiving than using, say SQL Server, as a session server, as there are additional serialization restrictions.

Another way to store the DataTable, if you only want to use it at page level, is in ViewState. ViewState["dtbl"] = DataTable;
And you can access it from the ViewState Simply DataTable dtbl = (DataTable)ViewState["dtbl"];

What is the efficient way to store multiple values in Sql Server Session database?

I have 10 - 12 variables on my aspx page which needs to be stored in SQL server Session database(load balancer environment).
What is the the efficient way of storing and retrieving them
group every thing in a class and and store in session ( i am thinking this will reduce the number of database calls)
Storing each variable in session ( I am thinking this will make database calls for every session object i stored)
Can any of you please suggest me the efficient way to do it?

It probably doesn't matter in terms of performance, so use whatever produces the cleanest code.
Each separate session item does not require a separate database call. The entire session is loaded from the db at the start of the request; the entire** session is then written back to the db at the end of the request (assuming that the request isn't using a read-only session).
There will be minor performance differences due to how native types are serialised compared to custom objects (size of the serialised data and/or CPU cost to serialise). These differences will usually be minor compared to the cost of the round-trips to the database.
** I must admit that I can't actually remember whether the entire session is written back to the db or only those items that have been added/modified during the current request.