I'm trying to prepare to build a database driven .net application and I have hit a roadblock early on due to my lack of knowledge on this topic. Searching around didn't yield anything so here I am asking for help.
I'm receiving weekly data in xml format that will be added to a database and then reports generated using that data. I have a limited license on the xml files so only I can download them and I need to get the results to my end users as well. As far as I can see, I have 2 options:
Feed the data from the xml files into a web hosted database and then have each user connect to the database.
Upload the xml data to a server, have each user download it and keep a local copy of their own database. I'm thinking this will invalidate my license to the original data.
Things / questions of note:
The database holds weekly sports historical data for about the last 10 years.
I need to limit access to the database to only subscribed users.
I'll need to decide how the database will be built.
I need to decide what kind of hosting I'll need.
As you can see, quite an ambitious project for someone new to this. I haven't asked any specific questions so far:
What kind of hosting solutions shall I look for?
Should I use SQL? (Complete newbie on this subject)
Should I use clickonce and then host the application?
Do you have any book or tutorial recommendations that would cover a project like this?
Do I need a script to feed the xml into the database if I go that route? Will that script reside on the server and do it automatically even if I'm not there to instigate it?
I hope the general topic isn't too vague. I tried to actually ask specific questions on it and I'm aware I don't have any code to show as it's just in the early stages of thinking.
The question is a bit vague since you are early on in the decision-making process. However, I do believe that I can offer some help in directing your thinking as you proceed. I think in the situation you are describing, one key thing you should consider is to host your data via JSON/WCF/REST. If you look into these technologies, you will see that there are different ways you can offer your data based upon your developing requirements. For example, how are you going to do authentication? Are you going to allow third-party clients?
What you really don't want to do is allow direct database access, even for authenticated users. Instead, put something in front of it. If you are working in the .NET space, look into all of the different things WCF offers and pick one based upon what fits best. Once you pick that, then you will know what you need for hosting and deployment. Even if you are going to provide the clients as well as the server, this is still a good way to protect your data and provide a way to expand your offering in the future.
Related
I'm part of a small team that currently uses an Access database for scheduling a larger team's availability. This has presented some issues with corruption of the Access database. Additionally, I want to implement additional functionality over time.
I've set out to create an application for the 4-5 of us to use that will solve the concurrent database issue, as well as give the team more functionality.
Since this is a shared network drive, I won't have access to SQL Server (from my guess). I thought maybe a web service would be the way to go, but I don't really want to front the bill for this. Additionally, when I eventually leave the team I don't want to maintain this.
Some ideas I've come up with is an application written in C# that acts as the front-end with SQLite embedded as the back-end. However, I've spent days trying to get the Entity Framework to work with SQLite and am at the point of giving up.
I'm trying to decide what else I can do to solve this issue. Is there another technology I can use?
As was said, it sounds like you try to reinvent the DMBS wheel.
If you have a Database that multiple clients can use at the same time, "sharing a access file on a network share" will simply not cut it. You need a proper DBMS. You have simply outgrown the scale Access was designed for. propably even the scale it was intended for.
You said cost might be an issue, but it is not really: There are dozens of DBMS out there, with a number being Freeware. MySQL is a shining example of a free DBMS. Conver that whole Access thing into a MySQL Database. Write a frontend for the MySQL Database. Done.
If you already have a computer providing the share across the network, that same computer can provide the MySQL Server. Setting up a DBMS with 1+ instances can be a bit more involved then just enabling a share, but not much more then programming a WebService.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I'm working on a collaborative real-time drawing tool. My client is already written in C# and I need a server. I've done a lot of search on google, but I don't know how to choose a database and a language for the server side. I'm a beginner and I would like to have your help. In my database I need to store users information and their pictures, also users can chat together, so I'll be using sockets. Is it possible to build my server with nodeJs and a mongoDB database or C# and SQL. Also, what makes this difficult for me to choose is how can I store pictures (that users are working on) in my database and always sync with changes. Is that possible with Json (image represented by an array of bytes?) or there is another way for storing pictures and update them whenever a change is made by a user?
Thank you.
A lot of opinions will go into into my answer, but here goes...
Some assumptions -
Since your asking, your team size is approximately 1.
Since the team size is ~1, the customer base is smallish.
Since chat is not the primary function of the product, price, simplicity and time are important.
You know your way around the existing app, thus you are already familiar with C#, and also comfortable with nodeJS, SQL and mongoDB.
So ask yourself -
Firstly, what are you best at? What are you most familiar with? Do you have time constraints that can be best served with a more familiar stack?
What about storing your images in AWS S3 or similar cloud storage? Persisting binary data into an SQL database is not ideal, and it quickly causes issues with managing the size of your database backups.
What are the requirements for uptime/availability and scale?
Is there an off-the-shelf alternative chat program that you can plug in so that you have NO back-end? (i.e. freshdesk)
You database decision comes down to requirements. Lots of possibly irrelevant questions here i.e.
Does the user need to see their old chat?
How many users in a chat room?
How long do chat messages have to stick around for?
If no one is available on the other end, how will this work? How do you match users?
What reporting or administrative requirements are there?
Are there lots of object types with complex relationships between them that are best stored in an SQL DB?
Some recommendations -
Keep your database small - don't persist chat logs or purge them as soon as you can; Keep files over a few KB out of the db
If you choose .NET, consider WCF or signalR for communications
If you don't have to store chat logs, consider a memory database like redis with pub-sub capability
If you have to support and maintain the server, consider instead a Linux OS and compatible stack so your not woken up in the middle of the night when a Windows Update fails and the machine wont boot.
Consider virtualization. i.e. open up a free-ish AWS account and sniff around what technologies you could cobble together to do much of the work for you - a few lambda functions and you may not even need a server!
Personal favorites -
Linux
Redis for pub-sub and chat
object store (mongo/dynamo whatever) for user profile
S3 or comparable for images
The stack you choose is up to you but if you coded client in C# then it would be simplest to stay with that stack.
For me right now to do this task I would create a web api in ASP.NET Core 2.0. This will handle all the user info and image uploading as well as authentication.
I would use Entityframework Core which is an ORM that can map to any database you want (although support may vary). The new SQL Server Express runs on windows and linux so I would probably choose that. For the chat you can use SignalR.
Really it doesn't matter so much what you use as long as it can accomplish the job but if you are just starting out it make sense to pick a reasonably new stack (although not bleeding edge or you may have difficulty finding adequate documentation).
I think a good place to start is make sure you understand the following:
Difference between a 'database', 'database management system(DBMS)' and 'SQL'
SQL vs NoSQL DBMS. How is data stored and accessed for each?
mangoDB is a NOoQL DBMS. MySQL is a SQL DBMS.
JSON is a file format for representing data. Think of XML. It has nothing to do with arrays of bytes.
When you have clients collaborating on a picture, your server application will probably want to keep the picture in memory to allow for fast editing. You will want to save it to the database every once in a while. You can do that in many ways. You could also decide you want to store the files as bitmaps on the servers hard disk.
Most applications have a single instance of the server handling multiple clients so you want your server application to be written in a faster language. That being said with today technology and the scope of your application, pretty much any language will do.
I am designing a program that will build and maintain a database, and act as a central server. This is the 'first stage' of a grander plan. Coming later will be 3-5 remote programs built around the information put into this database.
The requirements are:
The remote programs must be able to access the information in the database.
The remote programs must be able to set alerts when information in the database changes.
The remote programs must be able to request the central server to go out and fetch new / different data.
So, the question is this: how do I expose this data and events to the outside world? My two choices are:
Have them communicate directly with my 'server' application. This seems easier to:
do event notifications (although I suppose I'm probably missing something in SQL).
It also seems like this is more 'upgradeable' - that is I don't need to worry about the database updating and crashing all my remote programs because something changed. I can account for this and transform it the data to a version the child program will understand.
Just go ahead and let them connect directly to the database.
This nice thing about this is that it's solved. I can use LINQ for SQL. The only thing the main server application needs to do is let the remote programs know where the database is.
I'm unsure how to trigger / relay 'events' for field changes in a database over different programs that may or may not be on the same computer.
Forgive my ignorance on this question. I feel woefully unprepared to ask it, but I'm having a hard time figuring out where to get started with this. It is my first real DB project :-/
Thanks!
If the other programs are going to need to know about updates to the database, then the best solution is to manage all db updates through your server application so it can alert clients of the changes. Otherwise it will be tough for the clients to be aware of changes to the db. This also has the advantage of hiding the implementation details of your storage solution from the clients, so you are free to change databases, etc...
My suggestion would be to go with option 1. Build out a web service that can provide the information they all need. This will be the most flexible and allow you to reduce duplicate backend code that would happen with direct communication with the database.
I would recommend looking at some Data Source design patterns first. This types of patterns will help you come up with solutions about how to manage the states of your data. Otherwise I think that I would require some more information about your requirements for the clients to make any further useful suggestions.
I recommend you learn about SQL Server and/or databases first. You don't appear to realize that most of what you want from your "central server" can all be done by SQL Server itself.
A central databse is the simplest option and the cheapest to both build and maintain.
There are however a few scenarios where a central database could cause problems:
High load on one of the systems: A high load on one of the systems could reduce performance on the other systems. For example someone running an internal report stops you being able to take orders on your eCommerce site.
With several systems writing to the same database there is a greater chance of locking.
With several systems dependent on the same database schema, how do you upgrade? All systems at the same time?
If you need to take down the database all systems stop.
Good afternoon - I have a pretty general question today - I've been tasked with creating a web application to manage some basic information on customers. It's a very simple application, but what I don't know is what to keep in mind to develop the site around supporting multiple users at their own domains or subdomains of our url?
How would I restrict users from logging in to each others portion of the app?
I've seen mention of database scoping in similar questions on Stack Overflow, could anybody elaborate on best practices for an implementation like this?
Are there any new features in MVC3 to support multi-tenancy? I am facing this issue with MVC2 and my eCommerce site where we decided we wanted it white-labeled and customizable for multiple shop owners, and don't know where to begin in implementing these features in an existing application. Any input is appreciated.
edit
To elaborate on multi-tenancy, what I mean - in the context of a store for example, multiple users sign up for their own store at www.mystore.com and are each given a unique subdomain to access their own instance of the store, at user1.mystore.com, user2.mystore.com etc. Each store would have customers with order histories, and those customers would have logins. I would need to restrict customers of user1.mystore.com from logging in at user2.mystore.com without a new account, and likewise prevent user2.mystore.com from accessing user1.mystore.com's customer history.
I implemented a complete MVC multi-tennant app. Here are some links I found handy and some sample apps:
http://msdn.microsoft.com/en-us/library/aa479086.aspx
http://codeofrob.com/archive/2010/02/14/multi-tenancy-in-asp.net-mvc-controller-actions-part-i.aspx
http://www.developer.com/design/article.php/10925_3801931_2/Introduction-to-Multi-Tenant-Architecture.htm
http://msdn.microsoft.com/en-us/library/aa479086.aspx#mlttntda_cc
http://lukesampson.com/post/303245177/subdomains-for-a-single-application-with-asp-net-mvc
http://code.google.com/p/multimvc/
http://www.paulstovell.com/widgets
http://www.agileatwork.com/bolt-on-multi-tenancy-in-asp-net-mvc-with-unity-and-nhibernate/
http://ayende.com/blog/3530/multi-tenancy-approaches-and-applicability
http://weblogs.asp.net/zowens/archive/tags/Multi-tenancy/default.aspx
http://cloudsamurai.codeplex.com/
http://cloudninja.codeplex.com/
http://msdn.microsoft.com/en-us/library/hh534484.aspx
http://blog.maartenballiauw.be/post/2009/05/20/ASPNET-MVC-Domain-Routing.aspx
http://blog.tonywilliams.me.uk/asp-net-mvc-2-routing-subdomains-to-areas
Even starting from scratch, you are in for a world of hurt. The MVC framework does very little to help you address the issues.
Most likely you are about to spend a fair amount of time restructuring your database.
The first step is that you are going to create a table to house your "Tenant" list. Then you need to add this TenantId to just about every table in your system to make sure no one steps on each other. You can skip any tables that are global in nature. One example might be a list of Status Codes.
However, everything from users to the data they have etc will have to have this ID. Also, modify all of your indexes to take tenantid into account.
Once you have that, you'll need to modify all of your queries to take the tenantid into account.
One column of the tenants table should be the portal url. Like customername.oursite.com or whatever. This way you could point multiple urls to the exact same code. When the site needs to use the current tenantid just look it up based on the URL the passed in.
If I was doing this, I'd plan to spend about 1 to 2 hours per table in the database to make it "multi-tenant". Obviously some tables (and their queries) will go faster; others will take longer.
Incidentally, this doesn't cover things like customizing the UI (look / feel) per tenant or anything of that nature. If you need to do this then you'll have to either create a directory on the server for each tenant to hold their style sheets or load it directly from the DB (which has it's own issues with regards to caching).
Typically, you design for this at the beginning of the project. Refitting an already (or almost) complete project is a PITA.
Finally, test, test, test and do more testing. You will have to make sure that every single query pulls only the data it absolutely needs to.
There has been some talk of multi-tenancy support in Sharp Architecture (based on MVC 3) found here: http://www.yellowfeather.co.uk/2011/02/multi-tenancy-on-sharp-architecture-revisited/
Not sure if that really helps you with your existing application, porting over would be a bit of a job.
I have a WPF application that stores a large amount of information in XML files and as the user uses the application they add more information to the XML files. It's basically using the XML files as a database. Since over the life of the program the XML files have gotten quite large, and I've been think about putting the data on a website, I've been looking into how to move all the information into an SQL database.
I've used SQL databases with web applications (PHP, Ruby, and ASP.NET) but never with a Desktop application. Ideally I'd like to be able to keep all the information in one database file and distribute it along with the application without requiring the user to connect to a remote database (so they don't need an internet connection - though eventually it would be nice if could compare the local file's version with one online somewhere and update if necessary) and without making them install a local database server on their computer. Is this possible?
I'd also like to use LINQ with any new database solution so switching to a database doesn't force to many changes (I read the XML with LINQ).
I'm sure this question has been asked and that there are already some good tutorials on the subject but I just can't find them.
SQLite is a good embedded database that you can ship along with your application. I have not done much more than some prototyping with it, so I personally cannot say with 100% certainty that it will meet your needs. But from what I have read, and what little I have written against it, it seems appropriate for the job.
SQLite Homepage
ADO.NET Provider
If you know how your objects are all going to fit together, you could serialize them/deserialize them to store them on disk as a set of ProtoBuf objects (depending on their size, of course). I've found that it's a pretty simple, elegant solution to storing a set of interconnected classes. Each class that should be savable, all your data, can be serialized using this method, and then restored as necessary.
Here's the .NET link to it.
This is a previous question I asked on SO, and got several good responses.