What is a data structure (like list, array, etc...) that could replace a database like SQL?
I would like it to have as many database-like features as possible, like query select and so on...
If is there none, suggest structure how it should be look like
edit:
datatable is good enough i think, thx for the answers
The simplest such data structure would be a record in F# or a class in C#. This would represent your table. A collection of tables would represent a database. You can query this with query expressions (aka Linq), and serialize it as pointed out above. You can also use a DataTable. If you are just looking for an in memory representation of a database you could have that with SQLite.
If you just want to access a database you can do it with the SQLProvider in F#, or Dapper in both F# and C#.
Here is an example with a list of records and a query expression:
open System
type Row = {
Id: bigint
Name: string
Address: string
}
let table = [
{Id = 100I; Name = "Joe"; Address = "NYC"}
{Id = 101I; Name = "Jane"; Address = "KC"}
{Id = 102I; Name = "Jim"; Address = "LA"}
]
let notInNYC =
query {
for user in table do
where (user.Address <> "NYC")
select user.Name
}
|> Seq.toList
//val notInNYC : string list = ["Jane"; "Jim"]
If you are looking to use an actual SQL database, then
(per MSDN):
private static void CreateCommand(string queryString, string connectionString)
{
using (SqlConnection connection = new SqlConnection(connectionString))
{
SqlCommand command = new SqlCommand(queryString, connection);
command.Connection.Open();
command.ExecuteNonQuery();
}
}
If you are looking to not use an actual SQL database and try to save data, relations, etc. directly in your code (not sure why you'd want to do that), you could create your own custom classes for it. You'd want to include some form of table, as well as a search method that could look through the instances of table, etc. There are so many functionalities that you'd have to implement though, so this would be difficult to do if you are trying to replicate all of the functionality of a real SQL db.
Assuming you already know about Entity Framework as an ORM and a gateway to access DBs, here are some alternatives you'd want to have in mind.
One straight forward and quick solution for small data amounts is serialization.
You can choose from:
Json
XML
Binary
Some others.
Serialization allows you to store and retrieve an object graph with not fuss of setting up DBs and connections. But doesn't give sophisticated search and update capabilities.
Another thing you might want to explore is NoSQL databases.
Check out LiteDB to get you started with the concept.
Related
I am very new to working with databases. Now I can write SELECT, UPDATE, DELETE, and INSERT commands. But I have seen many forums where we prefer to write:
SELECT empSalary from employee where salary = #salary
...instead of:
SELECT empSalary from employee where salary = txtSalary.Text
Why do we always prefer to use parameters and how would I use them?
I wanted to know the use and benefits of the first method. I have even heard of SQL injection but I don't fully understand it. I don't even know if SQL injection is related to my question.
Using parameters helps prevent SQL Injection attacks when the database is used in conjunction with a program interface such as a desktop program or web site.
In your example, a user can directly run SQL code on your database by crafting statements in txtSalary.
For example, if they were to write 0 OR 1=1, the executed SQL would be
SELECT empSalary from employee where salary = 0 or 1=1
whereby all empSalaries would be returned.
Further, a user could perform far worse commands against your database, including deleting it If they wrote 0; Drop Table employee:
SELECT empSalary from employee where salary = 0; Drop Table employee
The table employee would then be deleted.
In your case, it looks like you're using .NET. Using parameters is as easy as:
string sql = "SELECT empSalary from employee where salary = #salary";
using (SqlConnection connection = new SqlConnection(/* connection info */))
using (SqlCommand command = new SqlCommand(sql, connection))
{
var salaryParam = new SqlParameter("salary", SqlDbType.Money);
salaryParam.Value = txtMoney.Text;
command.Parameters.Add(salaryParam);
var results = command.ExecuteReader();
}
Dim sql As String = "SELECT empSalary from employee where salary = #salary"
Using connection As New SqlConnection("connectionString")
Using command As New SqlCommand(sql, connection)
Dim salaryParam = New SqlParameter("salary", SqlDbType.Money)
salaryParam.Value = txtMoney.Text
command.Parameters.Add(salaryParam)
Dim results = command.ExecuteReader()
End Using
End Using
Edit 2016-4-25:
As per George Stocker's comment, I changed the sample code to not use AddWithValue. Also, it is generally recommended that you wrap IDisposables in using statements.
You are right, this is related to SQL injection, which is a vulnerability that allows a malicioius user to execute arbitrary statements against your database. This old time favorite XKCD comic illustrates the concept:
In your example, if you just use:
var query = "SELECT empSalary from employee where salary = " + txtSalary.Text;
// and proceed to execute this query
You are open to SQL injection. For example, say someone enters txtSalary:
1; UPDATE employee SET salary = 9999999 WHERE empID = 10; --
1; DROP TABLE employee; --
// etc.
When you execute this query, it will perform a SELECT and an UPDATE or DROP, or whatever they wanted. The -- at the end simply comments out the rest of your query, which would be useful in the attack if you were concatenating anything after txtSalary.Text.
The correct way is to use parameterized queries, eg (C#):
SqlCommand query = new SqlCommand("SELECT empSalary FROM employee
WHERE salary = #sal;");
query.Parameters.AddWithValue("#sal", txtSalary.Text);
With that, you can safely execute the query.
For reference on how to avoid SQL injection in several other languages, check bobby-tables.com, a website maintained by a SO user.
In addition to other answers need to add that parameters not only helps prevent sql injection but can improve performance of queries. Sql server caching parameterized query plans and reuse them on repeated queries execution. If you not parameterized your query then sql server would compile new plan on each query(with some exclusion) execution if text of query would differ.
More information about query plan caching
Two years after my first go, I'm recidivating...
Why do we prefer parameters? SQL injection is obviously a big reason, but could it be that we're secretly longing to get back to SQL as a language. SQL in string literals is already a weird cultural practice, but at least you can copy and paste your request into management studio. SQL dynamically constructed with host language conditionals and control structures, when SQL has conditionals and control structures, is just level 0 barbarism. You have to run your app in debug, or with a trace, to see what SQL it generates.
Don't stop with just parameters. Go all the way and use QueryFirst (disclaimer: which I wrote). Your SQL lives in a .sql file. You edit it in the fabulous TSQL editor window, with syntax validation and Intellisense for your tables and columns. You can assign test data in the special comments section and click "play" to run your query right there in the window. Creating a parameter is as easy as putting "#myParam" in your SQL. Then, each time you save, QueryFirst generates the C# wrapper for your query. Your parameters pop up, strongly typed, as arguments to the Execute() methods. Your results are returned in an IEnumerable or List of strongly typed POCOs, the types generated from the actual schema returned by your query. If your query doesn't run, your app won't compile. If your db schema changes and your query runs but some columns disappear, the compile error points to the line in your code that tries to access the missing data. And there are numerous other advantages. Why would you want to access data any other way?
In Sql when any word contain # sign it means it is variable and we use this variable to set value in it and use it on number area on the same sql script because it is only restricted on the single script while you can declare lot of variables of same type and name on many script. We use this variable in stored procedure lot because stored procedure are pre-compiled queries and we can pass values in these variable from script, desktop and websites for further information read Declare Local Variable, Sql Stored Procedure and sql injections.
Also read Protect from sql injection it will guide how you can protect your database.
Hope it help you to understand also any question comment me.
Old post but wanted to ensure newcomers are aware of Stored procedures.
My 10ยข worth here is that if you are able to write your SQL statement as a stored procedure, that in my view is the optimum approach. I ALWAYS use stored procs and never loop through records in my main code. For Example: SQL Table > SQL Stored Procedures > IIS/Dot.NET > Class.
When you use stored procedures, you can restrict the user to EXECUTE permission only, thus reducing security risks.
Your stored procedure is inherently paramerised, and you can specify input and output parameters.
The stored procedure (if it returns data via SELECT statement) can be accessed and read in the exact same way as you would a regular SELECT statement in your code.
It also runs faster as it is compiled on the SQL Server.
Did I also mention you can do multiple steps, e.g. update a table, check values on another DB server, and then once finally finished, return data to the client, all on the same server, and no interaction with the client. So this is MUCH faster than coding this logic in your code.
Other answers cover why parameters are important, but there is a downside! In .net, there are several methods for creating parameters (Add, AddWithValue), but they all require you to worry, needlessly, about the parameter name, and they all reduce the readability of the SQL in the code. Right when you're trying to meditate on the SQL, you need to hunt around above or below to see what value has been used in the parameter.
I humbly claim my little SqlBuilder class is the most elegant way to write parameterized queries. Your code will look like this...
C#
var bldr = new SqlBuilder( myCommand );
bldr.Append("SELECT * FROM CUSTOMERS WHERE ID = ").Value(myId);
//or
bldr.Append("SELECT * FROM CUSTOMERS WHERE NAME LIKE ").FuzzyValue(myName);
myCommand.CommandText = bldr.ToString();
Your code will be shorter and much more readable. You don't even need extra lines, and, when you're reading back, you don't need to hunt around for the value of parameters. The class you need is here...
using System;
using System.Collections.Generic;
using System.Text;
using System.Data;
using System.Data.SqlClient;
public class SqlBuilder
{
private StringBuilder _rq;
private SqlCommand _cmd;
private int _seq;
public SqlBuilder(SqlCommand cmd)
{
_rq = new StringBuilder();
_cmd = cmd;
_seq = 0;
}
public SqlBuilder Append(String str)
{
_rq.Append(str);
return this;
}
public SqlBuilder Value(Object value)
{
string paramName = "#SqlBuilderParam" + _seq++;
_rq.Append(paramName);
_cmd.Parameters.AddWithValue(paramName, value);
return this;
}
public SqlBuilder FuzzyValue(Object value)
{
string paramName = "#SqlBuilderParam" + _seq++;
_rq.Append("'%' + " + paramName + " + '%'");
_cmd.Parameters.AddWithValue(paramName, value);
return this;
}
public override string ToString()
{
return _rq.ToString();
}
}
I need to cache some look up tables in memory from sql database. I have hundreds of them. The tables are pretty simple with the following structure.
Tablename= "l_lookupobjectname"
column1Name: ID
Column2Name: Code
Code is mostly a string but can also be an integer in a few cases.
I use entity framework and would like a generic way to load those tables into my web application memory. I do not want to individually load each table by specifying its name.
I'm thinking along having a list of dictionary<in id, dynamic code>.
My problem is:
How do I generate the data access code that will pull all the data to my List of dictionary without having to write repetitive code for all my hundreds of table.
"select ID, Code from all the tables" instead calling this statement for each table.
I'm not concerned about the code for caching the data. This is quite trivial.
Your issue might be types, unless you declare everything is a string or an object (and cast it as needed).
Other than that going with some nested dictionaries seems like your best bet. You can build SQL queries ("select * from {0}"), and just provide a list of tables. Then read each one into a dictionary.
You could use DataSet, but that is quite cumbersome. Probably SqlDataReader is better bet.
You can get column names from it by:
var reader = cmd.ExecuteReader();
var columns = new List<string>();
for(int i=0;i<reader.FieldCount;i++)
{
columns.Add(reader.GetName(i));
}
and then just read it all as strings or objects.
With "table names" I mean just the name of normal (not queries or stuff like that), plain old tables. This is because I'm working on a project that currently connects to the Jet engine and among other features, it shows a list of tables that the user would double click to see some specific's table contents. But now I want the user to be able to change the engine from a list of installed engines. But for my program to work with other engines it will need to get the table names in a way that will work for every SQL engine (or at least most of them). I actually also need to be able to get all the column names for a specific table, and also, be able to create a "CREATE TABLE" query in a way that will work with every possible engine (since the user can create tables from a wizard, and my program generates the query). I'm actually very doubtful that this is possible but, as far as I know, Visual Studio can create tables from a wizard for different database engines. How do they manage to do this? Will I have to have a different "CREATE TABLE" query for every possible SQL engine?
I'm wondering if ADO can help with this as it seems to have everything somehow standardized.
No, unfortunately there is no general way to do these things as far as I know. All DB engines have slightly different dialects of DDL and SQL, support different sets of data types, and different ways of managing their metadata etc. If you keep to the absolute lowest denominator of features I guess you could rely on standard SQL/DDL but that will be very limited.
Usually this is solved by creating an abstract data layer with several different implementations which handles the differences.
ADO only solves part of the problem as it offers a common interface for sending queries to a database but the SQL in the queries have to be specified by the client.
If you want any back-end, there will always be one that does not work, but nearly every back-end will allow:
select table_name from information_schema.tables
Your basic create table commands, with keys and indexes, are easily coded to be compatible with nearly every back-end, execpt for auto-incremented integers keys, which have a different syntax on every back end.
So the answer, "mostly yes, probably more than you think, but not 100%". Because the quirks are small, it is possible to write some general code with some tweaks for the particular back-end.
This should do it for you in MSSQL. I imagine it would be very similar for other SQL implementations.
SELECT DISTINCT Name FROM sysobjects WHERE xtype='U'
You can use the GetSchema ADO function to get a DataTable with almost all schema data.
This example use a SQLConnection, but the function can be used in any ODBCConnection.
using System;
using System.Data;
using System.Data.SqlClient;
class Program
{
static void Main()
{
string connectionString = GetConnectionString();
sing (SqlConnection connection = new SqlConnection(connectionString))
{
// Connect to the database then retrieve the schema information.
connection.Open();
DataTable table = connection.GetSchema("Tables");
// Display the contents of the table.
DisplayData(table);
Console.WriteLine("Press any key to continue.");
Console.ReadKey();
}
}
private static string GetConnectionString()
{
// To avoid storing the connection string in your code,
// you can retrieve it from a configuration file.
return "Data Source=(local);Database=AdventureWorks;" +
"Integrated Security=true;";
}
private static void DisplayData(System.Data.DataTable table)
{
foreach (System.Data.DataRow row in table.Rows)
{
foreach (System.Data.DataColumn col in table.Columns)
{
Console.WriteLine("{0} = {1}", col.ColumnName, row[col]);
}
Console.WriteLine("============================");
}
}
}
In Visual Studio it is implemented using the Data Designer Extensibility (DDEX) where a specific provider should expose GetSchema method to help retrieving metadata. You can get some ideas here.
I'm working on a Silverlight project trying to access a database using LINQ To DataSet and then sending data over to Silverlight via .ASMX web service.
I've defined my DataSet using the Server Explorer tool (dragging and dropping all the different tables that I'm interested in). The DataSet is able to access the server and database with no issues.
Below is code from one of my Web Methods:
public List<ClassSpecification> getSpecifications()
{
DataSet2TableAdapters.SpecificationTableAdapter Sta = new DataSet2TableAdapters.SpecificationTableAdapter();
return (from Spec in Sta.GetData().AsEnumerable()
select new ClassSpecification()
{
Specification = Spec.Field<String>("Specification"),
SpecificationType = Spec.Field<string>("SpecificationType"),
StatusChange = Spec.Field<DateTime>("StatusChange"),
Spec = Spec.Field<int>("Spec")
}).ToList<ClassSpecification>();
}
I created a "ClassSpecification" data class which is going to contain my data and it has all the table fields as properties.
My question is, is there a quicker way of doing the assignment than what is shown here? There are actually about 10 more fields, and I would imagine that since my DataSet knows my table definition, that I would have a quicker way of doing the assignment than going field by field. I tried just "select new ClassSpecification()).ToList
Any help would be greatly appreciated.
First, I'll recommend testing out different possibilities using LINQPad, which is free and awesome.
I can't quite remember what you can do from the table adapter, but you should be able to use the DataSet to get at the data you want, e.g.
string spec = myDataSet.MyTable.Rows[0] // or FindBy... or however you are choosing a row
.Specification;
So you might be able to do
foreach(var row in myDataSet.MyTable.Rows) {
string spec = row.Specification;
...
}
Or
return (from row in myDataSet.Specification
select new ClassSpecification()
{
Specification = row.Specification,
SpecificationType = row.SpecificationType,
StatusChange = row.StatusChange,
Spec = row.Spec,
}).ToList<ClassSpecification>();
Or even
return myDataSet.Specification.Cast<ClassSpecification>()
Not sure if the last one will work, but you can see that there are several ways to get what you want. Also, in my tests the row is strongly typed, so you shouldn't need to create a new class in which to put the data - you should just be able to use the existing "SpecificationRow" class. (In fact, I believe that this is the anemic domain model anti-pattern.)
So are you using a dataset for lack of a Linq provider to the database? That is the only reason I would consider this move. For instance, with Linq to Sql you can drag the table out and drop it. Then you have instant objects with the shape you want for it.
I have an Access 2003 file that contains 200 queries, and I want to print out their representation in SQL. I can use Design View to look at each query and cut and paste it to a file, but that's tedious. Also, I may have to do this again on other Access files, so I definitely want to write a program to do it.
Where are queries stored an Access db? I can't find anything saying how to get at them. I'm unfamiliar with Access, so I'd appreciate any pointers. Thanks!
Procedures are what you're looking for:
OleDbConnection conn = new OleDbConnection(connectionString);
conn.Open();
DataTable queries = conn.GetOleDbSchemaTable(OleDbSchemaGuid.Procedures, null);
conn.Close();
This will give you a DataTable with the following columns in it (among others):
PROCEDURE_NAME: Name of the query
PROCEDURE_DEFINITION: SQL definition
So you can loop through the table like so:
foreach(DataRow row in queries.Rows)
{
// Do what you want with the values here
queryName = row["PROCEDURE_NAME"].ToString();
sql = row["PROCEDURE_DEFINITION"].ToString();
}
you can put this together using the OleDbConnection's GetSchema method along with what Remou posted with regards to the ADO Schemas
oops forgot link: MSDN
In case you wanted to do a quick query by hand.
SELECT MSysObjects.Name
FROM MSysObjects
WHERE type = 5
Not in C#, but may be a good place to start:
http://www.datastrat.com/Code/DocDatabase.txt