Is there any way to get a simple list of strings from a Dapper Query? I don't want to create an object that contains all my field names by type. My query returns one row of data. Sometimes with 2 columns other times with 5 or 20 or 100 and I just want all the values returned as a single list of strings.
Dapper would make it easy to work with multiple rows, single column, via db.Query<string>(...).
For multiple columns single row, you could try:
var x = db.QuerySingle<(string,string)>(...)
(for two columns; add more items to the tuple for more)
This uses the value-tuple approach to read the data column-wise.
However, this is only good for a handful of columns. If you have hundreds of columns and a single row then I suggest transposing your query (perhaps via PIVOT).
Related
I have a view which I've created by joining several tables whose records can be changed so the content of the columns of the view can also be changed.
Columns of the view contain data like address,random numbers,date,some random string etc.
I'm accepting search text from user and returns rows if any of its column contain text entered by the user.
My view have millions of records so normal like query won't work(takes long time) ?
What is the most efficient way to search this view as it changes as its tables get changed ?
I'm using oracle database, C#, entityframework.
For better performance you should properly add index in the original table .. these indexes are automatically refreshed by rdbms engine on each change .. so is impossible that you obtain wrong data by the index value .. the index value and the table data contain the same values..
You don't need to reindex every time ... sometimes (monthly) you can updated the related statistcs ..
so the index can change you performance in better a lot .. and this also for the view
The view in create on the top of the original table on fly and is not a stored copy of the original tables .. so the indexes help the view to render more fastly the expected result ..
the indexes Indexes when properly designed, serve for important purposes in a database server:
They let the rdbms
find groups of adjacent rows instead of single rows.
avoid sorting by reading the rows in a desired order.
let the server satisfy (sometimes) entire queries from the index alone, avoiding (when possible) the need to access the table at all.
from mysql https://dev.mysql.com/doc/refman/5.5/en/mysql-indexes.html
https://dev.mysql.com/doc/refman/5.5/en/column-indexes.html
https://dev.mysql.com/doc/refman/5.5/en/multiple-column-indexes.html
http://code.tutsplus.com/tutorials/top-20-mysql-best-practices--net-7855
http://use-the-index-luke.com
I have a table that has about 1 million rows. One of the columns is a string, let's call it column A.
Now I need to work on a list L of about 1,000 strings, mostly one or two words, and I need to find all the records in the table where column A contains one of the 1,000 strings in the list L.
The only way I can think of is to use each string in L to do a full table scan, find if the string is a substring of column A content of each row. But that will be O(n2), and for a million rows it will take a very long time.
Is there a better way? Either in SQL or in C# code?
One million rows is a relatively small number these days. You should be able to pull all strings from column A, along with your table's primary key, into memory, and do a regex search using a very long regex composed from your 1000 strings:
var regex = new Regex("string one|string two|string three|...|string one thousand");
Since regex gets compiled into a final automaton, you would get reasonably fast scanning times for your strings. Once your filtering is complete, collect the IDs, and query full rows from the table using them.
The best way to do is is using linq. Lets say that you have your list
List<string> test = new List<string>{"aaa","ddd","ddsc"};
then using Linq you can constract
var match = YourTable.Where (t=> test.Contains(t.YourFieldName);
I suggest looking into full text search, it won't decrease the count of the operations you have to perform but it will increase the performance.
Assuming you use Sql server (you should always use the relevant tag to specify the rdbms),
you can create a DataTable from your List<string> and send it to a stored procedure as a table valued parameter.
Inside the stored procedure you can use a simple join of that table valued parameter to your table on database_table.col contains(table_parameter.value) (using full text search).
Of course, things will go a lot faster if you create a full text index as suggested in the comments by Glorfindel
I have a checkboxlist in my C#/asp.net project and I'm populating it with a dataTable that gets data from a query to my database. The query returns a large amount of data and I want to restrict the number of elements that it shows initially before I filter the data. (To, say, the top 1000). How would I go about doing this?
There are two places where you can limit the number of data.
In the database (assuming you use SQL Server) you can modify the query to return the top 1000 rows.
SELECT TOP 1000 * FROM SomeTable
Or you can filter the data after it arrives using Linq.
var newData = dataTable.AsEnumerable().Take(1000);
I would prefer the first method, so you don't truck around useless data. But the second definitely works as well if you need that data elsewhere.
You can use the Take<> generic IEnumerable method:
var data = someQuery.Exec();
var limitedData = data.Take(1000).ToArray();
I have an object structure that is mimicking the properties of an excel table. So i have a table object containing properties such as title, header row object and body row objects. Within the header row and each body row object, i have a cell object containing info on each cell per row. I am looking for a more efficient way to store this table structure since in one of my uses for this object, i am printing its structure to screen. Currently, i am doing an O(n^2) complexity for printing each row for each cell:
foreach(var row in Table.Rows){
foreach(var cell in row.Cells){
Console.WriteLine(cell.ToString())
}
}
Is there a more efficient way of storing this structure to avoid the n^2? I ask this because this printing functionality exists in another n^2 loop. Basically i have a list of tables titles and a list of tables. I need to find those tables whose titles are in the title list. Then for each of those tables, i need to print their rows and the cells in each row. Can any part of this operation be optimized by using a different data structure for storage perhaps? Im not sure how exactly they work but i have heard of hashing and dictionary?
Thanks
Since you are looking for tables with specific titles, you could use a dictionary to store the tables by title
Dictionary<string,Table> tablesByTitle = new Dictionary<string,Table>();
tablesByTitle.Add(table.Title, table);
...
table = tablesByTitle["SomeTableTitle"];
This would make finding a table an O(1) operation. Finding n tables would be an O(n) operation.
Printing the tables then of cause depends on the number of rows and columns. There is nothing, which can change that.
UPDATE:
string tablesFromGuiElement = "Employees;Companies;Addresses";
string[] selectedTables = tablesFromGuiElement.Split(';');
foreach (string title in selectedTables) {
Table tbl = tablesByTitle[title];
PrintTable(tbl);
}
There isn't anything more efficient than an N^2 operation for outputting an NxN matrix of values. Worst-case, you will always be doing this.
Now, if instead of storing the values in a multidimensional collection that defines the graphical relationship of rows and columns, you put them in a one-dimensional collection and included the row-column information with each cell, then you would only need to iterate through the cells that had values. Worst-case is still N^2 for a table of N rows and N columns that is fully populated (the one-dimensional array, though linear to enumerate, will have N^2 items), but the best case would be that only one cell in that table is populated (or none are) which would be constant-time.
This answer applies to the, printing the table part, but the question was extended.
for the getting the table part, see the other answer.
No, there is not.
Unless perhaps your values follow some predictable distribution, then you could use a function of x and y and store no data at all, or maybe a seed and a function.
You could cache the print output in a string or StringBuider if you require it multiple times.
If there is enough data I guess you might apply some compression algorithm but I wouldn't say that was simpler or more efficient.
I have a table with 4 columns and N rows. At the beginning N will be around 1000 and will have tendency to grow up to 3000.
1st: string unique
2nd: int with N/5 unique values
3rd: int with 5 unique values
4th: data value
The objective is to get to the value of the 4th column with different queries, ex: "get the value, where the 1st column is 17", or: "get all values where the 2nd column is 7", or: "does any row has this data". ~40% of queries will be done against the 4th column, ~30% against 3rd, ~20% 2nd and ~10% for 1st.
Since there would be around 100 queries per second, and around 2 changes (add/update/remove) per second against this table, I was wondering, what would be the fastest way (in C#) to manage this data? Memory is not an issue
I'm currently using a SortedDictionary, where the key is the 4th data value; and the dictionary's value is a class containing the first three values. Verifying the "4th column" is now easy by just using ContainsKey; and when querying by other values I use:
foreach(var object in Objects) if(Objects[Data].2nd==object.Value.2nd) {...}
Any suggestions appreciated.
This is the equivalent problem of how much indexing to use on a table in a database. If you want fast lookups on all 4 columns, you could created SortedDictionarys of each column type, and use the corresponding dictionary for lookups, but this will increase your add/update/remove time by having to update all 4 dictionaries (not to mention locking as well). It all depends on how fast you want updates and lookups on different columns to be.
However, given that multiple columns can have the same data, and SortedDictionary depends on unique key values, you may want to either write your own datastructure or use one of the MultiSet classes available in C# collection libraries (C5 springs to mind, but there are several others)