Time out while i fetch data from SQL in Asp.net MVC - c#

I hava a database which only have 2 table Object , User ,
Obviously user table has information about all user and the object table have millions of records
ObjectTable
Id (int)
Text1 (nvarchar(max))
Text2 (nvarchar(max))
I am trying to make a translator , first of all i put all data in database like following
1 : Good , Well
2 : Bad , NotWell
3 : Man , Husband Of Women
if suppose i havae 2 text boxes in my site user enter following text
Good Bad Man
then i will split that string on space and then i will have an array of string now i will took first element of array and go to server to find that wheather there is any match of Good in my database if i found match then i will replace that value with that Text2 like we have Well for Good which took too much time to translate and sometime it gives Request Timed Out . So , what is the best way to deal with it

You do not provide a lot of information to help you about your timeout problem.
Only things I can tell you are :
first of all, check that there are indexes on your Text1 and Text2 columns. If not, add it
look at your sql query. It should be SELECT * FROM OBJECT WHERE TEXT1='Good' (maybe add a top 100 to avoid returning to many things with LIKE ).
For the index to run smoothly, there should be no function called on column TEXT1 (like uppercase or trim). If you use a like, it should be TEXT1 LIKE 'good%' (% at the end)
if you use a like, beware of not returning your whole table on a empty entry, % or _ entry ( LIKE '%', LIKE '%%', LIKE '_%' can be bad things)
why use nvarchar(max) if you are just storing words ?
don't forget to sanitize your entries to avoid sql injection

Related

C#: How to check user console input against a column in a .csv?

Yo,
I am fairly new to C# and I'm making a small text based console RPG game just for practice and I was wondering if there was anyway I could make something similar to this:
Ask user for name
Ask user for password
Check if the name is in column 1 of a .csv file
If it is, check if the password is in column 2 and in the same row as the name
If everything checks out, load the values from columns 3 and 4 in the same row as
the name and password into corresponding values like health and level
Here's an example .csv if I didn't really explain it well:
"charactername", "password123", "health", "level"
"Oswald", "498562a", "100", "4"
"Hammerfist", "98457813", "77", "6"
So just for the sake of clarity, this is what I'm thinking:
Enter your username: Oswald
Enter your password: 498562a
Program reads through column 1 of the file looking for "Oswald" and finds a match
Program then checks to see if the password value is in the same row but in
column 2 of the file and checks to see if it matches the user input, it matches it
Program then loads the health and level values into character variables
Entrance Successful
Is there a way to do this? Am I overlooking a simple solution? Or should I try and go about it a different way? I'm just don't want to hard code the values into a Dictionary or List or Array or whatever. I'm not looking for security at this stage either, it's just for practice and fun.
Thanks in advance! Let me know if you need a better understanding of what I'm trying to do and I'll try to explain the best I can.
There are two ways to go about reading csv
Just like standard text file, line by line. Then parse your line into array of columns and get your values
Using Microsoft.ACE.OLEDB.version
OleDbconnection connection = new OleDbConnection();
connection.ConnectionString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=C:\Documents\file.csv;Extended Properties=\"Text;HDR=Yes;FORMAT=Delimited\"";
This way you can load DataTable on the start of your app and query this table just like Sql database.
notice : HDR=Yes means that first row is a header row

How to arrange this data in columns?

I have a mysql database. In which there are 50 columns of detail.
Detail 1, Detail2, Detail3...... Detail50.
I have the website locally so i am scrapping from myself. The site is not in order no tags and names data is just in form of text line by line, so this was the only option to take what i get line by line and save to DB. So every line gets a column from 1-50....
Some pages have 10 columns other have 50 and the data is in no order now i have the DB,how can i sort them any suggestion ,idea is welcome.
This image will make it more clear:
So You can see Sometimes its Inner Diameter in Detail4 and sometime in 1, these are just examples i would have hard coded but there are too many possibilities, but the repeating words all have the same staring name just values different .Any chance to atleast make 50 % of the data in order the ones with same 4-5 starting words like
part,inner,diameter,oil filter etc.
Any suggestion or ideas can it be done in mysql or C# code.....
Thank you
your approach is totally wrong, but if you want to go this way, just make a table with two columns 'id' and 'details' ... make an insert for each column for the specific product ID.
After that you can use a SELECT like that:
select SUBSTRING(details, 14) from products where details like 'Inner Diameter%' and id = 'my_product_id';

Search only numbers with mysql

i have project in ASP.NET MVC 3 and a mysql database that contains a table of string values for phone numbers (this phone numbers can be stored as 123 456-789 or 12345 6789 or 123456789 or any ways the user enter his number) and other table with a keywords data for that users.
The thing is that i have a search that will find in the keywords table (fulltext table) for the users, but i'm writing a method that search in the phone table if the search query matches against certain regular expression.
I have 2 questions:
- How could be that regular expression from the C# code side that tells what method to execute (SearchByKeyword or SearchByNumber)?
- Using the same regular expression i think, i must do the mysql query to search in the phones table... how can i do it?
I hope i have explained well and sorry if my english is a little bit bad.
It's best when you capture the data to standardise the format it is saved in, IE:
111-111-1-111-11
1111-1111-1111
111111111111
1-1-1-1-1-1-1-1-1-1-1-1
All will save as
111111111111
You can then do a simple LIKE query:
SELECT * FROM tblNumbers WHERE number LIKE '%111%'
I would say that its bad idea to feed a phone number from the input -> directly to query variable.
better way would be:
input -> parser, error check procedure -> MySQL query request.
and before adding your phone number to the query - you can remove all extra symbols, like (, ), -, "space", etc.

How can I sort an SQLite query ignoring articles ("the", "a", etc.)?

I'm using C# to display a list of movie titles that I am calling from an SQLite database. Currently, I'm using a custom ListBox class that has a function to sort the text stripping the word 'The' from the beginning of every item. However, it doesn't exactly seem to be the simplest way to do it, since it calls from the SQLite database and then sorts. I'd prefer to cut it down to just one step, hopefully sorting straight from the database in my "SELECT" query.
I've done some searching on this, and have found some suggestions, including creating an extra sort-by column in the database. While this is certainly a possibility, I'm wondering if there's any simpler options that don't require inserting almost identical duplicate information (especially if the database becomes larger). I'm pretty new to SQLite, but I've read something about creating a collate function that can be used to create custom ordering. However, I'm not sure if this is appropriate use for it and can't seem to find any help with implementing it in C#.
Was hoping someone might be able to share some guidance. If an extra sorting column is the best way to go, then that is what I shall do.
To avoid inserting duplicate data, what about having two columns: TITLE_PREFIX (usually empty, but sometimes contains "The ", or "A "; no index on this column) and TITLE (contains the title without "The " or "A "; this is the column you create the index on). To display the data, you have to combine TITLE_PREFIX and TITLE. But you just search on TITLE.
Here is the solution:
ORDER BY (CASE
WHEN sortTitle LIKE 'the %' THEN substr(sortTitle,5)
WHEN sortTitle LIKE 'a %' THEN substr(sortTitle,3)
WHEN sortTitle LIKE 'an %' THEN substr(sortTitle,4)
ELSE sortTitle END)
You could store each title in 2 parts: title and prefix.
With SQLite you can combine 2 string values via the || operator also known as the concatenate operator.
Here's an example:
SELECT prefix || ' ' || title FROM movies ORDER BY title
You can also use ltrim in case prefix is empty, so you don't have a space at the front:
SELECT ltrim(prefix || ' ' || title) FROM movies ORDER BY title
Another alternative is to store the prefix at the end of the title. For example at a lot of movie stores you will see something like:
Three Musketeers, The
Within C# Code
If you wanted to do this within C#, use LINQ to do the ordering for you. I've posted a full sample on PasteBin. This will allow you to:
avoid duplicating data in your database
take advantage of DB indexes as you normally would, no matter which RDBMS
put in noise words in a config file, thereby reducing downtime/rebuild/redeploy when modifying the list
ensure a solution is more readable in your client code
DropDownList1.DataSource = myBooks.OrderBy(n => ReplaceNoise(n.Title))
public string ReplaceNoise(string input)
{
string[] noise = new string[] { "the", "an", "a" };
//surely this could be LINQ'd
foreach (string n in noise)
{
if (input.ToLower().StartsWith(n))
{
return input.Substring(n.Length).Trim();
}
}
return input;
}
Within your SQLite statement
How about simply replacing the noise words with blanks in the order by? It's an ugly first step, but strongly consider a new column to store this value for sorting purposes.
ORDER BY REPLACE(REPLACE([title],'the',''), 'a', '')
Admittedly, this gets ugly when you end up with this:
REPLACE(REPLACE(REPLACE(REPLACE([title],'The ',''),'a',''),'of',''),'by','')
You could try building a table that supports full-text searching (using the FTS module) on the title. Then you'll be able to do fast searches on any words in the title without requiring lots of extra work on your part. For example, a user query of good bad ugly might produce “The Good, the Bad and the Ugly” as one of its first results. The extra cost of all this is about a quarter of the length of the text itself in general, but might be more for your dataset, as titles aren't full english text. You also need to spend the time building those extra indices – you don't want to build them on your main dataset on a live system (obviously) – but that shouldn't be too big a problem.
Create a virtual column (result of a function that can be implemented in C#) and sort on this virtual column. The function could move "The" to the end as in "Three Musketeers, The" or discard "The", whatever you want it to do.

c# - SQL - speed up code to DB

I have a page with 26 sections - one for each letter of the alphabet. I'm retrieving a list of manufacturers from the database, and for each one, creating a link - using a different field in the Database. So currently, I leave the connection open, then do a new SELECT by each letter, WHERE the Name LIKE that letter. It's very slow, though.
What's a better way to do this?
TIA
Since you are going to fetch them all anyway, you might find it faster to fetch them in one go and split them into letter-groups in the code.
Looking at it from the other end, why do you need to fetch all the lists just to build a set of links? Shouldn't you fetch a single letter when its link is clicked?
It sounds like you are doing up to 26 queries, which will never be fast. Often a single db query can take at least 40 ms, due to network latency, establishing connection, etc. So, doing this 26 times means that it will take around 40 x 26 ms, or more than one second. Of course, it can take much longer depending on your schema, data set, hardware, etc., but this is a rule of thumb that gives you a rough idea of the impact of queries on overall page render time.
One way I deal with this kind of situation is to use a DataTable. Fetch all the records into the DataTable, and then you can iterate through the alphabet, and use the Select method to filter.
DataTable myData = GetMyData();
foreach(string letter in lettersOfTheAlphabet)
{
myData.Filter(String.Format("Name like '{0}%'", letter));
//create your link here
}
Depending on your model layer you may wish to filter in a different way, but this is the basic idea that should improve the performance a lot.
Assuming you are querying to determine which letters are used, so that you know which links to render, you could actually just query for the letters themselves, like this:
select distinct substring(ManufacturerName, 1, 1) as FirstCharacter
from MyTable
order by 1
get one result set from one query and split that up. There is quite a lot of overhead going out the the database 26 times to do basically the same work!
You could probably do it smarter with a stored procedure. Let the SP return all the information you need in one call, and suddenly you only have one database interaction instead of 26...
Bring back all the items in one set (dataset, etc..), either through stored procedure or query, including the field left(col1,1), and sorting by that field..
select left(col1,1) as LetterGroup, col1, url_column from table1 order by left(col1,1)
Then look through the whole resultset, changing sections when the letter changes.
First letter in the alphabet sucks (sorry) as discriminator. You do not neet to split them actually (you could just ask for "where name like 'a%'), but whatever you run for that gives you on average a 1/26 or so split of the names. Not extremely efficient.
What do you mean with "creating a link - using a different field in the Database" - this sounds like a bad design to me.
there are a couple ways u can do this. 1) create a view in your db that has all the manufactures and their website link and then continue to hit the view for each letter. 2) select all the manufactures once and store it in a .net dataset and then use that dataset to populate your links.
This seems dirty to me, but you could create a first letter CHAR column and trigger to populate it. Have the first letter from the manufacturer name stored in that column and index it. Then select * from table where FirstLetter = 'A'.
Or create a lookup table with rows A - Z and set up foreign key in the manufacturer table. Again you would probably need a trigger to update this information. Then you could inner join the lookup table to the manufacturer table.
Then instead of putting 26 datasets in the page, have a list of links (A-Z) which select and show each dataset one at a time.
If I read you right, you're making a query for every manufacturer to get the "different field" you need to construct the link. If so, that's your problem, not the 26 alphabetic queries (though that's no help).
In a case like that, the faster way is this one query:
SELECT manufacturer_name, manufacturer_id, different_field
FROM manufacturers m
INNER JOIN different_field_table d
ON m.manufacturer_id = d.manufacturer_id
ORDER BY manufacturer_name
In your server code, loop through the records as usual. If you want, emit a heading when the first letter of the manufacturer_name changes.
For additional speed:
Put that in a stored procedure.
Index different_field_table on manufacturer_id.

Categories