I have a row that contains a field defined as varchar(MAX). I'm confused about the limit of the field: in some places, I read that varchar(MAX) has a size limit of 8K and in other places it seems that the limit is 2GB.
I have a string that I want to save to a database; it's about 220K. I'm using linq-to-sql and when the write query submits to the database, the row gets written without any exceptions generated. However, when I open the database table in SSMS, the cell that should contain the long string is empty. Why is that and how do I take advantage of the 2GB limit that I read about?
This is the property in the linq-to-sql model:
All MAX datatypes--VARCHAR(MAX), NVARCHAR(MAX), and VARBINARY(MAX)--have a limit of 2 GB. There is nothing special you need to do. Without specifying MAX, the limit for VARCHAR and VARBINARY are 8000 and the limit for NVARCHAR is 4000 (due to NVARCHAR being double-byte). If you are not seeing any data come in at all, then something else is going on.
Are you sure that the column is even in the INSERT statement? If you submit test data of only 20 characters, does that get written? If you want to see what SQL is actually submitted by Linq, try running SQL Profiler and look at the SQL Statement: Statement Ended event, I believe.
Also, when you say that the "long string is empty", do you mean an actual empty string or do you mean NULL? If it is not NULL, you can also wrap the field in a LEN() function to see if there are blanks for returns at the beginning that push any non-whitespace characters out of view. Meaning, SELECT LEN(stringField), * FROM Table. Another thing to try is to use "Results to Text" instead of "Results to Grid" (this is a Query option).
EDIT:
Seeing that the field is marked as NOT NULL, are you sure that you are setting the ClientFileJS property of your object correctly? Is it possible that the empty string is due to that property being initialized as string ClientFileJS = ""; and is never updated?
Related
So I have a table with a column of type VARCHAR (100) and I'm wondering if there's a way to configure SQL Server 2012 (T-SQL) so that if a transaction tries to submit a string of 101+ characters then it takes the first 100.
Is this possible, or should I be doing the truncation in the C# side of things ???
Normally, SQL Server will present an error on any attempt to insert more data into a field than it can hold
String or binary data would be truncated. The statement has been terminated.
SQL Server will not permit a silent truncation of data just because the column is too small to accept the data. But there are other ways that SQL Server can truncate data that is about to be inserted into a table that will not generate any form of error or warning.
By default, ANSI_WARNINGS are turned on, and certain activities such as creating indexes on computed columns or indexed views require that they be turned on. But if they are turned off, SQL Server will truncate the data as needed to make it fit into the column. The ANSI_WARNINGS setting for a session can be controlled by
SET ANSI_WARNINGS { ON|OFF }
Unlike with an insert into a table, SQL Server will quietly cut off data that is being assigned to a variable, regardless of the status of ANSI_WARNINGS. For instance:
declare #smallString varchar(5)
declare #testint int
set #smallString = 'This is a long string'
set #testint = 123.456
print #smallString
print #testint
Results is:
This
123
This can occasionally show itself in subtle ways since passing a value into a stored procedure or function assigns it to the parameter variables and will quietly do a conversion. One method that can help guard against this situation is to give any parameter that will be directly inserted into a table a larger datatype than the target column so that SQL Server will raise the error, or perhaps to then check the length of the parameter and have custom code to handle it when it is too long.
For instance, if a stored procedure will use a parameter to insert data into a table with a column that is varchar(10), make the parameter varchar(15). Then if the data that is passed in is too long for the column, it will rollback and raise a truncation error instead of silently truncating and inserting. Of course, that runs the risk of being misleading to anyone who looks at the stored procedures header information without understanding what was done.
Source: Silent Truncation of SQL Server Data Inserts
Do this on code level. When you are inserting the current field check field length and Substring it.
string a = "string with more than 100 symbols";
if(a.Length > 100)
a = a.Substring(0, 100);
After that you are adding a as sql parameter to the insert query.
The other way is to do it in the query, but again I don't advice you to do that.
INSERT INTO Table1('YourColumn') VALUES(LEFT(RTRIM(stringMoreThan100symbols), 100))
LEFT is cutting the string and RTRIM is performing Trim operation of the string.
My suggestion would be to make the application side responsible for validating the input before calling any DB operation.
SQL Server silently truncates any varchars you specify as stored procedure parameters to the length of the varchar. So you should try considering stored procedures for you requirements. So it will get handled automatically.
If you have entity classes (not necessarily from EF) you can use StringLength(your field length) attribute to do this.
If i do a query like this
SELECT * from Foo where Bar = '42'
and Bar is a int column. Will that string value be optimized to 42 in the db engine? Will it have some kind of impact if i leave it as it is instead of changing it to:
Select * from Foo where Bar = 42
This is done on a SQL Compact database if that makes a difference.
I know its not the correct way to do it but it's a big pain going though all code looking at every query and DB schema to see if the column is a int type or not.
SQL Server automatically convert it to INT that because INT has higher precedence than VARCHAR.
You should also be aware of the impact that implicit conversions can
have on a query’s performance. To demonstrate what I mean, I’ve created and populated the following table in the AdventureWorks2008 database:
USE AdventureWorks2008;
IF OBJECT_ID ('ProductInfo', 'U') IS NOT NULL
DROP TABLE ProductInfo;
CREATE TABLE ProductInfo
(
ProductID NVARCHAR(10) NOT NULL PRIMARY KEY,
ProductName NVARCHAR(50) NOT NULL
);
INSERT INTO ProductInfo
SELECT ProductID, Name
FROM Production.Product;
As you can see, the table includes a primary key configured with the
NVARCHAR data type. Because the ProductID column is the primary key,
it will automatically be configured with a clustered index. Next, I
set the statistics IO to on so I can view information about disk
activity:
SET STATISTICS IO ON;
Then I run the following SELECT statement to retrieve product
information for product 350:
SELECT ProductID, ProductName
FROM ProductInfo
WHERE ProductID = 350;
Because statistics IO is turned on, my results include the following
information:
Table 'ProductInfo'. Scan count 1, logical reads 6, physical reads 0,
read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob
read-ahead reads 0.
Two important items to notice is that the query performed a scan and
that it took six logical reads to retrieve the data. Because my WHERE
clause specified a value in the primary key column as part of the
search condition, I would have expected an index seek to be performed,
rather than I scan. As the figure below confirms, the database engine performed a scan, rather than a seek. Figure below shows the details of that scan (accessed by hovering the mouse over the scan icon).
Notice that in the Predicate section, the CONVERT_IMPLICIT function is
being used to convert the values in the ProductID column in order to
compare them to the value of 350 (represented by #1) I passed into the
WHERE clause. The reason that the data is being implicitly converted
is because I passed the 350 in as an integer value, not a string
value, so SQL Server is converting all the ProductID values to
integers in order to perform the comparisons.
Because there are relatively few rows in the ProductInfo table,
performance is not much of a consideration in this instance. But if
your table contains millions of rows, you’re talking about a serious
hit on performance. The way to get around this, of course, is to pass
in the 350 argument as a string, as I’ve done in the following
example:
SELECT ProductID, ProductName
FROM ProductInfo
WHERE ProductID = '350';
Once again, the statement returns the product information and the statistics IO data, as shown in the following results:
Now the index is being properly used to locate the record. And if you
refer to Figure below, you’ll see that the values in the ProductID
column are no longer being implicitly converted before being compared
to the 350 specified in the search condition.
As this example demonstrates, you need to be aware of how performance
can be affected by implicit conversions, just like you need to be
aware of any types of implicit conversions being conducted by the
database engine. For that reason, you’ll often want to explicitly
convert your data so you can control the impact of that conversion.
You can read more about Data Conversion in SQL Server.
If you look into the MSDN chart which tells about the implicit conversion you will find that string is implicitly converted into int.
both should work in your case but the norme is to use quote anyway.
cuz if this work.
Select * from Foo where Bar = 42
this not
Select * from Foo where Bar = %42%
and this will
SELECT * from Foo where Bar = '%42%'
ps: you should anyway look at entity framework and linq query it make it simple...
If i am not mistaken, the SQL Server will read it as INT if the string will only contains number (numeric) and you're comparing it to the INTEGER column datatype, but if the string is is alphanumeric , then that is the time you will encounter an error or have an unexpected result.
My suggestion is , in WHERE clause, if you are comparing integer, do not put single quote. that is the best practice to avoid error and unexpected result.
You should use always parameters when executing sql by code, to avoid security lacks (EJ: Sql injection).
I wanted to see what others have experienced when working with types like List<> or Dictionary<> and having in turn storing and retrieving that data?
Here's an example scenario: users will be creating their own "templates", where these templates is essentially a collection of Dictionary, e.g. for user1, values are (1, Account), (2, Bank), (3, Code), (4, Savings), and for user2, values (unrelated) could be (1, Name), (2, Grade), (3, Class), and so on. These templates/lists could be of varying length but they will always have an index and a value. Also, each list/ template will have one and only one User linked to it.
What types did you choose on the database side?
And pain-points and/or advice I should be aware of?
As far as the types within the collection go, there is a fairly 1-to-1 mapping between .Net types and SQL types: SQL Server Data Type Mappings. You mostly need to worry about string fields:
Will they always be ASCII values (0 - 255)? Then use VARCHAR. If they might contain non-ASCII / UCS-2 characters, then use NVARCHAR.
What is their likely max length?
Of course, sometimes you might want to use a slightly different numeric type in the database. The main reason would be if an int was chosen on the app side because it "easier" (or so I have been told) to deal with than Int16 and byte, but the values will never be above 32,767 or 255, then you should most likely use SMALLINT or TINYINT respectively. The difference between int and byte in terms of memory in the app layer might be minimal, but it does have an impact in terms of physical storage, especially as row counts increase. And if that is not clear, "impact" means slowing down queries and sometimes costing more money when you need to buy more SAN space. But, the reason I said to "most likely use SMALLINT or TINYINT" is because if you have Enterprise Edition and have Row Compression or Page Compression enabled, then the values will be stored in the smallest datatype that they will fit in.
As far as retrieving the data from the database, that is just a simple SELECT.
As far as storing that data (at least in terms of doing it efficiently), well, that is more interesting :). A nice way to transport a list of fields to SQL Server is to use Table-Valued Parameters (TVPs). These were introduced in SQL Server 2008. I have posted a code sample (C# and T-SQL) in this answer on a very similar question here: Pass Dictionary<string,int> to Stored Procedure T-SQL. There is another TVP example on that question (the accepted answer), but instead of using IEnumerable<SqlDataRecord>, it uses a DataTable which is an unnecessary copy of the collection.
EDIT:
With regards to the recent update of the question that specifies the actual data being persisted, that should be stored in a table similar to:
UserID INT NOT NULL,
TemplateIndex INT NOT NULL,
TemplateValue VARCHAR(100) NOT NULL
The PRIMARY KEY should be (UserID, TemplateIndex) as that is a unique combination. There is no need (at least not with the given information) for an IDENTITY field.
The TemplateIndex and TemplateValue fields would get passed in the TVP as shown in my answer to the question that I linked above. The UserID would be sent by itself as a second SqlParameter. In the stored procedure, you would do something similar to:
INSERT INTO SchemaName.TableName (UserID, TemplateIndex, TemplateName)
SELECT #UserID,
tmp.TemplateIndex,
tmp.TemplateName
FROM #ImportTable tmp;
And just to have it stated explicitly, unless there is a very specific reason for doing so (which would need to include never, ever needing to use this data in any queries, such that this data is really just a document and no more usable in queries than a PDF or image), then you shouldn't serialize it to any format. Though if you were inclined to do so, XML is a better choice than JSON, at least for SQL Server, as there is built-in support for interacting with XML data in SQL Server but not so much for JSON.
List or any collection's representation in databases are supposed to be tables. Always think of it as a collection and relate it to what a database offers.
Though you can always serialize a collection, i do not suggest it since updating or inserting records, you'd always update the whole record or data whereas having a table, you'd only have to query for the KEY wherein Dictionary, you already have it.
I am developing a project using VS 2008 and SQL-Server 2005.
I have used varchar(150)-field for saving a normal string, which have to save
softcopy
hardcopy
both (softcopy & hardcopy)
It works when saving as softcopy or hardcopy but for both (softcopy & hardcopy) it throws the following error:
string or binary data would be truncate. The statement has been terminated.
When I restart the application after this error had occurred everything works perferctly.
I tried to use Nvarchar(Max) for the same field but the error was all the same.
Please give me suggestion to avoid this error.
The error says you are updating the column with larger size than what it can accomodate. Check for blank space in the column value
This happens if you are trying to insert too much data into a field that has a limited size, in your case 150.
The error string or binary data would be truncate. The statement has been terminated comes when the size is exceeded. Try to update your size from varchar(150) to varchar(500).
Or check your data length in your query or stored procedure where you are assigning value to the filed saved in db (Ref: as you change it to nvarchar(max) but its not working).
This exception occures when your datatype,size does not match with database field attributes.
It may be because your column length is less than your inserting data.
so if you increase your column length to that which can occupy the data you input then issue will get resolved.
If you use Nvarchar (Unicode character data) data type then you need to insert the data with N' as a Prefix like this:
Set #var = N'Hello World'
I am using Visual Studios 2010 for this C# program I am making where I am generating a cache that I want to store in a local database made by the SqlCeEngine class. I want to use the SqlCeEngine class to make this database in a 100% programmatic manner, but I am having problems finding CREATE TABLE examples using MS SQL data types while I am still fairly new to databases and what experience I have had before was with sqlite3.
I want to make a table with a UniqueID column as the PRIMARY KEY, but ideally using a smallint as the datatype for it as this generated cache shouldn't be exceeding 2^15 entries, or even 1500 entries for that matter. But let's say I also want to make a second column that holds some text, but I have a hard time determining how long that will be ahead of time. On average, I am expecting 8 characters or less, but I might have the occasional entry that is over 260 characters long. I could just make the string size for the entries set to the maximum length the string is allowed to be (4000 characters I believe), but for this project, space consumption by the cache needs to be minimized and I think that would waste a lot of space. I noticed there were nchar and nvarchar data types, but I didn't really understand the difference as they both needed a length to be declared. It just seemed like one was hinting towards that it might be more friendly with strings of different lengths under the hood of the SqlCeEngine than the other.
TL;DR
Is there a way to declare a SQL database in C# using the SqlCeEngine class that uses a smallint data type for the primary key column and variable length text in the second column without wasting space?
You want to use a varchar, unless you're storing unicode characters, in that case use nvarchar.
The difference between char(50) and varchar(50) is that char(50) will grab and hold onto 50 characters, whether they're used or not. varchar(50) will allow up to 50 characters, but if 0 are used, it doesn't reserve any space.
So if you had a field char(20) and the value was 'Name' the size of that field would be 20 characters, the 4 you specified and an additional 16 to fill it out. If you had a varchar(20) and the value was 'Name' the size would be 4.