We Keep Coding

C# "anyString".Contains('\0', StringComparison.InvariantCulture) returns true in .NET5 but false in older versions

I encountered an incompatible problem while I was trying to upgrade my projects from .NET core 3.1 to the latest .NET 5.
My original code has a validation logic to check invalid file name characters by checking each character returned from Path.GetInvalidFileNameChars() API.
var invalidFilenameChars = Path.GetInvalidFileNameChars();
bool validFileName = !invalidFilenameChars.Any(ch => fileName.Contains(ch, StringComparison.InvariantCulture));
Suppose you give a regular value to fileName such as "test.txt" that should be valid. Surprisingly, however, the above code gives the file name is invalid if you run it with 'net5' target framework.
After spend some time on debugging, what I found is that the returned invalid character set contains '\0', null ASCII character and "text.txt".Contains("\0, StringComparison.InvariantCulture) gives true.
class Program
{
static void Main(string[] args)
{
var containsNullChar = "test".Contains("\0", StringComparison.InvariantCulture);
Console.WriteLine($"Contains null char {containsNullChar}");
}
}
If you run in .NET core 3.1, it never says regular string contains null character. Also, if I omit the second parameter (StringComparison.InvariantCulture) or if I use StringComparison.Ordinal, the strange result is never returned.
Why this behavior is changed in .NET5?
EDIT:
As commented by Karl-Johan Sjögren before, there is indeed a behavior change in .NET5 regarding string comparison:
Behavior changes when comparing strings on .NET 5+
Also see the related ticket:
string.IndexOf get different result in .Net 5
Though this issue should be related to above, the current result related to '\0' still looks strange to me and might still be considered to be a bug as answered by #xanatos.
EDIT2:
Now I realized that the actual cause of this problem was my confusion between InvariantCulture and Ordinal string comparison. They are actually quite different things. See the ticket below:
Difference between InvariantCulture and Ordinal string comparison
Also note that this should be unique problem of .NET as other major programming languages such as Java, C++ and Python treat ordinal comparison by default.

not a bug, a feature
The issue that I've opened has been closed, but they gave a very good explanation. Now... In .NET 5.0 they began using on Windows (on Linux it was already present) a new library for comparing strings, the ICU library. It is the official library of the Unicode Consortium, so it is "the verb". That library is used for CurrentCulture, InvariantCulture (plus the respective IgnoreCase) and and any other culture. The only exception is the Ordinal/OrdinalIgnoreCase. The library is targetted for text and it has some "particular" ideas about non-text. In this particular case, there are some characters that are simply ignored. In the block 0000-00FF I would say the ignored characters are all control codes (please ignore the fact that they are shown as €‚ƒ„†‡ˆ‰Š‹ŒŽ‘’“”•–—™š›œžŸ, at a certain point these characters have been remapped somewhere else in the Unicode, but the glyps shown don't reflect it, but if you try to see their code, like doing char ch = '€'; int val = (int)ch; you'll see it), and '\0' is a control code.
Now... My personal thinking is that to compare string from today you'll need a master's degree in Unicode Technologies 😥, and I do hope that they'll do some shenanigans in .NET 6.0 to make the default comparison Ordinal (it is one of the proposals for .NET 6.0, the Option B). Note that if you want to make programs that can run in Turkey you already needed a master's degree in Unicode Technologies (see the Turkish i problem).
In general I would say that to look for words that aren't keywords/fixed words (for example column names), you should use Culture-aware comparisons, while to look for keywords/fixed words (for example column names) and symbols/control codes you should use Ordinal comparisons. The problem is when you want to look for both at the same time. Normally in this case you are looking for exact words, so you can use Ordinal. Otherwise it becames hellish. And I don't even want to think how Regex works internally in a Culture-aware environment. That I don't want to think about. Becasue in that direction there can only be folly and nightmares 😁.
As a sidenote, even before the "default" Culture-aware comparisons had some secret shaeaningans... for example:
int ix = "ʹ$ʹ".IndexOf("$"); // -1 on .NET Framework or .NET Core <= 3.1
what I had written before
I'll say that it is a bug. There is a similar bug with IndexOf. I've opened an Issue on github to track it.
As you have written, the Ordinal and OrdinalIgnoreCase work as expected (probably because they don't need to use the new ICU library for handling Unicode).
Some sample code:
Console.WriteLine($"Ordinal Contains null char {"test".Contains("\0", StringComparison.Ordinal)}");
Console.WriteLine($"OrdinalIgnoreCase Contains null char {"test".Contains("\0", StringComparison.OrdinalIgnoreCase)}");
Console.WriteLine($"CurrentCulture Contains null char {"test".Contains("\0", StringComparison.CurrentCulture)}");
Console.WriteLine($"CurrentCultureIgnoreCase Contains null char {"test".Contains("\0", StringComparison.CurrentCultureIgnoreCase)}");
Console.WriteLine($"InvariantCulture Contains null char {"test".Contains("\0", StringComparison.InvariantCulture)}");
Console.WriteLine($"InvariantCultureIgnoreCase Contains null char {"test".Contains("\0", StringComparison.InvariantCultureIgnoreCase)}");
Console.WriteLine($"Ordinal IndexOf null char {"test".IndexOf("\0t", StringComparison.Ordinal)}");
Console.WriteLine($"OrdinalIgnoreCase IndexOf null char {"test".IndexOf("\0", StringComparison.OrdinalIgnoreCase)}");
Console.WriteLine($"CurrentCulture IndexOf null char {"test".IndexOf("\0", StringComparison.CurrentCulture)}");
Console.WriteLine($"CurrentCultureIgnoreCase IndexOf null char {"test".IndexOf("\0", StringComparison.CurrentCultureIgnoreCase)}");
Console.WriteLine($"InvariantCulture IndexOf null char {"test".IndexOf("\0", StringComparison.InvariantCulture)}");
Console.WriteLine($"InvariantCultureIgnoreCase IndexOf null char {"test".IndexOf("\0", StringComparison.InvariantCultureIgnoreCase)}");
and
Console.WriteLine($"Ordinal Contains null char {"test".Contains("\0test", StringComparison.Ordinal)}");
Console.WriteLine($"OrdinalIgnoreCase Contains null char {"test".Contains("\0test", StringComparison.OrdinalIgnoreCase)}");
Console.WriteLine($"CurrentCulture Contains null char {"test".Contains("\0test", StringComparison.CurrentCulture)}");
Console.WriteLine($"CurrentCultureIgnoreCase Contains null char {"test".Contains("\0test", StringComparison.CurrentCultureIgnoreCase)}");
Console.WriteLine($"InvariantCulture Contains null char {"test".Contains("\0test", StringComparison.InvariantCulture)}");
Console.WriteLine($"InvariantCultureIgnoreCase Contains null char {"test".Contains("\0test", StringComparison.InvariantCultureIgnoreCase)}");
Console.WriteLine($"Ordinal IndexOf null char {"test".IndexOf("\0t", StringComparison.Ordinal)}");
Console.WriteLine($"OrdinalIgnoreCase IndexOf null char {"test".IndexOf("\0test", StringComparison.OrdinalIgnoreCase)}");
Console.WriteLine($"CurrentCulture IndexOf null char {"test".IndexOf("\0test", StringComparison.CurrentCulture)}");
Console.WriteLine($"CurrentCultureIgnoreCase IndexOf null char {"test".IndexOf("\0test", StringComparison.CurrentCultureIgnoreCase)}");
Console.WriteLine($"InvariantCulture IndexOf null char {"test".IndexOf("\0test", StringComparison.InvariantCulture)}");
Console.WriteLine($"InvariantCultureIgnoreCase IndexOf null char {"test".IndexOf("\0test", StringComparison.InvariantCultureIgnoreCase)}");

Cannot interpret token '!' at position 5

I have a column named 'Jul' which contains some values like '123' and '-'. I'm trying to get sum of column by adding values and neglecting '-'. I tried:
object julysum;
julysum = Convert.ToDouble(FormattedPurchaseSummaryTable.Compute("SUM(Jul)", "Jul != '-'"));
but i'm getting error: Cannot interpret token '!' at position 5.
How can i resolve this error?

There is no != operator.
You likely want <>.
Please read here for more information on what operators are available: https://learn.microsoft.com/en-us/dotnet/api/system.data.datacolumn.expression?redirectedfrom=MSDN&view=netframework-4.7.2#operators

Blam's answer already pointed you towards the expression language. Having fixed the first issue, we need to fix the second which is that Jul is a string, not an integer (or float, etc). So we need to convert it as well.
But, per Compute I don't think it'll support a complex expression within the SUM to do the CONVERT.
So I think what you need to do is to add a new DataColumn with an expression something like:
IIF(Jul != '-',CONVERT(Jul,'System.Int32'),0)
And then compute your SUM over this new column instead.

Conditional breakpoint not working

The above code has a conditional breakpoint set at its bottom line in yellow followed by the Breakpoint Settings dialog which should work with:
item.Value == "aday"
However I get the below error, I have searched online for this and can't find any reason why this should fail. Im using VS 2015 Pro.
EDIT- Thank you for pointing out the obvious error on my part, I do normally code in C#.
But now using a single '=' I get this???????
I assume that I it equates to an assignment, and adding parenthesis didn't help either?

Just tested with a sample VB.NET project.
The problem is the ==. This is C# syntax but since you have a VB.NET application you should use a single equal
item.Value = "aday"
(I have always something new to learn from SO)

If item.Value.Equals("aday") Then 'Temp If please remove
Debugger.Break()
end if
Actually works in strict mode, Gasp!!!!
Thanks to all contributions, greatly appreciated :)

I am using the C# in Visual Studio 2017.
After search in an hour, conclusion was:
rewrite the conditional expression from:
item.Value == "aday"
to:
item != null && item.Value == "aday"
MAKE SURE item was not null. so that you can refer to field of value with item.Value

Convert increment value

I tried to convert incremented value to string.
i++.ToString()
The above statement working fine.
++i.ToString()
This one showing error line. Why?

Because in the second expression the ToString() method of i is called first and the ++ operator is meaningless for strings. On the other hand, in your first expression, the integer i is get incremented by one and the method ToString() is called. So you get the string representation of your number. You could conver it, as already ulugbek-umirov pointed out in his comment:
(++i).ToString();
Please have a look here.

It is caused by the different operator precendence of i++ and ++i.
i++ belongs to "Primary" whereas ++i belongs to the "Unary" group. But since the . belongs also to "Primary" is has a higher precendence . So .ToString is executed first. You cannot use ++ operator on a string.
You can wrap it in paranthesis:
(++i).ToString()
Reference: Operators

Because the second means:
++(i.ToString()) // Oops, compiler error
and string type can not be incremented.

(++i).ToString() will do the trick.

The error is caused by the precedence of the operators being applied.
In your instruction you're trying to apply an int-increment to a string.
If you want to post increment you should do:
(++i).ToString();

Answers and comments saying that this is due to anything "executing first" are misleading. This problem is due to the higher precedence of . relative to ++(prefix). Order of evaluation has nothing to do with it really - especially since ++i.ToString() does not compile and therefore there is no evaluation at all (hence no order in which it happens).
See also precedence vs associativity vs order.

You have to options
make it in 2 times:
i++;
i.ToString();
Use parenthesis to set priority (first increase then convert to string)
(++i).ToString();
or
(i++).ToString();

Because when ++ is used as a postfix, it will increase its value immediately then covert it to string using ToString();
Instead if you use ++ as prefix operator then it will convert the existing value to string using ToString() then it will try to increment the value, so in this case you are getting error on increment a string value...
so try using parenthesis for increasing its precedence as (++i).ToString();

What is wrong with ToLowerInvariant()?

I have the following line of code:
var connectionString = configItems.
Find(item => item.Name.ToLowerInvariant() == "connectionstring");
VS 2010 code analysis is telling me the following:
Warning 7 CA1308 : Microsoft.Globalization : In method ... replace the call to 'string.ToLowerInvariant()' with String.ToUpperInvariant().
Does this mean ToUpperInvariant() is more reliable?

Google gives a hint pointing to CA1308: Normalize strings to uppercase
It says:
Strings should be normalized to uppercase. A small group of characters, when they are converted to lowercase, cannot make a round trip. To make a round trip means to convert the characters from one locale to another locale that represents character data differently, and then to accurately retrieve the original characters from the converted characters.
So, yes - ToUpper is more reliable than ToLower.
In the future I suggest googling first - I do that for all those FxCop warnings I get thrown around ;) Helps a lot to read the corresponding documentation ;)

Besides what TomTom says, .net is optimized for string comparison in upper case. So using upper invariant is theoretically faster than lowerinvariant.
This is indeed stated in CLR via C# as pointed out in the comments.
Im not sure if this is of course really true since there is nothing to be found on MSDN about this topic. The string comparison guide on msdn mentions that toupperinvariant and tolowerinvariant are equal and does not prefer the former.