We Keep Coding

Replace Unicode character "�" with a space

I'm a doing an massive uploading of information from a .csv file and I need replace this character non ASCII "�" for a normal space, " ".
The character "�" corresponds to "\uFFFD" for C, C++, and Java, which it seems that it is called REPLACEMENT CHARACTER. There are others, such as spaces type like U+FEFF, U+205F, U+200B, U+180E, and U+202F in the C# official documentation.
I'm trying do the replace this way:
public string Errors = "";
public void test(){
string textFromCsvCell = "";
string validCharacters = "^[0-9A-Za-z().:%-/ ]+$";
textFromCsvCell = "This is my text from csv file"; //All spaces aren't normal space " "
string cleaned = textFromCsvCell.Replace("\uFFFD", "\"")
if (Regex.IsMatch(cleaned, validCharacters ))
//All code for insert
else
Errors=cleaned;
//print Errors
}
The test method shows me this text:
"This is my�texto from csv file"
I try some solutions too:
Trying solution 1: Using Trim
Regex.Replace(value.Trim(), #"[^\S\r\n]+", " ");
Try solution 2: Using Replace
System.Text.RegularExpressions.Regex.Replace(str, #"\s+", " ");
Try solution 3: Using Trim
String.Trim(new char[]{'\uFEFF', '\u200B'});
Try solution 4: Add [\S\r\n] to validCharacters
string validCharacters = "^[\S\r\n0-9A-Za-z().:%-/ ]+$";
Nothing works.
How can I replace it?
Sources:
Unicode Character 'REPLACEMENT CHARACTER' (U+FFFD)
Trying to replace all white space with a single space
Strip the byte order mark from string in C#
Remove extra whitespaces, but keep new lines using a regular expression in C#
EDITED
This is the original string:
"SYSTEM OF MONITORING CONTINUES OF GLUCOSE"
in 0x... notation
SYSTEM OF0xA0MONITORING CONTINUES OF GLUCOSE
Solution
Go to the Unicode code converter. Look at the conversions and do the replace.
In my case, I do a simple replace:
string value = "SYSTEM OF MONITORING CONTINUES OF GLUCOSE";
//value contains non-breaking whitespace
//value is "SYSTEM OF�MONITORING CONTINUES OF GLUCOSE"
string cleaned = "";
string pattern = #"[^\u0000-\u007F]+";
string replacement = " ";
Regex rgx = new Regex(pattern);
cleaned = rgx.Replace(value, replacement);
if (Regex.IsMatch(cleaned,"^[0-9A-Za-z().:<>%-/ ]+$"){
//all code for insert
else
//Error messages
This expression represents all possible spaces: space, tab, page break, line break and carriage return
[ \f\n\r\t\v​\u00a0\u1680​\u180e\u2000​\u2001\u2002​\u2003\u2004​\u2005\u2006​\u2007\u2008​\u2009\u200a​\u2028\u2029​​\u202f\u205f​\u3000]
References
Regular expressions (MDN)

Using String.Replace:
Use a simple String.Replace().
I've assumed that the only characters you want to remove are the ones you've mentioned in the question: � and you want to replace them by a normal space.
string text = "imp�ortant";
string cleaned = text.Replace('\u00ef', ' ')
.Replace('\u00bf', ' ')
.Replace('\u00bd', ' ');
// Returns 'imp ortant'
Or using Regex.Replace:
string cleaned = Regex.Replace(text, "[\u00ef\u00bf\u00bd]", " ");
// Returns 'imp ortant'
Try it out: Dotnet Fiddle

Define a range of ASCII characters, and replace anything that is not within that range.
We want to find only Unicode characters, so we will match on a Unicode character and replace.
Regex.Replace("This is my te\uFFFDxt from csv file", #"[^\u0000-\u007F]+", " ")
The above pattern will match anything that is not ^ in the set [ ] of this range \u0000-\u007F (ASCII characters (everything past \u007F is Unicode)) and replace it with a space.
Result
This is my te xt from csv file
You can adjust the range provided \u0000-\u007F as needed to expand the range of allowed characters to suit your needs.

If you just want ASCII then try the following:
var ascii = new ASCIIEncoding();
byte[] encodedBytes = ascii.GetBytes(text);
var cleaned = ascii.GetString(encodedBytes).Replace("?", " ");

Replace("this string that is a rule", "with this") in c#

I have this variable with an address:
var address = "Portugal, RUA ENG.CAMILO MENDONCA-LOTE 139 - APARTADO 80-ZONA INDUSTRIAL, 5300-426 Bragança";
And I want to know if is there a way to instead using a fixed string in the first parameter of .Replace(,), to use a rule, like : from "- APARTADO" to ",", and then a fixed string as second paramenter.

From what you indicated your rule to be (anything from "- APARTADO" to a ,), you could use regex to achieve this.
var address = "Portugal, RUA ENG.CAMILO MENDONCA-LOTE 139 - APARTADO 80-ZONA INDUSTRIAL, 5300-426 Bragança";
var regex = new Regex("- APARTADO[^']*,");
var replacedAddress = regex.Replace(address, /* Your replacement value */);
In the example above the regex pattern breaks down as follows:
"- APARTADO": matches the exact string you wanted to start with
[^']*: Matches any number of characters that is not a '

Consider an approach using string.split. It takes a string, and stores it into a string array made up of fragments of your original string which is separated by a specified string. Then find the fragment of the string you want to replace, edit it, and then concatenate the string using string.join!
MSDN has some good documentation on this: https://msdn.microsoft.com/en-us/library/ms228388.aspx

Finding Link Text with Regular Expressions

Team:
I need some help with some regular expressions. The goal is to be able to identify three different ways that users might express links in a note, and those are as follows.
MSN
possibilities
http://www.msn.com OR
https://www.msn.com OR
www.msn.com
Then by being able to find them I can change each one of them to real A tags as necessary. I realize the first example is already an A tag but I need to add some attributes to it specific to our application -- such as TARGET and ONCLICK.
Now, I have regular expressions that can find each one of those individually, and those are as follows, respective to the examples above.
<a?\w+((\s+\w+(\s*=\s*(?:".*?"|'.*?'|[^'">\s]+))?)+\s*)/?>
(http|https):\/\/[\w\-_]+(\.[\w\-_]+)+([\w\-\.,#?^=%&:/~\+#]*[\w\-\#?^=%&/~\+#])?
[\w\-_]+(\.[\w\-_]+)+([\w\-\.,#?^=%&:/~\+#]*[\w\-\#?^=%&/~\+#])?
But the problem is that I can't run all of them on the string because the second one will match a part of the first one and the third one will match a part of both the first and second. At any rate -- I need to be able to find the three permutations distinctly so I can replace each one of them individually -- because the third expression for example will need http:// added to it.
I look forward to everybodys assistance!

Assuming that the link starts or ends either with a space or at beginnd/end of line (or inside an existing A tag) I came up with the following code, which also includes some sample texts:
string regexPattern = "((?:<a (?:.*?)href=\")|^|\\s)((?:http[s]?://)?(?:\\S+)(?:\\.(?:\\S+?))+?)((?:\"(?:.*?)>(.*?)</a>)|\\s|$)";
string[] examples = new string[] {
"some text MSN more text",
"some text http://www.msn.com/path/file?page=some.page&subpage=9#jump more text",
"some text http://www.msn.com/path/file?page=some.page&subpage=9#jump more text",
"some text https://www.msn.com/path/file?page=some.page&subpage=9#jump more text",
"some text www.msn.com/path/file?page=some.page&subpage=9#jump",
"www.msn.com/path/file?page=some.page&subpage=9#jump more text"
};
Regex re = new Regex(regexPattern);
foreach (string s in examples) {
MatchCollection mc = re.Matches(s);
foreach (Match m in mc) {
string prePart = m.Groups[1].Value;
string actualLink = m.Groups[2].Value;
string postPart = m.Groups[3].Value;
string linkText = m.Groups[4].Value;
MessageBox.Show(" prePart: '" + prePart + "'\n actualLink: '" + actualLink + "'\n postPart: '" + postPart + "'\n linkText: '" + linkText + "'");
}
}
As this code uses groups with numbers it should be possible to use the regular expression in JavaScript too.
Depending on what you need to do with the existing A tag you need to parse the particular first group as well.
Update:
Modified the regex as requested so that the link Text becomes group no. 4
Update 2:
To better catch malformed links you might try this modified version:
pattern = "((?:<a (?:.*?)href=\"?)|^|\\s)((?:http[s]?://)?(?:\\S+)(?:\.(?:[^>\"\\s]+))+)((?:\"?(?:.*?)>(.*?)</a>)|\\s|$)";

Well, if we want to do in a single pass, you could create name groups for each scenario:
(?<full><a?\w+((\s+\w+(\s*=\s*(?:".*?"|'.*?'|[^'">\s]+))?)+\s*)/?>.*</a>)|
(?<url>(http|https)://[\w\-_]+(\.[\w\-_]+)+([\w\-\.,#?^=%&:/~\+#]*[\w\-\#?^=%&/~\+#])?)|
(<?www>[\w\-_]+(\.[\w\-_]+)+([\w\-\.,#?^=%&:/~\+#]*[\w\-\#?^=%&/~\+#])?)
Then you would have to check which was the matched group:
Match match = regex.Match(pattern);
if (match.Success)
{
if (match.Groups["full"].Success)
Console.WriteLine(match.Groups["full"].Value);
else if (match.Groups["url"].Success)
....
}

Regex string issue in making plain text urls clickable

I need a working Regex code in C# that detects plain text urls (http/https/ftp/ftps) in a string and make them clickable by putting an anchor tag around it with same url. I have already made a Regex pattern and the code is attached below.
However, if there is already any clickable url is present in the input string then the above code puts another anchor tag over it. For example the existing substring in the below code: string sContent: "ftp://www.abc.com'>ftp://www.abc.com" has another anchor tag over it when the code below is run. Is there any way to fix it?
string sContent = "ttt <a href='ftp://www.abc.com'>ftp://www.abc.com</a> abc ftp://www.abc.com abbbbb http://www.abc2.com";
Regex regx = new Regex("(http|https|ftp|ftps)://([\\w+?\\.\\w+])+([a-zA-Z0-9\\~\\!\\#\\#\\$\\%\\^\\&\\*\\(\\)_\\-\\=\\+\\\\\\/\\?\\.\\:\\;\\'\\,]*)?", RegexOptions.IgnoreCase);
MatchCollection mactches = regx.Matches(sContent);
foreach (Match match in mactches)
{
sContent = sContent.Replace(match.Value, "<a href='" + match.Value + "'>" + match.Value + "</a>");
}
Also, I want a Regex code to make emails as clickable with "mailto" tag. I can do it myself but the above mentioned issue of double anchor tag will also appear in it.

I noticed in your example test string that if a duplicate link e.g. ftp://www.abc.com is in the string and is already linked then the result will be to double anchor that link. The Regular Expression that you already have and that #stema has supplied will work, but you need to approach how you replace the matches in the sContent variable differently.
The following code example should give you what you want:
string sContent = "ttt <a href='ftp://www.abc.com'>ftp://www.abc.com</a> abc ftp://www.abc.com abbbbb http://www.abc2.com";
Regex regx = new Regex("(?<!(?:href='|<a[^>]*>))(http|https|ftp|ftps)://([\\w+?\\.\\w+])+([a-zA-Z0-9\\~\\!\\#\\#\\$\\%\\^\\&\\*\\(\\)_\\-\\=\\+\\\\\\/\\?\\.\\:\\;\\'\\,]*)?", RegexOptions.IgnoreCase);
MatchCollection matches = regx.Matches(sContent);
for (int i = matches.Count - 1; i >= 0 ; i--)
{
string newURL = "<a href='" + matches[i].Value + "'>" + matches[i].Value + "</a>";
sContent = sContent.Remove(matches[i].Index, matches[i].Length).Insert(matches[i].Index, newURL);
}

Try this
Regex regx = new Regex("(?<!(?:href='|>))(http|https|ftp|ftps)://([\\w+?\\.\\w+])+([a-zA-Z0-9\\~\\!\\#\\#\\$\\%\\^\\&\\*\\(\\)_\\-\\=\\+\\\\\\/\\?\\.\\:\\;\\'\\,]*)?", RegexOptions.IgnoreCase);
It should work for your example.
(?<!(?:href='|>)) is a negative lookbehind, that means the pattern matches only if it is not preceeded by "href='" or ">".
See lookarounds on regular-expressions.info
and the especially the zero-width negative lookbehind assertion on msdn
See something similar on Regexr. I had to remove the alternation from the look behind, but .net should be able to handle it.
Update
To ensure that there are also (maybe possible) cases like "<p>ftp://www.def.com</p>" correctly handled, I improved the regex
Regex regx = new Regex("(?<!(?:href='|<a[^>]*>))(http|https|ftp|ftps)://([\\w+?\\.\\w+])+([a-zA-Z0-9\\~\\!\\#\\#\\$\\%\\^\\&\\*\\(\\)_\\-\\=\\+\\\\\\/\\?\\.\\:\\;\\'\\,]*)?", RegexOptions.IgnoreCase);
The lookbehind (?<!(?:href='|<a[^>]*>)) is now checking that there is not a "href='" nor a tag starting with "
The output of the teststring
ttt <a href='ftp://www.abc.com'>ftp://www.abc.com</a> abc <p>ftp://www.def.com</p> abbbbb http://www.ghi.com
is with this expression
ttt <a href='ftp://www.abc.com'>ftp://www.abc.com</a> abc <p><a href='ftp://www.def.com'>ftp://www.def.com</a></p> abbbbb <a href='http://www.ghi.com'>http://www.ghi.com</a>

I know I arrived late to this party, but there are several problems with the regex that the existing answers don't address. First and most annoying, there's that forest of backslashes. If you use C#'s verbatim strings, you don't have to do all that double escaping. And anyway, most of the backslashes weren't needed in the first place.
Second, there's this bit: ([\\w+?\\.\\w+])+. The square brackets form a character class, and everything inside them is treated either as a literal character or a class shorthand like \w. But getting rid of the square brackets isn't enough to make it work. I suspect this is what you were trying for: \w+(?:\.\w+)+.
Third, the quantifiers at the end of the regex - ]*)? - are mismatched. * can match zero or more characters, so there's no point making the enclosing group optional. Also, that kind of arrangement can result in severe performance degradation. See this page for details.
There are other, minor problems, but I won't go into them right now. Here's the new and improved regex:
#"(?n)(https?|ftps?)://\w+(\.\w+)+([-a-zA-Z0-9~!##$%^&*()_=+/?.:;',\\]*)(?![^<>]*+(>|</a>))"
The negative lookahead - (?![^<>]*+(>|</a>)) is what prevents matches inside tags or in the content of an anchor element. It's still very crude, though. There are several areas, like inside <script> elements, where you don't want it to match but it does. But trying to cover all the possibilities would result in a mile-long regex.

Check out: Detect email in text using regex and Regex URL Replace, ignore Images and existing Links, just replace the regex for links, it will never replace a link inside a tag, only in contents.
http://html-agility-pack.net/?z=codeplex
Something like:
string textToBeLinkified = "... your text here ...";
const string regex = #"((www\.|(http|https|ftp|news|file)+\:\/\/)[_.a-z0-9-]+\.[a-z0-9\/_:#=.+?,##%&~-]*[^.|\'|\# |!|\(|?|,| |>|<|;|\)])";
Regex urlExpression = new Regex(regex, RegexOptions.IgnoreCase | RegexOptions.ExplicitCapture);
HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(textToBeLinkified);
var nodes = doc.DocumentNode.SelectNodes("//text()[not(ancestor::a)]") ?? new HtmlNodeCollection();
foreach (var node in nodes)
{
node.InnerHtml = urlExpression.Replace(node.InnerHtml, #"$0");
}
string linkifiedText = doc.DocumentNode.OuterHtml;

Get the number of an href url parameter from downloaded html page?

I am trying to get an ID from a url parameter inside an href that looks like this:
MyItemName
I want the 71312 only and at the momment I am trying to do it using regex (but if you have a better approch I would be glad to try):
string html,itemID;
using (var client = new WebClient())
{
html = client.DownloadString("http://www.mysite.com/search.php?search_text=" + myItemName);
}
string pattern = "" + myItemName + "";
Match m = Regex.Match(html, pattern, RegexOptions.IgnoreCase);
if (m.Success)
{
itemID = m.Groups[1].Value;
MessageBox.Show(itemID);
}
Example of the html:
more html body
<h1>Items - List</h1>
<p>MyItemNameTest, MyItemNameTestB, MYItemNameOther</p>
</div>
more html body

To show where your regex went wrong:
. and ? are special characters in regular expressions. . means "any character" and ? means "zero or one occurences of the previous expression". Therefore your regex fails to match. Also, you need to use verbatim strings in C# (unless you want to escape every backslash):
#"" + myItemName + "";
will probably work.
That said, unless all the links you're examining follow exactly this format, you might run into problems. It's kind of a running gag here on SO that parsing HTML with regular expressions will earn you the wrath of Cthulhu.

Use:
Uri u = new Uri("http://www.mysite.com/myitem.php?id=12313");
string s = u.Query;
HttpUtility.ParseQueryString(s).Get("id");
In variable id you have the number. Figure out the rest of the function :)