We Keep Coding

String split on dynamic separator

I am to deal with the following problem.
I have to extract messages from a communication buffer. Sadly the communication protocol is lousy and not well-structured. The only way I came up with to distinguish packets in the buffer is an intermediate "ack" command that is transmitted by the server.
Example:
[Packet1][ACK][Packet2][ACK][Packet3]
I could have used String.Split(ACK), but the separator is also not consistent. Though, there are 3 rules to identify the ack packet.
Starts with "AK".
Ends with "0" or "1".
Total length is 5 characters.
Ack example:
"AKxxy" where:
xx: (01 to 99)
y: (0 or 1)
I hope that there may be a regular expression that can solve my problem, but I lack the needed knowledge and time.
Is there any RegEx "expert" that may possible help me? Feel free to suggest any solution.
Thank you.
Edit:
Example packet (I really had to remove the packet information):
AK010CONFIDENTIALPACKET1AK011CONFIDENTIALPACKET2AK020AK011CONFIDENTIALPACKET3AK021CONFIDENTIALPACKET4AK050
Sadly, each packet in the protocol does not start or end with a specific character so I cannot distinguish them. To identify each one I have to split them using the ack packet and then perform different checks on each one.

The direct translation would be
\bAK\d{2}[01]\b
That is
\b # a word boundary
AK # AK literally
\d{2} # two digits
[01] # one of 0 or 1
\b # another word boundary
The expression needs to be tested though (see a demo on regex101.com).

EDIT:
looking at the other answers, this has probably merely ornamental value.
The solution of #Jan and #ThymosK
var packets = Regex.Split(buffer, #"AK\d{2}[01]");
seems much more elegant.
But I think it might be nice to see how all the parsing can be moved inside the regex. Even if it is way too unreadable :P
I have designed a regex that can give you messages and delimiters as groups:
(?s)(AK[0-9][0-9][0,1])|((?:(?!AK[0-9][0-9][0,1]).)*)
It can analyze text like this:
You can test it here.
As usual, regexes are write only. I can hardly read this myself. But I'll try and go through it:
The first group is straightforward and simply catches your ack command:
(AK[0-9][0-9][0,1])
The second group contains a negative lookahead (?! ... ) which matches anything not followed by the regex specified by .... Here we insert your ack syntax, so anything not followed by ack is matched. Then we add a single character, to extend this to actually match anything up to ack.
Basically this second part asserts that we are currently not followed by ack and then adds a single character. This is repeated as long as possible, until we find the ack. I turn this into the second group.
Since I don't have C# currently, I can't wrap this in code with the C# regex engine. But python works nicely with it and offers a useful findall method, that gives you all those groups.

string interim = Regex.Replace(buffer, "AK\d{2}[01]", "|");
var commands = interim.Split('|');
Assuming that | is not a valid input char. You can pick something very exotic.

c#, find multiline message by regex request

Main task
Find all DEBUG messages and select message fully (no matter single line message or multiline with unknown length)
I wrote such regex code:
\d{13}\t.*DEBUG.*(?=\d{13})
its search perfectly, but only single-line messages
Also I tried such code:
string myReg1 = #"\d{13}\t.*DEBUG.*(?=\d{13})";
MatchCollection match1 = Regex.Matches(logData, myReg1, RegexOptions.Singleline);
but this code found only one mach, where must be 147 matches....
I have logs like this:
1426174736798 addons.manager DEBUG Registering shutdown blocker for OpenH264Provider
1426174736799 addons.manager DEBUG Registering shutdown blocker for PluginProvider
*** Blocklist::_preloadBlocklistFile: blocklist is disabled

Try using this non-greedy regex instead (EDIT: tweaked a bit for input):
\d{13}\t.{0,100}DEBUG.+?(?=\d{13}|$)
Now this is tweaked a bit more closely to your input data. I can't really think of an ideal way to keep that first dot before the DEBUG from eating up other rows that you don't want. In a perfect world, you could write a phrase to say something like, "any character except a row of 13 digits", but this is not really something that regex does well. Maybe someone else can make this better. In the meantime, I have restricted the first dot to consume no more than 100 characters. If it goes more than 100 characters past the 13 digit number and has not found the string "DEBUG" yet, it is fairly safe to assume it is on a row we don't care about. You may need to tweak this number up or down a bit to fit your data (and I hate imperfect solutions like this), but hopefully this will get you in the neighborhood.
Changing .* to .+? makes the dot non-greedy. I also added an or to the last non-capturing group with a $ to match end-of-line (RegexOptions.SingleLine will treat the entire input as one line) to ensure that your last record is captured, since there will be no 13 digit number following the end of it.
This appears to work correctly in Expresso, which uses the same regex engine as .NET

Parse directories from a string

Firstly i have spent Three hours trying to solve this. Also please don't suggest not using regex. I appreciate other comments and can easily use other methods but i am practicing regex as much as possible.
I am using VB.Net
Example string:
"Hello world this is a string C:\Example\Test E:\AnotherExample"
Pattern:
"[A-Z]{1}:.+?[^ ]*"
Works fine. How ever what if the directory name contains a white space? I have tried to match all strings that start with 1 uppercase letter followed by a colon then any thing else. This needs to be matched up until a whitespace, 1 upper letter and a colon. But then match the same sequence again.
Hope i have made sense.

How about "[A-Z]{1}:((?![A-Z]{1}:).)*", which should stop before the next drive letter and colon?
That "?!" is a "negative lookaround" or "zero-width negative lookahead" which, according to Regular expression to match a line that doesn't contain a word? is the way to get around the lack of inverse matching in regexes.

Not to be too picky, but most filesystems disallow a small number of characters (like <>/\:?"), so a correct pattern for a file path would be more like [A-Z]:\\((?![A-Z]{1}:)[^<>/:?"])*.
The other important point that has been raised is how you expect to parse input like "hello path is c:\folder\file.extension this is not part of the path:P"? This is a problem you commonly run into when you start trying to parse without specifying the allowed range of inputs, or the grammar that a parser accepts. This particular problem seems pretty ad hoc and so I don't really expect you to come up with a grammar or to define how particular messages are encoded. But the next time you approach a parsing problem, see if you can first define what messages are allowed and what they mean (syntax and semantics). I think you'll find that once you've defined the structure of allowed messages, parsing can be almost trivial.

Regex to validate domain name with port

I am new developer and don't have much exposure on Regular Expression. Today I assigned to fix a bug using regex but after lots of effort I am unable to find the error.
Here is my requirement.
My code is:
string regex = "^([A-Za-z0-9\\-]+|[A-Za-z0-9]{1,3}\\.[A-Za-z0-9]{1,3}\\.[A-Za-z0-9] {1,3}\\.[A-Za-z0-9]{1,3}):([0-9]{1,5}|\\*)$";
Regex _hostEndPointRegex = new Regex(regex);
bool isTrue = _hostEndPointRegex.IsMatch(textBox1.Text);
It's throwing an error for the domain name like "nikhil-dev.in.abc.ni:8080".
I am not sure where the problem is.

Your regex is a bit redundant in that you or in some stuff that is already included in the other or block.
I just simplified what you had to
(?:[A-Za-z0-9-]+\.)+[A-Za-z0-9]{1,3}:\d{1,5}
and it works just fine...
I'm not sure why you had \ in the allowed characters as I am pretty sure \ is not allowed in a host name.
Your problem is that your or | breaks things up like this...
[A-Za-z0-9\\-]+
or
[A-Za-z0-9]{1,3}\\.[A-Za-z0-9]{1,3}\\.[A-Za-z0-9]{1,3}\\.[A-Za-z0-9]{1,3}
or
\*
Which as the commentor said was not including "-" in the 2nd block.
So perhaps you intended
^((?:[A-Za-z0-9\\-]+|[A-Za-z0-9]{1,3})\.[A-Za-z0-9]{1,3}\.[A-Za-z0-9]{1,3}\.[A-Za-z0-9]{1,3}):([0-9]{1,5}|\*)$
However the first to two or'ed items would be redundant as + includes {1-3}.
ie. [A-Za-z0-9\-]+ would also match anything that this matches [A-Za-z0-9]{1,3}
You can use this tool to help test your Regex:
http://regexpal.com/
Personally I think every developer should have regexbuddy
The regex above although it works will allow non-valid host names.
it should be modified to not allow punctuation in the first character.
So it should be modified to look like this.
(?:[A-Za-z0-9][A-Za-z0-9-]+\.)(?:[A-Za-z0-9-]+\.)+[A-Za-z0-9]{1,3}:\d{1,5}
Also in theory the host isn't allowed to end in a hyphen.
it is all so complicated I would use the regex only to capture the parts and then use Uri.CheckHostName to actually check the Uri is valid.
Or you can just use the regex suggested by CodeCaster

Split text file at sentence boundary

I have to process a text file (an e-book). I'd like to process it so that there is one sentence per line (a "newline-separated file", yes?). How would I do this task using sed the UNIX utility? Does it have a symbol for "sentence boundary" like a symbol for "word boundary" (I think the GNU version has that). Please note that the sentence can end in a period, ellipsis, question or exclamation mark, the last two in combination (for example, ?, !, !?, !!!!! are all valid "sentence terminators"). The input file is formatted in such a way that some sentences contain newlines that have to be removed.
I thought about a script like s/...|. |[!?]+ |/\n/g (unescaped for better reading). But it does not remove the newlines from inside the sentences.
How about in C#? Would it be remarkably faster if I use regular expressions like in sed? (I think not). Is there an other faster way?
Either way (sed or C#) is fine. Thank you.

Regex is a good option that I was using for a long time.
A very good regex that worked fine for me is
string[] sentences = Regex.Split(sentence, #"(?<=['""A-Za-z0-9][\.\!\?])\s+(?=[A-Z])");
However, regex is not efficient. Also, though the logic works for ideal cases, it does not work good in production environment.
For example, if my text is,
U.S.A. is a wonderful nation. Most people feel happy living there.
The regex method will classify it as 5 sentences by splitting at each period. But we know that logically that it should be split as only two sentences.
This is what made me to look for a Machine Learning Technique and at last the SharpNLP worked pretty fine for me.
private string mModelPath = #"C:\Users\ATS\Documents\Visual Studio 2012\Projects\Google_page_speed_json\Google_page_speed_json\bin\Release\";
private OpenNLP.Tools.SentenceDetect.MaximumEntropySentenceDetector mSentenceDetector;
private string[] SplitSentences(string paragraph)
{
if (mSentenceDetector == null)
{
mSentenceDetector = new OpenNLP.Tools.SentenceDetect.EnglishMaximumEntropySentenceDetector(mModelPath + "EnglishSD.nbin");
}
return mSentenceDetector.SentenceDetect(paragraph);
}
Here in this example, I have made use of SharpNLP, in which I have used EnglishSD.nbin - a pre-trained model for sentence detection.
Now if I apply the same input on this method, it will perfectly split text into two logical sentences.
You can even tokenize, POSTag, Chuck etc., using the SharpNLP project.
For step by step integration of SharpNLP into your C# application, read through the detailed article I have written. It will explain to you the integration with code snippets.
Thanks

Sentence splitting is a non-trivial problem for which machine learning algorithms have been developed. But splitting on whitespace between [.\?!]+ and a capital letter [A-Z] might be a good heuristic. Remove the newlines first with tr, then apply the RE:
tr '\r\n' ' ' | sed 's/\([.?!]\)\s\s*\([A-Z]\)/\1\n\2/g'
The output should be one sentence per line. Inspect the output and refine the RE if you find errors. (E.g., mr. Ed would be handled incorrectly. Maybe compile a list of such abbreviations.)
Whether C# or sed is faster can only be determined experimentally.

You could use something like this to extract the sentences:
var sentences = Regex.Matches(input, #"[\w ,]+[\.!?]+")
foreach (Match match in sentences)
{
Console.WriteLine(match.Value);
}
This should match sentences containing words, spaces and commas and ending with (any number of) periods, exclamation and question marks.

You can check my tutorial http://code.google.com/p/graph-expression/wiki/SentenceSplitting
Basic idea is to have split chars and impossible pre/post condition at every split. Tjis simple heuristic works very well.

The task you're interested in is often referred to as 'sentence segmentation'. As larsmans said, it's a non-trivial problem, but heuristic approaches often perform reasonably well, at least for English.
It sounds like you're primarily interested in English, so the regex heuristics already presented may perform adequately for your needs. If you'd like a somewhat more accurate solution (at the cost of just a little more complexity), you might consider using LingPipe, an open-source NLP framework. I've had pretty good luck with LingPipe, the few times I've used it.
See http://alias-i.com/lingpipe/demos/tutorial/sentences/read-me.html for a detailed tutorial on sentence segmentation.