We Keep Coding

Implement Language Auto-Completion based on ANTLR4 Grammar

I am wondering if are there any examples (googling I haven't found any) of TAB auto-complete solutions for Command Line Interface (console), that use ANTLR4 grammars for predicting the next term (like in a REPL model).
I've written a PL/SQL grammar for an open source database, and now I would like to implement a command line interface to the database that provides the user the feature of completing the statements according to the grammar, or eventually discover the proper database object name to use (eg. a table name, a trigger name, the name of a column, etc.).
Thanks for pointing me to the right direction.

Actually it is possible! (Of course, based on the complexity of your grammar.) Problem with auto-completion and ANTLR is that you do not have complete expression and you want to parse it. If you would have complete expression, it wont be any big problem to know what kind of element is at what place and to know what can be used at such a place. But you do not have complete expression and you cannot parse the incomplete one. So what you need to do is to wrap the input into some wrapper/helper that will complete the expression to create a parse-able one. Notice that nothing that is added only to complete the expression is important to you - you will only ask for members up to last really written character.
So:
A) Create the wrapper that will change this (excel formula) '=If(' into '=If()'
B) Parse the wrapped input
C) Realize that you are in the IF function at the first parameter
D) Return all that can go into that place.
It actually works, I have completed intellisense editor for several simple languages. There is much more infrastructure than this, but the basic idea is as I wrote it. Only be careful, writing the wrapper is not easy if not impossible if the grammar is really complex. In that case look at Papa Carlo project. http://lakhin.com/projects/papa-carlo/

As already mentioned auto completion is based on the follow set at a given position, simply because this is what we defined in the grammar to be valid language. But that's only a small part of the task. What you need is context (as Sam Harwell wrote: it's a semantic process, not a syntactic one). And this information is independent of the parser. And since a parser is made to parse valid input (and during auto completion you have most of the time invalid input), it's not the right tool for this task.
Knowing what token can follow at a given position is useful to control the entire process (e.g. you don't want to show suggestions if only a string can appear), but is most of the time not what you actually want to suggest (except for keywords). If an ID is possible at the current position, it doesn't tell you what ID is actually allowed (a variable name? a namespace? etc.). So what you need is essentially 3 things:
A symbol table that provides you with all possible names sorted by scope. Creating this depends heavily on the parsed language. But this is a task where a parser is very helpful. You may want to cache this info as it is time consuming to run this analysis step.
Determine in which scope you are when invoking auto completion. You could use a parser as well here (maybe in conjunction with step 1).
Determine what type of symbol(s) you want to show. Many people think this is where a parser can give you all necessary information (the follow set). But as mentioned above that's not true (keywords aside).
In my blog post Universal Code Completion using ANTLR3 I especially addressed the 3rd step. There I don't use a parser, but simulate one, only that I don't stop when a parser would, but when the caret position is reached (so it is essential that the input must be valid syntax up to that point). After reaching the caret the collection process starts, which not only collects terminal nodes (for keywords) but looks at the rule names to learn what needs to be collected too. Using specific rule names is my way there to put context into the grammar, so when the collection code finds a rule table_ref it knows that it doesn't need to go further down the rule chain (to the ultimate ID token), but instead can use this information to provide a list of tables as suggestion.
With ANTLR4 things might become even simpler. I haven't used it myself yet, but the parser interpreter could be a big help here, as it essentially doing what I do manually in my implementation (with the ANTLR3 backend).

This is probably pretty hard to do.
Fundamentally you want to use some parser to predict "what comes next" to display as auto-completion. This has to at least predict what the FIRST token is at the point where the user's input stops.
For ANTLR, I think this will be very difficult. The reason is that ANTLR generates essentially procedural, recursive descent parsers. So at runtime, when you need to figure out what FIRST tokens are, you have to inspect the procedural source code of the generated parser. That way lies madness.
This blog entry claims to achieve autocompletion by collecting error reports rather than inspecting the parser code. Its sort of an interesting idea, but I do not understand how his method really works, and I cannot see how it would offer all possible FIRST tokens; it might acquire some of them. This SO answer confirms my intuition.
Sam Harwell discusses how he has tackled this; he is one of the ANTLR4 implementers and if anybody can make this work, he can. It wouldn't surprise me if he reached inside ANTLR to extract the information he needs; as an ANTLR implementer he would certainly know where to tap in. You are not likely to be so well positioned. Even so, he doesn't really describe what he did in detail. Good luck replicating. You might ask him what he really did.
What you want is a parsing engine for which that FIRST token information is either directly available (the parser generator could produce it) or computable based on the parser state. This is actually possible to do with bottom up parsers such as LALR(k); you can build an algorithm that walks the state tables and computes this information. (We do this with our DMS Software Reengineering Toolkit for its GLR parser precisely to produce syntax error reports that say "missing token, could be any of these [set]")

C# how to find the state someone was born in with their social security number

I am currently working on an assignment in which I am to validate various formats using regular expressions (phone numbers, birth date, email address, Social Security). One of the features our teacher has suggested would be to have a method that returns the state an individual was born using their Social Security Number.
xxx-xx-xxxx
The first 3 digits correspond to a state/area as outlined here:
http://socialsecuritynumerology.com/prefixes.php
If I've isolated the first 3 numbers as an integer already, is there anyway I could quickly match the number with its corresponding area code?
Currently I'm only using if-else statements but its getting pretty tedious.
Example:
if (x > 0 && x<3)
return "New Hampshire";
else if (x <= 7)
return "Maine";
...

You have a few options here:
50 if statements, one for each state, as you are doing.
A switch with 999 conditions, matching each option with a state. It probably looks cleaner and you can generate it with a script and interject the return statements wherever necessary. Maybe worse than option 1 in terms of tediousness.
Import the file as text, parse it into a Dictionary and do a simple lookup. The mapping is most likely not going to change in the near future, so the robustness argument is rather moot, but it is probably "simpler" in terms of amount of effort*. And it's another chance to practice regex to parse lines in the file you linked.
*Where "effort" is measured purely in the amount of tedious gruntwork prone to annoying human error and hand fatigue. Energy consumed within the brain due to engineering and implementing a solution where the computer does the ugly stuff for you is not included. :)

It's hard to tell your level of skill and what your course has taught you so far which is why it's difficult answering these kinds of questions, and also for the most part that's why you will get a negative response from people - they assume that you would have had the answer in your course materials already and will assume that you are being lazy.
I'm going to assume that you are at a basic level and that you already know how to solve the problem the brute force way (your if/else construct) and that you are genuinely interested in how to make your code better and not simply asking for a solution you can copy/paste.
Now, while your if/else idea will work, that is a procedural way of thinking. You are working with an object oriented language, so I would suggest to you to think about how you could use the principles of OO to make this work better. A good starting point would be to make a collection of state objects that contain all the parameters you need. You could then loop through your state collection and use their properties to find the matching one. You could create the state collection by reading from a file or database or even just hard coding it for the purposes of your assignment.

Your intuition that a long chain of if and else if statements might be unwieldy is sound. Not only is it tedious, but the search to find the correct interval is inefficient. However, something needs to be in charge of this tedium. A simple solution would be to use a Dictionary to store key/value pairs that you could build up once, and reuse throughout the application. This has the downside of requiring more space than necessary, as every individual mapping becomes an element of the data structure. You could instead follow the advice from this question and use a data structure more suited to ranged values for look ups.
It's difficult to tell from your question as it's written what your level of expertise is, and going into any real detail here would essentially mean completing your assignment for you. It should be noted though, that there's nothing inherently wrong with your solution, and depending on where you are in your education it may be what's expected.

Parse numbers from large text, possibly without regex (performance critical)

I'm extremely familiar with regex before you all start answering with variations of: /d+
I want to know if there are alternatives to regex for parsing numbers out of a large text file.
I'm parsing through tons of huge files and need to do some group/location analysis on the positions of keywords. I'm now at the point where i need to start finding groups of numbers as well nested closely to my content of interest. I want to avoid regex if at all possible because this needs to be a speedy process.
It is possible to take chunks of a file to inspect for the numbers of interest. That however would require more work and add hard coded limits for searching. (i'd like to avoid this)
I'm open to any suggestions.
UPDATE
Sorry for the lack of sample data. For HIPAA reasons I'd rather not even consider scrambling the text and posting it.
A great substitute would be the HTML source of any stackoverflow.com question page. Imagine I needed to grab the reputation (score) of all people that posted an answer to a question. This also means that the comma (,) is needed as well. I can't remove the html to simplify the content because I'm using some density analysis to weed out unrelated content. Removing the HTML would mix content too close together.

Unless the file is some sort of SGML, then I don't know of any method (which is not to say there isn't, I just don't know of one)
However, it's not to say that you can't create your own parser; you could eliminate some of the overheads of the .Net regex library by writing something that only finds ranges of numbers.
Fundamentally, I guess that that's all any library would do, at the most basic level.
Might help if you can post a sample of the sort of data you'll be processing?

Sentence generator using Thesaurus

I am creating an application in .NET.
I got a running application name http://www.spinnerchief.com/. It did what I needed it to do but but I did not get any help from Google.
I need functional results for my application, where users can give one sentence and then the user can get the same sentence, but have it worded differently.
Here is an example of want I want.
Suppose I put a sentence that is "Pankaj is a good man." The output should be similar to the following one:
Pankaj is a great person.
Pankaj is a superb man.
Pankaj is a acceptable guy.
Pankaj is a wonderful dude.
Pankaj is a superb male.
Pankaj is a good human.
Pankaj is a splendid gentleman

To do this correctly for any arbitrary sentence you would need to perform natural language analysis of the source sentence. You may want to look into the SharpNLP library - it's a free library of natural language processing tools for C#/.NET.
If you're looking for a simpler approach, you have to be willing to sacrifice correctness to some degree. For instance, you could create a dictionary of trigger words, which - when they appear in a sentence - are replaced with synonyms from a thesaurus. The problem with this approach is making sure that you replace a word with an equivalent part of speech. In English, it's possible for certain words to be different parts of speech (verb, adjective, adverb, etc) based on their contextual usage in a sentence.
An additional consideration you'll need to address (if you're not using an NLP library) is stemming. In most languages, certain parts of speech are conjugated/modified (verbs in English) based on the subject they apply to (or the object, speaker, or tense of the sentence).
If all you want to do is replace adjectives (as in your example) the approach of using trigger words may work - but it won't be readily extensible. Before you do anything, I would suggest that you clearly defined the requirements and rules for your problem domain ... and use that to decide which route to take.

For this, the best thing for you to use is WordNet and it's hyponym/hypernym relations. There is a WordNet .Net library. For each word you want to alternate, you can either get it's hypernym (i.e. for person, a hypernym means "person is a kind of...") or hyponym ("X is a kind of person"). Then just replace the word you are alternating.
You will want to make sure you have the correct part-of-speech (i.e. noun, adjective, verb...) and there is also the issue of senses, which may introduce some undesired alternations (sense #1 is the most common).

I don't know anything about .Net, but you should look into using a dictionary function (I'm sure there is one, or at least a library that streamlines the process if there isn't).
Then, you'd have to go through the string, and ommit words like "is" or "a". Only taking words you want to have synonyms for.
After this, its pretty simple to have a loop spit out your sentences.
Good luck.

C# - Show the differences when comparing strings

In my asp.net project, I have two strings (actually, they are stored in a Session object, then i do a .ToString() )
This project is part of my free Japanese language exercises on my website (Italian only for now, so i won't link/spam)
For now i do an if (original == inputted.ToLower()) , but I would like to compare the strings and highlight the differences on the screen
like this:
original: hiroyashi
wrote by user: hiroyoshi
i was thinking to compare the two strings and save the differences in another variable, with HTML tags, and then show it on a Literal control... but... if the differences are many, or the input is shorter... how to do that?
It looks there is the needing of an huge amount of coding... or not?

I seem to remember someone asking this not too long ago, and essentially they were pointed at difference engines.
A quick search on codeplex brings up:
http://www.codeplex.com/site/search?projectSearchText=diff
May be worth a hunt through some of those that come up - you may be able to plug something into your existing code?
Cheers,
Terry

John Resig wrote a javascript diff algorithm, but he's removed the page explaining what it does from his site. It's still available through the google cache though. Apologies if linking that is bad John. It should do what you want, someone else took it, tweaked it and put an article up about it here - complete with a test page

I am not sure if this would be helpful, but this is a way I would do:
I would use a hashmap, and store all words seperate by space there.
Then using that I would map with the original.
You can add html tags or whatever if they are different.
There is bound to be a performance issue here on a large dictionary of words
The coding itself would not be long though.