We Keep Coding

Filter elements in Revit and set parameter

Hello everybody,
around two or three month ago I started to learn Dynamo for Revit... finally :)
After learning and testing a lot, I got a few own scripts working. Then I learned Python, because I couldn't create the next script only with Dynamo-Nodes.
Then I thought "Let's see how difficult it is to get something done as a PlugIn".
I watched some Videos and read a lot of stuff.
Finally I got the Revit-AddIn-Wizard installed and made my first small Test-PlugIn.
Great...
Now I have a few problems which I do not understand... so I thought I will try my luck here... because I got so much information and help, reading through this site.
My goal was/is the following: (I tell you what I have now)
A form with a few buttons, comboboxes and a DataGridView.
I can load an Excelfile, click on "Show" to show it in the DataGridView.
The header of each row will be automatically put into 3 comboboxes.
In the first combobox you select the first search-parameter, in the second you CAN select another search-parameter and in the third combobox you select the parameter you want to set.
I have a checkbox to switch from type- to instance-parameter for the search- and the set-operation.
There is also a button which shows another small form with a list of categories (I won't search for ALL, only nearly all modelcategories).
PlugIn
I took me a lot of "watching Videos, reading through the internet, testing, testing and testing".
Thanks to this site here and a few others... I managed to get this whole PlugIn nearly 100% working.
But now I have a few strange issues and I have absolutely no clue on how to fix them or if it is possible. And I really hope that someone can help me.
First... I just tell you my problems and perhaps someone can say "this really IS an issue!" or that it is possible to get it done. Then I would post some code.
So... what do I do?!
1. I have a FilteredElementCollector which filters ALL elements.
2. Depending on my "Type/Instance-Checkboxes" I do .WhereElementIsElementType OR .WhereElementIsNotElementType.
3. Then it passes a MultiCategoryFilter to get the big list down to only the modelcategories.
4. Next, the collection passes one of ten different "methods" depending on all settings. There I filter this collection depending on the searchlists-comboboxes. When the combobox says "Familie" or "Typ" then it filters for ".BuiltInParameter.SymbolFamilyName" or ".Name" otherwise it just uses ".LookupParameter".
After that I have a collection with only the elements of selected categories which contains the values from the Excellist.
5. Depending on what my search- and set-settings are (e.g. search for type and set instance) I have to get the instances from the collected types or the other way around.
6. Then I pass it down to another method where I finally set the parameter.
So... Excelheader goes into comboboxes, depending on what you select in there it creates lists with the values of the selected rows.
I hope you all understand.
Now... where are my problems?
When I search for type-familynames or instance-parameter and set a typeparameter it works for ALL categories without any error.
1. When I try to set an instanceparameter (doesn't matter what my search-setting are) it works for all "normal" families but not for the systemfamilies (e.g. walls, floors, pipes etc.). No error, just nothing happens WHY? It seems that I cannot set an instance-parameter for system-families.
2. Roofs, Stairs, CurtainPanels and GenericModel make problems when I search for a typeparameter Error is something like "The object reference was not set to an object instance". Only with these 4 categories and it doesn't matter what I want to set... but when I search for family-/typeNAME or Instance-Parameter, then I can set type or instance and it works (except instance for sysfam).
3. When I try to search AND set an instance-parameter it works for ALL categories EXCEPT if one wall does not contain a search value... it really is enough that ONE wall does not have a search-param-value that everything will be cancelled.
I have a few other small problems... but I hope someone can help me with these problems... I would be extremely thankfull
greetings and have a nice day or night :)
Philipp

Tl; dr.
The three problems you describe sound like your own. I have no heard anybody else runAsk three separate questions and provide three separate minimal code snippets describing how they arise,. into those. I suggest that you create three separate independent minimal reproducible cases to demonstrate all three issues. Chances are, when you simplify and minimalise your code, the problem will go away. If it does not, it might just possibly be in a small and manageable enough state for other people to help you take a look at it. Given the long-winded description above, nobody in the world can help you.

Thank you for your answer Jeremy,
as I said, as a first start it is ok for me if you don't say "With theses categories, there are indeed some issues!"
I think I've managed to create 3 small examples of my problems.
For each problem I made a zip-file containing the complete visual-studio folder, a small exampleproject and a readme.txt with (I hope) enough information to understand everything in detail.
Problem1
Problem3
You only need to compile them or copy the .addin and .ddl files into the Revit AddIn folder. Then you get the new ribbons.
Short problem summary = I get problems when searching for parametervalues and setting values to another parameter.
Edit: I just solved the 2. problem when searching for familynames and setting system-families-parameter.
I used:
ElementClassFilter ecf = new ElementClassFilter(typeof(FamilyInstance));
FilteredElementColletor colle2 = new FilteredElementCollector(doc);
colle2.WherePasses(ecf);
I simply deleted the ClassFilter and do it now like in the other cases where I need instances.
FilteredElementCollector colle2 = new FilteredElementCollector(doc);
colle2.WhereElementIsNotElementType();
The 1. and 3. problem still exist :/
I would be thankful for any help someone can provide :)

Roslyn: Continuously shadow-copy & syntax-transform a document as it is being changed

I'm writing a script editor that needs to transform the user's input (which is similar to script syntax) into valid C# according to some rules I define. For example, if the users puts in
using System;
public string hello()
{
return "Hi!" // whoops, semicolon here is missing!
}
I'd need to transform this to
using System;
public class ContainerClass
{
public string hello()
{
return "Hi!" // whoops, semicolon here is missing!
}
}
my transformation will insert new nodes (such as the class declaration) and might move around existing ones, but it will never modify or remove existing ones. (I know SourceCodeKind = Script does something vaguely similar, but I can't use that for a variety of reasons).
Now I need to come up with a way to do this transformation given the following considerations:
Since I need to run the transformation each time the user changes the original document (i.e. just types a single letter), I can't afford to re-parse the entire thing every time from a performance perspective. For example, if the user inserts the missing semicolon after ";", ideally I would just insert the same (or cloned) node into my already transformed document, instead of re-parsing everything. I suppose that rules out standard ways of modification such as DocumentEditor.
I need to have a way to re-map locations from my transformed document to locations in the original document. Since I will never delete nodes, I think theoretically this should be possible (but how?).
This is necessary e.g. as I would end up with diagnostic messages (and intellisense information etc.) pointing to locations in the transformed document, and need to get the original document's location for these to actually show them to the user.
Can anyone thing of a more or less direct way to do this? Is there maybe even some Roslyn helper classes for use cases like this?
My ideas below. I'm not quite sure they'd work, and I think they'd be very hard to implement, so I'm hoping there is some easier way to be honest;
For #1, my only idea was to get get the text changes (Document.GetTextChangesAsync) of the original document after its source code changes; and then somehow try to find out what nodes have been affected by this (maybe get nodes that intersect the edited area in the old and new document, then compute which ones have been deleted, added or modified) - and then to apply these changes in my transformed document. This seems awfully complex though.
For #2, my only idea so far was to enable tracking for nodes of the original document. Then I would find whatever node a location points to in the transformed document, and find the node in the original document this originated from (and then find the location of that node).
But the problem is that e.g. the code above would produce a diagnostic error pointing towards the location right after "Hi!", wich the location span's length of 0, as there's a semicolon missing. So the location doesn't really point to a node at all. Maybe I could try finding adjacent nodes in that case?!

How do I improve cyclomatic complexity of some options?

So do I just ignore this code analysis warning by suppressing it? Or is there a way to truly fix it?
Here is a user story that comes close to mine, but I changed it slightly so that company information isn't on the site...
Say I have a website for a company that ships to 15 countries, and they want to show the names of those countries in the user's language of choice from the appropriate resources.resx file.
Now my "options" in a list are more complex than just a name/value or key/value pair. So the current code has a method that returns all the options, so it might look like:
return new[]
{
new CountryOption(code1, resourceKey1, someOtherValue1),
new CountryOption(code2, resourceKey2, someOtherValue2),
new CountryOption(code3, resourceKey3, someOtherValue3),
... (repeat 12 more times so I have 15 countries)
};
Thus I get a list (IEnumerable<CountryOption>) of all the countries from which to choose.
Most such applications will read this sort of information from a database, but this data rarely changes and putting it into a database will slow the performance of the site. One could put this into a flat file to read, but again, compiled in code will be faster. Finally, we do have some unit tests to make sure this information is correct that run with each build (harder to do for information in a database).
Is the only way to reduce cyclomatic complexity for a list of known values to read it from some source outside the code? (If so, the suppressing the message is probably the right thing to do.)

As others pointed out, the creation of a collection by itself had a cyclomatic complexity of 1. But the object that was in the collection was using a func<string> and that drove the complexity up - basically 2 for each item in the collection! Thanks for the help.

Better method of handling/reading these files (HCFA medical claim form)

I'm looking for some suggestions on better approaches to handling a scenario with reading a file in C#; the specific scenario is something that most people wouldn't be familiar with unless you are involved in health care, so I'm going to give a quick explanation first.
I work for a health plan, and we receive claims from doctors in several ways (EDI, paper, etc.). The paper form for standard medical claims is the "HCFA" or "CMS 1500" form. Some of our contracted doctors use software that allows their claims to be generated and saved in a HCFA "layout", but in a text file (so, you could think of it like being the paper form, but without the background/boxes/etc). I've attached an image of a dummy claim file that shows what this would look like.
The claim information is currently extracted from the text files and converted to XML. The whole process works ok, but I'd like to make it better and easier to maintain. There is one major challenge that applies to the scenario: each doctor's office may submit these text files to us in slightly different layouts. Meaning, Doctor A might have the patient's name on line 10, starting at character 3, while Doctor B might send a file where the name starts on line 11 at character 4, and so on. Yes, what we should be doing is enforcing a standard layout that must be adhered to by any doctors that wish to submit in this manner. However, management said that we (the developers) had to handle the different possibilities ourselves and that we may not ask them to do anything special, as they want to maintain good relationships.
Currently, there is a "mapping table" set up with one row for each different doctor's office. The table has columns for each field (e.g. patient name, Member ID number, date of birth etc). Each of these gets a value based on the first file that we received from the doctor (we manually set up the map). So, the column PATIENT_NAME might be defined in the mapping table as "10,3,25" meaning that the name starts on line 10, at character 3, and can be up to 25 characters long. This has been a painful process, both in terms of (a) creating the map for each doctor - it is tedious, and (b) maintainability, as they sometimes suddenly change their layout and then we have to remap the whole thing for that doctor.
The file is read in, line by line, and each line added to a
List<string>
Once this is done, we do the following, where we get the map data and read through the list of file lines and get the field values (recall that each mapped field is a value like "10,3,25" (without the quotes)):
ClaimMap M = ClaimMap.GetMapForDoctor(17);
List<HCFA_Claim> ClaimSet = new List<HCFA_Claim>();
foreach (List<string> cl in Claims) //Claims is List<List<string>>, where we have a List<string> for each claim in the text file (it can have more than one, and the file is split up into separate claims earlier in the process)
{
HCFA_Claim c = new HCFA_Claim();
c.Patient = new Patient();
c.Patient.FullName = cl[Int32.Parse(M.Name.Split(',')[0]) - 1].Substring(Int32.Parse(M.Name.Split(',')[1]) - 1, Int32.Parse(M.Name.Split(',')[2])).Trim();
//...and so on...
ClaimSet.Add(c);
}
Sorry this is so long...but I felt that some background/explanation was necessary. Are there any better/more creative ways of doing something like this?

Given the lack of standardization, I think your current solution although not ideal may be the best you can do. Given this situation, I would at least isolate concerns e.g. file read, file parsing, file conversion to standard xml, mapping table access etc. to simple components employing obvious patterns e.g. DI, strategies, factories, repositories etc. where needed to decouple the system from the underlying dependency on the mapping table and current parsing algorithms.

You need to work on the DRY (Don't Repeat Yourself) principle by separating concerns.
For example, the code you posted appears to have an explicit knowledge of:
how to parse the claim map, and
how to use the claim map to parse a list of claims.
So there are at least two responsibilities directly relegated to this one method. I'd recommend changing your ClaimMap class to be more representative of what it's actually supposed to represent:
public class ClaimMap
{
public ClaimMapField Name{get;set;}
...
}
public class ClaimMapField
{
public int StartingLine{get;set;}
// I would have the parser subtract one when creating this, to make it 0-based.
public int StartingCharacter{get;set;}
public int MaxLength{get;set;}
}
Note that the ClaimMapField represents in code what you spent considerable time explaining in English. This reduces the need for lengthy documentation. Now all the M.Name.Split calls can actually be consolidated into a single method that knows how to create ClaimMapFields out of the original text file. If you ever need to change the way your ClaimMaps are represented in the text file, you only have to change one point in code.
Now your code could look more like this:
c.Patient.FullName = cl[map.Name.StartingLine].Substring(map.Name.StartingCharacter, map.Name.MaxLength).Trim();
c.Patient.Address = cl[map.Address.StartingLine].Substring(map.Address.StartingCharacter, map.Address.MaxLength).Trim();
...
But wait, there's more! Any time you see repetition in your code, that's a code smell. Why not extract out a method here:
public string ParseMapField(ClaimMapField field, List<string> claim)
{
return claim[field.StartingLine].Substring(field.StartingCharacter, field.MaxLength).Trim();
}
Now your code can look more like this:
HCFA_Claim c = new HCFA_Claim
{
Patient = new Patient
{
FullName = ParseMapField(map.Name, cl),
Address = ParseMapField(map.Address, cl),
}
};
By breaking the code up into smaller logical pieces, you can see how each piece becomes very easy to understand and validate visually. You greatly reduce the risk of copy/paste errors, and when there is a bug or a new requirement, you typically only have to change one place in code instead of every line.

If you are only getting unstructured text, you have to parse it. If the text content changes you have to fix your parser. There's no way around this. You could probably find a 3rd party application to do some kind of visual parsing where you highlight the string of text you want and it does all the substring'ing for you but still unstructured text == parsing == fragile. A visual parser would at least make it easier to see mistakes/changed layouts and fix them.
As for parsing it yourself, I'm not sure about the line-by-line approach. What if something you're looking for spans multiple lines? You could bring the whole thing in a single string and use IndexOf to substring that with different indices for each piece of data you're looking for.
You could always use RegEx instead of Substring if you know how to do that.

While the basic approach your taking seems appropriate for your situation, there are definitely ways you could clean up the code to make it easier to read and maintain. By separating out the functionality that you're doing all within your main loop, you could change this:
c.Patient.FullName = cl[Int32.Parse(M.Name.Split(',')[0]) - 1].Substring(Int32.Parse(M.Name.Split(',')[1]) - 1, Int32.Parse(M.Name.Split(',')[2])).Trim();
to something like this:
var parser = new FormParser(cl, M);
c.PatientFullName = FormParser.GetName();
c.PatientAddress = FormParser.GetAddress();
// etc
So, in your new class, FormParser, you pass the List that represents your form and the claim map for the provider into the constructor. You then have a getter for each property on the form. Inside that getter, you perform your parsing/substring logic like you're doing now. Like I said, you're not really changing the method by which your doing it, but it certainly would be easier to read and maintain and might reduce your overall stress level.

Efficient algorithm for finding related submissions

I recently launched my humble side project and would like to add a "related submissions" section when viewing a submission. Exactly like what SO is doing here - see right column, titled "Related"
Considering that each submission has a title and a set of tags, what is most effective (optimum result), most efficient (fast, memory friendly) way to query the database for related submissions?
I can think of one way to do this (which I'll post as an answer) but I'm very interested to see what others have to say. Or perhaps there's already a standard way of achieving this?

Here's my two cent solution:
To achieve the best output, we need to put “weight” on the query results.
To start with, each submission in the database is assumed to have a weight of zero.
Then, if a submission in the "pool" shares one tag with the current submission, we'd add +3 to the found submission. Hence, if another submission is found that shares two tags with the current submission, we add +6 to the weight.
Next, we split/tokenize the title of the current submission and remove “stop words”.
I’ve seen a list of stop words from google, but for now I’ll define my stop words to be: [“of”, “a”, “the”, “in”]
Example:
Title “The Best Submission of All Times”
Result the array: ["The", “Best”, “Submission”, “of”, “All”, “Times”]
Remove stop words: [“Best”, “Submission”, “All”, “Times”]
Then we query the database for submissions containing any of the mentioned titles, and for each result we add the weight: +2
And finally sort the list descending by weight and take the top N results.
What do you think? (be gentle!)

If I understand well, you need a technique to find whether two posts are "similar" one to each other. You may want to use a probabilistic model for that:
http://en.wikipedia.org/wiki/Mutual_information
The idea would be to say that if two posts share a lot of "uncommon" words, they are probably speaking on the same topic. For detecting uncommon words, depending on your application, you may use a general table of frequencies, or maybe better, build it yourself on the universe of the words of your posts (but you will need to have enough of them to have something relevant).
I would not limit myself on title and tags, but I would overweight them in the research.
This kind of ideas is very common in spam filtering. I unfortunately the time to make a full review, but a quick google search gives:
http://www.aclweb.org/anthology/P/P04/P04-3024.pdf
karlmicha.googlepages.com/acl2004_poster.pdf