The software company I work for offers data conversion as a service for new clients who have previously used other similar software. I've written a VB.NET application to automate common conversion operations, and have separate logic between known vendors. A particular file layout has become rather common, but due to the nature of how this particular competitor's application stores information, certain fields mean one thing for one client, and another for a different client.
Certain elements within this vendor's format change every time, so I've written the application to account for that. Because some data fields mean different things to different vendors, I have to change my mapping code every time. This wasn't an issue when I had one of these every six months or so, but these are becoming much more common and I would much rather find a way to further automate this process.
I was thinking of implementing a simple interpreted scripting language that would allow me to save the conversion settings for the client in a text file, so I'm not having to feed the settings into my app every time I run it. My goal is to have the time spent in implementing this pay off in faster conversions in the long run.
This is the example I had in my head:
# this is a comment
RECORD INCLUDE(source[Class] != 'I' && source[Category] != 99);
FIELDMAP
destination[name] == source[name];
desintation[address] == source[mailingaddress1];
...
END FIELDMAP
BEGIN
# logic goes here
END
I don't want to make this more complicated than it needs to be. I admit that I've never even looked into scripting and am kinda clueless as to where to start or what other considerations need to be made.
Are there already scripting libraries for .NET that will do most of what I want? Is scripting the wrong way to go about this? I really don't want to reinvent the wheel if there is a better way.
Thank you for any help. :)
IronPython (one of the the DLR languages Jake referred to) is a .NET implementation of the Python scripting language. It allows you to access .NET code (including framework classes and class you've written) and would be an excellent choice for this sort of task.
It doesn't have any real dependencies (other than the .NET Framework) and you won't need to learn much learn much Python-specific syntax in order to get the job done.
(If you decide to go with IronPython, I recommend using Notepad++ as your editor.)
Rather than write a custom language, you could embed any of the DLR languages (IronPython, IronRuby, or JScript), or you can use compiler services to compile VB.NET from your application.
Another idea ... if you only need to change the mappings between variable names, maybe you could just come up with a little XML file that would define the mappings.
Related
Environment
Windows 8.1 64bit
Microsoft Visual Studio
C#
What I'm trying to do
To implement a mini language in C#
My job in my company is to make a software which automates evaluation of the performance of company products. The evaluation involves opening valves, flowing chemicals, fetching signals from sensors, calculating some values, etc...
The issue is when a user wants to change the procedure and parameters like how long the reaction time is, the order of opening valves, whether reaction conduit is flushed out or not, the programmer has to change the source code and build another software every time the user requires it.
So I'm trying to implement a mini language in C#. A user can organize a procedure of measurement with the easy-to-understand language. The user writes a code in a txt file, my software reads it, parse each line of code, extract commands and parameters, call corresponding method in the software.
For example, a user writes a line of code like open_valve(3), my software reads the code and call an internal method send_commands_to_ADconverter(VALVE, 3) which sends appropriate commands to a target valve via an A/D converter connected to the computer. This is my conception.
However, due to lack of my programming skill, I have no idea where to start. What technology is available? What are the keywords to achieve the goal? How can I achieve this objective? Any help would be appreciated. Thanks.
Edit 1
c# - Adding scripting functionality to .NET applications - Stack Overflow doesn't refer to accessing external devices like sensors, valves via A/D converter which is crucial for my purpose, so it is unclear this is a duplicate question of the link above.
In order to create a language you need a "parser" of some sort. You will need to define a "grammar". Parsing your "progam" via the grammar will result in a structure that you can then call methods in your code that implement each feature of your language.
You are on a big learning curve here :) lots of strange things like EBNF. You will probably see lots of references to things like Gold and ANTLR. These are hugely capable but involve things like "compiler compilers" and add a level of complexity that can be confusing and require extra steps in you build pipeline.
Here are a couple of libraries I've used that allow you to define you grammar in c#.
https://github.com/picoe/Eto.Parse
https://irony.codeplex.com/
Irony is very good (I've used it for various projects) but hasn't been maintained for a while.
Eto.Parse is more recent. It also has built in parsers that allow you to create a Grammer by parsing BNF. This is very cool
If I understand, your goal is to parse a syntax written by your user and take a decision with what he typed.
I suggest you to use regular expression to match and split the user input.
There are several scripting languages which can be run directly from C#.
As your users doesn't seem to have programming knowledge it might help to use a verbose language like VBScript.
To run VBScript you can use the Scripting Control. See this question for samples: Use Microsoft Scripting Control to evaluate 'If' expressions (via c#)
IIRC the script control must be run as a 32bit application.
Well, the easiest option would be to use what you already have - C#.
You can compile C# code on the fly, and it's very easy and cheap with the new compiler, Roslyn. Then all you need to do is define your interface so that it's easy to use for the users, and put that code in a method you can call from your code. This allows you to easily support all the various development tools for C# as well, including Visual Studio/VS Code. In the simplest form, all you need to do is make your "script" implement some base class that has all the methods you want the users to have - there's certainly better approaches, but this one is very simple, which sounds like something you're looking for.
Of course, this only makes sense if you can trust your users. Making safely sand-boxed code in C# is a lot more complicated (though certainly possible), and you'd probably be better off using some ready scripting solution.
I am trying to convert a large (200K lines) VB6 project into C#. I only want to convert the form design because I want to rewrite all the actual code in C#.
I want to use the free Artinsoft converter but this only allows projects of 10K lines. I am thinking I could break the VB6 project into 20 separate projects and then generate the C#.NET forms by using Artinsoft, then bringing the converted forms back into one C# project.
Is this feasible?
EDIT: Will this work? What are the potential problems?
I will try it in following order
1) Write automated test cases at least for major business operations
this will save you a lot of bother verifying the results later on.
2) Refactor your VB (try to move out your business logic in separate Classes/DLLs).
3) you can verify your re-factoring using automated test cases written in step 1.
4) Now you have small chunks this will also help you create chunks then you can use Artinsoft.
5) Your new C# application will be nicely layered.
6) Run tests created in step 1 on your new application and on your old application and compare results and keep improving unless they match :)
I would advise against using a converter for the UI. It may seem trivial, but there are features that simply do not natively exist in .NET (control arrays without a wrapper for a big one). And the control wrappers it creates are not fun to use nor maintain.
VB6 does not port well; the UI nor the code. If you were to continue down this path, there are things that can help.
Remove all unused forms and code
Ensure there are no control arrays and if there are, correct related code
Get all logic wrapped in classes
Ensure you are not calling/loading/referencing forms directly and that you are using an instantiated object
Ensure good encapsulation practices are in use and good object oriented design patterns
I would seriously reconsider conversion. Converted code/UI sounds great, but in my bad experiences with VB6 conversion, there are so many problems because of how bad VB6 allowed developers to be and features that simply don't exist anymore.
Do yourself a huge favor... Start a new project, create a custom control library that inherits from the standard controls you use, and use those everywhere. Don't like the way Cliptext works on a masked textbox in C# versus VB6? Extend it or override it. Want all your listviews to be sortable? Easy, single-point implementation. Find out in a year that the business wants spell checking on 90% of textboxes? Easy.
I intend to write a small application to scratch a personal itch and probably make the life of some colleagues easier. Here is what I have:
10+ years of experience in C
Plenty of experience in programming against the Win16/32 API in C from the Win3.1 to 2000 days.
C library written by myself already doing about 75% of what the application shall do.
What the application shall do:
open a binary, feed it into the mentioned library.
take the resulting text output and feed it into a new Excel Workbook.
apply some formating.
integrate nicely with the Windows environment (availability in "Open With...", remember some stuff using the registry etc.)
(maybe later) before giving the CSV data to Excel, parse it by looking up the meaning of some values in an XML file.
Except for the XML parsing part I have done all of that stuff before including COM / Office Automation in C/Win32. There is a lot of boilerplate code involved, but it is doable and the result will be a pretty small application without the need for an installer.
So why even think about C# / .Net?
no experience with parsing XML
the promise of less boilerplate code for the Windows and Excel stuff (yes, I have done C++ with OWL, MFC, ATL etc. but I am not going there anymore - not for free/fun)
Since I have also experience with C++, VB(not .Net) and a little Java / Objective-C I suppose learning C# will all be about the .Net libraries and not actually about the language.
My considerations so far:
Learning .NET might be fun and might result in less code / first steps in a more modern environment.
Sticking with what I know will lead to a predictable outcome in terms of effort and function (except for the optional XML stuff)
VB looked great at the beginning until the projects where about 80% done, then the pain started and the DLL coding in C. I am concerned history could repeat itself if I choose .Net.
My primary objective is the functionality. Effort is a concern. The XML parsing is optional.
Please advice.
Update: one thing I forgot to mention explicitly is that I am also worried about easy deployment of the tool to my co-workers. With Win32 I am pretty sure I can come up with an EXE file < 1Mb that can be easily emailed and does not require installation. With .Net not so much. Can I create the necessary MSI or whatever in Visual Studio Express (free) or do I need 3rd party tools?
as others have your question mostly covered, I'd just like to quickly comment on your considerations:
Learning .NET might be fun and might result in less code / first steps in a more modern environment.
Totally agreed. It is definitely fun and usually it does result in less code. The investment you make now will certainly benefit you in future projects. It is way faster to program in .Net than in C. Not only it is easier, but it is also safer. You are isolated from many programming errors common in C mostly related to memory mismanagement. You also get a very complete managed API to do stuff you would usually need to build your own framework.
Sticking with what I know will lead to a predictable outcome in terms of effort and function (except for the optional XML stuff)
Hence your indecision. :-)
VB looked great at the beginning until the projects where about 80% done, then the pain started and the DLL coding in C. I am concerned history could repeat itself if I choose .Net. My primary objective is the functionality. Effort is a concern. The XML parsing is optional.
.Net is an entirely different beast from VB. Most of the things you wouldn't be able to do in VB, or at least do them easily, are supported by .Net. For instance, Windows Services are a snap to build in .Net. Socket programming is also supported, but there are very few reasons to do it yourself, as you've got loads of communication APIs with .Net. You've got web-services, .Net Remoting, MSMQ management, and more recently WCF. Proper multithreading is supported by .Net, unlike the idiotic apartment model in VB. In case you really need to go low level, you can also actually use pointers in C#, inside of unsafe code blocks, even though I would never advise to do so.
If you really need to do things in C, then integrating is also relatively easy. You can create COM objects and use interop to work with them from .Net. You can also interact directly with plain ol' dlls using DllImport. Using www.pinvoke.net makes it easier.
When I developed in VB, sometimes I also had to go back to C++ to do stuff that I wasn't able of doing in VB. Since I began programming in .Net, the only extremely rare scenarios I would need to go back to C++ were when I needed to use legacy COM components that used types I was having a hard time to marshal via interop. I wouldn't worry about history repeating itself.
If you're using COM, you may be interested in using C# 4.0 instead of earlier versions - the downside being that it's only in beta. But basically it makes COM stuff somewhat less ugly for various reasons.
I'd expect there to be plenty of good C libraries for XML parsing by now. I would expect the main benefit to actually be the knowledge gained. I doubt that you'll actually produce the code faster for this project, but the next one may well be a lot quicker.
How much do you care about learning new stuff?
It sounds like an ideal project for learning C# & .NET.
You know most of what you need to do so you can use that to gain a base level of understanding of C# & .NET which you can then apply to the stuff you need to learn.
As Rune says though, a key driver could be the timescales. If this is something you need in a hurry then coding it in C & using win32 directly might be the answer.
Sorry I couldn't be more definite.
I think you should use C#. With your experience the learning curve won't be too steep. The code will ultimately be cleaner (and less of it) than you probably could with C/Win32.
There is probably going to be no problem using your existing C-library with the [DllImport] attribute.
It depends. :-) It depends on whether you want to do this quickly or if you want to learn something new. It depends on whether you will be the only maintainer of the code or if others will maintain it in the future. It depends on how complex your xml handling will be and on how complex the COM automation is.
You will probably get a working application quicker if you do it in C than in C#. Both since you have much of the stuff needed already in place and since you know C well.
But this project sounds like a good match for C# and .Net. .Net has great support for XML and COM interop is easy but clumsy in C# (much better in the next version!). So if you are interested in learning C# and .Net this would be a good project to do so.
I would definitely do this in .Net and probably C# (but I am biased). Using .Net would probably result in code that is easier to read and maintain and most probably easier to write. So if you are interested in learning C# I would suggest you go for it!
Edit:
You worry about the size of the executable if you write it in .Net. I doubt that will be a problem, for most if not all of the libraries you will use for a project like this will already be installed on your computer. 1 Mb is rather large for a .Net executable, event for a big project.
a short notice on the installation. .NET is as default xcopy-able so you wouldn't need an installer for the exe to be usable. Mail it around (or with the next release of the .NET framework optionaly leave it on a network share)
You could look at building a hybrid system that uses C++/CLI and C#. C++/CLI provides a nice bridge between the two and lets you easily split different parts of the system between the managed and unmanaged worlds.
Not sure if the setup projects are included in the free versions of visual studio. But you could use clickonce (included with the framework) or WIX (open source XML based msi creation tool).
learning C# will all be about the .Net libraries and not actually about the language
No there are many things you need to learn about the language (delegates , events , generics ...) and also it is object oriented and it manages the memory by itself and yes no pointers :)
anyway C# and .NET are great all you need is some effort to get up to speed
I have some old C 32 Bit DLLs that are using Oracle's Pro C precompiler library (proc.exe) to expose a hundred or so sproc/func calls to an even older VB6 GUI which references these functions through explicit Declare statements like so:
Declare Function ConnectToDB Lib "C:\windows\system32\EXTRACT32.DLL" (CXN As CXNdets, ERR As ERRdets) As Long
All the structures in the C header files are painstakingly replicated in the VB6 front end. At least the SQL is precompiled.
My question is, is it worth trying to impose a .Net interface (by conversion to an assembly) onto the the C code and upgrade the VB6 to C# or do you think I should just abandon the whole thing and start from scratch. As always, time is of the essence hence my appeal for prior experience. I know that if I keep the Declares in .Net I will have to add lots of complicated marshalling decorations which I'd like to avoid.
I've never had to Convert C to .Net before so my main question if everything else is ignored is are there any porting limitations that make this inadvisable?
... At least the SQL is precompiled.
Is this the only reason you've got code in C? If so, my advice is to abandon that and simply rewrite the entire thing in C# (or even VB6 if that's what your app is written in) ... unless you've profiled it and can prove a measurable difference, you won't be getting any perf benefits from having sql/sproc calls in C. You will only get increased maintenance costs due to the complexity of having to maintain this interop bridge.
You should continue to use the DLL in .NET by creating an assembly around the Declares. That one assembly probably would go a little quicker in VB.NET than C#. Then have your new UI reference that assembly. Once you have that going then you have bought yourself time to convert the C code into .NET. You do this by initially keeping the assembly and replacing the the declares with new .NET code. Soon you will have replaced everything and can refactor it to a different design.
The time killer is breaking behavior. The closer you can preserve the behavior of the original application the faster the conversion will be. Remember there nothing wrong with referencing a traditional DLL. .NET is built on many layers of APIs which ultimately drill down to the traditional DLLs that continue to be used by Windows. Again once you have the .NET UI working then you have more time to work on the core and bring everything into .NET.
I always advise extreme caution before setting out to rewrite anything. If you use a decent tool to upgrade the VB6 to .NET, it will convert the Declare statements automatically, so don't stress about them too much!
It's a common pitfall to start out optimistically rewriting a large piece of software, make good early progress fixing some of the well-known flaws in the old architecture, and then get bogged down in the functionality that you've just been taking for granted for years. At this point your management begin to get twitchy and everything can get very uncomfortable. I have been there and it's no fun. Sounds like your users are already twitchy, which is a bad sign.
...and here's a blog post by a Microsofty that agrees with me:
Many companies I worked with in the early days of .NET looked first at rewriting driven in part by a strong desire to improve the underlying architecture and code structures at the same time as they moved to .NET. Unfortunately many of those projects ran into difficulty and several were never completed. The problem they were trying to solve was too large
...and some official advice from Microsoft UK regarding migrating from VB6 to .NET
Performing a complete rewrite to .NET is far more costly and difficult to do well [than converting] ... we would only recommend this approach for a small number of situations.
Maybe your program is small, and you have a great understanding of the problems it solves, and you are great at estimating accurately and keeping your projects on track, and it will all be fine.
If you move from VB6 to VB.net or C#, throw away the C code and use the appropriate ODP.net classes or LINQ to access those stored procedures. Since the C layer (as I understand it) has no logic other than exposing the stored procedures, it's not useful anymore after the switch. By doing that, you get (at least) much better exception handling (i.e. exceptions at all instead of magic return codes), maintainability etc.
See also: Automatically create C# wrapper classes around stored procedures
Currently the project I'm working with does not have completely fixed models (due to an external influence) and hence I'd like some flexibility in writing them. Currently they are replicated across three different layers of the application (db, web api and client) and each has similar logic in it (ie. validation).
I was wondering if there is an approach that would allow me to write a model file (say in ruby), and then have it convert that model into the necessary c# files. Currently it seems I'm just writing a lot of boilerplate code that may change at any stage, whereas this generated approach would allow me to focus on much more important things.
Does anyone have a recommendation for something like this, a dsl/language I can do this in, and does anyone have any experience regarding something like this?
This can be easily done with ANTLR. If the output is similar enough you can simply use the text templating mechanism—otherwise it can generate an abstract syntax tree for you to traverse.
I have seen a system that used partial classes and partial methods to allow for regeneration of code without affecting custom code. The "rules engine" if you will was completely generated from a Visio state diagram. This is basically poor mans workflow but very easy to modify. The Viso diagram was exported to XML which was read in using powershell and T4 to generate the classes.
The above example is of an external DSL. I.E. external to the programming language that the application runs in. You could on the other hand create an internal DSL which is implemented and used in a programming language.
This and the previous article on DSLSs from Code-Magazine are quite good.
In the above link Neal Ford shows you how to create an internal DSL in C# using a fluent interface.
One thing he hasn't mentioned yet is that you can put this attribute [EditorBrowsable(EditorBrowsableState.Never)] on your methods so that they don't appear to intellisense. This means that you can hide the non-DSL (if you will) methods on the class from the user of the DSL making the fluent API much more discoverable.
You can see a fluent interface being written live in this video series by Daniel Cazzulino on writing an IoC container with TDD
On the subject of external DSLs you also have the option of Oslo (CTP at the moment) which is quite powerful in it's ability to let you create external DSLs that can be executed directly rather than for the use of code generation which come to think of it isn't really much of a DSL at all.
I think you are on the right track.
What I usually do in a situation like this is design a simple language that captures my needs and write a LL1 (Recursive Descent) parser for it.
If the language has to have non-trivial C# syntax in it, I can either quote that, or just wrap it in brackets that I can recognize, and just pass it through to the output code.
I can either have it generate a parse tree structure, and generate say 3 different kinds of code from that, or I can just have it generate code on the fly, either using a mode variable with 3 values, or just simultaneously write code to 3 different output files.
There's more than one way to do it. If you are afraid of writing parsers (as some programmers are), there is lots of help elsewhere on SO.