We Keep Coding

Import data into sql server from files with different format

I have a program, which watches a folder on the server. when new files (flat file) come in, the program (C#) read data, bulk insert into the table. it works fine.
Now, we extend the system. It means the data files could be in different formats (flat file, csv, txt, excel..), or with different columns (we need map them to the columns in the table).
my question is: is C# the best choice for this? or, SSIS is a better choice?
Thanks

I wouldn’t necessarily choose one or the other but choose depending on the file type and the amount of processing. For some file types its probably easier to go with C# and for some other SSIS works better.
Do you have someone on your team who is good with SSIS? It’s much easier to find a C# dev to do the job for you than to find someone who knows SSIS.
How likely is that requirements/formats are going to be updated in the future? That’s also important thing to keep in mind.
I do agree with what others said that SSIS is more powerful and offers support for more complex transformations but the questions is do you really need this?

It's depends on your context. Different format should not decision go to SSIS. With solution C# program: you can continue go with it because it run stable before. Easy to deployment, specific into your domain, easy to configuration as well.
With solution SSIS: The configuration more complicate required developer has deep knowledge into SSIS. The administration fee required more than C# program. However it easy to visual (has diagram for you see the flow integration more easier).
From my viewpoint, if the integration process does not required complicated about business rule you should go with C# program. Otherwise, SSIS more powerful if integration process required rules complicated. Hope this help.

In C# application I guess you are using the SqlbulkCopy component and compared to SSIS its not that powerful. So if your data size becomes huge,then C# application will become slower.
If you are familiar with SSIS,my suggestion is to go with SSIS. In SSIS,you can implement end-to-end solution as you have developed in C#,right from checking the files in a specific folder to loading the data into database.

Automated deployments with Kentico

Does anybody have experience automating deployments with Kentico? E.g. the difficulty of synchronizing document types, bizforms etc to another server?

I've used the built in content staging module to do this sort of thing. Unfortunately it's not all Unicorns and Rainbows. There were definitely some bugs in the module which essentially serializes the data from one server, and deserializes on the target server.
That was back in version 5.5 or 5.5R2 though, and they released version 6 a few months ago. I would take some time and look at the documentation for it's limitations, and then maybe give it a test before committing to it. It can definitely work for some, but it may not be Content Editor friendly.
Kentico Developer Documentation on Content Staging Module

Another possibility would be to utilize a tool that does database comparisons and syncing. I've used the SQL Examiner Suite before, but I've heard that Red Gate makes good tools too.
SQL Examiner
SQL Data Exminer
Red Gate Tools SQL Compare
While this probably isn't the best method, it can work. If you're not making significant changes on a regular basis this can be good for one off syncs between your local/dev server and production. This probably wouldn't be a good solution for "content staging", but more for changes that occurred due to development oriented tasks.

Another option is to use the Export/Import feature in Kentico: http://devnet.kentico.com/docs/6_0/devguide/index.html?export_and_import_overview.htm.
I haven't automated this process, but you can have a look at the ExportManager class in Kentico's API Reference: http://devnet.kentico.com/Documentation.aspx.
Hope this helps

With Kentico 10 you could use the Continuous Integration Feature. It is now working much better than in Kentico 9.
With the Continuous Integration Feature Database objects could be deployed together with the code files and are serialized automatically into the target database.
If you do not want to use this module, you need to use the Object Export Feature in Kentico (Site => Export site or objects).
In both scenarios you have to know, that content (Pages) are difficult to stage between different servers. Content staging is only usefull if you have a "real" staging server, where contend editors prepare the contet that should be staged to the live server on time.
In case you want to stage from a DEV server to the LIVE server, the pages will be overwritten by the dev version, if the GUID of the page is matching.
If you use Continuous Integration, all pages which are not in the DEV server instance will be deleted!
All other objects (Develop objects like Templates, Web Parts, Page Types, etc.) could be imported without any issues.

Which RDBMS and development tool should I choose to re-write my character-based app?

I have a pawnshop CRUD app written 20 years ago with INFORMIX-SQL/SE (DOS) which is currently running on DOS 6.22 within Microsoft Virtual PC 2007 on Windows Vista. I would like to modernize this app with a GUI, SQL-based engine and retain its existing functionality. It doesn't require any networking or multi-user capability. I would prefer a product which is royalty-free.
I also would like to quickly re-write it with as little effort possible. Which tool would you recommend?
I'm debating whether to re-write my INFORMIX-SQL app with I4GL (character-based) or another Windows/GUI-based tool.
My app is very robust and has some incredible features which my users are very happy with. Only obstacle which is keeping me from effectively acheiving market penetration is, believe it, my app is char-based and I would like to duplicate the same functionality with a GUI. My feeling is that its quicker for a user to process a transaction with my char-based app vs. having to focus a cursor with a mouse, but cosmetics is hurting me!
I would like to know specific instances of limitations, bugs or drawbacks of using another development tool before I invest considerable amount of time evaling another product. Answers to this question could save me a lot of time and money!
If you visit www.frankcomputer.com you can view a video-demo of my pawnshop app. (CAVEAT: The website's in Spanish, use google translate to get a more-or-less decent translation of the text. Start the video at the two-minute mark, with 720p resolution and full-screen to best comprehend my app.)

If I were doing it, I would probably choose to write a WPF GUI in C# with a SQL Server Express backend database. An embedded database like SQLite might work as well. But the main reason I would choose that is because that's what I'm most familiar with. Someone else would likely choose something else...
I might also choose ASP.NET MVC and make it a web application if that were an option (you say that multi-user is not required, but I say it's not required yet).
Also, if you're not the one who's going to be developing it (i.e. you're going to hire someone to build it for you) then I would say that you should find the developer first and let them choose (or at least have a say in) the technology. If you choose the technology up-front then you're simply limiting the field of developers who'll be able to work with you and there's really not much point in that.

I'd recommend you use Python with a PostgreSQL backend. Now some will think this is overkill, but after watching your video and reading your site (I had to use a translator), I suspect the added flexibility is something you will truly enjoy by going this route.
The reasons I'd argue for this solution are:
Python and PostgreSQL are both great products with amazing communities when you need them.
Both products have a bright outlook in their development paths. Since you obviously spent a lot of time and effort tweaking SPACE, I'm betting you will do the same over the next 40 years. So, the tools you choose now need to be there for you as you continue your development cycle.
They are both free with friendly licenses.
Cross-platform support.
Scalability. You can use PostgreSQL installed locally and connect via socket or scale it all the way up to several servers using load balanced connection pooling.
Security.
Data integrity. This includes how easy it is to make your whole environment easy to backup and thus easy to restore in the event of a catastrophe.
Whatever tools you end up choosing. I wish you the best in this project. I can tell you are working on something you truly love and that is something more of us should strive for!!

Based upon your answers and your emphasis upon time to make the changes and that you don't seem to want to change the Application at all but it is being forced upon you by then you should certainly evaluate Genero from 4js.
This will allow you to utilise your existing code but provide a nicer looking front-end. You can also maintain a single codebase supporting both character and "Gui" clients.

Choose whatever language and technology is easiest for you. If you need DB access and a short lead time it sounds like Java or Visual Basic would be best. Both have plenty of free tools to get you started.

The top languages tags in StackOverflow are C# (by a long margin), then Java, PHP and DotNet, followed by C++ and Python. Some of that will be skewed by the Joel & Jeff origin of the site, but any of those is more than capable of the task. Personally, I'd go with Java or Python but I don't like being tied to the Microsoft stack.
wxWidgets and QT might be options for the GUI components.
Of the databases, mysql, SQL Server Express or Oracle Express Edition are all free and robust. SQLite is good enough for most single user applications though. I'd put this at the bottom of the 'importance' list. For small-scale single user apps, you should be able to chop and change DB platforms without much hassle. The biggest relevance would be in how you actually backup/copy/restore data in the event of disk failure or corruption.

Should I use WPF or Windows Forms Application for my project in C#?

I am developing a Client-Server based application in which client application will access server database to store billing information. It will also have report generation facility. Windows Forms is good in document printing & I don't see such facility or controls in WPF. If I am wrong then please correct me.
I want database security, which DB should I use, SQL Server, MySQL or Oracle. I would like to use free DB but security is my priority.
Please suggest how I can implement a Client-Server architecture with multiple clients in C#?
Thank you Geeks!!!

Using WPF will allow you to deliver a more innovative User Experience. There's a decent high level overview of Working with Documents in WPF on MSDN.
Any database should be able to provide decent security. If you're using C#, I would recommend one of the versions of SQL Server (If SQL Server Express works for you...go for it).
Windows Communication Foundation (WCF).

Regarding 1.
WPF has a handy PrintDialog control with a PrintVisual method on it. This can be used to print any visual elements from a WPF controls hierarchy and may be able to satisfy your printing requirements.
I have developed many applications in WinForms and only a few in WPF, but I would lean very heavily in the WPF direction for any new development. It is very flexible and powerful.

1- WPF is awesome when you want to implement a windows application, and consider the freedom it gives and the separation of concern you can eventually get, I would say 100% go with WPF. There are many ways to control your printing procedure that some have been already explained in above.
2- For database, depending on your load and concurrency, you can choose a different database. Oracle is an awesome one, but probably the most expensive one. So, at the end you need to compare MySQL which is a fantastic db and at the same time free, and SQL server express edition which is a free version of the enterprise/professional version of Microsoft SQL server. Express edition is basically the limited edition of the same database, and if you need o expand your network and support, you might not be able to afford the fees of an enterprise solution. So one idea can be to stick to the best free database from the beginning if you foresee such a demand in the future. The security of these both databases are somehow in the same level, but you need to work on your network/server securities as well, because I found them harder to break.
3-Using Windows Communication Foundation (WCF) is very simple and easy especially when you want to use something like WPF. You basically create some data contracts and publish them, and that would be your communication end points. You basically call them, and deal with them as they are your local methods. Highly recommended. So what do you mean exactly by multiple clients? Does it mean different types of client (like web-based, mobile apps, etc.)? I also suggest taking a look at RESTful web services as it has the same kind of communication concepts. I just mentioned some names, so you can start reading something about each of them.

1 - Actually, WPF provides better/easier document/control printing that winforms does. Additionally, WPF still provides support for RDLC and other reporting solutions for printing & reporting. Asking if you should use WPF or WinForms is like asking if you should eat lemons or limes for vitamin C. Regardless of which one you use, you'll get where you're trying to go, but the flavor and experience will be wildly different.
2 - The database is not your security. The DB you choose will have little to no impact on your security. Personally, I would look at Active Directory and Domain security, and basically allowing the system to use the user's domain account and domain groups to control authentication and authorization.
3 -There are many possible solutions to a client-server framework, and each one has its own strengths and drawback, and then there's also the possibility of using webforms which you've not even touched upon.

Excel macro run from the web?

I have been handed a critical macro that takes an old school file full of invoices which thankfully is quite consistent. The macro reads this file, moves the data around to make it consistant and then generates a three tab speadsheet which is pretty much three CSV's. It then generates off these three CSV's another speadsheet which has a tab for each invoice. The amount of invoices can really vary.
It works, everyone is happy. We would like to put this out on the web with some security. For now, have it so that the user:
1) Logs in, uploads the old school file and presses process which will then spit out the same speadsheet with each tab being an invoice.
2) Store the data in a database for future growth and use of this data, as well as reporting.
I'm teaching myself ASP.NET and C# and think this would be a great learning project. Before I jump into it, can this realistically be done and what would others recommend in this case? Should I simply re-write based off the logic in the macro or is there a way to port over existing VBA code?

You can do it with an Excel COM API. But this tends to lead to memory leaks, I would not recomend it.
Microsoft has Excel Services which allow you to run Excel Spreadsheets on the server. But it is very expensive and may not support Macros.
SpreadSheetGear may be able to do it. But I have not tested it myself.
I would recommend that you rewrite the application in C#, you would get a better solution, and it may not take you any longer than getting the spreadsheet running on the server.

Using the Excel COM API from a web application is difficult. There are security issues which are non-trivial to address. If you wanted to retain the excel processing then you could build some sort of an out of band process which monitors an upload directory and, when it detects a new file, kicks off a process of transforming the excel file as the old macro use to.
There is no easy transition from VBA to C# since all the VBA code assume the existence of excel which may not be the case. However you can call macros in workbooks using the COM API.

Driving Excel from C# is surprisingly hard to get 100% right. Conversely, driving Excel from a VB6 application is surprisingly easy. But, calling this from a web application makes it harder, since you need to deal both with security and concurrency (2 users at once will trip over each other).
Microsoft don't support the use of Excel on the server (apart from Excel Services), so don't expect any help there. SpreadsheetGear is suited to this, but you'd have to pay for it.
You say this would make a good learning project - I'd disagree; it's likely to put you off programming altogether. This particular mix doesn't have a "nice" solution - it's a case of finding the least-unpleasant hack. If you want to learn ASP.NET & C#, I'd say find another pet project.