We Keep Coding

Securing VBA or moving into VB.NET, C#, other? [duplicate]

This question already has answers here:
Best way to protect Excel VBA code?
(2 answers)
Closed 2 years ago.
I consider myself to be quite a competent Excel VBA programmer. Over a few years of my professional finance career I created quite a lot of useful procedures (UDF's, some small or big apps with multiple user-forms and a lot of small, easy to do stuff).
I decided that I will take a step forward and try to monetize my work. I have looked at VBA obfuscators, VBA compilers and stuff like that but sadly, the protection from obfuscators seems to be weak (at least I think so, I cannot comment as I know only VBA language, I am not master in computer science) or in case of compilators, compilation modifies my code so much, that 1/3 of it is not working properly (I have tried to debug it, but no matter what I do it does not work).
I checked the amount of code I have and it is about 13-14 thousand lines.
Now - is it worth or even possible for me to translate all that stuff into VB.NET or C# starting from scratch?
I have no experience with any of them, I have programmed only in VBA. I have been reading and it seems like it is nearly impossible to convert and re-write the code due to differences in user-forms, no UDF's, which I use in my subroutines and stuff like that. If it is possible, can someone point me in the right direction, what would be a better choice?
I am quite lost at the moment, as I have no experience and I know no developer who could "guide" me into this or that direction. What would be a good way to "translate" my VBA code and make a working app from it?
I know it is not easy, but if it will take me a few months of after-hours work, I will do it. The thing is that I do not want to completely waste the time and I want to do it right.
Thank you kindly for all ideas and your help. I hope my message is not completely off-topic.

It feels like an of topic question because it can't be factually answered; we can only give you our opinion for the most part. I'll pick out those parts that can be answered with facts:
Now - is it worth or even possible for me to translate all that stuff into VB.NET or C# starting from scratch?
Worth, I can't answer. Value is contextual - look at the number of people who get a kick out of providing stuff for free.
Possible; certainly. Your VBA code encapsulates a process, a notion, it does useful work and makes your life easier. It could make someone else's life easier too; would they pay you to make their life easier? Would they be able to integrate your work into theirs? Do you know how to make your work available to them in a way that means that they can actually use it?
You can put all this stuff into .NET, but don't think it makes it any harder to reverse engineer your work. If you want your stuff to be immune to reverse engineering, sell the data processing it does rather than selling the code. Create a webservice that calculates all the things people want, don't sell them a DLL that they could integrate and use but then their kid could decompile and release as a rival offering. That's not to say that this happens in business as much - businesses tend to be staffed with conscientious types that recognise when they license someone's work and use it, if they just ripped it and took it for free they would be stabbing one of their own in the back. If you're creating components for businesses to license and use, you'll probably have fewer piracy problems than shareware type licensing to end home users. But you can probably still sell the processing as a service rather than the software as a thing..
impossible to convert and re-write the code due to differences in user-forms, no UDF's, which I use in my subroutines and stuff like that
It might well be impossible to drop your code into something .netty, press play and have it working in an hour.. But I didn't think we were doing that. I thought we were re-implementing the process/software in .NET. Maybe whatever you pick doesn't have some blah UDF (user defined function?) this, or input box that - but at the end of it all, those things do some simple job and you can use something else in .NET that does that simple job. If anyone told me "seriously, there are things you can do in VBA that are flat out impossible in C#" I'd laugh for a while, then go back to playing javascript quake
14 thousand lines isn't an overbearing amount of code that you couldn't just sit down and start bashing it out in the evenings, and probably have reimplemented it all in something else in about a month - you already know what you want the code to do, so the hard part is getting info on how you do that in (your chosen language) - it's on topic for SO though, so fee free to search and ask how to translate various bits of VBA to C# etc.. People won't take kindly to questions that are a dump of 500 lines of VBA and a "can someone translate this to C# for me" though

A New and Full Implementation of Generic Intellisense

I am interested in writing a generic Intellisense enabled editor for SQL and C# (et al. if possible!). I would like to do this in C# as an overridden or extended WPF richTextBox-type control. I know there are many example projects available and I have implemented a basic version of my own; but most of the examples that I have come across (and indeed my own) are just that, basic.
A couple of code examples are:
DIY Intellisense By yetanotherchris
CodeTextBox - another RichTextBox control with syntax highlighting and Intellisense By Tamas Honfi
I have however, found a great example of an SQL editor with Intellisense QueryCommander SQL Editor By Mikael Håkansson which seems to work well. Microsoft must use a XML library of command keywords, but my question is: How (in detail) do Microsoft implement their Intellisense (as-you-type Intellisense) and how hard would it be for me to create my own of the same standard?
Edit A: A year on and I have managed to develop my own editor control with basic intellisense mainly for my own "enjoyment". I thought I would come back provide a list of freely available .NET projects that helped me with my own development and can be used out-of-the-box and free of charge:
ICSharpCode (WinForms)
AvalonEdit (WPF)
ScintillaNET (WinForms)
Query Commander [for example of intellisense implementation] (WinForms)
Edit B: 15 months after the question was asked I am still looking for new improved editors. This one is nice...
RoslynPAD is cool!
Edit C: 2 years+ on from the question, I have found the following projects, both using WPF and backed by AvalonEdit.
CodeCompletion for AvalonEdit using NRefactory. This project is really nice and has a full implementation of intellisense using NRefactory.
ScriptCS ScriptCS makes it easy to write and execute C# with a simple text editor.

How (in detail) do Microsoft implement their as-you-type Intellisense?
I can describe it to any level of detail you care to name, but I don't have the time for more than a brief explanation. I'll explain how we do it in Roslyn.
First, we build an immutable model of the token stream using a data structure that can efficiently represent edits, since obviously edits are precisely what there are going to be a lot of.
The key insight to making it efficient for persistent reuse is to represent the character lengths of the tokens but not their character positions in the edit buffer; remember, a token at the end of the file is going to change position on every edit but the length of the token does not change. You must at all costs minimize the number of total re-lexings if you want to be efficient on extremely large files.
Once you have an immutable model that can handle inserts and deletions to build up an immutable token stream without re-lexing the entire file every time, you then have to do the same thing, but for grammatical analysis. This is in practice a considerably harder problem. I recommend that you obtain an undergraduate or graduate degree in computer science with an emphasis on parser theory if you have not already. We obtained the help of people with PhDs who did their theses on parser theory to design this particular bit of the algorithm.
Then, obviously, build a grammatical analyzer that can analyze C#. Remember, it has to analyze broken C#, not correct C#; IntelliSense has to work while the program is in a non-compiling state. So start by coming up with modifications to the grammar that have good error-recovery characteristics.
OK, so now you've got a parser that can efficiently do grammatical analysis without re-lexing or re-parsing anything but the edited region, most of the time, which means that you can do the work between keystrokes. I forgot to mention, of course you will need to come up with some mechanism to not block the UI thread while doing all of these analyses should the analysis happen to take longer than the time between two keystrokes. The new "async/await" feature of C# 5 should help with that. (I can tell you from personal experience: be careful with the proliferation of tasks and cancellation tokens. If you are careless, it is possible to get into a state where there are tens of thousands of cancelled tasks pending, and that is not fast.)
Now that you've got a grammatical analysis you need to build a semantic analyzer. Since you are only doing IntelliSense, it does not need to be a particularly sophisticated semantic analyzer. (Our semantic analyzer must do an analysis suitable for generating code from correct programs and correct error analysis from incorrect programs.) But of course, again it has to do good semantic analysis on broken programs, which does increase the complexity considerably.
My advice is to start by building a "top level" semantic analyzer, again using an immutable model that can persist the state of the declared-in-source-code types from edit to edit. The top level analyzer deals with anything that is not a statement or expression: type declarations, directives, namespaces, method declarations, constructors, destructors, and so on. The stuff that makes up the "shape" of the program when the compiler generates metadata.
Metadata! I forgot about metadata. You'll need a metadata reader. You need to be able to produce IntelliSense on expressions that refer to types in libraries, obviously. I recommend using the CCI libraries as your metadata reader, and not Reflection. Since you are only doing IntelliSense, obviously you don't need a metadata writer.
Anyway, once you have a top-level semantic analyzer, then you can write a statement-and-expression semantic analyzer that analyzes the types of the expressions in a given statement. Pay particular attention to name lookup and overload resolution algorithms. Method type inference will be particularly tricky, especially inside LINQ queries.
Once you've got all that, an IntelliSense engine should be easy; just work out the type of the expression at the current cursor position and display a dropdown appropriately.
how hard would it be for me to create my own of the same standard?
Well, we've got a team of, call it ten people, and it'll probably take, call it five years all together to get the whole thing done from start to finish. But we have lots more to do than just the IntelliSense engine. That's maybe only 40% of the work. Oh, and half those people work on VB, now that I think about it. But those people have on average probably five or ten years experience in doing this sort of work, so they're faster at it than you will be if you've never done this before.
So let's say it should take you about ten to twenty years of full time work, working alone, to build a Roslyn-quality IntelliSense engine for C# that can do acceptably-close-to-correct analysis of large programs in the time between keystrokes.
Longer if you need to do that PhD first, obviously.
Or, you could simply use Roslyn, since that's what it's for. That'll take you probably a few hours, but you don't get the fun of doing it yourself. And it is fun!
You can download the preview release here:
http://www.microsoft.com/download/en/details.aspx?id=27746

This is an area where Microsoft typically produces great results - Microsoft developer tools really are awesome. And there is a clear commercial advantage for sales of their developer tools and for sales of Windows to having the best intellisense so it makes sense for Microsoft to devote the kind of resources Eric describes in his wonderfully detailed answer. Still, I think it's worth pointing out a few of things:
Your customers may not actually need all the features that Microsoft's implementation provides. The Microsoft solution might be incredibly over-engineered in terms of the features that you need to provide to your customers/users. Unless you're actually implementing a generic coding environment that is intended to be competitive with Visual Studio, it is likely that there are aspects of your intended use that either simplify the problem, or that allow you to make compromises on the solution that Microsoft feels they cannot make. Microsoft will likely spend resources decreasing response times that are already measured in hundreds of milliseconds. That may not be something you need to do. Microsoft is spending time on providing an API for others to use for code analysis. That's likely not part of your plan. Prioritize your features and decide what "good enough" looks like for you and your customers then estimate the cost of implementing that.
In addition to bearing the obvious costs of implementing requirements that you may not actually have, Microsoft also carries some costs that may not be obvious if you haven't worked in a team. There are huge communication costs associated with teams. It's actually incredibly easy to have five smart people take longer to produce a solution than it takes for a single smart person to produce the equivalent solution. There are aspects of Microsoft's hiring practices and organizational structure that make this scenario more likely. If you hire a bunch of smart people with egos and then empower all of them to make decisions, you too can get a 5% better solution for 500% of the cost. That 5% better solution might be profitable for Microsoft, but it could be deadly for a small company.
Going from a 1 person solution to a 5 person solution increases the costs, but that's just the intra-team development costs. Microsoft has separate teams that are devoted to (roughly) design, development, and testing even for a single feature. The project-related communication between peers across these boundaries has higher friction than within each of the disciplines. This not only increases communication costs between individuals, but it also results in larger team sizes. And more than that - since it's not a single team of 12 individuals, but is instead 3 teams of 5 individuals, there is 3x the upward communication cost. More costs that Microsoft has chosen to carry that may not translate to similar costs for other companies.
My point here is not to describe Microsoft as an inefficient company. My point is that Microsoft makes a ton of decisions about everything from hiring, to team organization, to design and implementation that start from assumptions about profitability and risk that simply do not apply to companies that are not Microsoft.
In terms of the intellisense thing, there are various ways of thinking about the problem. Microsoft is producing a very generic, reusable solution that doesn't just solve intellisense, but also targets code navigation, refactoring, and various other uses for code analysis. You don't need to do things the same way if your sole goal is to make it easy for developers to enter code without having to type much. Targeting that feature doesn't take years of effort and there are all sorts of creative things you can do if you're not just providing an API, but you actually control the UI too.

Application design for 5 years

My question is not technical. It's more of a philosophical and really down to individual preference. I am designing and developing an application (web + desktop) and this just occurred to me and was wondering if you guys (programmers and designers) ever came across this before:
Some designers believe in making applications that will run 3-5 years down the line and any changes that come to them will be reflected on them without any need to resort to system core changes. As a programmer I know for a fact that this is never the case. Small cosmetic changes do occur but usually they die away after a year or two, as time progresses there will be changes that will require core changes and eventually you will make a new application.
Given the fast paced changes of technologies, designing an application for 5 years down the line is rather absurd, IMHO. Well I mean not designing, but the idea that this application will run for 5 years and the belief that we will not need to create a new one I think is living in a fool's paradise. I mean really, fellow programmers, most mission critical or basic small application that have a running flow usually are re-made/re-structured/re-organized/re-coded few years down the line anyway.
So my question is why keep to this attitude of having this perfect application that will run for a decade. It's stupid really, because you know for a fact that technology will change every year; new frameworks, new methods, new technologies will emerge and your client will want them. So, if you forgive my use of this phrase, WTF is the point?
I keep telling my designer that the application will be redesigned in a few years anyway, there is no point in trying to make it shoot lighting from its #ss because it just won't, ever. There is no such thing as a perfect application.
I hope you guys get my drift. Have you guys felt the same way too. BTW I have been in the software programming business for about 7 years now. If you really think about it, do you really think Facebook will remain the same 5 years down the line, for sure the design will change every year or so to remain "funky" but the core will change every couple of years. I am dead sure of that. Am I paranoid or what? Please tell me there are other programmers in the same road as I am. Anyone?

My approach is to design for change. This means writing the most maintainable code I can, keeping things loosely coupled and modular, trying to do things in as standard a way as possible so that other developers can quickly pick up on the code, etc.
I usually put a little more effort into future-proofing the database design, as changes there can be a lot more difficult than code changes in many cases.

I have worked at two different companies which had software products that were nearing 10+ years old. Although they had been extended with tons of new functionality and had been given many face-lifts, the core of the application was essentially untouched since the first stable release. This may not be typical, but if the architect is skilled enough, the system can be built to be modular and extensible enough to accommodate a surprising amount of growth.

"The biggest roadblock to a great plan, is the dream of a perfect plan."
I agree - designing the perfect system which can be elegantly resilient to every possible future change, is fruitless. Every successful project is a tradeoff: building flexibility where you're confident it will be needed (or where it's easy to do anyway); and building somewhat quick-n-dirty code where you believe flexibility/change is unlikely. If you analyze the system well and the client has a good idea of their needs/requirements (not always a given), you'll get that balance right at least most of the time.
However, the idea that the whole system will be replaced every 3-5 years by some newer technology is also a fallacy. For every client who wants the latest, newest, sexiest system, there's 5 clients that are afraid to part with (or can't afford to replace) their legacy COM/VB/MS-Access/whatever system that is a morass of spaghetti logic built-up haphazardly without regard to maintainability, flexibility, or extensibility. You don't want to be the one building that system; if you are, then you're doing your client/employer a disservice.

Personally, I would never set out designing any sort of app assuming it is going to be replaced within a few years. It is all to often those 'updates' and 'rewrites' get pushed off for quick fixes desired by customers who don't want to wait for an entire new application. Sure the requirements change, features are needed, but often they will be desired in the current iteration.
I mean, there are a lot of apps, languages, and design patterns out there that thought the same thing, and are still in use today. One that pops into my head is the y2k bug. Programmers in the 70's never thought their code would last 30 years, and surely someone would expand those year values to 4 numbers before the turn of the century. We all remember how that thinking turned out...

I beleive that if the core of the system is built on principles that haven't changed much in many many years, then the system fundamentally will not change much, most changes would be primarially aesthetically.
Some tried and tested principles that remain strong to this day are for example database normalisation, modularisation of code etc etc.
So it depends what you define as the core. For me, the core means the design of the system, which if done properly probably wont change much at all in the future.

It really depends on the environment. I find that our internal corporate applications are fully overhauled (often retired) within 18 months of launch. It isn't anyone's fault - business priorities change, requirements change, new systems come online. At the same time other system run for a very long time.
We certainly don't develop any application with the expectation that it will soon be retired but there are business needs that need immediate solutions and sometimes it is better to get the app online and in the hands of the end-users as soon as possible. We update, iterate, and determine the next best steps.

The data and data structures (often) need to be legitimately designed to last for decades. The algorithms, UI and everything else is expected to evolve rapidly.
If your data represents legal documents, financial records, you might be required to hold on to it for decades.
This, too, though can be taken to extremes. There are subsets of data that probably no one will care about in 50 years, like memory performance counters that might have been stored in the database.

I remember a quote from James Kovac I believe, in one of the .Net Rocks' podcasts where he described our industry as:
one where the only constant is change
which is why you should design for flexibility so that when changes come, and it inevitably will in most cases* you will have an easier job adapting/updating your application. That's not to say you shouldn't try to build your application on solid grounds, but just more important to have a flexible solution which you can change easily than the perfect solution the first time around.
*I know in lots of banks people are still using ancient applications because it's too risky to change them and they simply don't have the expertise to change them anymore..

Well I think the ideal design is completely orthogonal, but you just have to accept that it rarely works out the way you envisage. If you've never read The Pragmatic Programmer, it talks a lot about future-proofing your code.

How do you handle large projects? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
I've just inherited a large project previously coded by about 4-5 people. The documentation consists of comments, and is not very well written. I have to get up to date on this project. How do I start? It consists of many different source files. Do you just dig in? Are there tools that can help visualize the structure/flow?

If you have a chance, I'd try and talk to the original designers and developers. Ask them about any major design issues or shortcomings of the project. Is the project in good shape and only needs maintenance or are there major components that need to be added or reworked? What are going to be the biggest roadblocks to maintaining the project? Take one or two of them to lunch (separately) if you have a budget for it as they might be more free to talk about problems outside of the office.
Talking to the users is also important for getting a feel for the current status of the project. Quite often they have a different opinion of how things stand then the developers do. Make sure, however, that they don't start giving you a list of all the things they want added or changed - you should take a few weeks to understand the project before you can start making major changes to it.
As for visualization tools, I'd start with the database design if there is a database involved. Tools like Microsoft Visio can create a diagram from an existing database. I find knowing the design of the database helps me wrap my head around what the programmers were trying to accomplish. Visio is also good for documenting program flow with some basic flowcharts though you'll have to create them yourself - it doesn't generate them automatically as far as I know.
Good luck.

I would encourage you to buy and read this book thoroughly. It provides you a LOT of information in this regard, much more than you will find here.

Brainstorming a little for you:
Step around in the application with a debugger, use a Static Code Analysis tool for which ever language you are working with...
Talk with people - both developers AND USERS to get a feel of the application.
Review the issue tracking system to see if you can see any recurring types of problem...

Are there tools that can help
visualize the structure/flow?
The latest Visual Studio 2010 allows you to generate architecture diagrams.
http://ajdotnet.wordpress.com/2009/03/29/visual-studio-2010-architecture-edition/

Try to find the starting point of the system and start digging from there. It sort of sucks to be in that situation, and chances are the comments might not be that helpful either. If the original developers didn't bother (or didn't have the chance) to document, chances are they never kept the comments up to date with code changes.
So time to bring the shovel... but don't just dig in blindly. One thing that is important is to understand what the system does from a users' perspective.
Concurrent with your code digging, you need to meet with a user (or the users' liason) and have him walk through the system, showing you how it is supposed to be used, for what purpose and what it and its subsystems are supposed to do. Moreover, attempt to understand what are the business pre-conditions and post-conditions of each major operation performed with this system.
Then map (or do a hierarchical) chart of the main functions of the system; classify them by category, purpose or module. If the system performs some sort of work flows or business transactions, attempt to chart some sort of state/transition diagram documenting each (and cross-referencing each state/transition to the subsystem or module in the system that is in charge for it.)
Once you have that, you can dig according to function. It will be best if you dig for a specific purpose, say, there is a bug fix to implement. You locate the logical module or category pertaining to that bug fix, you have the pre-conditions and post-conditions; then you can dig precisely on (or around) that bug fix.
If you just dig in without a guide (at least a high level one), you can be digging for months without getting anywhere (I'm telling you from painful experience.)
If there is no user manual, implement a draft according to your meetings with the users/users' liason. That could serve as a guide for implementing a developer's/administrator's manual for the system you just inherited (if there is ever a chance to implement one.)
If code is not on source control, put it on it. Doesn't matter what SCS you pick (could even be CVS, yuck!) What matters is to put it under source control asap.
Those developers didn't exist in a vacuum, they must have had exchanged emails. Identify other tech liasons they work with. Attempt to identify what other systems, if any, this system interfaces to (.ie. your databases, other's peoples databases, cron jobs, etc.)
But this could come at a later time. I think you should, for starters, focus on understanding how to use the system and what it is for. Let's call it understanding its business/knowledge architecture. Then dig according to that... or better yet, according to that and with the purpose of fixing a bug.
Good luck.

Use Profiler to see main functions and events in your project (the fastest way to learn framework)
Learn business logic very well to better understand the code
Documenting every new thing you learn - setup wiki (you will be surprised how quickly things are forgotten)
You can use Visio to draw Database Model Diagrams. (keep them close to you while studying the code)
These are the things that helped me when I inherited the previous project (50+ developers, 70+ GB database, 1 GB of source code and not even a single line of comments in code (maybe few :), and everything written in foreign language )

Use the debugger to walk through the application. That will let you go both deep and wide. You'll also be able to learn about how the code handles specific scenarios.
When you're ready to change something as #Jaxidian said, Working Effectively with Legacy Code is a great resource.

I was recently in a similar situation. What helped in my case was focusing on the changes I needed to perform on the project, and in the process of making those changes I learned about how the project is structured and so on. Sure, the first few tasks took a bit longer, but look on the bright side: I got stuff done and I got familiar with the project at the same time.

I'd suggest two things that may help:
Be productivity-driven. In other words, find a change that needs doing and use this to learn how that bit of the system works. Your changes may not be the most elegant without a whole-picture understanding of the software, but you will get work done within days/weeks.
Follow things from the user-interface. I.e if a change involves things a user does on a dialog, find that dialog in the code (relatively easy) and then work backwards to see what bits of the code provide data to the dialog, how the dialog interacts with the system, etc. Trying to find "where does X happen in the code" is very hard without good documentation, but finding "where is the code relating to this dialog" is quite easy and gives you an entry-point into the code.

Whenever I start a new project, I spend 2-3 days skim reading the code and making notes. I basically go through the entire solution from top to bottom and make a map in a text editor of each (significant) class in each project and what it appears to do.
The aim in doing this is not to completely understand the entire codebase, so don't worry if you feel you are not getting your head around it completely. The aim is that you end up with an index of where to go when you need to start on your first piece of work. You should also end up with a cursory picture of the solution in the back of your brain that will get filled in over the next couple of months. I always do this on the first few days as your superiors will not expect you to be productive during this time and you may never get another opportunity where you have the time to do so.
Also, do not rely on code comments for direction. Even with the best intentions they are often unmaintained and may lead to incorrect conclusions about what a class or section of code may do: a comment may lie but the code always tells the truth.

If you already have a team, you could charge each with a part of framework, and the result of their exploration should be registered somewhere, like a wiki. After that, give to each a task similar to something which is already done in the system (from the functional point of view)
For example: if a list of products is displayed in your app, you could display a list of orders (the complexity should be approximately the same), in the same manner it's done actually in the app. Than make it more interesting: try to edit it and save into DB.
Than switch the tasks and let the questions appear and than the first person who made the same task will show & explain how things are done.
Like that you'll see how the things are done pretty easy + your team will be up to date with this knowledge.

Presuming there is a database, start with the data model. Somewhere (Mythical Man-Month?) it was written "if I have your tables, I don't need to see your code."

Regarding potential tools, you may want to look into NDepend. It is a code-analysis tool, with an emphasis on highlighting the internal organization and dependencies of the code base (see this post for typical outputs), and spotting code quality issues. I have not used it personally, but Patrick Smacchia, one of the developers of the product, has a few posts where he applies NDepend to some classic apps (here is NUnit for instance) and discusses what it means, and I found them interesting.

Go and speak to the users or, read the manual and / or if one exists, go on a training course for the system (internal training departments will sometimes have put them together if there are lots of users).
If you don't know what it's meant to be doing then the chances of you being able to work out how it does it are close to zero.

Personal Project Planning [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
I want to design a 2D game idea with C#/XNA. Between school, project inexperience, limited resources, and other things that may cause me to bail on the project I am going to try to plan it out before I jump in:
What are some key aspects you've noticed in a successful personal project?
How did you (successfully) draw revenue the project (besides asking for donations)?
(Note: this is not the main purpose of my project but would be nice!)
How do you plan non-programming aspects like window layouts (UI), flow of game, and look and feel?
How do you bring an abstract idea of a game and making into a real product. What tools do you use in your sketching and planning?
With limited resources (artistic talent and money), how do I create some of the art resources I need (besides stock art)? Or should I not worry about that until I have a finished product?
Finally, how do you break barriers in your code (may it be knowledge, time, or perseverance)?

What are some key aspects you've noticed in a successful personal project?
I suppose this largely depends on your definition of success. I consider a game project successful if:
It's fun. Enough that I want to play it after I've finished.
I learn something in the process of making it.
I actually complete the project, where complete is defined as having art, audio, and any other assets adequate for the style of game that I'm making.
It works under 'field conditions', rather than simply on my own machine/hardware configuration.
In my case, every project that I've considered successful has been one that I've done some serious planning on and committed to seeing through to completion. That's the biggest step for me to get past. A casual project that is the product of a bored weekend usually won't get finished.
How did you (successfully) draw revenue the project (besides asking for donations)? (Note: this is not the main purpose of my project but would be nice!)
Heh. Haven't managed this yet. But then, that hasn't been a concern of mine enough to do the work required to support it. Someone already mentioned the potential revenue from XBLA for XNA projects, which is a very easy avenue to success if your project is the sort of thing that translates well to the console. I've looked into it, but it isn't really appropriate for my projects so far.
How do you plan non-programming aspects like window layouts (UI), flow of game, and look and feel?
Pen. And paper. I draw a lot of screenshots as guides and I'll usually have some kind of artistic theme in mind to go along with the gameplay. Rapid prototypes help here too; everything from making mockups of menus and various screens in a drawing program to making some simple interactive stuff in a throwaway project.
Write everything down. I take lots of notes and I've been known to have a laptop with notepad open on it sitting next to my development machine, to save me the trouble of alt-tabbing away from my code to type a quick note on what I'm doing. Some great ideas come out of the process of trying to make other things work and if you don't write it down, it didn't happen.
How do you bring an abstract idea of a game and making into a real product. What tools do you use in your sketching and planning?
Work iteratively as much as possible. I hate to sound cliche, but the whole agile model works really well for games. Start with that core 'fun' idea you have that inspired you to make a game in the first place and get that working in code. Maybe it's a single mechanic that you want to build puzzles/levels around or maybe it's more of a look/feel that you can capture in a single environment/level/whatever. Either way, if you can get a single example of it working right, it becomes a good check on whether the idea is actually viable and will likely give you some ideas of the different ways it can be implemented. That might seem kind of vague, but more specific advice would depend on any number of other factors (your choice of genre, audience, the specific ideas you have in mind, etc.).
With limited resources (artistic talent and money), how do I create some of the art resources I need (besides stock art)? Or should I not worry about that until I have a finished product?
Peter Molyneux of Lionhead has said that he used to prototype all of his games with ascii art. If they were fun with that limited level of graphics, then they were worth continuing. While you might not be able to get that simple with some genres (and an ascii 3d shooter might be novel for it's own sake...), certainly worrying about finished art, audio, or other assets is something that can be saved for much later. In the end, if the game is fun to play, then getting an artist willing to work with you becomes a lot more trivial. I have, on many occasions, used copyrighted assets during my prototyping phase when I just couldn't stand to look at my own art any longer. Just be sure that you have anything like that replaced before you start distributing your project...
Finally, how do you break barriers in your code (may it be knowledge, time, or perseverance)?
Not sure exactly what you're thinking about here, but breaking barriers is what writing code is all about really. You're finding creative solutions to interesting problems. The web is a great resource for general knowledge, but in the end, it's your own time and perseverance that will create something interesting.
Best of luck.

What are some key aspects you've noticed in a successful personal project?
Don't worry about how it will turn out
Do it for fun
Don't plan it
Start small, don't think big
Actually code instead of procrastinating on so or other similar sites

If you want somebody else to use this game, make it non personal. Get somebody else who you can give demos to and get feedback from. Make the game very very simple. Build that first before moving onto something very simple. Then go for simple!
not sure :)
Plan with pen and paper sketches for window layouts and game flow. For look and feel, browse the web for slick looking sites you can take inspiration from.
Use notepad, and write the shortest list of things that need doing to make the simplest working demo. Only work on the things you wrote down otherwise you can get carried away on some detail and end up not producing anything useful.
Art is an important part of a game, and you can't have a finished product without it. How about searching the web for a collaborator who'd be interested in doing it for you.
If your stuck with something in your code, ask for help on StackOverflow! :)

Key aspects: Primarily, something that you are excited about. Make a game that REALLY gets you going, and you'll be motivated by your desire to share the finished product with others -- your end product's quality will probably be improved by your passion, as well.
Profitability: Well, since this is XNA and all, if you release a version on Xbox Live, you will earn 70% of the sales revenue that your game earns. Games can be priced for 200, 400, or 800 points (roughly $2.50-$10).
UI/Usability: For this, I typically go through a lot of playability testing...booting it up, seeing how things "feel," then tweaking things if they just don't feel right. I came across a lot of things that I never really thought about before (fade-ins/fade-outs for menus, repeat rates for keys, and so forth), and play-testing is one of the more effective ways that I've been able to identify issues like that. If you've got friends, you can ask them to give you feedback as well.
Planning Tools: Truthfully, I just make a list of the things that I want to include in the overall game concept, then slowly get more granular as I iron out more and more of the fine details. Certain ideas inevitably change after playtesting may reveal them to not be as fun as you had imagined they would be, of course. I haven't ever seen a huge need for any formal tools for this process, though, but I will give the disclaimer that all of my projects have been fairly small in scope and have not been professional-sized undertakings or anything.
Art: This may not always be an option for you, but I have a few friends online that are gifted with art skills -- I solicited interest, and got some volunteers. I presented the option to earn a cut of whatever royalties may come in, but most of them were happy with an attribution in the credits and a chance to see their work in a game that's playable by people all over the world. Regarding small things (icons, etc), stock/royalty-free art can do the job in certain cases, too, but I wouldn't recommend relying on it.
Breaking Barriers: Not sure what's quite meant by this question, but generally speaking if I've come up against some sort of unforeseen issue (technical/implementation challenge,
gameplay-related, or what-have-you), I'm able to either solve the problem or come up with a suitable work-around if I put my mind to it for long enough. Did you mean something else by this?

Some great thoughts given in these articles:
http://www.flatredball.com/frb/docs/index.php?title=Tips_for_Completing_an_Independent_Project
http://makeitbigingames.com/2006/06/five-realistic-steps-to-starting-a-game-development-company/
http://makeitbigingames.com/2006/02/five-foundational-steps-to-surviving-as-a-game-developer/

First of all - I know universities don't tend to teach this but you really must do it... get source control going. I would suggest installing TortoiseSvn from here: http://tortoisesvn.net/downloads
With TortoiseSvn (a subversion client) - you can create a repository on your USB Flash Drive and carry it around with you, so you always have a copy of your project files. More importantly you can have more confidence when you make changes - because it keeps a history of the changes you've made. So if you break something, you can revert to the most up to date version on your usb flash drive... or even an earlier version if you wish. It also provides ways of showing differences between file revisions, all from the explorer shell extension.
For a personal project the key thing is to keep motivated. You keep motivated by setting yourself attainable tasks. Aim for something attainable. Always make sure you can build your project. Concentrate on adding new functionality while keeping the old stuff still working rather than building up huge nicely designed framework that might end up going no where. If you write your code well, you can refactor and abstract later after you figured out what game it is you actually want to build.
I would advocate a todo list of some kind... something like the spreadsheet found here on Joels site: http://www.joelonsoftware.com/articles/fog0000000245.html
If you want to get fancy, I would suggest something like Trac: http://trac.edgewall.org/
Trac is a wiki, to-do / bug list and repository viewer all in one. If you get someone else working with you on a project, collaboration with source control and a shared wiki becomes more useful. It's also good to work with others... you can keep each other motivated.
The Wolf fire blog recently had some interesting design ideas and cited serval sources here: http://blog.wolfire.com/2009/02/design-principles-from-tufte/
As for getting money, others have already covered the community games thing.
Art assets - there are a few places to find these. Scout the forums at www.gamedev.net. Def look at this blog here for some excellent art (and game design ideas) http://lostgarden.com/
Others have suggested keeping tools simple - I would agree. The simplier and more accessible things are when it comes to project planning, prototyping and design - the more likely you are to use them. As long as you can communicate with yourself when you revisit something... or other people when collaborating, it's a good tool. Set yourself attainable goals, and get on with it :-)

What are some key aspects you've noticed in a successful personal project? Focus on something that actually works.
How did you (successfully) draw revenue the project (besides asking for donations)? Not directly. However, skills get you a job. Build skills.
How do you plan non-programming aspects like window layouts (UI), flow of game, and look and feel?
Build the inside -- the core data model first. If the data model works, everything else is far simpler.
Build unit tests for the model to be absolutely sure it always works.
Work out "flow" and other control issues based on that model.
Build the GUI later, once you have a solid foundation.
How do you bring an abstract idea of a game and making into a real product? What tools do you use in your sketching and planning?
I do the following.
I use Argo UML for technical UML diagrams.
I write documents with my end-state in mind. What the API will be like. How it will work. As part of writing the documents, I clarify my thinking.
I write code in pieces, with unit tests, and refactor ruthlessly.
With limited resources (artistic talent and money), how do I create some of the art resources I need (besides stock art)? Or should I not worry about that until I have a finished product?
Find an artist who wants to help.
Finally, how do you break barriers in your code (may it be knowledge, time, or perseverance)?
Build the inside -- the core data model first.
Spike elements of the technology separately from the main development. Build separate proofs of concept for each new technology. Do not try to integrate unknown technology into the final product. (Many of my customers try this and it doesn't work.)

Having had an experience with game development in my teenage years what I can suggest is the following:
Make sure you have the story ready, or at least the basic points before you start implementing. Also make sure that you do not change the genre of the game while in dev stage. (yeah, I know, this is stupid, but could happen ;-) )
We actually drew some revenue when we managed to sell the game to a software house, however because we had already spent too much in the dev process, we just broke even (i.e. to little profit). So be careful what you spend beforehand
From what you said, you're still in school, so take advantage of that. There will surely be folk out there with artistic talent, most probably friends of yours. Make a team. That's what we did and it was all the way back in 1993! At that time it was even harder. We introduced our talented friends to 3D-Studio (v3 back then) and some even to computers in general, and believe me, before you knew it they were making brilliant art out of it!
Use pen and paper to create the story boards (even better with the friends mentioned above. Decide on the development approach. I.e. what libraries will you use etc. Read a lot about game dev. Make a plan of the project. Give it your best ;-)
Unlike us back then, you have the web to help you. For whatever problems you have with your code it is easy to look it up. StackOverflow didn't exist back then!
And of course make sure you have fun in the process otherwise it will eat you alive!

What are some key aspects you've
noticed in a successful personal
project?
Just do it!