We Keep Coding

How can I find out what is creating garbage? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
This is a really general learning-based question, not a technical problem.
I'm making a game in Unity. The game involves a lot of pretty complex, high-object-count processes, and as such, I generate a lot of garbage. As everyone who makes games that use managed code knows, garbage collection spikes are a HUGE buzzkill. I am also trying to target mobile devices, so this pain is amplified.
My goal is simple: track down the procedures that generate the most garbage and pool and reuse the objects involved in order to reduce the load on the garbage collector. I have tackled all of the obvious classes, but my GC issues persist. If anyone can offer some greater wisdom here, please bring it on.
What I have been unable to track down in my research is a good way to measure GC load and track down problem spots in the code itself. I can get metrics from Unity and from diagnostics classes about the size of the memory pool (though I'm no expert on what any of those numbers really mean) and I can display the total number of GC collections that have happened since the start, but my knowledge of debugging this issue kind of ends there.
I know there MUST be better ways to solve this problem. Can anyone point me at the right tools or framework classes (or libraries?) that can help me out here? I've seen some mention of a debugger program (sgen?) that's part of Mono, but I can't find a download or let alone a guide for how to hook it up to a Unity game.
Any help would be greatly appreciated. Thanks.

For C# and Unity3D specifically, the profiler built into Unity3D is the best tool you'll have available to you. You should be able to get very far with the Unity Profiler. Specifically the CPU Profiler:
I have highlighted the most important column - GC Alloc - Which, on a frame-by-frame basis, shows memory that has been allocated for garbage collection. The aim of the game here is to make that column as consistently close to zero as possible.
I'd suggest spending a few hours in there, playing your game, pausing it and digging into areas of your code where GC Alloc is showing numbers.
Finally, I strongly suggest you check out this video tutorial by one of the developers of Unity on memory debugging and using the Unity Editor profiler properly.

Caveat: it's all down to details: general principles are nice but you're right - real data is the right thing. Unfortunately it's hard to get for Unity: I've seen reports that DotTrace works for some aspects of Unity games but that most of Unity looks like a black box to it.
A useful rule of thumb is to look for new statements. It's a rule of thumb, not a 100% scientific principle (structs are created with new but they go on the stack... mostly...) but it's a good leading indicator of possible causes of fragmentation. If you "new" something up and let if just disappear ( go out of scope, be dereferenced, etc) it will end up in the garbage collector. Moreover if you new up a batch of items and dereference some of them you'll also fragment the heap which makes the collection spikes worse as the collector tries to defragment the memory.
This is why most recommendations in this space will be to use pools, as you're already doing.
Here are some general purpose links that might be useful in tackling memory management in Unity:
http://www.gamasutra.com/blogs/WendelinReich/20131109/203841/C_Memory_Management_for_Unity_Developers_part_1_of_3.php
http://andrewfray.wordpress.com/2013/02/04/reducing-memory-usage-in-unity-c-and-netmono/

How can I obfuscate my c# code, so it can't be deobfuscated so easily? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Closed 9 years ago.
Improve this question
I release a bunch of tools for free, but recently I have began to sell an application, that has private research, and people have been able to get my source code, and one person publically released my entire source. I spent a lot of time on this program, just to have someone crack it, and release my entire source.
How can I go about protecting my program? I have tried HWID, but people are still able to crack it. I know I am limited by C#, but it the most convenient to use. I just need a way to protect my programs from people trying to do this.

You could have a look at the many obfuscation tools that are out there, for example these:
http://gapotchenko.com/eazfuscator.net
http://orangeheap.blogspot.nl
http://confuser.codeplex.com (succeeded by https://yck1509.github.io/ConfuserEx/ and then again by https://github.com/XenocodeRCE/neo-ConfuserEx)
http://ntoolbox.com)

Well, the problem with languages like C#/Java is that they are generally much easier to de-obfuscate. The way to secure this is generally to put this stuff into a webservice, but you said you couldn't really do that. What about porting specific non-trivial functions over to a language like C, and obfuscate that. Your C# program could then use reflection to make calls to this external/unmanaged dll. It would increase the difficulty for de-obfuscating, but the problem is that if someone wants it bad enough, they can figure it out as it is client-side.
It may be that legal action is the only real solution here, but this is not a site for legal advice, and I am not qualified to give it if it were.
Additionally, this could be a business decision. Consider Making your software open-source and post a donation link. I am also not qualified to give business advice, but this is worth considering. It may actually increase your revenue, not to mention the other benefits that come with releasing open-source software.

There's Dotfuscator (http://www.preemptive.com/products/dotfuscator/overview). But the best solution in some cases is to offer what you do as a web site (Software as a Service).
Consider this tool for example to convert VB to C#: http://converter.telerik.com/
Or this tool to format JSON: http://jsonformatter.curiousconcept.com/
This may or may not work for you. I don't know what the nature of the software you're trying to protect is.

No, your code needs to contain the information needed to decrypt itself, its an impossible problem to solve.
Your best solution is to put your intellectual property on a server in the cloud that only you have access to. Give your customers a unique login, audit their access to check for abuse, off load as much grunt work onto your customers machines, but keep your algorithms locked into the cloud.

Steps to open source a small project [closed]

Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 9 years ago.
Improve this question
I've been working for a couple of years on a small project, almost by myself, with the eventual help of some colleagues. The project is getting out of my hands, because the size of the code is growing (around 20K lines now) and the initial expectations I had for it have outgrown my own ability and time. So now I want to open source it, with the hope to attract some contributors. My motivations for going open source are these:
The project is rather academic (a library of algorithms for scientific computing), and I don't really have any economic interest in it.
The project is getting too big for me to handle it by myself, and the number of features I've planned are enough to keep a small team motivated (I think).
It needs a lot of testing, not just unit testing, but testing in the real world to see if the API is easy to use, the performance is as expected, etc.
I'm sure it has a lot of bugs, but I can only find a few, since its me alone testing it.
It needs proper documentation, because the API is getting a bit complex.
Other than that, I think that the project could benefit from a comunity in terms of deciding which features are most needed, and creating a set of guidelines for the future development.
I'm using Git, so my first thought was to publish it on Github and/or Codeplex. Besides that, what would be the steps to help to slowly grow a community of users and perhaps developers around it? Do I need a domain of my own, or should I stick to Github/Codeplex? How do I set up a platform for collaboration between developers potentially geographically separated? Should I set up a mailing list? And most important, how do I attract people to use it and collaborate with it?
The project is a .NET library for optimization and machine learning, written in C#.

There is only one piece of advice I can give here; use Github. It is common, (pretty much) everyone knows about it, it is easy to use, and the community who you are trying to attract is already on it. It has a ton of tools which you may not have even thought about, but may come in handy. It it pretty much the perfect solution for what you're looking to do, so don't overthink it.
As for attracting people to use it and contribute, if it is something that is useful and good, people will find it. I have found a ton of obscure projects with a simple google. If someone googles for something related to your project (and it is appropriate named and such) they will likely find it. There isn't really much you can do to force a demand though, just let it happen. As for contributors, people who are using it will likely contribute they're additions back. Just be sure to stay actively involved in managing it (monitoring pull requests, etc). If no one is accepting requests or managing versions, contributors will likely start to give up on your project.

Usage of GC.Collect Method in c# [duplicate]

This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
When is it acceptable to call GC.Collect?
Is Using of GC.Collect Method necessary and is it a good practice?
If yes, when should I use this method?
If No, Why?
Edit: Accourding to MSDN What is the meaning of: Use this method to try to reclaim all memory that is inaccessible.

As a general rule... don't use it.
GC.Collect force a collection by the garbage collector, which, among other things, means pausing all the threads of the program so that the garbage collector can verify which objects are no longer referenced and claim the unused memory.
Usually the GC will automatically decide when he should collect memory, considering when your program is idle or if the allocated memory (virtual memory) is getting low so in order to allocate more it needs to free some. In my experience the .NET GC (as in Microsoft) is pretty intelligent and does what it does ratter well. Other GCs (as in mono) I don't have experience with them, but still probably will do a better job than the developer deciding when to perform a collection.
That, of course, has a good impact in performance and, as with many other situations, it knows better than you do the best time to perform a collection (in 99% of the cases that's true). So no, it's not a good practice and you should only do it when you have a really good reason to do it... a deep understanding of what it does and the possible consequences it may have.

Not generally recommended; not generally considered good practice.
In general the GC knows better than you what needs cleaning up, and when. Best practice is to leave the GC alone to do its business unhindered unless you have concrete proof that the GC's strategy is causing problems that can be solved by forcing it to behave differently.

It is usually not necessary, and usually not a good idea to use it.
There are times when you need to use it but usually you are forcing a collect at a not optimal time, which have a performance hit.
There is good discussion on the matter here and here.

I have explained a few things relevant here:
WPF memory leak

Stack vs. Heap in .NET [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
In your actual programming experience, how did this knowledge of STACK and HEAP actually rescue you in real life? Any story from the trenches? Or is this concept good for filling up programming books and good for theory?

The distinction in .NET between the semantics of reference types and value types, is a much more important concept to grasp.
Personally, I have never bothered thinking about the stack or heap in all my years of coding (just CLR based).

To me it is the difference between being a "developer/programmer" and a "craftsman". Anyone can learn to write code and see how things just "magically happen" for you not knowing why/how. To really be valuable at what you do, I think there is a great importance to find out as much as you can about the Framework you're using. Remember it's not just a language, it's a Framework that you leverage to create the best application to your abilities.
I've analyzed many memory dumps over the years and found it extremely helpful knowing the internals and differences between the two. Most of these have been OutOfMemory conditions and unstable applications. This knowledge is absolutely necessary to use WinDbg when looking at dumps. When investigating a memory dump, knowing how memory is allocated between the kernel/user-mode process and the CLR can at least tell you where to begin your analysis.
For example, let's take an OOM case:
The allocated memory you see in the Heap Sizes, Working Set, Private Memory, Shared Memory, Virtual Memory, Committed Memory, Handles, and Threads can be a big indicator of where to start.
There about 8 different heaps that the CLR uses:
Loader Heap: contains CLR structures and the type system
High Frequency Heap: statics, MethodTables, FieldDescs, interface map
Low Frequency Heap: EEClass, ClassLoader and lookup tables
Stub Heap: stubs for CAS, COM wrappers, P/Invoke
Large Object Heap: memory allocations that require more than 85k bytes
GC Heap: user allocated heap memory private to the app
JIT Code Heap: memory allocated by mscoreee (Execution Engine) and the JIT compiler for managed code
Process/Base Heap: interop/unmanaged allocations, native memory, etc
Finding what heap has high allocations can tell me if I have memory fragmentation, managed memory leaks, interop/unmanaged leaks, etc.
Knowing that you have 1MB (on x86)/ 4MB (on x64) of stack space allocated for each thread that your app uses reminds me that if I have 100 threads you will have an additional 100MB of virtual memory usage.
I had a client that had Citrix servers crashing with OutOfMemory problems, being unstable, slow responsiveness when their app was running on it in multiple sessions. After looking at the dump (I didn't have access to the server), I saw that there were over 700 threads being used by that instance of the app! Knowing the thread stack allocation, allowed me to correlate the OOMs were caused by the high thread usage.
In short, because of what I do for my "role", it is invaluable knowledge to have. Of course even if you're not debugging memory dumps it never hurts either!

It certainly is helpful to understand the distinction when one is building compilers.
Here are a few articles I've written about how various issues in memory management impact the design and implementation of the C# language and the CLR:
http://blogs.msdn.com/ericlippert/archive/tags/Memory+Management/default.aspx

I don't think it matters if you're just building average business applications, which I think most .NET programmers are.
The books I've seen just mention stack and heap in passing as if memorizing this fact is something of monumental importance.

Personally, this is one of the very few technical questions that I ask every person I'm going to hire.
I feel that it is critical to understanding how to use the .NET framework (and most other languages). I never hire somebody who doesn't have a clear understanding of memory usage on the stack vs. the heap.
Without understanding this, it's almost impossible to understand the garbage collector, understand .NET performance characteristics, and many other critical development issues.

The important distinction is between reference types and value types. It's not true that "value types go on the stack, reference types go on the heap". Jon Skeet has written about this and so has Eric Lippert.

We had a Claim Entity (business Object) which contained data for an entire claim. One of the requirements of the application was to create an audit trail of every single value changed by the user. In order to this without hitting the database twice we would maintain Original Claim Entity in the form and a Working Claim Entity. The Working Claim Entity would get updated when the user clicked Save and we would then compare the Original Claim Entity properties with corresponding Working Claim Entity properties to determine what changed. One day we noticed hey our compare method is never finding a difference. This is where my understanding of the Stack and Heap saved my rear end (specifically value types vs reference types). Because we needed to maintain to copies of the same object in memory the developer simply created two objects
Dim originalClaim As ClaimBE
Dim workingClaim As ClaimBE
then called the business layer method to return the claim object and assigned the same claimBE to both variables
originalClaim = BLL.GetClaim()
workingClaim = originalClaim
hence two reference types pointing to the same value type. Nightmare averted.