I have dictionary of lists of objects, and for each list I need to load a bitmap using the associated key, and for each object in the list, calculate a transform, clone the bitmap using the transform, and save the result. The amount of objects in each list varies from 60,000 to one sometimes, and doing it serially will most likely take forever. So, I tried parallelizing the for each loop in each list, but unfortunately, the Bitmap class holds a global lock, and so, to avoid an exception, I have to place a lock on the cloning and saving code, which really defeats the point of using a parallel loop in the first place, as the cloning and saving is the most time-consuming part of the process. Is there a way I can get a subimage of a bitmap in multiple threads?
It turns out the real answer, for me at least, is to drop GDI+, and use Magick.NET, which is better designed for this. Now it works perfectly, with multithreading and the expected speedup.
Related
In my game I can use a list of game objects or tags to iterate but i prefer knows what is the most efficient way.
Save more memory using tags or unity requires many resources to do a search by tag?
public List<City> _Citys = new List<City>();
or
foreach(GameObject go in GameObject.FindGameObjectsWithTag("City"))
You're better of using a List of City objects and doing a standard for loop to iterate over the 'City' objects. The List just simply holds references to the 'City' objects, so impact on memory should be minimal - you could use an array of GameObjects[] instead of a List (which is what FindGameObjectsWithTag returns).
It's better for performance to use a populated List/Array rather than searching by Tags and of course you're explicitly pointing to an object rather than using 'magic' strings -- if you change the tag name later on then the FindGameObjectsWithTag method will silently break, as it will no longer find any objects.
Also, avoid using a foreach loop in Unity as this unfortunately creates a lot of garbage (the garbage collector in Unity isn't great so it's best to create as little garbage as possbile), instead just use a standard for loop:
Replace the “foreach” loops with simple “for” loops. For some reason, every iteration of every “foreach” loop generated 24 Bytes of garbage memory. A simple loop iterating 10 times left 240 Bytes of memory ready to be collected which was just unacceptable
EDIT: As mentioned in pid's answer - measure. You can use the built-in Unity profiler to inspect memory usage: http://docs.unity3d.com/Manual/ProfilerMemory.html
Per Microsoft's C# API rules, verbs such as Find* or Count* denote active code while terms such as Length stand for actual values that require no code execution.
Now, if the Unity3D folks respected those guidelines is a matter of debate, but from the name of the method I can already tell that it has a cost and should not be taken too lightly.
On the other side, your question is about performance, not correctness. Both ways are correct per se, but one is supposed to have better performance.
So, the main rule of refactoring for performance is: MEASURE.
It depends on memory allocation and garbage collection, it is impossible to tell which really is faster without measuring.
So the best advice I could give you is pretty general. Whenever you feel the need to enhance performance of code you have to actually measure what you are about to improve, before and after.
Your code examples are 2 distinctly different things. One is instantiating a list, and one is enumerating over an IEnumerable returned from a function call.
I assume you mean the difference between iterating over your declared list vs iterating over the return value from GameObject.FindObjectsWithTag() in which case;
Storing a List as a member variable in your class, populating it once and then iterating over it several times is more efficient than iterating over GameObject.FindObjectsWithTag several times.
This is because you keep your List and your references to the objects in your list at all times without having to repopulate it.
GameObject.FindObjectsWithTag will search your entire object hierarchy and compile a list of all the objects that it finds that matches your search criteria. This is done every time you call the function, so there is additional overhead even if the amount of objects it finds is the same as it still searches your hierarchy.
To be honest, you could just cache your results with a List object using GameObject.FindObjectWithTag providing the amount of objects returned will not change. (As in to say you are not instantiating or destroying any of those objects)
I'm making a game and need to store a fair bit of animation data. For each frame I have about 15 values to store. My initial idea was to have a list of "frame" objects that contain those values. Then I thought I could also just have separate lists for each of the values and skip the objects altogether. The data will be loaded once from an XML file when the game is started.
I'm just looking for advice here, is either approach at all better (speed, memory usage, ease of use, etc) than the other?
Sorry if this is a dumb question, still pretty new and couldn't find any info on stuff like this.
(PS: this is a 2D game with sprites, so 1 frame != 1 frame of screen time. I estimate somewhere around 500-1000 frames total)
If the animation data is not changing, you could use a struct instead of an class, which combines the "namespacing" of objects with the "value-typeness" of primitives. That will make all the values of a single frame reside in the same space, and save you some memory and page faults.
Just make sure that the size of your arrays doesn't get you into LOH if you intend to allocate and deallocate them often.
you are creating an animation So you should keep in mind a few basic things of.net:
Struct is created on stack. So, it is faster to instantiate (and destroy) a struct than a class.
On the other hand, every time you assign a structure or pass it to a function, it gets copied.
you can still pass struct by reference simply by specifying ref in the function parameter list
So it will be better to keep in mind this things
In my case I have 2D ArrayList full of objects that have data which is used when drawing horizontal lines, vertical lines and filled rectangles. In different cases the amount of objects is different and the image has a different amount of drawn lines and rectangles. But the image needs to be redrawn sometimes. The more stuff there is to redraw, the stronger the flashes on the screen (I hope you can figure out what I mean with "flashes").
I have already tried double buffering, but I don't think I'm doing it right- the type of the flashes only changes... But that's not the current case.
I have read that it's easy to add and remove elements (no matter the type) anywhere from the ArrayList, but the access to them is hard. I have also read that the List performs better that the ArrayList (does that mean that the program will require less resources fro the computer?), but it's only easy to add and remove elements from the end. But I'm not sure if the Array's performance is the fastest. And this is making me think that the flashes can become weaker if I change the ArrayList with an Array or List.
It won't be a problem in my case because all the elements that I store in the ArrayList are form the same class.
My question is: Can I weaken the flashes if I replace the ArrayList with an Array or List?
OK, two problems here. The ArrayList vs. List question has no effect on "flashing", since the difference in performance is small. If your list items are all of the same type or if they all derive from a common base type other than object or if they all implement a common interface, then List<T> is the better choice. This is because less castings or even boxings/unboxings are involved. In addition, it will be easier to access your items.
Flashing: Do all your drawing in a paint-event method (of your form or of some control). Then call the Invalidate method of this form or control. Don't draw "directly". As an improvement, you can pass a rectangle structure to Invalidate, telling which part has to be redrawn. In the paint method, you can then check e.ClipRectangle, which tells you which part has to be redrawn. This gives you the opportunity to make some improvements here. However, be aware of the fact, that the Windows OS itself can trigger an Invalidate, which results in arbitrary ClipRectangles.
Comparing drawing performance with reading from ArrayList performance, you can't solve flickering problem by using another data container. So, alternative container of ArrayList is not the answer. You need to use some other methods like;
Using SDL, DirectX, OpenGL like graphics library.
Drawing to a bitmap buffer and showing the bitmap after it is completed (similar to double buffering but sometimes it works better).
Subclassing some components and overriding some methods like background drawing.
There are even more methods but the answer of your question is definitely not much to do with ArrayList.
You should use a List<T> instead of ArrayList, just to get rid of the unnecessary casting when you read objects from the list. However, the performance gain from that is so small compared to drawing graphics that it won't have any noticable effect on your update problem.
You could consider drawing your graphics to a bitmap, then draw the bitmap when you need to update the screen.
A List<T> and an ArrayList behave the same when it comes to adding and removing elements; it's cheap to add or remove elements at the end of the list, but more expensive to insert or remove elements at the beginning of the list.
An array is the fastest of the lists, and both List<T> and ArrayList uses arrays to store its elements internally. However, you can't resize an array so you might still want to use a List<T> as it does the work of allocating arrays as needed and keeps track of how much of the array is used.
To fix the "flashes", you need to enable double-buffering.
You should not use ArrayList because it encourages mistakes and/or poor design.
You should not have lists that contain multiple types of objects.
Instead, you should use List<T>, perhaps with a polymorphic base class.
My code has to generate millions object to perform some algorithm (millions objects will be created and at the same time 2/3 of them should be destroyed).
I know that object creation causes performance problems.
Could someone recommend how to manage so huge amount of objects, garbage collection and so on?
Thank you.
Elaborating a bit on my "make them a value type" comment above.
If you have a struct Foo, then preparing for the algorithm with e.g. var storage = new Foo[1000000] will only allocate one big block of memory (I 'm assuming the required amount of contiguous memory will be available).
You can then manually manage the memory inside that block to avoid performing more memory allocations:
Keep a count of how many slots in the array are actually used
To "create" a new Foo, put it at the first unused slot and increment the counter
To "delete" a Foo, swap it with the one in last used slot and decrement the counter
Of course making an algorithm work with value types vs reference types is not as simple as changing class to struct. But if workable it will allow you to side-step all of this overhead for an one-time startup cost.
If it is possible in your algorithm then try to reuse objects - if 2/3 are destroyed immedietly then you can try to use them again.
You can implement IDisposable interface on the type whose object is been created. Then you can implment using keyword and write whatever logic involving the object within the using scope. The following links will give you a fair idea of what i am trying to say. Hope they are of some help.
http://www.codeguru.com/csharp/csharp/cs_syntax/interfaces/article.php/c8679
Am I implementing IDisposable correctly?
Regards,
Samar
I'm using the System.Drawing classes to generate thumbnails and watermarked images from user-uploaded photos. The users are also able to crop the images using jCrop after uploading the original. I've taken over this code from someone else, and am looking to simplify and optimize it (it's being used on a high-traffic website).
The previous guy had static methods that received a bitmap as a parameter and returned one as well, internally allocating and disposing a Graphics object. My understanding is that a Bitmap instance contains the entire image in memory, while Graphics is basically a queue of draw operations, and that it is idempotent.
The process currently works as follows:
Receive the image and store it in a temporary file.
Receive crop coordinates.
Load the original bitmap into memory.
Create a new bitmap from the original, applying the cropping.
Do some crazy-ass brightness adjusting on the new bitmap, maybe (?) returning a new bitmap (I'd rather not touch this; pointer arithmetics abound!), lets call this A.
Create another bitmap from the resulting one, applying the watermark (lets call this B1)
Create a 175x175 thumbnail bitmap from A.
Create a 45x45 thumbnail bitmap from A.
This seems like a lot of memory allocations; my question is this: is it a good idea to rewrite portions of the code and reuse the Graphics instances, in effect creating a pipeline? In effect, I only need 1 image in memory (the original upload), while the rest can be written directly to disk. All the generated images will need the crop and brightness transformations, and a single transformation that is unique to that version, effectively creating a tree of operations.
Any thought or ideas?
Oh, and I should probably mention that this is the first time I'm really working with .NET, so if something I say seems confused, please bear with me and give me some hints.
Reusing Graphics objects will probably not result in significant performance gain.
The underlying GDI code simple creates a device context for the bitmap you have loaded in RAM (a Memory DC).
The bottleneck of your operation appears to be in loading the image from disk.
Why reload the image from disk? If it is already in a byte array in RAM, which it should be when it is uploaded - you can just create a memory stream on the byte array and then create a bitmap from that memory stream.
In other words, save it to the disk, but don't reload it, just operate on it from RAM.
Also, you shouldn't need to create a new bitmap to apply the watermark (depending on how it'd done.)
You should profile the operation to see where it needs improvement (or even if it needs to be improved.)
The process seems reasonable. Each image has to exist in memory before it is saved to disk - so each version of your thumbnails will be in memory first. The key to making sure this works efficiently is to Dispose your Graphics and Bitmap objects. The easiest way to do that is with the using statement.
using( Bitmap b = new Bitmap( 175, 175 ) )
using( Graphics g = Graphics.FromBitmap( b ) )
{
...
}
I completed a similar project a while ago and did some practical testing to see if there was a difference in performance if I reused the Graphics object rather than spin up a new one for every image. In my case, I was working on a steady stream of large numbers of images (>10,000 in a "batch"). I found that I did get a slight performance increase by reusing the Graphics object.
I also found I got a slight increase by using GraphicsContainers in the Graphics object to easily swap different states into/out of the object as it was used to perform various actions. (Specifically, I had to apply a crop and draw some text and a box (rectangle) on each image.) I don't know if this makes sense for what you need to do. You might want to look at the BeginContainer and EndContainer methods in the Graphics object.
In my case, the difference was slight. I don't know if you would get more or less improvement in your implementation. But since you will incur a cost in rewriting your code, you might want to consider finishing the current design and doing some perf tests before rewriting. Just a thought.
Some links you might find useful:
Using Nested Graphics Containers
GraphicsContainer Class
I am only going to throw this out there casually, but if you wanted a quick 'guide' to best practices for working with images, look at the Paint.NET project. For free high-proformance tools for doing image manipulation, look at AForge.NET.
The benefit of AForge is to allow you to do alot of these steps without creating a new bitmap every time. If this is for a website, I can almost guarentee that the code you are working with will be the performance bottleneck for the application.