It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 10 years ago.
I'm writing trading software and I need every single microsecond in speed.
What can I do? I was thinking to use ngen but wikipedia tells that JIT might be even better. What can I tune? May I force it to use Xeon E5 instructions somehow? Will changing priority in Windows to highest help and if so how to run program always with highest priority? May I add this program to "trusted" so .NET will not check security etc.
I.e. I want complete list of configurations in hardware/software/.net and actions (like running ngen) which can affect and help run program faster.
The time comes stop guessing and find where the problem(s) actually are.
Use EQATEC free preformance analyzer to figure out your bottlenecks and fix them.
NGEN is usefull to boost the startup time of the application , but it's definitely not a golden key for that problem. It's most probabble that you will fix it profiling your app.
What about runtime optimization:
checkout DB accesses (if any), optimize your queries and minimize the data retrived to the amount you really need
look on disk access operations
look on CPU consuption. After profiling yoi can use Process Explorer to check CPU and Memory consuption from your application behavioural point of view
after profiling identify unnecessary or heavy iteration you made (if any), and make use of dictionary (just example) for O(1) access
... and more...
Like a literature for reading on performance can suggest definitely the monster blog of one of the greatest performance specialists in IT industry: Rico Mariani's Performance Tidbits
Hope this helps.
Not windows, pick a more real-time focused OS, as speed has several components, consistent speed or best possible?
Tuning code is nice and all, but don't always go for that. Try code reviews, correcting large logical errors or extra work is much more beneficial than low level tuning.
Understand that every microsecond counts, but your CPU and logic are the fastest part, network, disk, paging, etc are more than likely the real enemies. So make sure you are focusing on the right thing. You must stop all I/O which is not needed.
Certain drivers, network cards, etc are faster than others.
Review your use of libraries and make sure you understand their performance aspects, so much run time is within libraries etc.
Sure there are more .NET centric answers and specific ones, but I tried answering the question behind the question.
Eliminate GC!
Another "trick" that people use is in trading systems is to write C# code that does not garbage collect. Obviously it is impossible to remove garbage collection entirely, but what you can do is minimize or eliminate GC once your program is running. You do all the work to setup your app, initialize your components, etc in an initialization phase and allow any GC here. But once you are setup and ready to run, you make sure that the code does not generate garbage and does not perform boxing/unboxing. (Have a search on SO for like "avoid GC" and you will find some useful information).
The term "trading system" can encompass a myriad of different things. Are you talking about writing an algo in .NET? Or is your algo written in something else and there is .NET framework that is hosting it? Are you just talking about the UI? Is your trading system distrubuted? Do you know how fast you program needs to be? If you are not trading a High Frequency model then what is "fast enough"? Don't just push for something that has to operate in the 1-2ms scale when your trading strategy does not need it.
And importantly, don't throw out the tried and trusted OO principles; SOLID still applies to trading systems although you may bend the rules in certain cases. Just make sure that you identify what needs to be performant and optimize that - don't think that you have to optimize everything and make sure you benchmark and measure everything so that you know what needs to be faster and by how much.
And Keep It Simple! A trading system does not have to be complex.
Related
It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 9 years ago.
I have a service, that I query once in a very long while, and i would like to "streamline", or improve the efficiency, of its memory allocation.
Most of the time it just sits and waits, and once in awhile it gets a request that requires allocating a lot of memory, and do some processing on it. I don't know the types or structure in advance - it depends on the request, and varies wildly.
Now, the big processing request is precluded by some chatter (other requests), that might take a few seconds.
What I want to do is, when the chatter (smaller requests) start, say to the .Net Framework: go to windows, and get yourself a couple of GB's of memory so it'll be available faster when i ask, and when I'm done, say to the .Net: everything I'm not currently using, you can give back, because I'm not going to need it for a while.
I'm starting profiling as we speak... but I suspect it would be part of the issues that could improve.
I'll try to clarify the situation.
I have a service that sits on a server and 95% of the time just does nothing. Once in a long while it gets a request to do some mostly memory intensive processing.
I know a little bit of time in advance that it's all going to happen.
All i want to do, is hint the GC "Were going to need a lot of memory soon" and later "Were not going to need anything special for a while"
OK.
I've done profiling, and decided I don't care about this.
The allocation does take some time (several to several dozens milliseconds), but it's insignificant versus the rest of the processing...
regarding the releasing part, it happens eventually, and doesn't really interfere with the rest of the server...
If you want to be able to reserve a chunk of memory for your uses then please see:
allocating "unmanaged" memory in c#
Note, doing so can be risky and the Garbage Collector and memory allocation in the .NET VM is already rather good.
If the memory allocation can be largely cached then I'd recommend caching what can be done so with WeakReference such that quick successive requests could benefit from accessing the cached data, but if a garbage collection comes in between requests spaced a decent amount apart then the data can be released and just re-created in the next request.
See: Weak reference benefits
And: http://msdn.microsoft.com/en-gb/library/system.weakreference.aspx
The GC is most of the time smart enough to do this for you. However, it is an architectural problem and can be dealt with by modifying the flow of activities in the service.
e.g. you can allocate the objects required for processing big request in advance before the request comes. However, for deallocating, either explicitly implement idisposible interface to them and destroy them once they are used or leave it to GC.
Further, you have to understand how memory allocation works. In order to get memory allocated for .Net objects, you must be knowing the type of object in advance. Just allocating plain block of memory is in no way helpful to you. Most of the time the object creation or cloning code is more resource consuming compared to the mallocs that the framework uses to allocate memory for the object.
Considering the details, I would say that even if you could successfully do this routine, it would become much more complex and might add few more bugs to your code. Better leave this to .Net framework. It is very good at allocating and deallocating memory.
It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 11 years ago.
We have a lot of old legacy Perl scripts that connect to a MS SQL db process some records, and make files. Over time, the daily transaction size grew and these scripts are becoming more and more expensive.
Moreover, the Databases grew with more and more tables, and modifying the old Perl scripts is cumbersome. Was thinking about redoing some of the major scripts under .NET (in C#)
Is there a speed advantage under a machine running Windows Server of using one vs the other?
Again the idea is
Execute Query
Process Results through some basic formatting
Write results to a file
Depends on how stupid the respective programmers are. When initialized properly they both should be comfortable saturating whatever bandwidth you throw at them disc system wise - and THERE is your bottleneck. Make large cached writes (.NET BufferedStream) and make ure you have a SSD or something fast ready. The perforamcne bottleneck with proper programming is the disc subsystem for this type of work.
Both tasks can be done equally fast in either language. They can also both be done horribly wrong and horribly slowly in both languages, so there is that to consider.
From another one of your comments, you mention that you do the formatting on the SQL server side. These queries would potentially be a lot less expensive if you did that on the app side, and then moved this script to a faster machine so as to impact the db server the least.
I'd guess that harddrive speed is your biggest problem now. You should monitor the resources while running this script -- Is it maxing out your cpu? is it reading/writing a lot to memory?(it shouldnt). Is it just waiting on disk i/o most of the time? If it is, you should look into upgrading your storage to either faster disks, a raid, or an ssd depending on what makes the most sense for your situation.
Even just something like defragging the disk might help.
If you have good cpu/memory to spare but cant avoid the slow disk, you could even look into compressing all of the output in memory before writing it (again, assuming this is a good idea, it really depends on where these reports are going and what format is needed).
I don't expect great differences between those languages when it comes to write performance (generally bottleneck will be hard disk, not processing power of CPU) . You should rather take a look on "costs of maintaining code", in C# you can get much cleaner code, not to mention that integration with MS SQL probably will be more efficient, not to mention that you can use threading.
Better Code, more maintainable, maybe faster. Yeah C# is good idea
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I am thinking about developing Winforms application that would use c library to preform all computations. I want to use C# because developing GUI with it is really easy, and C to increase performance. Are there other advantages or any disadvantages of combining those two languages?
Edit: As computations I mean mainly (but not limited to) graph algorithms, like coloring, dijakstra, maximum flow; I expect graphs to be huge, and performance is really crucial.
and C to increase performance
You could write pretty well performing applications using pure managed code. Take for example this very same site. It's pretty darn fast, don't you think?
The way I see things is the following: mix those two technologies only if you are really some kind of a performance maniac (someone that tries to solve the scaling problems that Google does for example) or if you have some existing C codebase that you cannot port to .NET immediately and that you have to interoperate with.
In other cases, managed code is safer. If you are sick about performance don't forget that in many cases the cost of marshaling between managed and unmanaged code will be higher than the performance you would gain from pure unmanaged code. Also the JITter gets smarter and smarter with each version of the framework and maybe one day it will generate almost as efficient code as unmanaged code.
So, yeah, go for managed and leave C to the maniacs :-)
I would not be so sure that C will be faster than C# for what you are going to do. The jitted code is extremely fast. Also, if you are doing lots of calculations, by using c#/.net, you could potentially take advantage of the Task Parallel Library, which would greatly ease parallelizing your calculations. And really, with all of the multi-core machines, I think that you will get a lot more bang for your bucks if you can take advantage of multiple cores (of course, straight C can thread as well, but the TPL in .net 4 is much more productive than the base WinApi)
One other thing that you might consider, and I am speaking from outside of my own experience, so definitely do your own research: I have heard that F# can be a very good choice for writing scientific sets/calculation libraries. I haven't written a line of F# myself, but from what I understand, the language design supports writing the calculations in a way that makes them extremely parallelizable.
(of course, now you have a new problem -- learning F# ;)
there is a post here that you might check out: F# performance in scientific computing
Getting fixed memory locations, and also converting your indexes to C pointers could have negative side effects. You can try both safe and unsafe code for sure and try to see the results but my guess is that safe code will perform better at the end. For sure, your computation is also important factor.
I have done this in the past, I would say that as other mentioned the cost of marshalling is not free so you need some pretty specialised need before this is a valid option.
My only advice to you would be to start in the most adapted language, which would be C# if you want to have managed code and other features that C# provide. And then if performance is an issue and do some profiling and optimise carefully by adding some code in C after weighting the pro/cons and considering whether you can get some optimisation by reworking the high level algorithm rather than optimising at a lower level.
For example, if you were to write scientific code, you would make your code multithreadable or checking that you cannot reuse previous computations before to attempt other kind of lower level optimisation, this will have a much larger impact on your efficiency rather than chasing tiny problems.
It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 11 years ago.
I have a first job interview for a software engineer position but in the email they state that I will have to write out a program at the interview stage. Does everyone do this?
What kind of program might it be for a graduate?
The job is for a .NET developer, but I can use any language, so I will stick with C#. I'm actually S**Ting it; I have no clue what they are going to ask me to do.
Many companies will spend much of the interview time asking candidates to write actual code (usually on a whiteboard or piece of paper though sometimes on a real computer) as this is a great way to see if they will be successful in the job. Some things to keep in mind:
Talk out loud. Often interviewers care about your thought process and approach to the problem as much or more than they care about the actual code you write.
Ask questions. Interviewers will often intentionally make a problem ambiguous just to see if you'll notice and seek clarification. Ask things like: "Who is the audience?" "Should I include exception handling?" "Shall I optimize for performance or just make sure it works?"
Don't get flustered if you're struggling. Tell the interview what you're thinking and they'll often point you in the right direction. Partial credit counts.
Get a good night's sleep before your interview. Lots of whiteboard coding and related discussion can be surprisingly grueling.
Good interviewers will be able to extract what you've learned over the course of your lifetime, so don't worry too much about last minute study sessions. It's too late. That said, it's not a bad idea to brush up on basic language syntax and core data structures and algorithms.
Here are some sample problems you may want to practice writing out on paper:
Write a function to calculate the nth number in the Fibonacci sequence.
Write a function to sort 2 arrays of numbers (without using existing libraries).
Design a Deck class and a Card class and write a function to shuffle a deck of cards.
Design a Circle class and write a function to determine if 2 circles intersect.
Design a LinkedList class and write a function to reverse the elements in the list.
At least be able to do this FizzBuzz
By the time the interview is scheduled, there's probably not too much you can do in the order of preparation.
Just remember, they want to know about your problem solving process. Just try to think out loud as much as possible and if you truly don't know something just say so.
No matter what type of question they ask, just go with the flow and do your best on it. The last thing any interviewer wants to see is someone who gets flustered or upset due to a particular question. I'll be the first to admit that some of the questions asked in an interview may be lame and unnecessary, but you are trying to get a job from these people and you will just have to humor them.
When you have more time to study, you should probably start looking at Questions every good .NET developer should be able to answer.
We issue programming tests all the time. There are many reasons for doing this, over and above the obvious one of testing coding ability. We look for
a) Coding style
b) Ability to develop and implement algorithms
c) Ability to follow instructions
d) Ability to communicate what has been done
But by far and away the most valuable thing about a programming test is discussing with the candidate why they did what they did. In this discussion it becomes obvious rather quickly how much the candidate really understood the test and their own design and implementation. It also roots out plagiarism very quickly.
Usually software development jobs give simple tests. I've never once interviewed for a job that required any more than a simple implementation of a function.
Her'es a few simple tests I know of:
FizzBuzz: http://www.geekschool.org/programming/fizzbuzz/
For a job at MS, I was asked to write a function to reverse the words in a string.
At a different job, I was asked to write an implementation of the Join function in c++.
A friend of mine got this one for game development: Write a function to test simple rectangle collision
More than likely it's something simple like FizzBuzz, just meant to weed out the totally unqualified people.
If the company don't ask for you to write code in the interview, that's really, really bad. Go for another company.
The type of test depends. I've done test that i had to write small C code, with pointers or recursive functions.
But generally, they ask for a basic asp.net application (I'm also C# developer), like just one form, inserting and reading from the DB.
It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 13 years ago.
Please can anyone one point me to a good tutorial that helps me to make my code fast and light.
I'm interested to know which Method is faster and when to use a method instead of other...
And how to evaluate if a code is good or bad?
My programming language is C#.
Hi all,
Thanks for your replies, they are very helpful.
I'm editing my question to be more specific specially that optimization is unlimited.
I want to know what is the best method in each condition.
For example using StringBuilder is better than string if i am appending lines to a string...
I only need this simple things.
Be aware of sub optimization.
Even though one specific function is faster than another replacing this isn't necessarily gonna make any difference in the runtime of your application. You need to understand which parts of your code that actually is a potential problem, and focus on optimizing these parts. Be aware of the O notation of your functions, and how often they are called. To identify parts that needs optimization a profiler can be of good help.
This question has some interesting points on why you shouldn't optimize until there is actually a need to do so.
Sure. Here's what we do:
Begin the journey by deciding when the journey is over. Set meaningful, customer-focussed, realistic performance goals. (For both speed and resource consumption.)
Carefully test your code frequently to see if you are meeting your performance targets.
If you are meeting your performance targets, don't worry about performance. Things are fine. Worry about bugs, or robustness, or features.
If you are not meeting your performance targets, run a profiler. Use it to identify what is the worst offending code. It only makes sense to fix the worst code; making something that is already incredibly fast and light slightly faster and lighter does not solve your performance problem.
Rewrite the slow code so that it's more performant. (This is the hard bit.) Make sure you test it to make sure it really is better.
If despite your best efforts you cannot make it good enough, either re-evaluate what your goals are, or cancel the project and spend your time on something that you can be successful at.
Keep iterating on that until you ship something.
Basically implement first, then test where to optimize.
If you are using Visual Studio Profissional you can use Analyze -> Launch Performance Wizard to analyze method performance. I am not sure about whether the other versions support this feature, however, there are also some commercial/free applications around ... look for profiler (see here for a list).
Type REALLY fast.
You should look at hidden features of c#, this post covers the most best practises in c# development
You can get a ton on advice of on this. But be aware of :
Premature optimization is root of all evil.
Aim first for correctness, next for clarity and only then performance.
As the old saying goes,
"No one cares how quickly you can calculate the wrong answer"
(on a practical level though, use a profiler)
If one method was always faster than another, they wouldn't bother including the slower one.
The only invariant when it comes to performance is that you need to profile. Everything follows from that.
If you get yourself a profiler, it will help you along, some will even give you great tips.
Example: ANTS Profiler
Usually you will find that reducing the number of times you create Strings to be the main performance boost you can get.
That and not messing with the Garbage Collector manually (unless you really really know what you're doing)
This link for Java design patterns is way too involved, don't get too put off by the word Java there, you can use what they teach for development in any language.
The thing is, if you want to know when to do what and what methods to use and so on, design patterns is what you are talking about.
I wish someone had pointed this to me earlier in my career.
In terms of general advice:
Try to use the fewest loops necessary
Move code out of loops where possible
Avoid copying things (like strings) in loops
Avoid creating objects in loops
Cache where warranted (generally small objects that take a lot of time to make), but make sure your cache has a good disposal policy or it turns into a memory leak
You can compile your program in native mode to improve the runtime performance.
One of the ways of figuring this out yourself is having a console app where you try running discrete pieces of code against each other and timing them. Like here.