We Keep Coding

How to best handle bit data in C#?

I'm trying to implement different signals containing different data and I implemented various datatypes in C# to manage the data cleanly (mainly structs, some enums). Most of these types are oddly sized, say some are 9 bit or 3 bit etc.
I implemented them as their closest equivalent basic C# type (most are byte, ushort or short, with some ints and uints).
What is the general way of handling such data types in C#? In the end I have to combine all the bits into one byte array, but I'm not sure how to combine them.
I thought about getting the byte array of each type with a BitConverter and putting all data as booleans into a BitArray which I than can convert back to a byte array. But I can't seem to split the byte array.
Another way to do it would be shifting every single variable I have, but that seems really dirty to do. If a type changed from 32 bit to 31 bit in the future that would seem like a hassle to change.
How is this usually done? Any best practices or something?
Basically I want to combine different sized data into one byte array. For example pack a 3 bit variable, a 2 bit boolean and a 11 bit value into 2 bytes.
Since I have the types implemented as C# types I can do BitArray arr = new BitArray(BitConverter.GetBytes((short)MyType)), but this would give me 16 bit while MyType might only have 9 bit.

My task is only to implement the data structures and the packing as
binary data in C#
To explicitly manipulate the in-memory layout of your structures:
Use StructLayout. Generally, this is only appropriate if dealing with interop or specialized memory constraints. See MSDN for examples/documentation. Note that you'll take a performance hit if your data is not byte-aligned.
To design a data structure:
Just use an existing solution like ASN.1 or Protobuf. This problem has already been addressed by experts; take advantage of their skills and knowledge. As an added bonus, using standard protocols makes it far easier for third parties to implement custom interfaces.

Passing an array pointer for placing in a hashmap in C++

I'm kinda new to C++ (coming from C#).
I'd like to pass an array of data to a function (as a pointer).
void someFunc(byte *data)
{
// add this data to a hashmap.
Hashtable.put(key, data)
}
This data will be added into a hashmap (some key-value based object).
In C#, i could just add the passed reference to a dictionary and be done with it.
Can the same be done in C++ ? or must i create a COPY of the data, and only add that to the data structure for storing it ?
I have seen this pattern in some code examples, but i am not 100% sure why it is needed, or whether it can be avoided at certain times.

Not sure where your key is coming from... but the std::map and the std::unordered_map are probably what you are looking for.
Now the underlying data structure of the std::map is a binary tree, while the std::unordered_map is a hash.
Furthermore, the std::unordered_map is an addition in the C++ 11 standard.

It all depends how you pass the data and how it is created. If the data is created on the heap(by using new) you can just put the pointer or reference you have to it in your table. On the other hand if the function takes the argument by value you will need to make a copy at some point, because if you store the address of a temp bad things will happen :).
As for what data structure to use, and how they work, I've found one of the best references is cppreference
Heap allocation should be reserved for special cases. stack allocation is faster and easier to manage, you should read up on RAII(very important). As for other reading try finding out stuff on dynamic vs. automatic memory allocation.
Just found this read specifically saying C# to C++ figured it'd be perfect for you, good luck c++ can be one of the more difficult languages to learn so don't assume anything will work the same as it does in C#. MSDN has a nice C# vs. C++ thing yet

Definition of C# data structures and algorithms

This may be a silly question (with MSDN and all), but maybe some of you will be able to help me sift through amazing amounts of information.
I need to know the specifics of the implementations of common data structures and algorithms in C#. That is, for example, I need to know, say, how Linked Lists are handled and represented, how they and their methods are defined.
Is there a good centralized source of documentation for this (with code), or should I just reconstruct it? Have you ever had to know the specifics of these things to decide what to use?
Regards, and thanks.

Scott Mitchell has a great 6-part article that covers many .NET data structures:
An Extensive Examination of Data Structures
For an algorithmic overview of data structures, I suggest reading the algorithm textbook: "Introduction to Algorithms" by Cormen, et al..
For details on each .NET data structure the MSDN page on that specific class is good.
When all of them fail to address issues, Reflector is always there. You can use it to dig through the actual source and see things for yourself.

If you really want to learn it, try making your own.
Googling for linked lists will give you a lot of hits and sample code to go off of. Wikipedia will also be a good resource.

Depends on the language. Most languages have the very basics now pre-built with them, but that doesn't mean their implementations are the same. The same named object--LinkedList in C# is completely different than the LinkedList in Java or C++. Even the String library is different. C# for instance is known to create a new String object every time you assign a string a new value...this becomes something you learn quickly when it brings your program to a crashing halt when you're working with substrings in C# for the first time.
So the answer to your question is massively complicated because I don't know quite what you're after. If you're just going to be teaching a class what a generic version of these algorithms and data structures are, you can present them without getting into the problems I mentioned above. You'll just need to select, lookup, read about a particular type of implementation of them. Like for LinkedList you need to be able to instantiate the list, destroy the list, copy the list, add to the list somewhere (usually front/back), remove from the list, etc. You could get fancy and add as many methods as you want.

Judy array for managed languages

Judy array is fast data structure that may represent a sparse array or a set of values. Is there its implementation for managed languages such as C#? Thanks

It's worth noting that these are often called Judy Trees or Judy Tries if you are googling for them.
I also looked for a .Net implementation but found nothing.
Also worth noting that:
The implementation is heavily designed around efficient cache usage, as such implementation specifics may be highly dependent on the size of certain constructs used within the sub structures. A .Net managed implementation may be somewhat different in this regard.
There are some significant hurdles to it that I can see (and there are probably more that my brief scan missed)
The API has some fairly anti OO aspects (for example a null pointer is viewed as an empty tree) so simplistic, move the state pointer to the LHS and make functions instance methods conversion to C++ wouldn't work.
The implementation of the sub structures I looked at made heavy use of pointers. I cannot see these efficiently being translated to references in managed languages.
The implementation is a distillation of a lot of very complex ideas that belies the simplicity of the public api.
The code base is about 20K lines (most of it complex), this doesn't strike me as an easy port.
You could take the library and wrap the C code in C++/CLI (probably simply holding internally a pointer that is the c api trie and having all the c calls point to this one). This would provide a simplistic implementation but the linked libraries for the native implementation may be problematic (as might memory allocation).
You would also probably need to deal with converting .Net strings to plain old byte* on the transition as well (or just work with bytes directly)

Judy really doesn't fit well with managed languages. I don't think you'll be able to use something like SWIG and get the first layer done automatically.
I wrote PyJudy and I ended up having to make some non-trivial API changes to fit well in Python. For example, I wrote in the documentation:
JudyL arrays map machine words to
machine words. In practice the words
store unsigned integers or pointers.
PyJudy supports all four mappings as
distinct classes.
pyjudy.JudyLIntInt - map unsigned
integer keys to unsigned integer
values
pyjudy.JudyLIntObj - map unsigned
integer keys to Python object values
pyjudy.JudyLObjInt - map Python
object keys to unsigned integer
values
pyjudy.JudyLObjObj - map Python
object keys to Python object values
I haven't looked at the code for a few years so my memories about it are pretty hazy. It was my first Python extension library, and I remember I hacked together a sort of template system for code generation. Nowadays I would use something like genshi.
I can't point to alternatives to Judy - that's one reason why I'm searching Stackoverflow.
Edit: I've been told that my timing numbers in the documentation are off from what Judy's documentation suggests because Judy is developed for 64-bit cache lines and my PowerBook was only 32 bits.
Some other links:
Patricia tries (http://www.csse.monash.edu.au/~lloyd/tildeAlgDS/Tree/PATRICIA/ )
Double-Array tries (http://linux.thai.net/~thep/datrie/datrie.html)
HAT-trie (http://members.optusnet.com.au/~askitisn/index.html)
The last has comparison numbers for different high-performance trie implementations.

This is proving trickier than I thought. PyJudy might be worth a look, as would be Tie::Judy. There's something on Softpedia, and something Ruby-ish. Trouble is, none of these are .NET specifically.

how to read and write a struct to file use C#?

I can use write(&stName,sizeof(stName),&FileName) and define a same struct in other program to read the file(XXX.h) when i use C, But I want do the same use C# and I should not use the unsafe mode. How do to solve the problem?
Edit:
thanks all. I will to try them
Edit:
Now if I want to use C write the Struct to file.h and use C# to read the struct from file.h, may I have chance solve that and not to count the offset? Because count the offset is not a good answer when I want to add some variable or other struct in the struct.

Look at the ISerializable interface and Serialization in general.

Even in C, this is a dangerous thing to do IMO. If you use a different compiler, operating system, architecture etc you can very easily "break" your data files - you're absolutely relying on the layout of the data in memory. It's a bit like exposing fields directly instead of properties - the in-memory layout should be an implementation detail which can be changed without the public form (the file) changing.
There are lots of ways of storing data, of course. For example:
Binary serialization (still pretty fragile, IMO)
XML serialization
Google's Protocol Buffers
Thrift
YAML
Hand-written serialization/deserialization e.g. using BinaryReader and BinaryWriter
There are balances in terms of whether the file is human readable, speed, size etc. See this question for answers to a similar question about Java.

You should take a look at the NetDataContractSerializer. You can markup those portions of the struct that you wish to serializer and use a file stream to write them out.

Look at the StructLayoutAttribute

Use Managed C++ or C++/CLI. It can read your .h file struct. It can read and write using:
read(in, &structure, sizeof(structure));
write(out, &structure, sizeof(structure));
and it can transfer that data very simply to anything else in .NET.

You'll have to convert each member of the struct individually using Bitconverter.convert(). This works well when your struct holds numeric data types, but you might have to do something more complex when using structs that contain more complicated data types like strings, arrays, and collections. For purposes like this, you will want to check out the .Net serialization facilities that other have mentioned.

You can look into Google Protocol buffers as well. You may not want to add another dependency into your code, but it's meant to allow you to do this sort of thing.