We Keep Coding

Efficient way to randomly select set bit

My current hobby project provides Monte-Carlo-Simulations for card games with French decks (52 cards, from 2 to Ace).
To simulate as fast as possible, I use to represent multiple cards as bitmasks in some spots. Here is some (simplified) code:
public struct Card
{
public enum CardColor : byte { Diamonds = 0, Hearts = 1, Spades = 2, Clubs = 3 }
public enum CardValue : byte { Two = 0, Three = 1, Four = 2, Five = 3, Six = 4, Seven = 5, Eight = 6, Nine = 7, Ten = 8, Jack = 9, Queen = 10, King = 11, Ace = 12 }
public CardColor Color { get; private set; }
public CardValue Value { get; private set; }
// ID provides a unique value for each card, ranging from 0 to 51, from 2Diamonds to AceClubs
public byte ID { get { return (byte)(((byte)this.Value * 4) + (byte)this.Color); } }
// --- Constructors ---
public Card(CardColor color, CardValue value)
{
this.Color = color;
this.Value = value;
}
public Card(byte id)
{
this.Color = (CardColor)(id % 4);
this.Value = (CardValue)((id - (byte)this.Color) / 4);
}
}
The structure which holds multiple cards as bitmask:
public struct CardPool
{
private const ulong FULL_POOL = 4503599627370495;
internal ulong Pool { get; private set; } // Holds all cards as set bit at Card.ID position
public int Count()
{
ulong i = this.Pool;
i = i - ((i >> 1) & 0x5555555555555555);
i = (i & 0x3333333333333333) + ((i >> 2) & 0x3333333333333333);
return (int)((((i + (i >> 4)) & 0xF0F0F0F0F0F0F0F) * 0x101010101010101) >> 56);
}
public CardPool Clone()
{
return new CardPool() { Pool = this.Pool };
}
public void Add(Card card)
{
Add(card.ID);
}
public void Add(byte cardID)
{
this.Pool = this.Pool | ((ulong)1 << cardID);
}
public void Remove(Card card)
{
Remove(card.ID);
}
public void Remove(byte cardID)
{
this.Pool = this.Pool & ~((ulong)1 << cardID);
}
public bool Contains(Card card)
{
ulong mask = ((ulong)1 << card.ID);
return (this.Pool & mask) == mask;
}
// --- Constructor ---
public CardPool(bool filled)
{
if (filled)
this.Pool = FULL_POOL;
else
this.Pool = 0;
}
}
I want to draw one or more cards at random from the second struct CardPool, but I cannot imagine how to do that without iterating single bits in the pool.
Is there any known algorithm to perfom this? If not, do you have any idea of doing this fast?
Update:
The structure is not intended to draw all cards from. It gets cloned frequently and cloning an array is no option. I really think of bitoperations for drawing one or multiple cards.
Update2:
I wrote a class which holds the cards as List for comparison.
public class CardPoolClass
{
private List<Card> Cards;
public void Add(Card card)
{
this.Cards.Add(card);
}
public CardPoolClass Clone()
{
return new CardPoolClass(this.Cards);
}
public CardPoolClass()
{
this.Cards = new List<Card> { };
}
public CardPoolClass(List<Card> cards)
{
this.Cards = cards.ToList();
}
}
Comparing 1.000.000 clone operations of full decks:
- struct: 17 ms
- class: 73 ms
Admitted: The difference is not as much as I thought.
But taken into account that I additionally give up the easy lookup of precalculated values, this makes a big difference.
Of course, it would be faster to draw a random card with this class, but I would have to calculate an index for lookup then, what just transfers the problem to another spot.
I repeat my initial question: Is there a known algorithm for choosing a random set bit from an integer value or has someone an idea for getting this done faster than to iterate all bits?
The discussion about using a class with a List or an Array takes us nowhere, this is not my question and I am able to elaborate on my own if I would be better off using a class.
Update3, the lookup-code:
CODE DELETED: This might be misleading because it does not refer to passages which performance suffers from what is subject of the thread.

Since a same card cannot be drawn twice in a row, you can place every card (in your case, the indices of Pool's set bits) in an array, shuffle it, and pop the cards one by one from any end of this array.
Here's a pseudo-code (because I don't know C#).
declare cards as an array of indices
for each bit in Pool
push its index into cards
shuffle cards
when a card needs to be drawn
pop an index from cards
look up the card with Card(byte id)
Edit
Here's an algorithm to get a random set bit once in a 64-bit integer, using a code from Bit Twiddling Hacks to get position of a bit with given rank (number of more significant set bits).
ulong v = this.Pool;
// ulong a = (v & ~0UL/3) + ((v >> 1) & ~0UL/3);
ulong a = v - ((v >> 1) & ~0UL/3);
// ulong b = (a & ~0UL/5) + ((a >> 2) & ~0UL/5);
ulong b = (a & ~0UL/5) + ((a >> 2) & ~0UL/5);
// ulong c = (b & ~0UL/0x11) + ((b >> 4) & ~0UL/0x11);
ulong c = (b + (b >> 4)) & ~0UL/0x11;
// ulong d = (c & ~0UL/0x101) + ((c >> 8) & ~0UL/0x101);
ulong d = (c + (c >> 8)) & ~0UL/0x101;
ulong t = (d >> 32) + (d >> 48);
int bitCount = ((c * (~0UL / 0xff)) >> 56);
ulong r = Randomizer.Next(1, bitCount+1);
ulong s = 64;
// if (r > t) {s -= 32; r -= t;}
s -= ((t - r) & 256) >> 3; r -= (t & ((t - r) >> 8));
t = (d >> (s - 16)) & 0xff;
// if (r > t) {s -= 16; r -= t;}
s -= ((t - r) & 256) >> 4; r -= (t & ((t - r) >> 8));
t = (c >> (s - 8)) & 0xf;
// if (r > t) {s -= 8; r -= t;}
s -= ((t - r) & 256) >> 5; r -= (t & ((t - r) >> 8));
t = (b >> (s - 4)) & 0x7;
// if (r > t) {s -= 4; r -= t;}
s -= ((t - r) & 256) >> 6; r -= (t & ((t - r) >> 8));
t = (a >> (s - 2)) & 0x3;
// if (r > t) {s -= 2; r -= t;}
s -= ((t - r) & 256) >> 7; r -= (t & ((t - r) >> 8));
t = (v >> (s - 1)) & 0x1;
// if (r > t) s--;
s -= ((t - r) & 256) >> 8;
s--; // s is now the position of a random set bit in v
The commented lines make another version, with branches. If you want to compare the two versions, uncomment these lines and comment the lines following them.
In the original code, the last line is s = 65 - s, but since you use 1 << cardID for manipulations on card pools, and r is random anyway, s-- gives the correct value.
The only thing to watch out for is a zero value for v. But executing this code on an empty pool would be meaningless anyway.

How can I convert IP range to Cidr in C#?

There has lots of example of convert CIDR to ip range. But I want to know how can I use start/end ip address to generate a/some cidr in C#?
for example:
I have start ip address(192.168.0.1) and end ip address(192.168.0.254). So use these two address to generate cidr list {192.168.0.0/31, 192.168.0.2/32}. Is there any C# code example?

CIDR class with static methods to split an IP range into a minimal set of disjoint CIDR ranges, which cover exactly the original IP range.
The split methods (the "real" one working on BigIntegers doing the actual work, and the wrapper for IP addresses and CIDR creation) are at the bottom.
Use with foreach (IPRangeToCidr.CIDR c in IPRangeToCidr.CIDR.split(first, last)) ...
Requires System.Numerics.dll in the references.
using System;
using System.Numerics;
using System.Net;
using System.Net.Sockets;
using System.Collections.Generic;
namespace IPRangeToCidr {
public struct CIDR {
private IPAddress address;
private uint network_length, bits;
public CIDR(IPAddress address, uint network_length) {
this.address = address;
this.network_length = network_length;
this.bits = AddressFamilyBits(address.AddressFamily);
if (network_length > bits) {
throw new ArgumentException("Invalid network length " + network_length + " for " + address.AddressFamily);
}
}
public IPAddress NetworkAddress {
get { return address; }
}
public IPAddress LastAddress {
get { return IPAddressAdd(address, (new BigInteger(1) << (int) HostLength) - 1); }
}
public uint NetworkLength {
get { return network_length; }
}
public uint AddressBits {
get { return bits; }
}
public uint HostLength {
get { return bits - network_length; }
}
override public String ToString() {
return address.ToString() + "/" + NetworkLength.ToString();
}
public String ToShortString() {
if (network_length == bits) return address.ToString();
return address.ToString() + "/" + NetworkLength.ToString();
}
/* static helpers */
public static IPAddress IPAddressAdd(IPAddress address, BigInteger i) {
return IPFromUnsigned(IPToUnsigned(address) + i, address.AddressFamily);
}
public static uint AddressFamilyBits(AddressFamily family) {
switch (family) {
case AddressFamily.InterNetwork:
return 32;
case AddressFamily.InterNetworkV6:
return 128;
default:
throw new ArgumentException("Invalid address family " + family);
}
}
private static BigInteger IPToUnsigned(IPAddress addr) {
/* Need to reverse addr bytes for BigInteger; prefix with 0 byte to force unsigned BigInteger
* read BigInteger bytes as: bytes[n] bytes[n-1] ... bytes[0], address is bytes[0] bytes[1] .. bytes[n] */
byte[] b = addr.GetAddressBytes();
byte[] unsigned = new byte[b.Length + 1];
for (int i = 0; i < b.Length; ++i) {
unsigned[i] = b[(b.Length - 1) - i];
}
unsigned[b.Length] = 0;
return new BigInteger(unsigned);
}
private static byte[] GetUnsignedBytes(BigInteger unsigned, uint bytes) {
/* reverse bytes again. check that now higher bytes are actually used */
if (unsigned.Sign < 0) throw new ArgumentException("argument must be >= 0");
byte[] data = unsigned.ToByteArray();
byte[] result = new byte[bytes];
for (int i = 0; i < bytes && i < data.Length; ++i) {
result[bytes - 1 - i] = data[i];
}
for (uint i = bytes; i < data.Length; ++i) {
if (data[i] != 0) throw new ArgumentException("argument doesn't fit in requested number of bytes");
}
return result;
}
private static IPAddress IPFromUnsigned(BigInteger unsigned, System.Net.Sockets.AddressFamily family) {
/* IPAddress(byte[]) constructor picks family from array size */
switch (family) {
case System.Net.Sockets.AddressFamily.InterNetwork:
return new IPAddress(GetUnsignedBytes(unsigned, 4));
case System.Net.Sockets.AddressFamily.InterNetworkV6:
return new IPAddress(GetUnsignedBytes(unsigned, 16));
default:
throw new ArgumentException("AddressFamily " + family.ToString() + " not supported");
}
}
/* splits set [first..last] of unsigned integers into disjoint slices { x,..., x + 2^k - 1 | x mod 2^k == 0 }
* covering exaclty the given set.
* yields the slices ordered by x as tuples (x, k)
* This code relies on the fact that BigInteger can't overflow; temporary results may need more bits than last is using.
*/
public static IEnumerable<Tuple<BigInteger, uint>> split(BigInteger first, BigInteger last) {
if (first > last) yield break;
if (first < 0) throw new ArgumentException();
last += 1;
/* mask == 1 << len */
BigInteger mask = 1;
uint len = 0;
while (first + mask <= last) {
if ((first & mask) != 0) {
yield return new Tuple<BigInteger, uint>(first, len);
first += mask;
}
mask <<= 1;
++len;
}
while (first < last) {
mask >>= 1;
--len;
if ((last & mask) != 0) {
yield return new Tuple<BigInteger, uint>(first, len);
first += mask;
}
}
}
public static IEnumerable<CIDR> split(IPAddress first, IPAddress last) {
if (first.AddressFamily != last.AddressFamily) {
throw new ArgumentException("AddressFamilies don't match");
}
AddressFamily family = first.AddressFamily;
uint bits = AddressFamilyBits(family); /* split on numbers returns host length, CIDR takes network length */
foreach (Tuple<BigInteger, uint> slice in split(IPToUnsigned(first), IPToUnsigned(last))) {
yield return new CIDR(IPFromUnsigned(slice.Item1, family), bits - slice.Item2);
}
}
}
}

It is difficult to determine the what exactly is being asked here (the CIDR list you give doesn't seem to correspond with the given input addresses), however the following code will allow you to find the smallest single CIDR that contains the specified start and end addresses.
You need to first convert the start and end IP addresses into 32 bit integers (e.g. 192.168.0.1 becomes 0xc0a80001), then apply the following algorithm:
var startAddr = 0xc0a80001; // 192.168.0.1
var endAddr = 0xc0a800fe; // 192.168.0.254
// Determine all bits that are different between the two IPs
var diffs = startAddr ^ endAddr;
// Now count the number of consecutive zero bits starting at the most significant
var bits = 32;
var mask = 0;
while (diffs != 0)
{
// We keep shifting diffs right until it's zero (i.e. we've shifted all the non-zero bits off)
diffs >>= 1;
// Every time we shift, that's one fewer consecutive zero bits in the prefix
bits--;
// Accumulate a mask which will have zeros in the consecutive zeros of the prefix and ones elsewhere
mask = (mask << 1) | 1;
}
// Construct the root of the range by inverting the mask and ANDing it with the start address
var root = startAddr & ~mask;
// Finally, output the range
Console.WriteLine("{0}.{1}.{2}.{3}/{4}", root >> 24, (root >> 16) & 0xff, (root >> 8) & 0xff, root & 0xff, bits);
Running it on the two addresses in your question gives:
192.168.0.0/24

Necromancing.
No, there wasn't, and I don't understand why people keep upvoting wrong answers.
Here's the code for IP-range to CIDR & vice-versa:
// https://dev.maxmind.com/geoip/
// https://stackoverflow.com/questions/461742/how-to-convert-an-ipv4-address-into-a-integer-in-c
public static string IPrange2CIDR(string ip1, string ip2)
{
uint startAddr = IP2num(ip1);
uint endAddr = IP2num(ip2);
// uint startAddr = 0xc0a80001; // 192.168.0.1
// uint endAddr = 0xc0a800fe; // 192.168.0.254
// uint startAddr = System.BitConverter.ToUInt32(System.Net.IPAddress.Parse(ip1).GetAddressBytes(), 0);
// uint endAddr = System.BitConverter.ToUInt32(System.Net.IPAddress.Parse(ip2).GetAddressBytes(), 0);
if (startAddr > endAddr)
{
uint temp = startAddr;
startAddr = endAddr;
endAddr = temp;
}
// uint diff = endAddr - startAddr -1;
// int bits = 32 - (int)System.Math.Ceiling(System.Math.Log10(diff) / System.Math.Log10(2));
// return ip1 + "/" + bits;
uint diffs = startAddr ^ endAddr;
// Now count the number of consecutive zero bits starting at the most significant
int bits = 32;
// int mask = 0;
// We keep shifting diffs right until it's zero (i.e. we've shifted all the non-zero bits off)
while (diffs != 0)
{
diffs >>= 1;
bits--; // Every time we shift, that's one fewer consecutive zero bits in the prefix
// Accumulate a mask which will have zeros in the consecutive zeros of the prefix and ones elsewhere
// mask = (mask << 1) | 1;
}
string res = ip1 + "/" + bits;
System.Console.WriteLine(res);
return res;
}
// https://www.digitalocean.com/community/tutorials/understanding-ip-addresses-subnets-and-cidr-notation-for-networking
public static void CIDR2IP(string IP)
{
string[] parts = IP.Split('.', '/');
uint ipnum = (System.Convert.ToUInt32(parts[0]) << 24) |
(System.Convert.ToUInt32(parts[1]) << 16) |
(System.Convert.ToUInt32(parts[2]) << 8) |
System.Convert.ToUInt32(parts[3]);
int maskbits = System.Convert.ToInt32(parts[4]);
uint mask = 0xffffffff;
mask <<= (32 - maskbits);
uint ipstart = ipnum & mask;
uint ipend = ipnum | (mask ^ 0xffffffff);
string fromRange = string.Format("{0}.{1}.{2}.{3}", ipstart >> 24, (ipstart >> 16) & 0xff, (ipstart >> 8) & 0xff, ipstart & 0xff);
string toRange = string.Format("{0}.{1}.{2}.{3}", ipend >> 24, (ipend >> 16) & 0xff, (ipend >> 8) & 0xff, ipend & 0xff);
System.Console.WriteLine(fromRange + " - " + toRange);
}
public static uint IP2num(string ip)
{
string[] nums = ip.Split('.');
uint first = System.UInt32.Parse(nums[0]);
uint second = System.UInt32.Parse(nums[1]);
uint third = System.UInt32.Parse(nums[2]);
uint fourth = System.UInt32.Parse(nums[3]);
return (first << 24) | (second << 16) | (third << 8) | (fourth);
}
public static void Test()
{
string IP = "5.39.40.96/27";
// IP = "88.84.128.0/19";
CIDR2IP(IP);
// IPrange2CIDR("88.84.128.0", "88.84.159.255");
IPrange2CIDR("5.39.40.96", "5.39.40.127");
System.Console.WriteLine(System.Environment.NewLine);
System.Console.WriteLine(" --- Press any key to continue --- ");
System.Console.ReadKey();
}

I use this for IpV4, let me know if there is a problem in it.
You can find the extraction source code from the following link:
https://blog.ip2location.com/knowledge-base/how-to-convert-ip-address-range-into-cidr/
using System;
using System.Collections.Generic;
using System.Net;
namespace ConsoleApp
{
public class IPNetwork
{
private readonly long _firstIpAddress;
private readonly long _lastIpAddress;
public static IPNetwork[] FromIpRange(IPAddress firstIpAddress, IPAddress lastIpAddress)
=> FromIpRange(IpAddressToLong(firstIpAddress), IpAddressToLong(lastIpAddress));
public static IPNetwork[] FromIpRange(long firstIpAddress, long lastIpAddress)
{
var result = new List<IPNetwork>();
while (lastIpAddress >= firstIpAddress)
{
byte maxSize = 32;
while (maxSize > 0)
{
long mask = IMask(maxSize - 1);
long maskBase = firstIpAddress & mask;
if (maskBase != firstIpAddress)
break;
maxSize--;
}
double x = Math.Log(lastIpAddress - firstIpAddress + 1) / Math.Log(2);
byte maxDiff = (byte)(32 - Math.Floor(x));
if (maxSize < maxDiff)
{
maxSize = maxDiff;
}
var ipAddress = IpAddressFromLong(firstIpAddress);
result.Add(new IPNetwork(ipAddress, maxSize));
firstIpAddress += (long)Math.Pow(2, 32 - maxSize);
}
return result.ToArray();
}
private static long IMask(int s)
{
return (long)(Math.Pow(2, 32) - Math.Pow(2, 32 - s));
}
public static long IpAddressToLong(IPAddress ipAddress)
{
var bytes = ipAddress.GetAddressBytes();
return ((long)bytes[0] << 24) | ((long)bytes[1] << 16) | ((long)bytes[2] << 8) | bytes[3];
}
public static IPAddress IpAddressFromLong(long ipAddress)
=> new IPAddress((uint)IPAddress.NetworkToHostOrder((int)ipAddress));
public IPNetwork(IPAddress prefix, int prefixLength = 32)
{
if (prefix.AddressFamily != System.Net.Sockets.AddressFamily.InterNetwork)
throw new NotSupportedException("IPv6 is not supported");
Prefix = prefix;
PrefixLength = prefixLength;
var mask = (uint)~(0xFFFFFFFFL >> prefixLength);
_firstIpAddress = IpAddressToLong(Prefix) & mask;
_lastIpAddress = _firstIpAddress | ~mask;
}
public static IPNetwork Parse(string value)
{
try
{
var parts = value.Split('/');
return new IPNetwork(IPAddress.Parse(parts[0]), int.Parse(parts[1]));
}
catch
{
throw new FormatException($"Could not parse IPNetwork from {value}");
}
}
public override string ToString() => $"{Prefix}/{PrefixLength}";
public IPAddress Prefix { get; }
public int PrefixLength { get; }
public IPAddress LastAddress => IpAddressFromLong(_lastIpAddress);
public IPAddress FirstAddress => IpAddressFromLong(_firstIpAddress);
public long Total => _lastIpAddress - _firstIpAddress + 1;
}
}
Usage 1:
var startAddress = IPAddress.Parse("192.168.0.0");
var endAddress = IPAddress.Parse("192.168.0.255");
foreach (var item in IPNetwork.FromIpRange(startAddress, endAddress))
Console.WriteLine(item);
Result
192.168.0.0/24
Usage 2:
var startAddress = IPAddress.Parse("192.168.0.1");
var endAddress = IPAddress.Parse("192.168.0.254");
foreach (var item in IPNetwork.FromIpRange(startAddress, endAddress))
Console.WriteLine(item);
Result:
192.168.0.1/32
192.168.0.2/31
192.168.0.4/30
192.168.0.8/29
192.168.0.16/28
192.168.0.32/27
192.168.0.64/26
192.168.0.128/26
192.168.0.192/27
192.168.0.224/28
192.168.0.240/29
192.168.0.248/30
192.168.0.252/31
192.168.0.254/32

I would recommend the use of IPNetwork Library https://github.com/lduchosal/ipnetwork.
As of version 2, it supports IPv4 and IPv6 as well.
Supernet
IPNetwork network = IPNetwork.Parse("192.168.0.1");
IPNetwork network2 = IPNetwork.Parse("192.168.0.254");
IPNetwork ipnetwork = IPNetwork.Supernet(network, network2);
Console.WriteLine("Network : {0}", ipnetwork.Network);
Console.WriteLine("Netmask : {0}", ipnetwork.Netmask);
Console.WriteLine("Broadcast : {0}", ipnetwork.Broadcast);
Console.WriteLine("FirstUsable : {0}", ipnetwork.FirstUsable);
Console.WriteLine("LastUsable : {0}", ipnetwork.LastUsable);
Console.WriteLine("Usable : {0}", ipnetwork.Usable);
Console.WriteLine("Cidr : {0}", ipnetwork.Cidr);
Output
Network : 192.168.0.0
Netmask : 255.255.255.0
Broadcast : 192.168.0.255
FirstUsable : 192.168.0.1
LastUsable : 192.168.0.254
Usable : 254
Cidr : 24
Have fun !

I found this C code and converted it into C# and it's working now.

Converting small C# checksum program into Java

I'm trying to build a simple ground control station for an RC airplane. I've almost finished it, but I'm having a LOT of trouble with the checksum calculation. I understand that the data types of Java and C# are different. I've attempted to account for that but I'm not sure I've succeeded. The program utilizes the CRC-16-CCITT method.
Here is my port:
public int crc_accumulate(int b, int crc) {
int ch = (b ^ (crc & 0x00ff));
ch = (ch ^ (ch << 4));
return ((crc >> 8) ^ (ch << 8) ^ (ch << 3) ^ (ch >> 4));
}
public byte[] crc_calculate() {
int[] pBuffer=new int[]{255,9,19,1,1,0,0,0,0,0,2,3,81,4,3};
int crcEx=0;
int clength=pBuffer.length;
int[] X25_INIT_CRC=new int[]{255,255};
byte[] crcTmp=new byte[]{(byte)255,(byte)255};
int crcTmp2 = ((crcTmp[0] & 0xff) << 8) | (crcTmp[1] & 0xff);
crcTmp[0]=(byte)crcTmp2;
crcTmp[1]=(byte)(crcTmp2 >> 8);
System.out.println("pre-calculation: 0x"+Integer.toHexString((crcTmp[0]&0xff))+" 0x"+Integer.toHexString((crcTmp[1]&0xff))+"; ushort: "+crcTmp2);
if (clength < 1) {
System.out.println("clength < 1");
return crcTmp;
}
for (int i=1; i<clength; i++) {
crcTmp2 = crc_accumulate(pBuffer[i], crcTmp2);
}
crcTmp[0]=(byte)crcTmp2;
crcTmp[1]=(byte)(crcTmp2 >> 8);
System.out.print("crc calculation: 0x"+Integer.toHexString((crcTmp[0]&0xff))+" 0x"+Integer.toHexString((crcTmp[1]&0xff))+"; ushort: "+crcTmp2);
if (crcEx!=-1) {
System.out.println(" extraCRC["+crcEx+"]="+extraCRC[crcEx]);
crcTmp2=crc_accumulate(extraCRC[crcEx], crcTmp2);
crcTmp[0]=(byte)crcTmp2;
crcTmp[1]=(byte)(crcTmp2 >> 8);
System.out.println("with extra CRC: 0x"+Integer.toHexString((crcTmp[0]&0xff))+" 0x"+Integer.toHexString((crcTmp[1]&0xff))+"; ushort: "+crcTmp2+"\n\n");
}
return crcTmp;
}
This is the original C# file:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace ArdupilotMega
{
class MavlinkCRC
{
const int X25_INIT_CRC = 0xffff;
const int X25_VALIDATE_CRC = 0xf0b8;
public static ushort crc_accumulate(byte b, ushort crc)
{
unchecked
{
byte ch = (byte)(b ^ (byte)(crc & 0x00ff));
ch = (byte)(ch ^ (ch << 4));
return (ushort)((crc >> 8) ^ (ch << 8) ^ (ch << 3) ^ (ch >> 4));
}
}
public static ushort crc_calculate(byte[] pBuffer, int length)
{
if (length < 1)
{
return 0xffff;
}
// For a "message" of length bytes contained in the unsigned char array
// pointed to by pBuffer, calculate the CRC
// crcCalculate(unsigned char* pBuffer, int length, unsigned short* checkConst) < not needed
ushort crcTmp;
int i;
crcTmp = X25_INIT_CRC;
for (i = 1; i < length; i++) // skips header U
{
crcTmp = crc_accumulate(pBuffer[i], crcTmp);
//Console.WriteLine(crcTmp + " " + pBuffer[i] + " " + length);
}
return (crcTmp);
}
}
}
I'm quite sure that the problem in my port lies between lines 1 and 5. I expect to get an output of 0x94 0x88, but instead the program outputs 0x2D 0xF4.
I would greatly appreciate it if someone could show me where I've gone wrong.
Thanks for any help,
Cameron

Alright, for starters lets clean up the C# code a little:
const int X25_INIT_CRC = 0xffff;
public static ushort crc_accumulate(byte b, ushort crc)
{
unchecked
{
byte ch = (byte)(b ^ (byte)(crc & 0x00ff));
ch = (byte)(ch ^ (ch << 4));
return (ushort)((crc >> 8) ^ (ch << 8) ^ (ch << 3) ^ (ch >> 4));
}
}
public static ushort crc_calculate(byte[] pBuffer)
{
ushort crcTmp = X25_INIT_CRC;
for (int i = 1; i < pBuffer.Length; i++) // skips header U
crcTmp = crc_accumulate(pBuffer[i], crcTmp);
return crcTmp;
}
Now the biggest problem here is that there are no unsigned numeric types in Java, so you have to work around that by using the next bigger numeric type instead of ushort and byte and masking off the high bits as needed. You can also just drop the unchecked because Java has no overflow checking anyway. The end result is something like this:
public static final int X25_INIT_CRC = 0xffff;
public static int crc_accumulate(short b, int crc) {
short ch = (short)((b ^ crc) & 0xff);
ch = (short)((ch ^ (ch << 4)) & 0xff);
return ((crc >> 8) ^ (ch << 8) ^ (ch << 3) ^ (ch >> 4)) & 0xffff;
}
public static int crc_calculate(short[] pBuffer) {
int crcTmp = X25_INIT_CRC;
for (int i = 1; i < pBuffer.length; i++) // skips header U
crcTmp = crc_accumulate(pBuffer[i], crcTmp);
return crcTmp;
}
For the input in your question ({ 255, 9, 19, 1, 1, 0, 0, 0, 0, 0, 2, 3, 81, 4, 3 }) the original C#, cleaned up C# and Java all produce 0xfc7e.

How to get little endian data from big endian in c# using bitConverter.ToInt32 method?

I am making application in C# which has a byte array containing hex values.
I am getting data as a big-endian but I want it as a little-endian and I am using Bitconverter.toInt32 method for converting that value to integer.
My problem is that before converting the value, I have to copy that 4 byte data into temporary array from source byte array and then reverse that temporary byte array.
I can't reverse source array because it also contains other data.
Because of that my application becomes slow.
In the code I have one source array of byte as waveData[] which contains a lot of data.
byte[] tempForTimestamp=new byte[4];
tempForTimestamp[0] = waveData[290];
tempForTimestamp[1] = waveData[289];
tempForTimestamp[2] = waveData[288];
tempForTimestamp[3] = waveData[287];
int number = BitConverter.ToInt32(tempForTimestamp, 0);
Is there any other method for that conversion?

Add a reference to System.Memory nuget and use BinaryPrimitives.ReverseEndianness().
using System.Buffers.Binary;
number = BinaryPrimitives.ReverseEndianness(number);
It supports both signed and unsigned integers (byte/short/int/long).

In modern-day Linq the one-liner and easiest to understand version would be:
int number = BitConverter.ToInt32(waveData.Skip(286).Take(4).Reverse().ToArray(), 0);
You could also...
byte[] tempForTimestamp = new byte[4];
Array.Copy(waveData, 287, tempForTimestamp, 0, 4);
Array.Reverse(tempForTimestamp);
int number = BitConverter.ToInt32(tempForTimestamp);
:)

If you know the data is big-endian, perhaps just do it manually:
int value = (buffer[i++] << 24) | (buffer[i++] << 16)
| (buffer[i++] << 8) | buffer[i++];
this will work reliably on any CPU, too. Note i is your current offset into the buffer.
Another approach would be to shuffle the array:
byte tmp = buffer[i+3];
buffer[i+3] = buffer[i];
buffer[i] = tmp;
tmp = buffer[i+2];
buffer[i+2] = buffer[i+1];
buffer[i+1] = tmp;
int value = BitConverter.ToInt32(buffer, i);
i += 4;
I find the first immensely more readable, and there are no branches / complex code, so it should work pretty fast too. The second could also run into problems on some platforms (where the CPU is already running big-endian).

Here you go
public static int SwapEndianness(int value)
{
var b1 = (value >> 0) & 0xff;
var b2 = (value >> 8) & 0xff;
var b3 = (value >> 16) & 0xff;
var b4 = (value >> 24) & 0xff;
return b1 << 24 | b2 << 16 | b3 << 8 | b4 << 0;
}

Declare this class:
using static System.Net.IPAddress;
namespace BigEndianExtension
{
public static class BigEndian
{
public static short ToBigEndian(this short value) => HostToNetworkOrder(value);
public static int ToBigEndian(this int value) => HostToNetworkOrder(value);
public static long ToBigEndian(this long value) => HostToNetworkOrder(value);
public static short FromBigEndian(this short value) => NetworkToHostOrder(value);
public static int FromBigEndian(this int value) => NetworkToHostOrder(value);
public static long FromBigEndian(this long value) => NetworkToHostOrder(value);
}
}
Example, create a form with a button and a multiline textbox:
using BigEndianExtension;
private void button1_Click(object sender, EventArgs e)
{
short int16 = 0x1234;
int int32 = 0x12345678;
long int64 = 0x123456789abcdef0;
string text = string.Format("LE:{0:X4}\r\nBE:{1:X4}\r\n", int16, int16.ToBigEndian());
text += string.Format("LE:{0:X8}\r\nBE:{1:X8}\r\n", int32, int32.ToBigEndian());
text += string.Format("LE:{0:X16}\r\nBE:{1:X16}\r\n", int64, int64.ToBigEndian());
textBox1.Text = text;
}
//Some code...

The most straightforward way is to use the BinaryPrimitives.ReadInt32BigEndian(ReadOnlySpan) Method introduced in .NET Standard 2.1
var number = BinaryPrimitives.ReadInt32BigEndian(waveData[297..291]);

If you won't ever again need that reversed, temporary array, you could just create it as you pass the parameter, instead of making four assignments. For example:
int i = 287;
int value = BitConverter.ToInt32({
waveData(i + 3),
waveData(i + 2),
waveData(i + 1),
waveData(i)
}, 0);

I use the following helper functions
public static Int16 ToInt16(byte[] data, int offset)
{
if (BitConverter.IsLittleEndian)
return BitConverter.ToInt16(BitConverter.IsLittleEndian ? data.Skip(offset).Take(2).Reverse().ToArray() : data, 0);
return BitConverter.ToInt16(data, offset);
}
public static Int32 ToInt32(byte[] data, int offset)
{
if (BitConverter.IsLittleEndian)
return BitConverter.ToInt32(BitConverter.IsLittleEndian ? data.Skip(offset).Take(4).Reverse().ToArray() : data, 0);
return BitConverter.ToInt32(data, offset);
}
public static Int64 ToInt64(byte[] data, int offset)
{
if (BitConverter.IsLittleEndian)
return BitConverter.ToInt64(BitConverter.IsLittleEndian ? data.Skip(offset).Take(8).Reverse().ToArray() : data, 0);
return BitConverter.ToInt64(data, offset);
}

You can also use Jon Skeet "Misc Utils" library, available at https://jonskeet.uk/csharp/miscutil/
His library has many utility functions. For Big/Little endian conversions you can check the MiscUtil/Conversion/EndianBitConverter.cs file.
var littleEndianBitConverter = new MiscUtil.Conversion.LittleEndianBitConverter();
littleEndianBitConverter.ToInt64(bytes, offset);
var bigEndianBitConverter = new MiscUtil.Conversion.BigEndianBitConverter();
bigEndianBitConverter.ToInt64(bytes, offset);
His software is from 2009 but I guess it's still relevant.

I dislike BitConverter, because (as Marc Gravell answered) it is specced to rely on system endianness, meaning you technically have to do a system endianness check every time you use BitConverter to ensure you don't have to reverse the array. And usually, with saved files, you generally know the endianness you're trying to read, and that might not be the same. You might just be handling file formats with big-endian values, too, like, for instance, PNG chunks.
Because of that, I just wrote my own methods for this, which take a byte array, the read offset and read length as arguments, as well as a boolean to specify the endianness handling, and which uses bit shifting for efficiency:
public static UInt64 ReadIntFromByteArray(Byte[] data, Int32 startIndex, Int32 bytes, Boolean littleEndian)
{
Int32 lastByte = bytes - 1;
if (data.Length < startIndex + bytes)
throw new ArgumentOutOfRangeException("startIndex", "Data array is too small to read a " + bytes + "-byte value at offset " + startIndex + ".");
UInt64 value = 0;
for (Int32 index = 0; index < bytes; index++)
{
Int32 offs = startIndex + (littleEndian ? index : lastByte - index);
value |= (((UInt64)data[offs]) << (8 * index));
}
return value;
}
This code can handle any value between 1 and 8 bytes, both little-endian and big-endian. The only small usage peculiarity is that you need to both give the amount of bytes to read, and need to specifically cast the result to the type you want.
Example from some code where I used it to read the header of some proprietary image type:
Int16 imageWidth = (Int16) ReadIntFromByteArray(fileData, hdrOffset, 2, true);
Int16 imageHeight = (Int16) ReadIntFromByteArray(fileData, hdrOffset + 2, 2, true);
This will read two consecutive 16-bit integers off an array, as signed little-endian values. You can of course just make a bunch of overload functions for all possibilities, like this:
public Int16 ReadInt16FromByteArrayLe(Byte[] data, Int32 startIndex)
{
return (Int16) ReadIntFromByteArray(data, startIndex, 2, true);
}
But personally I didn't bother with that.
And, here's the same for writing bytes:
public static void WriteIntToByteArray(Byte[] data, Int32 startIndex, Int32 bytes, Boolean littleEndian, UInt64 value)
{
Int32 lastByte = bytes - 1;
if (data.Length < startIndex + bytes)
throw new ArgumentOutOfRangeException("startIndex", "Data array is too small to write a " + bytes + "-byte value at offset " + startIndex + ".");
for (Int32 index = 0; index < bytes; index++)
{
Int32 offs = startIndex + (littleEndian ? index : lastByte - index);
data[offs] = (Byte) (value >> (8*index) & 0xFF);
}
}
The only requirement here is that you have to cast the input arg to 64-bit unsigned integer when passing it to the function.

public static unsafe int Reverse(int value)
{
byte* p = (byte*)&value;
return (*p << 24) | (p[1] << 16) | (p[2] << 8) | p[3];
}
If unsafe is allowed... Based on Marc Gravell's post

This will reverse the data inline if unsafe code is allowed...
fixed (byte* wavepointer = waveData)
new Span<byte>(wavepointer + offset, 4).Reverse();

C# Language: Changing the First Four Bits in a Byte

In order to utilize a byte to its fullest potential, I'm attempting to store two unique values into a byte: one in the first four bits and another in the second four bits. However, I've found that, while this practice allows for optimized memory allocation, it makes changing the individual values stored in the byte difficult.
In my code, I want to change the first set of four bits in a byte while maintaining the value of the second four bits in the same byte. While bitwise operations allow me to easily retrieve and manipulate the first four bit values, I'm finding it difficult to concatenate this new value with the second set of four bits in a byte. The question is, how can I erase the first four bits from a byte (or, more accurately, set them all the zero) and add the new set of 4 bits to replace the four bits that were just erased, thus preserving the last 4 bits in a byte while changing the first four?
Here's an example:
// Changes the first four bits in a byte to the parameter value
public void changeFirstFourBits(byte newFirstFour)
{
// If 'newFirstFour' is 0101 in binary, make 'value' 01011111 in binary, changing
// the first four bits but leaving the second four alone.
}
private byte value = 255; // binary: 11111111

Use bitwise AND (&) to clear out the old bits, shift the new bits to the correct position and bitwise OR (|) them together:
value = (value & 0xF) | (newFirstFour << 4);
Here's what happens:
value : abcdefgh
newFirstFour : 0000xyzw
0xF : 00001111
value & 0xF : 0000efgh
newFirstFour << 4 : xyzw0000
(value & 0xF) | (newFirstFour << 4) : xyzwefgh

When I have to do bit-twiddling like this, I make a readonly struct to do it for me. A four-bit integer is called nybble, of course:
struct TwoNybbles
{
private readonly byte b;
public byte High { get { return (byte)(b >> 4); } }
public byte Low { get { return (byte)(b & 0x0F); } {
public TwoNybbles(byte high, byte low)
{
this.b = (byte)((high << 4) | (low & 0x0F));
}
And then add implicit conversions between TwoNybbles and byte. Now you can just treat any byte as having a High and Low byte without putting all that ugly bit twiddling in your mainline code.

You first mask out you the high four bytes using value & 0xF. Then you shift the new bits to the high four bits using newFirstFour << 4 and finally you combine them together using binary or.
public void changeHighFourBits(byte newHighFour)
{
value=(byte)( (value & 0x0F) | (newFirstFour << 4));
}
public void changeLowFourBits(byte newLowFour)
{
value=(byte)( (value & 0xF0) | newLowFour);
}

I'm not really sure what your method there is supposed to do, but here are some methods for you:
void setHigh(ref byte b, byte val) {
b = (b & 0xf) | (val << 4);
}
byte high(byte b) {
return (b & 0xf0) >> 4;
}
void setLow(ref byte b, byte val) {
b = (b & 0xf0) | val;
}
byte low(byte b) {
return b & 0xf;
}
Should be self-explanatory.

public int SplatBit(int Reg, int Val, int ValLen, int Pos)
{
int mask = ((1 << ValLen) - 1) << Pos;
int newv = Val << Pos;
int res = (Reg & ~mask) | newv;
return res;
}
Example:
Reg = 135
Val = 9 (ValLen = 4, because 9 = 1001)
Pos = 2
135 = 10000111
9 = 1001
9 << Pos = 100100
Result = 10100111

A quick look would indicate that a bitwise and can be achieved using the & operator. So to remove the first four bytes you should be able to do:
byte value1=255; //11111111
byte value2=15; //00001111
return value1&value2;

Assuming newVal contains the value you want to store in origVal.
Do this for the 4 least significant bits:
byte origVal = ???;
byte newVal = ???
orig = (origVal & 0xF0) + newVal;
and this for the 4 most significant bits:
byte origVal = ???;
byte newVal = ???
orig = (origVal & 0xF) + (newVal << 4);

I know you asked specifically about clearing out the first four bits, which has been answered several times, but I wanted to point out that if you have two values <= decimal 15, you can combine them into 8 bits simply with this:
public int setBits(int upperFour, int lowerFour)
{
return upperFour << 4 | lowerFour;
}
The result will be xxxxyyyy where
xxxx = upperFour
yyyy = lowerFour
And that is what you seem to be trying to do.

Here's some code, but I think the earlier answers will do it for you. This is just to show some sort of test code to copy and past into a simple console project (the WriteBits method by be of help):
static void Main(string[] args)
{
int b1 = 255;
WriteBits(b1);
int b2 = b1 >> 4;
WriteBits(b2);
int b3 = b1 & ~0xF ;
WriteBits(b3);
// Store 5 in first nibble
int b4 = 5 << 4;
WriteBits(b4);
// Store 8 in second nibble
int b5 = 8;
WriteBits(b5);
// Store 5 and 8 in first and second nibbles
int b6 = 0;
b6 |= (5 << 4) + 8;
WriteBits(b6);
// Store 2 and 4
int b7 = 0;
b7 = StoreFirstNibble(2, b7);
b7 = StoreSecondNibble(4, b7);
WriteBits(b7);
// Read First Nibble
int first = ReadFirstNibble(b7);
WriteBits(first);
// Read Second Nibble
int second = ReadSecondNibble(b7);
WriteBits(second);
}
static int ReadFirstNibble(int storage)
{
return storage >> 4;
}
static int ReadSecondNibble(int storage)
{
return storage &= 0xF;
}
static int StoreFirstNibble(int val, int storage)
{
return storage |= (val << 4);
}
static int StoreSecondNibble(int val, int storage)
{
return storage |= val;
}
static void WriteBits(int b)
{
Console.WriteLine(BitConverter.ToString(BitConverter.GetBytes(b),0));
}
}