I need to create a text file (alphanumeric) with size (exact or nearly) in MB equal to my input number, such as 1 MB. I tried to generated a string, with think that one char is 16 bit or 2 byte, so:
1KB = 1024 Bytes = 1024 x 8 Bits = 1024 x 8 / 16 chars = 512 chars.
But the generated file is not very seem to be right :(
If I generate byte array and write it to file, the size is pretty correct!
public static void generateDummyFileBySize(long sizeInMb, string filePath) {
byte[] data = new byte[sizeInMb * 1024 * 1024];
Random rng = new Random();
rng.NextBytes(data);
File.WriteAllBytes(filePath, data);
}
Help me please. This is the code I used to generated text file
public static String generateStringSize(long sizeByte)
{
StringBuilder sb = new StringBuilder();
Random rd = new Random();
/**
* Byte -> Bit -> Divide by 16 to num of char
* */
var numOfChars = sizeByte * 8 ;
string allows = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
int maxIndex = allows.Length - 1;
for (int i = 0; i < numOfChars; i++)
{
int index = rd.Next(maxIndex);
char c = allows[index];
sb.Append(c);
}
return sb.ToString();
}
public static void generateTextFileBySize(long size, string fileName)
{
long sizeOld = size;
try
{
String s;
String path = Directory.GetCurrentDirectory();
var physicPath = path + Path.DirectorySeparatorChar + fileName;
File.Delete(physicPath);
if (size <= MAX_SIZE)
{
s = StringUtil.generateStringSize(size);
Console.WriteLine("Generated a string with length " + size);
File.WriteAllText(physicPath, s, Encoding.UTF8);
}
else
{
while (size > MAX_SIZE)
{
s = StringUtil.generateStringSize(MAX_SIZE);
Console.WriteLine("Appending " + MAX_SIZE + " to file");
File.AppendAllText(physicPath, s, Encoding.UTF8);
size -= MAX_SIZE;
}
s = StringUtil.generateStringSize(size);
File.AppendAllText(physicPath, s, Encoding.UTF8);
}
Console.WriteLine("Created file named " + fileName + " with size " + sizeOld + " bytes.");
}
catch (Exception ex)
{
Console.WriteLine("Error when generating file");
Console.WriteLine(ex.Message);
}
}
The problem lies here:
var numOfChars = sizeByte * 8 ;
You create eight times the characters you need. Instead just use
var numOfChars = sizeByte;
Each of the characters you use occupies one byte when stored as UTF-8, so you don't need to use a different number of characters than the number of bytes you want.
Made following changes to your code:
Removed 8 from var numOfChars = sizeByte*8;, since in ASCII encoding, each character is 1 byte also changed the Encoding to ASCII instead of UTF8 to be precise for characters.
Following is the working code, generates precise 1 MB file, I have tested for other values like 8 MB, 7 MB, result is accurate
String s = generateStringSize(1024 * 1024);
File.WriteAllText("D:\\Test1.txt", s, Encoding.ASCII);
Related
Im trying to read a raw image file with C# and keep the last 2 bytes, and last 8 bytes in a variable. However there is something wrong in my if-statement so the variables just appends.. Like this:
twoBytes= eightBytes=
twoBytes=00 eightBytes=00
twoBytes=0000 eightBytes=0000
twoBytes=000000 eightBytes=000000
twoBytes=00000000 eightBytes=00000000
twoBytes=0000000000 eightBytes=0000000000
twoBytes=000000000000 eightBytes=000000000000
twoBytes=00000000000000 eightBytes=00000000000000
twoBytes=0000000000000000 eightBytes=0000000000000000
twoBytes=000000000000000000 eightBytes=000000000000000000
twoBytes=00000000000000000000 eightBytes=00000000000000000000
twoBytes=0000000000000000000000 eightBytes=0000000000000000000000
twoBytes=000000000000000000000000 eightBytes=000000000000000000000000
twoBytes=00000000000000000000000000 eightBytes=00000000000000000000000000
twoBytes=0000000000000000000000000000 eightBytes=0000000000000000000000000000
twoBytes=000000000000000000000000000000 eightBytes=000000000000000000000000000000
twoBytes=00000000000000000000000000000000 eightBytes=00000000000000000000000000000000
twoBytes=0000000000000000000000000000000000 eightBytes=0000000000000000000000000000000000
twoBytes=000000000000000000000000000000000000 eightBytes=000000000000000000000000000000000000
twoBytes=00000000000000000000000000000000000000 eightBytes=00000000000000000000000000000000000000
I want something like "twoBytes=55AA", and eightBytes="55AA454649205041"
My code:
// Read file, byte at the time (example 00, 5A)
FileStream fs = new FileStream("C:\\Users\\user\\image_files\\usb_guid_exfat.001", FileMode.Open);
int hexIn;
String hex;
String twoBytes = "";
String eightBytes = "";
for (int i = 0; (hexIn = fs.ReadByte()) != -1; i++)
{
hex = string.Format("{0:X2}", hexIn);
Console.WriteLine("twoBytes=" + twoBytes + " eightBytes=" + eightBytes);
// Transfer two bytes
twoBytes = twoBytes + hex;
if (twoBytes.Length < 4)
{
if (twoBytes.Length > 6) {
twoBytes = twoBytes.Substring(2, 4);
}
}
// Transfer eight bytes
eightBytes = eightBytes + hex;
if(eightBytes.Length < 8)
{
if (twoBytes.Length > 10) {
eightBytes = eightBytes.Substring(2, 8);
}
}
}
Your if statements are wrong. A value can't be less than 4 and greater than 6 at the same time.
If length is <=4, you have 1 or 2 bytes, so you need to inspect only if length is grater than 4 (6,8,etc). Otherwise the value stays the same.
The code for inspecting string bigger than 4:
twoBytes = twoBytes + hex;
if (twoBytes.Length > 4) {
twoBytes = twoBytes.Substring(twoBytes.Length-4, 4);
}
The similar with eightBytes.
Good luck!! :)
I'm building a project that includes file transferring through sockets. My server is written in python and my client is in C#. Since Python isn't very friendly when it comes to charsets, I'm transferring a file I'm uploading from the client by converting it into base64, and decoding it in python. It works perfectly. For some reason, when I do the opposite, when I encode the text in python and decode after transferring it throws errors.
Have a look -
This is the correct sending from client to server:
List<byte> b = Encoding.ASCII.GetBytes(Convert.ToBase64String(cont)).ToList();
int size = Encoding.ASCII.GetByteCount(st) + b.Count;
string pack = size + ":" + st;
buffer = Encoding.ASCII.GetBytes(pack);
List<byte> a = buffer.ToList();
a.AddRange(b);
connection.Send(a.ToArray());
And python:
base64.b64decode(params[2])
And this works.
When I do opposite(with the same libraries it makes an error):
string res = SendRecv("1?" + basepath + v[0]);
res = res.Remove(res.Length - 1).Substring(1);//because it is sent quoted
byte[] converted = Convert.FromBase64String(res.Replace(" ",String.Empty));
saved.Write(converted, 0, converted.Length);
saved.Close();
The SendRecv(send, and recv) methods:
private void Send(string st)
{
int size = Encoding.ASCII.GetByteCount(st);
string pack = size + ":" + st;
buffer = Encoding.ASCII.GetBytes(pack);
connection.Send(buffer);
}
private string Recv()
{
try
{
buffer = new byte[2];
connection.Receive(buffer, 2, SocketFlags.Partial);
string header = Encoding.ASCII.GetString(buffer, 0, 2);
while (!header.Contains(":"))
{
connection.Receive(buffer, 2, SocketFlags.Partial);
header += Encoding.ASCII.GetString(buffer, 0, 2);
}
int size = int.Parse(header.Split(':')[0]);
string mes0 = header.Split(':')[1];
buffer = new byte[size];
int b = 0;
int s = (size >= 2048) ? 2048 : size;
while(size-s > 0)
{
connection.Receive(buffer,b,s,SocketFlags.None);
size -= s;
s = (size >= 2048) ? 2048 : size;
b += s;
}
connection.Receive(buffer, size, SocketFlags.None);
string fullmes = mes0 + Encoding.ASCII.GetString(buffer);
return fullmes;
}
catch(Exception e)
{
MessageBox.Show(e.ToString());
}
return "";
}
private string SendRecv(string a)
{
Send(a);
return Recv();
}
Python:
return base64.b64encode(self.finalResult.getContent())
And it throws this exception:
The input is not a valid Base-64 string as it contains a non-base 64 character, more than two padding characters, or a non-white space character among the padding characters
I have a random integer value which I need to represent in String as a Byte array. For example:
int value = 32;
String strValue = getStringByteArray(value);
Console.WriteLine(strValue); // should write: " \0\0\0"
If value = 11 then getStringByteArray(value) shuld return "\v\0\0\0".
If value = 13 then getStringByteArray(value) shuld return "\r\0\0\0".
And so on.
Any idea on how to implement the method getStringByteArray(int value) in C#?
UPDATE
This is the code that receives the data from the C# NamedPipe Server:
bool CFilePipe::ReadString(int m_handle, string &value)
{
//--- check for data
if(WaitForRead(sizeof(int)))
{
ResetLastError();
int size=FileReadInteger(m_handle);
if(GetLastError()==0)
{
//--- check for data
if(WaitForRead(size))
{
value=FileReadString(m_handle,size);
return(size==StringLen(value));
}
}
}
//--- failure
return(false);
}
Don't take this approach at all. You should be writing to a binary stream of some description - and write the binary data for the length of the packet/message, followed by the message itself. For example:
BinaryWriter writer = new BinaryWriter(stream);
byte[] data = Encoding.UTF8.GetBytes(text);
writer.Write(data.Length);
writer.Write(data);
Then at the other end, you'd use:
BinaryReader reader = new BinaryReader(stream);
int length = reader.ReadInt32();
byte[] data = reader.ReadBytes(length);
string text = Encoding.UTF8.GetString(data);
No need to treat binary data as text at all.
Well. First of all you should get bytes from integer. You can do it with BitConverter:
var bytes = BitConverter.GetBytes(value);
Next, here is three variants. First - if you want to get result in binary format. Just take all your bytes and write as it is:
var str = string.Concat(bytes.Select(b => Convert.ToString(b, 2)));
Second variant. If you want convert your byte array to hexadecimal string:
var hex = BitConverter.ToString(array).Replace("-","");
Third variant. Your representation ("\v\0\0\0") - it is simple converting byte to char. Use this:
var s = bytes.Aggregate(string.Empty, (current, t) => current + Convert.ToChar(t));
This should help with that.
class Program
{
static void Main(string[] args)
{
Random rand = new Random();
int number = rand.Next(1, 1000);
byte[] intBytes = BitConverter.GetBytes(number);
string answer = "";
for (int i = 0; i < intBytes.Length; i++)
{
answer += intBytes[i] + #"\";
}
Console.WriteLine(answer);
Console.WriteLine(number);
Console.ReadKey();
}
}
Obviously, you should implement two steps to achieve the goal:
Extract bytes from the integer in the appropriate order (little-endian or big-endian, it's up to you to decide), using bit arithmetics.
Merge extracted bytes into string using the format you need.
Possible implementation:
using System;
using System.Text;
public class Test
{
public static void Main()
{
Int32 value = 5152;
byte[] bytes = new byte[4];
for (int i = 0; i < 4; i++)
{
bytes[i] = (byte)((value >> i * 8) & 0xFF);
}
StringBuilder result = new StringBuilder();
for (int i = 0; i < 4; i++)
{
result.Append("\\" + bytes[i].ToString("X2"));
}
Console.WriteLine(result);
}
}
Ideone snippet: http://ideone.com/wLloo1
I think you are saying that you want to convert each byte into a character literal, using escape sequences for the non printable characters.
After converting the integer to 4 bytes, cast to char. Then use Char.IsControl() to identify the non-printing characters. Use the printable char directly, and use a lookup table to find the corresponding escape sequence for each non-printable char.
my question is a continuation of this: (loop for reading different data types & sizes off very large byte array from file)
I have a raw byte stream stored on a file (rawbytes.txt or bytes.data) that I need to parse and output to a CSV-style text file.
The input of raw bytes (when read as characters/long/int etc.) looks something like this:
A2401028475764B241102847576511001200C...
Parsed it should look like:
OutputA.txt
(Field1,Field2,Field3) - heading
A,240,1028475764
OutputB.txt
(Field1,Field2,Field3,Field4,Field5) - heading
B,241,1028475765,1100,1200
OutputC.txt
C,...//and so on
Essentially, it's a hex-dump-style input of bytes that is continuous without any line terminators or gaps between data that needs to be parsed. The data, as seen above, consists of different data types one after the other.
Here's a snippet of my code - because there are no commas within any field, and no need arises to use "" (i.e. a CSV wrapper), I'm simply using TextWriter to create the CSV-style text file as follows:
if (File.Exists(fileName))
{
using (BinaryReader reader = new BinaryReader(File.Open(fileName, FileMode.Open)))
{
while (reader.BaseStream.Position != reader.BaseStream.Length)
{
inputCharIdentifier = reader.ReadChar();
switch (inputCharIdentifier)
case 'A':
field1 = reader.ReadUInt64();
field2 = reader.ReadUInt64();
field3 = reader.ReadChars(10);
string strtmp = new string(field3);
//and so on
using (TextWriter writer = File.AppendText("outputA.txt"))
{
writer.WriteLine(field1 + "," + field2 + "," + strtmp); // +
}
case 'B':
//code...
My question is based on the fact that some of the raw byte data contains null values, which are difficult to parse through - because there are an unknown number of null bytes (or non-null, out-of-place bytes) between consecutive data blocks (each starting with A, B or C if the data blocks are not corrupt).
QUESTION
So, how do I add a default case or some other mechanism to continue with the loop despite an error that might arise because of corrupt or faulty data? Is the following code something that would work?
inputCharIdentifier = reader.ReadChar();
...
case default:
//I need to know what to add here, instead of default
//(i.e. the case when the character could not be read)
while (binReader.PeekChar() != -1)
{
filling = binReader.readByte();
//filling is a single byte
try {
fillingChar = Convert.ToChar(filling);
break;
}
catch (Exception ex) { break; }
if (fillingChar == 'A' || fillingChar == 'B')
break;
The remaining part - adding code to each switch case (eg 'A') to continue without stopping the program - is there a way to do this without multiple try-catch blocks? [i.e. the code block character identifier is A, but the bytes after A are corrupt - in which case i need to exit the loop OR read (i.e. skip over) a defined number of bytes - which here, would be known if the message header correctly identifies the remaining bytes.
[Note: Case A, B and so on have different sized input - in other words, A might be 40 bytes total, while B is 50 bytes. So the use of a fixed size buffer, say inputBuf[1000], or [50] for instance - if they were all the same size - wouldn't work well either, AFAIK.]
Any suggestions? Please help! I'm relatively new to C# (2 months)...
Update: my entire code is as follows:
class Program
{
const string fileName = "rawbytes.txt";
static void Main(string[] args)
{
try
{
var program = new Program();
program.Parser();
}
catch (Exception e)
{
Console.WriteLine(e);
}
Console.ReadLine();
}
public void Parser()
{
char inputCharIdentifier = 'Z';
//only because without initializing inputCharIdentifier I ended up with an error
//note that in the real code, 'Z' is not a switch-case alphabet
//it's an "inconsequential character", i.e. i have defined it to be 'Z'
//just to avoid that error, and to avoid leaving it as a null value
ulong field1common = 0;
ulong field2common = 0;
char[] charArray = new char[10];
char char1;
char char2;
char char3;
int valint1 = 0;
int valint2 = 0;
int valint3 = 0;
int valint4 = 0;
int valint5 = 0;
int valint6 = 0;
int valint7 = 0;
double valdouble;
/*
char[] filler = new char[53];
byte[] filling = new byte[4621];
byte[] unifiller = new byte[8];
//these values above were temporary measures to manually filter through
//null bytes - unacceptable for the final program
*/
if (File.Exists(fileName))
{
using (BinaryReader reader = new BinaryReader(File.Open(fileName, FileMode.Open)))
{
while (reader.BaseStream.Position != reader.BaseStream.Length)
{
//inputCharIdentifier = reader.ReadChar();
//if (inputCharIdentifier != null)
//{
try
{
inputCharIdentifier = reader.ReadChar();
try
{
switch (inputCharIdentifier)
{
case 'A':
field1common = reader.ReadUInt64();
field2common = reader.ReadUInt64();
//unifiller = reader.ReadBytes(8);
//charArray = reader.ReadString();
//result.ToString("o");
//Console.WriteLine(result.ToString());
charArray = reader.ReadChars(10);
string charArraystr = new string(charArray);
char1 = reader.ReadChar();
valint1 = reader.ReadInt32();
valint2 = reader.ReadInt32();
valint3 = reader.ReadInt32();
valint4 = reader.ReadInt32();
using (TextWriter writer = File.AppendText("A.txt"))
{
writer.WriteLine(field1common + "," + /*result.ToString("o")*/ field2common + "," + charArraystr + "," + char1 + "," + valint1 + "," + valint2 + "," + valint3 + "," + valint4);
writer.Close();
}
break;
case 'B':
case 'C':
field1common = reader.ReadUInt64();
field2common = reader.ReadUInt64();
//charArray = reader.ReadString();
charArray = reader.ReadChars(10);
string charArraystr2 = new string(charArray);
char1 = reader.ReadChar();
valint1 = reader.ReadInt32();
valint2 = reader.ReadInt32();
using (TextWriter writer = File.AppendText("C.txt"))
{
writer.WriteLine(field1common + "," + result2.ToString("o") + "," + charArraystr2 + "," + char1 + "," + valint1 + "," + valint2);
writer.Close();
}
break;
case 'S':
//market status message
field1common = reader.ReadUInt64();
char2 = reader.ReadChar();
char3 = reader.ReadChar();
break;
case 'L':
filling = reader.ReadBytes(4);
break;
case 'D':
case 'E':
field1common = reader.ReadUInt64();
field2common = reader.ReadUInt64();
//charArray = reader.ReadString();
charArray = reader.ReadChars(10);
string charArraystr3 = new string(charArray);
//char1 = reader.ReadChar();
valint1 = reader.ReadInt32();
valint2 = reader.ReadInt32();
valint5 = reader.ReadInt32();
valint7 = reader.ReadInt32();
valint6 = reader.ReadInt32();
valdouble = reader.ReadDouble();
using (TextWriter writer = File.AppendText("D.txt"))
{
writer.WriteLine(field1common + "," + result3.ToString("o") + "," + charArraystr3 + "," + valint1 + "," + valint2 + "," + valint5 + "," + valint7 + "," + valint6 + "," + valdouble);
writer.Close();
}
break;
}
}
catch (Exception ex)
{
Console.WriteLine("Parsing didn't work");
Console.WriteLine(ex.ToString());
break;
}
}
catch (Exception ex)
{
Console.WriteLine("Here's why the character read attempt didn't work");
Console.WriteLine(ex.ToString());
continue;
//continue;
}
//}
}
}
}
}
The error I receive is as follows:
Here's why the character read attempt didn't work
System.ArgumentException: The output char buffer is too small to contain the decoded characters, encoding 'Unicode (UTF-8)' fallback 'System.Text.DecoderReplacementFallback'.
Parameter name: chars
at System.Text.Encoding.ThrowCharsOverflow()
at System.Text.Encoding.ThrowCharsOverflow(DecoderNLS decoder, Boolean nothingDecoded)
at System.Text.UTF8Encoding.GetChars(Byte* bytes, Int32 byteCount, Char* chars, Int32 charCount, DecoderNLS baseDecoder)
at System.Text.DecoderNLS.GetChars(Byte* bytes, Int32 byteCount, Char* chars, Int32 charCount, Boolean flush)
at System.Text.DecoderNLS.GetChars(Byte[] bytes, Int32 byteIndex, Int32 byteCount, Char[] chars, Int32 charIndex, Boolean flush)
at System.Text.DecoderNLS.GetChars(Byte[] bytes, Int32 byteIndex, Int32 byteCount, Char[] chars, Int32 charIndex)
at System.IO.BinaryReader.InternalReadOneChar()
at System.IO.BinaryReader.Read()
at System.IO.BinaryReader.ReadChar()
at line 69: i.e. inputCharIdentifier = reader.ReadChar();
Update: A sample file that generates the same error above is at the following link:
http://www.wikisend.com/download/106394/rawbytes.txt
Notice in particular the 8 unexpected null bytes between successive data blocks, even though the data block header - i.e. inputCharIdentifier - is valid. The number of bytes that follows such a header is always unpredictable and generally varies. My issue is that I need to be able to either delete or skip over such a situation when it arises to the next non-corrupt data block available - in the case of the sample file, the last (single) data block that occurs after the 8 out-of-place null bytes.
The 8 null bytes can be located in the file as follows:
Byte Counter: 1056
Line 2, Column 783 (according to Notepad++)
The crux of the problem is that the 8 null bytes can be any size - 3, 7, 15, 50, etc. It is always unknown - as a direct result of data corruption. But unlike "traditional" data corruption, i.e. where a fixed number of bytes, say 50, inside a data block that might be unreadable and can therefore be skipped over (by that exact number of bytes) - the data corruption i face consists of an unknown number of bytes between valid data blocks.
You cannot assign a case for these situations because the target variable (inputCharIdentifier) is null; thus it is enough with a condition avoiding these cases. I have also included a try...catch, just to make completely sure (any error while performing all the given actions would make the code to automatically skip to the following iteration).
try
{
using (BinaryReader reader = new BinaryReader(File.Open(fileName, FileMode.Open), Encoding.ASCII))
{
while (reader.BaseStream.Position != reader.BaseStream.Length)
{
inputCharIdentifier = reader.ReadChar();
if(inputCharIdentifier != null)
{
switch (inputCharIdentifier)
case 'A':
field1 = reader.ReadUInt64();
field2 = reader.ReadUInt64();
field3 = reader.ReadChars(10);
string strtmp = new string(field3);
//and so on
using (TextWriter writer = File.AppendText("outputA.txt"))
{
writer.WriteLine(field1 + "," + field2 + "," + strtmp);
}
case 'B':
//code...
}
}
}
}
catch
{
}
Have a problem, much like this post: How to read a .NET Guid into a Java UUID.
Except, from a remote svc I get a hex str formatted like this: ABCDEFGH-IJKL-MNOP-QRST-123456.
I need to match the GUID.ToByteArray() generated .net byte array GH-EF-CD-AB-KL-IJ-OP-MN- QR- ST-12-34-56 in Java for hashing purposes.
I'm kinda at a loss as to how to parse this. Do I cut off the QRST-123456 part and perhaps use something like the Commons IO EndianUtils on the other part, then stitch the 2 arrays back together as well? Seems way too complicated.
I can rearrange the string, but I shouldn't have to do any of these. Mr. Google doesn't wanna help me neither..
BTW, what is the logic in Little Endian land that keeps those last 6 char unchanged?
Yes, for reference, here's what I've done {sorry for 'answer', but had trouble formatting it properly in comment}:
String s = "3C0EA2F3-B3A0-8FB0-23F0-9F36DEAA3F7E";
String[] splitz = s.split("-");
String rebuilt = "";
for (int i = 0; i < 3; i++) {
// Split into 2 char chunks. '..' = nbr of chars in chunks
String[] parts = splitz[i].split("(?<=\\G..)");
for (int k = parts.length -1; k >=0; k--) {
rebuilt += parts[k];
}
}
rebuilt += splitz[3]+splitz[4];
I know, it's hacky, but it'll do for testing.
Make it into a byte[] and skip the first 3 bytes:
package guid;
import java.util.Arrays;
public class GuidConvert {
static byte[] convertUuidToBytes(String guid) {
String hexdigits = guid.replaceAll("-", "");
byte[] bytes = new byte[hexdigits.length()/2];
for (int i = 0; i < bytes.length; i++) {
int x = Integer.parseInt(hexdigits.substring(i*2, (i+1)*2), 16);
bytes[i] = (byte) x;
}
return bytes;
}
static String bytesToHexString(byte[] bytes) {
StringBuilder buf = new StringBuilder();
for (byte b : bytes) {
int i = b >= 0 ? b : (int) b + 256;
buf.append(Integer.toHexString(i / 16));
buf.append(Integer.toHexString(i % 16));
}
return buf.toString();
}
public static void main(String[] args) {
String guid = "3C0EA2F3-B3A0-8FB0-23F0-9F36DEAA3F7E";
byte[] bytes = convertUuidToBytes(guid);
System.err.println("GUID = "+ guid);
System.err.println("bytes = "+ bytesToHexString(bytes));
byte[] tail = Arrays.copyOfRange(bytes, 3, bytes.length);
System.err.println("tail = "+ bytesToHexString(tail));
}
}
The last group of 6 bytes is not reversed because it is an array of bytes. The first four groups are reversed because they are a four-byte integer followed by three two-byte integers.