We Keep Coding

Convert MemoryStream to Array C#

I have implemented a code block in order to convert Stream into Byte Array. And code snippet is shown below. But unfortunately, it gives OutOfMemory Exception while converting MemoryStream to Array (return newDocument.ToArray();). please could someone help me with this?
public byte[] MergeToBytes()
{
using (var processor = new PdfDocumentProcessor())
{
AppendStreamsToDocumentProcessor(processor);
using (var newDocument = new MemoryStream())
{
processor.SaveDocument(newDocument);
return newDocument.ToArray();
}
}
}
public Stream MergeToStream()
{
return new MemoryStream(MergeToBytes());
}

Firstly: how big is the document? if it is too big for the byte[] limit: you're going to have to use a different approach.
However, a MemoryStream is already backed by an (oversized) array; you can get this simply using newDocument.TryGetBuffer(out var buffer), and noting that you must restrict yourself to the portion of the .Array indicated by .Offset (usually, but not always, zero) and .Count (the number of bytes that should be considered "live"). Note that TryGetBuffer can return false, but not in the new MemoryStream() scenario.
If is also interesting that you're converting a MemoryStream to a byte[] and then back to a MemoryStream. An alternative here would just have been to set the Position back to 0, i.e. rewind it. So:
public Stream MergeToStream()
{
using var processor = new PdfDocumentProcessor();
AppendStreamsToDocumentProcessor(processor);
var newDocument = new MemoryStream();
processor.SaveDocument(newDocument);
newDocument.Position = 0;
return newDocument;
}

How can I write a Integer to a FileStream in C#?

I was wondering if someone can help me solve a issue I have run into while playing with FileStreams. I have been trying to send an integer, 50, to a FileStream and write its value onto a File. However, it writes 2 to the file instead of 50. I know the ASCII representation of 50 is 2, so am not sure if this is part of the issue. If anyone has any pointers, I'd really appreciate it!
Here is my relevant code:
From the main function:
string testMessage = "Testing writing some arbitrary string to a streama";
int tmL = testMessage.Length;
byte bb = Convert.ToByte(tmL);
SendByteStrem(bb);
And here is my streaming function:
public static void SendByteStrem(byte c){
using (Stream ioStream = new FileStream(#"C:\Users\db0201\Desktop\stream.txt", FileMode.OpenOrCreate)){
ioStream.WriteByte(c);
}
}

As you haven't explicitly stated your goal, i will answer the question for what it is.
The easiest way to write to a file would be to use File.WriteAllText which essentially opens a StreamWriter (which in-turn is open a FileStream) and calls Write
Creates a new file, write the contents to the file, and then closes
the file. If the target file already exists, it is overwritten.
File.WriteAllText(fileName, "50")
or
var myInt = 50;
File.WriteAllText(fileName, myInt.ToString())
If you wanted to use the StreaWriter exclusively
using (varwriter = new StreamWriter(fileName))
writer.Write(myInt.ToString());
If you wanted more configuration over the underlying FileStream
using (var writer = new StreamWriter(new FileStream(fileName, FileMode.CreateNew)))
writer.Write(myInt.ToString());
if you just want to use a FileStream then things get a bit more manual as you will need to convert things to bytes
using (var stream = new FileStream(fileName, FileMode.CreateNew))
{
var bytes = Encoding.UTF8.GetBytes(myInt.ToString());
stream.Write(bytes, 0, bytes.Length);
}

how to append data to a file [duplicate]

I would like to append a byte array to an already existing file (C:\test.exe). Assume the following byte array:
byte[] appendMe = new byte[ 1000 ] ;
File.AppendAllBytes(#"C:\test.exe", appendMe); // Something like this - Yes, I know this method does not really exist.
I would do this using File.WriteAllBytes, but I am going to be using an ENORMOUS byte array, and System.MemoryOverload exception is constantly being thrown. So, I will most likely have to split the large array up into pieces and append each byte array to the end of the file.
Thank you,
Evan

One way would be to create a FileStream with the FileMode.Append creation mode.
Opens the file if it exists and seeks to the end of the file, or
creates a new file.
This would look something like:
public static void AppendAllBytes(string path, byte[] bytes)
{
//argument-checking here.
using (var stream = new FileStream(path, FileMode.Append))
{
stream.Write(bytes, 0, bytes.Length);
}
}

Create a new FileStream.
Seek() to the end.
Write() the bytes.
Close() the stream.

You can also use the built-in FileSystem.WriteAllBytes Method (String, Byte[], Boolean).
public static void WriteAllBytes(
string file,
byte[] data,
bool append
)
Set append to True to append to the file contents; False to overwrite the file contents. Default is False.

I'm not exactly sure what the question is, but C# has a BinaryWriter method that takes an array of bytes.
BinaryWriter(Byte[])
bool writeFinished = false;
string fileName = "C:\\test.exe";
FileStream fs = new FileString(fileName);
BinaryWriter bw = new BinaryWriter(fs);
int pos = fs.Length;
while(!writeFinished)
{
byte[] data = GetData();
bw.Write(data, pos, data.Length);
pos += data.Length;
}
Where writeFinished is true when all the data has been appended, and GetData() returns an array of data to be appended.

you can simply create a function to do this
public static void AppendToFile(string fileToWrite, byte[] DT)
{
using (FileStream FS = new FileStream(fileToWrite, File.Exists(fileToWrite) ? FileMode.Append : FileMode.OpenOrCreate, FileAccess.Write)) {
FS.Write(DT, 0, DT.Length);
FS.Close();
}
}

No map for object error when deserializing object

I have the following C# code which is supposed to serialize arbitrary objects to a string, and then of course deserialize it.
public static string Pack(Message _message)
{
BinaryFormatter formatter = new BinaryFormatter();
MemoryStream original = new MemoryStream();
MemoryStream outputStream = new MemoryStream();
formatter.Serialize(original, _message);
original.Seek(0, SeekOrigin.Begin);
DeflateStream deflateStream = new DeflateStream(outputStream, CompressionMode.Compress);
original.CopyTo(deflateStream);
byte[] bytearray = outputStream.ToArray();
UTF8Encoding encoder = new UTF8Encoding();
string packed = encoder.GetString(bytearray);
return packed;
}
public static Message Unpack(string _packed_message)
{
UTF8Encoding encoder = new UTF8Encoding();
byte[] bytearray = encoder.GetBytes(_packed_message);
BinaryFormatter formatter = new BinaryFormatter();
MemoryStream input = new MemoryStream(bytearray);
MemoryStream decompressed = new MemoryStream();
DeflateStream deflateStream = new DeflateStream(input, CompressionMode.Decompress);
deflateStream.CopyTo(decompressed); // EXCEPTION
decompressed.Seek(0, SeekOrigin.Begin);
var message = (Message)formatter.Deserialize(decompressed); // EXCEPTION 2
return message;
}
But the problem is that any time the code is ran, I am experiencing an exception. Using the above code and invoking it as shown below, I am receiving InvalidDataException: Unknown block type. Stream might be corrupted. at the marked // EXCEPTION line.
After searching for this issue I have attempted to ditch the deflation. This was only a small change: in Pack, bytearray gets created from original.ToArray() and in Unpack, I Seek() input instead of decompressed and use Deserialize(input) instead of decompressed too. The only result which changed: the exception position and body is different, yet it still happens. I receive a SerializationException: No map for object '201326592'. at // EXCEPTION 2.
I don't seem to see what is the problem. Maybe it is the whole serialization idea... the problem is that somehow managing to pack the Message instances is necessary because these objects hold the information that travel between the server and the client application. (Serialization logic is in a .Shared DLL project which is referenced on both ends, however, right now, I'm only developing the server-side first.) It also has to be told, that I am only using string outputs because right now, the TCP connection between the servers and clients are based on string read-write on the ends. So somehow it has to be brought down to the level of strings.
This is how the Message object looks like:
[Serializable]
public class Message
{
public MessageType type;
public Client from;
public Client to;
public string content;
}
(Client right now is an empty class only having the Serializable attribute, no properties or methods.)
This is how the pack-unpack gets invoked (from Main()...):
Shared.Message msg = Shared.MessageFactory.Build(Shared.MessageType.DEFAULT, new Shared.Client(), new Shared.Client(), "foobar");
string message1 = Shared.MessageFactory.Pack(msg);
Console.WriteLine(message1);
Shared.Message mess2 = Shared.MessageFactory.Unpack(message1); // Step into... here be exceptions
Console.Write(mess2.content);
Here is an image showing what happens in the IDE. The output in the console window is the value of message1.
Some investigation unfortunately also revealed that the problem could lie around the bytearray variable. When running Pack(), after the encoder creates the string, the array contains 152 values, however, after it gets decoded in Unpack(), the array has 160 values instead.
I am appreciating any help as I am really out of ideas and having this problem the progress is crippled. Thank you.
(Update) The final solution:
I would like to thank everyone answering and commenting, as I have reached the solution. Thank you.
Marc Gravell was right, I missed the closing of deflateStream and because of this, the result was either empty or corrupted. I have taken my time and rethought and rewrote the methods and now it works flawlessly. And even the purpose of sending these bytes over the networked stream is working too.
Also, as Eric J. suggested, I have switched to using ASCIIEnconding for the change between string and byte[] when the data is flowing in the Stream.
The fixed code lies below:
public static string Pack(Message _message)
{
using (MemoryStream input = new MemoryStream())
{
BinaryFormatter bformatter = new BinaryFormatter();
bformatter.Serialize(input, _message);
input.Seek(0, SeekOrigin.Begin);
using (MemoryStream output = new MemoryStream())
using (DeflateStream deflateStream = new DeflateStream(output, CompressionMode.Compress))
{
input.CopyTo(deflateStream);
deflateStream.Close();
return Convert.ToBase64String(output.ToArray());
}
}
}
public static Message Unpack(string _packed)
{
using (MemoryStream input = new MemoryStream(Convert.FromBase64String(_packed)))
using (DeflateStream deflateStream = new DeflateStream(input, CompressionMode.Decompress))
using (MemoryStream output = new MemoryStream())
{
deflateStream.CopyTo(output);
deflateStream.Close();
output.Seek(0, SeekOrigin.Begin);
BinaryFormatter bformatter = new BinaryFormatter();
Message message = (Message)bformatter.Deserialize(output);
return message;
}
}
Now everything happens just right, as the screenshot proves below. This was the expected output from the first place. The Server and Client executables communicate with each other and the message travels... and it gets serialized and unserialized properly.

In addition to the existing observations about Encoding vs base-64, note you haven't closed the deflate stream. This is important because compression-streams buffer: if you don't close, it may not write the end. For a short stream, that may mean it writes nothing at all.
using(DeflateStream deflateStream = new DeflateStream(
outputStream, CompressionMode.Compress))
{
original.CopyTo(deflateStream);
}
return Convert.ToBase64String(outputStream.GetBuffer(), 0,
(int)outputStream.Length);

Your problem is most probably in the UTF8 encoding. Your bytes are not really a character string and UTF-8 is a encoding with different byte lengths for characters.
This means the byte array may not correspond to a correctly encoded UTF-8 string (there may be some bytes missing at the end for instance.)
Try using UTF16 or ASCII which are constant length encodings (the resulting string will likely contain control characters so it won't be printable or transmitable through something like HTTP or email.)
But if you want to encode as a string it is customary to use UUEncoding to convert the byte array into a real printable string, then you can use any encoding you want.

When I run the following Main() code against your Pack() and Unpack():
static void Main(string[] args)
{
Message msg = new Message() { content = "The quick brown fox" };
string message1 = Pack(msg);
Console.WriteLine(message1);
Message mess2 = Unpack(message1); // Step into... here be exceptions
Console.Write(mess2.content);
}
I see that the bytearray
byte[] bytearray = outputStream.ToArray();
is empty.
I did modify your serialized class slightly since you did not post code for the included classes
public enum MessageType
{
DEFAULT = 0
}
[Serializable]
public class Message
{
public MessageType type;
public string from;
public string to;
public string content;
}
I suggest the following steps to resolve this:
Check the intermediate results along the way. Do you also see 0 bytes in the array? What is the string value returned by Pack()?
Dispose of your streams once you are done with them. The easiest way to do that is with the using keyword.
Edit
As Eli and Marc correctly pointed out, you cannot store arbitrary bytes in a UTF8 string. The mapping is not bijective (you can't go back and forth without loss/distortion of information). You will need a mapping that is bijective, such as the Convert.ToBase64String() approach Marc suggests.

writing and reading an arraylist object to and from file

this is simple I know, but i don't have internet access and this netcafes keyboard sucks, so if someone can answer this question please.
what would be the class ? just give me a kick in the right direction. there is simple arraylist object that I want to write and read to/ from file.
thanks

There's no single definitive answer to this question. It would depend on the format of the file and the objects in the list. You need a serializer. For example you could use BinaryFormatter which serializes an object instance into a binary file but your objects must be serializable. Another option is the XmlSerializer which uses XML format.
UPDATE:
Here's an example with BinaryFormatter:
class Program
{
static void Main()
{
var list = new ArrayList();
list.Add("item1");
list.Add("item2");
// Serialize the list to a file
var serializer = new BinaryFormatter();
using (var stream = File.OpenWrite("test.dat"))
{
serializer.Serialize(stream, list);
}
// Deserialize the list from a file
using (var stream = File.OpenRead("test.dat"))
{
list = (ArrayList)serializer.Deserialize(stream);
}
}
}

Since you did not mention what type of data this array contains, I would suggest writing the file in binary format.
Here is a good tutorial on how to read and write in binary format.
Basically, you need to use BinaryReader and BinaryWriter classes.
[Edited]
private static void write()
{
List<string> list = new List<string>();
list.Add("ab");
list.Add("db");
Stream stream = new FileStream("D:\\Bar.dat", FileMode.Create);
BinaryWriter binWriter = new BinaryWriter(stream);
binWriter.Write(list.Count);
foreach (string _string in list)
{
binWriter.Write(_string);
}
binWriter.Close();
stream.Close();
}
private static void read()
{
List<string> list = new List<string>();
Stream stream = new FileStream("D:\\Bar.dat", FileMode.Open);
BinaryReader binReader = new BinaryReader(stream);
int pos = 0;
int length = binReader.ReadInt32();
while (pos < length)
{
list.Add(binReader.ReadString());
pos ++;
}
binReader.Close();
stream.Close();
}

If your objects in the arraylist are serializable, you can opt for binary serialization. But this means any other application need to know the serialization and then only can use this files. You may like to clarify your intent of using the serialization. So the question remains, why do you need to do a serialization? If it is simple, for you own (this application's) use, you can think of binary serialization. Be sure, your objects are serializable. Otherwise, you need to think of XML serialization.
For Binary serialization, you can think of some code like this:
Stream stream = File.Open("C:\\mySerializedData.Net", FileMode.Create);
BinaryFormatter bformatter = new BinaryFormatter();
bformatter.Serialize(stream, myArray);
stream.Close();