SO I am trying to get a simple dataset from the strInstallDataSet string into the dataset, using the code below, when I have the debugger connected I can see that strInstallDataSet has data, byteArray has data, but even after reading msDataset has nothing, length just sits at 0, I have tried setting the position before and after reading but it still just doesn't pick up any data. Any ideas?
MemoryStream msDataset = new MemoryStream();
if (strInstallDataSet != null)
{
// Convert string to byte array.
byte[] byteArray = Encoding.ASCII.GetBytes(strInstallDataSet);
msDataset.Read(byteArray, 0, byteArray.Length);
// Put stream back into dataset object.
dsInstallData.ReadXml(msDataset);
msDataset.Close();
msDataset.Dispose();
}
You probably want to be doing the following:
using(StringReader reader = new StringReader(strInstallDataSet))
{
dsInstallData.ReadXml(reader);
}
You are not writing anything to the stream, only reading msDataset.Read...
Side note 1: you are using very low level methods - there are Reader/Writer classes that will correctly take care of encoding already.
Side note 2: use "using" instead of manually calling Close or Dispose (and don't call 2 of them together as both do exactly the same things).
You're misunderstanding what the MemoryStream.Read() does, it reads into the byte array, not into the memorystream.
You want MemoryStream.Write() where you have MemoryStream.Read()
Or better yet...
MemoryStream xmlMemoryStream = new MemoryStream(byteArray);
You must use Write method instead of read. I think you want to write your bytearray into your memory stream.
Related
I'm making a chat system thing with tcp which requires to send things in byte arrays, but when I convert an image into a byte array, send it and then convert back it gives this error: 'End of Stream encountered before parsing was completed.'. With strings it works just fine.
public byte[] ObjectToByteArray(object obj)
{
BinaryFormatter formatter = new BinaryFormatter();
using (var stream = new MemoryStream())
{
formatter.Serialize(stream, obj);
return stream.ToArray();
}
}
public object ByteArrayToObject(byte[] bytes)
{
using (var stream = new MemoryStream())
{
var binForm = new BinaryFormatter();
stream.Write(bytes, 0, bytes.Length);
stream.Position = 0;
var obj = binForm.Deserialize(stream);
return obj;
}
}
There's two separate things here; firstly, and I cannot emphasize this enough; do not use BinaryFormatter. Ever. It will hurt you. Lots of serializers exist, and BinaryFormatter (and the cousin NetDataContractSerializer) is literally the absolute last you should use. I can expand on that if you like, or I can suggest alternatives if you like.
Now; as for the actual problem: I strongly suspect that it isn't what you think it is. I have a hunch, based on decades of working on network code, that the real problem here is "framing". By which I mean: TCP is a stream protocol, not a message/packet protocol. I strongly suspect that you have not correctly deframed the exact bytes that were sent. I can't say this for sure without seeing your socket code, but... as I say: it is an hunch based on lots of experience. To investigate this: note the length of the bytes you send, and note the length of the bytes you've received. I'm pretty sure you'll find they are different. If there's still doubt: get the base-64 or hex string of the sent payload and the received payload (Convert.ToBase64String, for example), and compare that string. I'm pretty sure they'll turn out to be different.
Ultimately, network code is hard; I could try and explain individual points, but "how to correctly send messages over a network" could fill a book. IMO, if you're not interested in specializing in writing network code for the next 5 years: use an existing tool that will do the job for you, for example gRPC. Lots and lots of other messaging RPC tools exist.
I would like to be able to get the length of the data available from a TCP network stream in C# to set the size of the buffer before reading from the network stream. There is a NetworkStream.Length property but it isn't implemented yet, and I don't want to allocate an enormous size for the buffer as it would take up too much space. The only way I though of doing it would be to precede the data transfer with another telling the size, but this seems a little messy. What would be the best way for me to go about doing this.
When accessing Streams, you usually read and write data in small chunks (e.g. a kilobyte or so), or use a method like CopyTo that does that for you.
This is an example using CopyTo to copy the contents of a stream to another stream and return it as a byte[] from a method, using an automatically-sized buffer.
using (MemoryStream ms = new MemoryStream())
{
networkStream.CopyTo(ms);
return ms.ToArray();
}
This is code that reads data in the same way, but more manually, which might be better for you to work with, depending on what you're doing with the data:
byte[] buffer = new byte[2048]; // read in chunks of 2KB
int bytesRead;
while((bytesRead = networkStream.Read(buffer, 0, buffer.Length)) > 0)
{
//do something with data in buffer, up to the size indicated by bytesRead
}
(the basis for these code snippets came from Most efficient way of reading data from a stream)
There is no inherent length of a network stream. You will either have to send the length of the data to follow from the other end or read all of the incoming data into a different stream where you can access the length information.
The thing is, you can't really be sure all the data is read by the socket yet, more data might come in at any time. This is try even if you somehow do know how much data to expect, say if you have a package header that contains the length. the whole packet might not be received yet.
If you're reading arbitrary data (like a file perhaps) you should have a buffer of reasonable size (like 1k-10k or whatever you find to be optimal for your scenario) and then write the data to a file as its read from the stream.
var buffer = byte[1000];
var readBytes = 0;
using(var netstream = GetTheStreamSomhow()){
using(var fileStream = (GetFileStreamSomeHow())){
while(netstream.Socket.Connected) //determine if there is more data, here we read until the socket is closed
{
readBytes = netstream.Read(buffer,0,buffer.Length);
fileStrem.Write(buffer,0,buffer.Length);
}
}
}
Or just use CopyTo like Tim suggested :) Just make sure that all the data has indeed been read, including data that hasn't gotten across the network yet.
You could send the lenght of the incoming data first.
For example:
You have data = byte[16] you want to send. So at first you send the 16 and define on the server, that this length is always 2 (because 16 has two characters). Now you know that the incomingLength = 16. You can wait now for data of the lenght incomingLength.
Which one is better : MemoryStream.WriteTo(Stream destinationStream) or Stream.CopyTo(Stream destinationStream)??
I am talking about the comparison of these two methods without Buffer as I am doing like this :
Stream str = File.Open("SomeFile.file");
MemoryStream mstr = new MemoryStream(File.ReadAllBytes("SomeFile.file"));
using(var Ms = File.Create("NewFile.file", 8 * 1024))
{
str.CopyTo(Ms) or mstr.WriteTo(Ms);// Which one will be better??
}
Update
Here is what I want to Do :
Open File [ Say "X" Type File]
Parse the Contents
From here I get a Bunch of new Streams [ 3 ~ 4 Files ]
Parse One Stream
Extract Thousands of files [ The Stream is an Image File ]
Save the Other Streams To Files
Editing all the Files
Generate a New "X" Type File.
I have written every bit of code which is actually working correctly..
But Now I am optimizing the code to make the most efficient.
It is an historical accident that there are two ways to do the same thing. MemoryStream always had the WriteTo() method, Stream didn't acquire the CopyTo() method until .NET 4.
The MemoryStream.WriteTo() version looks like this:
public virtual void WriteTo(Stream stream)
{
// Exception throwing code elided...
stream.Write(this._buffer, this._origin, this._length - this._origin);
}
The Stream.CopyTo() implementation like this:
private void InternalCopyTo(Stream destination, int bufferSize)
{
int num;
byte[] buffer = new byte[bufferSize];
while ((num = this.Read(buffer, 0, buffer.Length)) != 0)
{
destination.Write(buffer, 0, num);
}
}
Stream.CopyTo() is more universal, it works for any stream. And helps programmers that fumble copying data from, say, a NetworkStream. Forgetting to pay attention to the return value from Read() was a very common bug. But it of course copies the bytes twice and allocates that temporary buffer, MemoryStream doesn't need it since it can write directly from its own buffer. So you'd still prefer WriteTo(). Noticing the difference isn't very likely.
MemoryStream.WriteTo: Writes the entire contents of this memory stream to another stream.
Stream.CopyTo: Reads the bytes from the current stream and writes them to the destination stream. Copying begins at the current position in the current stream.
You'll need to seek back to 0, to get the whole source stream copied.
So I think MemoryStream.WriteTo better option for this situation
If you use Stream.CopyTo, you don't need to read all the bytes into memory to start with. However:
This code would be simpler if you just used File.Copy
If you are going to load all the data into memory, you can just use:
byte[] data = File.ReadAllBytes("input");
File.WriteAllBytes("output", data);
You should have a using statement for the input as well as the output stream
If you really need processing so can't use File.Copy, using Stream.CopyTo will cope with larger files than loading everything into memory. You may not need that, of course, or you may need to load the whole file into memory for other reasons.
If you have got a MemoryStream, I'd probably use MemoryStream.WriteTo rather than Stream.CopyTo, but it probably won't make much difference which you use, except that you need to make sure you're at the start of the stream when using CopyTo.
I think Hans Passant's claim of a bug in MemoryStream.WriteTo() is wrong; it does not "ignore the return value of Write()". Stream.Write() returns void, which implies to me that the entire count bytes are written, which implies that Stream.Write() will block as necessary to complete the operation to, e.g., a NetworkStream, or throw if it ultimately fails.
That is indeed different from the write() system call in ?nix, and its many emulations in libc and so forth, which can return a "short write". I suspect Hans leaped to the conclusion that Stream.Write() followed that, which I would have expected, too, but apparently it does not.
It is conceivable that Stream.Write() could perform a "short write", without returning any indication of that, requiring the caller to check that the Position property of the Stream has actually been advanced by count. That would be a very error-prone API, and I doubt that it does that, but I have not thoroughly tested it. (Testing it would be a bit tricky: I think you would need to hook up a TCP NetworkStream with a reader on the other end that blocked forever, and write enough to fill up the wire buffers. Or something like that...)
The comments for Stream.Write() are not quite unambiguous:
Summary:
When overridden in a derived class, writes a sequence of bytes to the current
stream and advances the current position within this stream by the number
of bytes written.
Parameters: buffer:
An array of bytes. This method copies count bytes from buffer to the current stream.
Compare that to the Linux man page for write(2):
write() writes up to count bytes from the buffer pointed buf to the file referred to by the file descriptor fd.
Note the crucial "up to". That sentence is followed by explanation of some of the conditions under which a "short write" might occur, making it very explicit that it can occur.
This is really a critical issue: we need to know how Stream.Write() behaves, beyond all doubt.
The CopyTo method creates a buffer, populates its with data from the original stream and then calls the Write method passing the created buffer as a parameter. The WriteTo uses the memoryStream's internal buffer to write. That is the difference. What is better - it is up to you to decide which method you prefer.
Creating a MemoryStream from a HttpInputStream in Vb.Net:
Dim filename As String = MyFile.PostedFile.FileName
Dim fileData As Byte() = Nothing
Using binaryReader = New BinaryReader(MyFile.PostedFile.InputStream)
binaryReader.BaseStream.Position = 0
fileData = binaryReader.ReadBytes(MyFile.PostedFile.ContentLength)
End Using
Dim memoryStream As MemoryStream = New MemoryStream(fileData)
I am making an application in c#. In that application I have one byte array and I want to write that byte array data to particular position.
Here i used the following logic.
using(StreamWriter writer=new StreamWriter(#"D:\"+ FileName + ".txt",true))
{
writer.WriteLine(Encoding.ASCII.GetString(Data),IndexInFile,Data.Length);
}
But whenever i am writing data in file, it starts writing from starting.
My condition is that suppose at the initial I have empty file and I want to start writing in file from position 10000. Please help me .Thanks in advance.
Never try to write binary data as strings, like you do. It won't work correctly. Write binary data as binary data. You can use Stream for that instead of StreamWriter.
using (Stream stream = new FileStream(fileName, FileMode.OpenOrCreate))
{
stream.Seek(1000, SeekOrigin.Begin);
stream.Write(Data, 0, Data.Length);
}
You can set the position inside the stream like this:
writer.BaseStream.Seek( 1000, SeekOrigin.Begin);
Add this before your WriteLine code.
Note that I have not included any code to check that there is at least 1000 chars inside the file to start with.
The WriteLine overload you are using is this one:
TextWriter.WriteLine Method (Char[], Int32, Int32)
In particular the IndexInFile argument supplied is actually the index in the buffer from which to begin reading not the index in the file at which to begin writing - this explains why you are writing at the start of the file not at IndexInFile as you expect.
You should obtain access to the underlying Stream and Seek to the desired position in the file first and then write to the file.
Maybe a bit hacky, but you can access the BaseStream property of the StreamWriter and use Seek(long offset, SeekOrigin origin) on it. (But be warned, as not every stream can use Seek.)
Im trying to write a single byte at a certain location in a file. This is what im using at the moment:
BinaryWriter bw = new BinaryWriter(File.Open(filename, FileMode.Open));
bw.BaseStream.Seek(0x6354C, SeekOrigin.Begin);
bw.Write(0xB0);
bw.Close();
The problem is that BinaryWriter.Write(args) writes a four-byte signed integer at the position. I wish to only write one byte at the particular location. And then later possibly two bytes else where, how I specify how many bytes to write?
change
bw.Write(0xB0);
to
bw.Write((byte)0xB0);
There is absolutely no need to use a high-level BinaryWriter just to write a simple byte to a stream - It's more efficient and tidy just to do this:
Stream outStream = File.Open(filename, FileMode.Open);
outStream.Seek(0x6354C, SeekOrigin.Begin);
outStream.WriteByte(0xb0);
(In general you also shouldn't really Seek after attaching a BinaryWriter to your stream - the BinaryWriter should be in control of the stream, and changing things "behind its back" is a bit dirty)
You could cast to byte:
bw.Write((byte)0xB0);
This should cause the correct overloaded version of Write to be invoked.