I'm using wcf to implement a distributed application using multiple services.
One of my services must use MSMQ binding in order to receive requests asynchronously, so the requests aren't ignored when it's offline. The problem with MSMQ binding is that it is one way only and therefore I can't have methods returning within the contract.
So I had to make another endpoint with a wshttpbinding.
Now I have a problem that it might even not bee related to wcf services.
The service with the msmq binding writes in a file which is read by the other service. The problem is that when the first service makes changes to the file I must restart both in order to the second one read those changes.
I'm writing the information from the file using the following:
string filename = "holder.txt";
if (!File.Exists(filename))
File.Create(filename);
Stream stream = File.Open(filename, FileMode.Create);
BinaryFormatter bFormatter = new BinaryFormatter();
bFormatter.Serialize(stream, requests_list);
stream.Close();
I close the stream so shouldn't the changes made been seen from the other service when reading it? using:
string filename = "holder.txt";
if (!File.Exists(filename))
return null;
Requests requests_list;
Stream stream = File.Open(filename, FileMode.Open);
BinaryFormatter bFormatter = new BinaryFormatter();
try
{
requests_list = (Requests)bFormatter.Deserialize(stream);
}
catch (Exception)
{
stream.Close();
return null;
}
stream.Close();
return requests_list;
Btw both methods are in a class shared by both services.
Thanks in advance
Related
I would like to send a stream of data(huge file > 2GB), to a WCF service, process it then return the processed data as a stream (transferMode = "Streamed" ), without buffering the entire stream in memory then sending it back.
I know the traditional approach of streaming data in and out(outside of a WCF operation) involves
At the consuming end, sending a stream to a (WCF service) void method with an input Stream and an output Stream as parameters.
At the consuming end, also having a receiving stream to get the
incoming,processed output Stream
In the method, processing that input Stream then writing the
processed bytes to the output Stream
those processed bytes are what is received through the output Stream
This way, the stream flow is not broken.
E.g from a Microsoft sample:
void CopyStream(System.IO.Stream instream, System.IO.Stream outstream)
{
//read from the input stream in 4K chunks
//and save to output stream
const int bufferLen = 4096;
byte[] buffer = new byte[bufferLen];
int count = 0;
while ((count = instream.Read(buffer, 0, bufferLen)) > 0)
{
outstream.Write(buffer, 0, count);
}
}
I would like to do the same, only that outputstream would be a WCF return type. Is that possible? How do I do it with transferMode = "Streamed"?
Using WCF, you cannot have more than one parameter(or message contract object) of Stream type when you want to use transferMode = "Streamed"
Hypothetically, with psuedocode like this:
Stream StreamAndReturn(System.IO.Stream instream)
{
Stream outstream = new MemoryStream();//instantiate outstream - probably should be buffered?
while ((count = instream.Read(buffer, 0, bufferLen)) > 0)
{
//some operation on instream that will
SomeOperation(instream,outstream);
}
return outstream; //obviously this will close break the streaming
}
I also tried NetTcpBinding with SessionMode set to SessionMode.Allowed, hoping to have a session that I can start,send the stream data to the service, get the results in a separate stream then using the OperationContext, retrieve whatever property will be associated with with that service instance. But it did not retain the session information, see below:
According to MSDN documentation I should also set InstanceContextMode = InstanceContextMode.PerSession and ConcurrencyMode = ConcurrencyMode.Multiple (see last paragraph)
For that, I Asked a question on SO, but still waiting for an answer. I am thinking maybe there is a better way to do it.
My recommendation would be for you to create two methods:
One that takes your input as a stream and returns an ID number generated on the server.
Another that takes the ID number, and returns the response stream.
You'll need some way of coordinating the input and output on the server, and it will make it a little more complicated if you put this on multiple servers behind a load balancer (shared state and all that), but it will allow you to use streaming for both the request side and the response side.
I have a question regarding the CopyTo() method of the Stream class:
https://learn.microsoft.com/en-us/dotnet/api/system.io.stream.copyto
This approach works for small file circa 15kb as I tried it, but anything higher (I tested with 2mbs, 4 mbs and so on) and it just hangs on the CopyTo() method. Can't really figure out why.
Code sample:
Server's handle client :
public void HandleClient(object c)
{
string path = "some path";
using (TcpClient client = (TcpClient)c)
{
using (NetworkStream netStream = client.GetStream())
{
using (FileStream fileStream = new FileStream(path, FileMode.Create))
{
netStream.CopyTo(fileStream);
}
}
}
}
Client send :
public void Send()
{
IPEndPoint remoteEndPoint = new IPEndPoint(IPAddress.Parse("some address"), 12345);
string path = "some path";
using (TcpClient client = new TcpClient(remoteEndPoint))
{
using (NetworkStream netStream = client.GetStream())
{
using (FileStream fileStream = new FileStream(path, FileMode.Open))
{
fileStream.CopyTo(netStream);
}
}
}
}
P.s. As I research my way into Network Programming, I often find people advising other people to switch to WCF for this kind of tasks since, apparently, WCF makes everything a lot easier. What do you guys suggest and could you provide some links for a WCF noob that would be useful in modeling a LAN file sharing application since that's what my goal is?
I managed to solve the CopyTo() issue. The problem was that I was sending the file on the main thread so the whole application just chocked for larger files that took more than an instant to transfer.
Put the sending operation in a separate thread and tested sending up to 3 GB, works as it should. Now, is there any way I could track the progress of the CopyTo() operation? Seems to me that I can't and that I should do manual transfer if I want to track the progress.
Thanks to everyone involved :)
I'm not sure you can do a CopyTo with a file larger than the buffer size. Maybe, you can try to write it by splitting your file every buffer size.
I had using the BinaryFormatter to Serialize an object through NetworkStream
The code like this
//OpenConnection ...
TCPClient client = server.AcceptTCPConnection();
Message message = new Message("bla bla"); // This is the serializable class
NetworkStream stream = client.GetStream(); // Get Stream
BinaryFormatter bf = new BinaryFormatter();
bf.Serialize(stream, message);
stream.Flush();
stream.Close(); //Close Connection
And in client Code, we just need to Read from stream
bf.Deserialize(stream) as Message
to get the object we just sent from Server.
But there is a problem here, if I delete the line stream.Close(); the client cannot read this Object. Or I can change to stream.Dispose();
However, I want to use this stream again to send another Message, how I can do? Please help, it make me feel so headache ##
UPDATE:
I found the reason of this issue. Because I used one machine to run both client and server. It definitely worked well in two different machines. Someone can tell me why? Get big problem with this for a couple day ago.
Sending multiple separate messages involves "framing" - splitting the single channel into separate chunks that don't ever require the client to "read to end". Oddly, though, I was under the impression that BinaryFormatter already implemented basic framing - but: I could be wrong. In the general case, when working with a binary protocol, the most common approach is to prefix each message with the length of the payload, i.e.
using(var ms = new MemoryStream()) {
while(...)
{
// not shown: serialize to ms
var len BitConverter.GetBytes((int)ms.Length);
output.Write(len, 0, 4);
output.Write(ms.GetBuffer(), 0, (int) ms.Length);
ms.SetLength(0); // ready for next cycle
}
}
the caller has to:
read exactly 4 bytes (at least, for the above), or detect EOF
determine the length
read exactly that many bytes
deserialize
repeat
If that sounds like a lot of work, maybe just use a serializer that does all this for you; for example, with protobuf-net, this would be:
while(...) { // each item
Serializer.SerializeWithLengthPrefix(output, PrefixStyle.Base128, 1);
}
and the reader would be:
foreach(var msg in Serializer.DeserializeItems<Message>(
input, PrefixStyle.Base128, 1))
{
// ...
}
(note: this does not use the same format / rules as BinaryFormatter)
I want to serialize/deserialize multiple object from/to a file. The syntax should be similar to this:
obj.Append(byteArray);
obj.Append(byteArray);
obj.Append(byteArray);
IEnumerable<byte[]> obj.Extract();
While this is very simple to accomplish (e.g., write a class that uses a filestream and protobuf-net internally), I'm wondering if there is any more elegant way of doing this. Is there any class (from a third party library), that uses a serializer to write to an filestream?
Edit: I need this as a filestream that captures video data that is sent through network. So the filestream must be open for a dedicated amount of time. My previous solution was to save every video frame to a new file, but it's not scalable (especially for hdd, and increasing video partners).
What about:
using(var stream = File.Open(filename, FileMode.Create))
{
var formatter = new BinaryFormatter();
formatter.Serialize(stream, firstObjectToSerialize);
formatter.Serialize(stream, secondObjectToSerialize);
}
I am planning to pass MemoryStream via WCF Streaming but it seems not working but when I slightly change the code to pass FileStream instead, it is working. In fact, my purpose is to pass large collection of business objects (serializable). I am using basicHttpBinding. Your suggestion would be much appreciated!
Edited:
The symptoms of the issue is that the incoming stream is empty. There is neither error nor exception.
You're not providing many details, however, I'm almost certain I know what the issue is as I've seen that happening a lot.
If you write something to a MemoryStream in order to return that one as the result of a WCF service operation, you need to manually reset the stream to its beginning before returning it. WCF will only read the stream from it current position, hence will return an empty stream if that position hasn't been reset.
That would at least explain the problem you're describing. Hope this helps.
Here some sample code:
[OperationContract]
public Stream GetSomeData()
{
var stream = new MemoryStream();
using(var file = File.OpenRead("path"))
{
// write something to the stream:
file.CopyTo(stream);
// here, the MemoryStream is positioned at its end
}
// This is the crucial part:
stream.Position = 0L;
return stream;
}