I'm investigating Microsoft Message Queues for doing inter-process cross-network messaging. But when I receive a message, I don't know a priori what type of object I'm getting, so the code
queue.Formatter = new XmlMessageFormatter(new Type[] { typeof(Wibble) });
can't be applied before I get the message because I don't know if it's a Wibble. So how do I receive different message types?
You're already using the constructor overload for XmlMessageFormatter that accepts an array of types. So just add all of the types that you're expecting to receive into that array, rather than just one type.
queue.Formatter = new XmlMessageFormatter(new Type[] {
typeof(Wibble),
typeof(Fleem),
typeof(Boo)
});
From TargetTypes:
The instance serialized in the message body must comply with one of the schemas represented in the type array. When you read the message using the Receive method, the method creates an object of the type that corresponds to the schema identified and reads the message body into it.
(Emphasis added)
You might consider not storing your object in the MSMQ message, but instead putting a reference to it's persistent location if you can. MSMQ has finite space on the message queues, so smaller messages are best.
If you can't do that, you can serialize your object to the messages BodyStream directly, using whatever serializer you like. Then store the type name as well, probably best in the message Label.
Something very similar to this (scratched it out here, no IDE on this computer) to put it in, and the analagous action on the way out:
public void FormatObject(object toFormat, Message message)
{
var serializer = new XmlSerializer(toFormat.GetType());
var stream = new MemoryStream();
serializer.Serialize(toFormat, stream);
//don't dispose the stream
message.BodyStream = stream;
message.Label = toFormat.GetType().AssemblyQualifiedName;
}
There is a great amount of misinformation running around on MSMQ primarily because the Microsoft documentation is frighteningly sparse on how to design a message send receive properly. I have both of the MSMQ books published on this subject and I'm still searching for sensible designs on the internet.
So, neither of these references say that there is a one message type to a queue requirement. And that would make PeakMessage and variants unnecessary and even stupid. Microsoft is vague and difficult in its documentation, but I've worked there and they are never stupid.
There is a constant irritating suggestion to use a CLSID as an identifier, a practice that is annoyingly short sighted. How about trying to embed the message type in the LABEL??? Then use PeadMessage to run up the Queue until you find a message meant expressly for your particular queue and with a message type that you can use to format the message properties to receive the message properly on the first try???
I know this makes for a complex code set, but would you rather do without? Or would you actually try to implement the suggestion of the responder above who is implying that if you have a system of 200 users with 200 message types that they should create 80,000 queues to manage all the one to one requirements? Some people just don't think these things through.
As joocer notes in a comment: use a different queue for different message types.
Alternatively you could agree with the message senders that all messages will be XML (anything that doesn't parse as XML is rejected). Then also agree to some basics of the XML schema: a header element with a message type (and version).
Then process (either yourself of via a serialiser) into the internal type.
Of course in many cases – where there is no real benefit to a deserialisation – just read the content of the XML as required.
Related
When writing classes for internal processing in .Net, I often use ArgumentException to indicate something is wrong with the given data and it can't be processed. Due to the nature of the program, the text I put in these exceptions is sometimes relevant to the user, and so it often gets shown on the UI.
However, I noticed that ArgumentException specifically overrides the Message property to append its own string to indicate which argument caused the exception. I don't want this extra text polluting the message, since the actual argument name is internal processing info that really doesn't need to be shown to the user, and the fact it adds a line break, and that it is localised, messes up the formatting of the message I show on the UI. The only way to get around this is to not give the exception the actual argument name, but I don't want to sabotage my own debugging / logging by removing that information, either.
I could use my own exception class, of course, but since a lot of these methods are for compression and decompression of proprietary file formats in old DOS games, and I want these methods to both be documented on a wiki and be generally easily usable by anyone else, I'd prefer keeping them portable and avoid reliance on other external classes. And, as a side note, subclassing ArgumentException would of course give the same issue.
The original source:
public override String Message
{
get {
String s = base.Message;
if (!String.IsNullOrEmpty(m_paramName)) {
String resourceString = Environment.GetResourceString("Arg_ParamName_Name", m_paramName);
return s + Environment.NewLine + resourceString;
}
else
return s;
}
}
(from referencesource.microsoft.com)
Since this actually overrides the Message property, there seems to be no normal way to get to the real message that's stored internally. Splitting on a line break seems messy and potentially unreliable depending on localisation differences (and the message I give it might potentially have line breaks already), and using reflection for this seems rather messy. Is there a clean way to recover the original message?
(Posting this here with solution for documenting reasons, since this behaviour really frustrated me when I encountered it)
Since I didn't want to dig into reflection, I figured a good way to get the original data without the associated class behaviour would be to serialize it. The names of the properties in the serialised info are very straightforward, and can be accessed without the ArgumentException getter mangling it with its own additions.
The code to accomplish this turned out to be pretty straightforward:
public static String RecoverArgExceptionMessage(ArgumentException argex)
{
if (argex == null)
return null;
SerializationInfo info = new SerializationInfo(typeof(ArgumentException), new FormatterConverter());
argex.GetObjectData(info, new StreamingContext(StreamingContextStates.Clone));
return info.GetString("Message");
}
I was going through the msdn post of system.messaging namespace and come across a doubt about object serialization. I want to know that is it always important to that object must be xmlserializable in order to be transferred over a MessageQueue instance.
Second, if we are invoking the MessageQueue.Send member is this also means that we are using default System.Messaging.XmlMessageFormatter.
An explanation would be appreciative.
Thanks
you don't have to use xml, you can use BinaryMessage like this
BinaryMessageFormatter formatter = new BinaryMessageFormatter();
System.Messaging.Message message = new System.Messaging.Message(YourObject, formatter);
second, xml message is the default, although I have always used binary, xml is to bulky for almost all kind of scenarios.
This problem seems simple and do-able enough, but I cannot for the life of me get it to work.
I have:
A PCAP file with a few packets I know are some type of ProtoBuf data (probably created with protobuf-csharp-port)
All the possible C# classes from an assembly decorated with:
[DebuggerNonUserCode, CompilerGenerated, GeneratedCode("ProtoGen", "2.4.1.473")]
public sealed class thing : GeneratedMessageLite<thing, thing.Builder>
All I want to do is parse those packets using what I know from the assembly file. Simple? Probably, but no matter what I do, nothing actually is getting parsed.
Here's an example of one of the many possible classes:
[DebuggerNonUserCode, CompilerGenerated, GeneratedCode("ProtoGen", "2.4.1.473")]
public sealed class Thing: GeneratedMessageLite<Thing, Thing.Builder>
{
// Fields
private static readonly string[] _thingFieldNames = new string[] { "list" };
private static readonly uint[] _thingFieldTags = new uint[] { 10 };
...
public static Builder CreateBuilder()
{
return new Builder();
}
...
public static thing ParseFrom(ByteString data)
{
return CreateBuilder().MergeFrom(data).BuildParsed();
}
...
public override void WriteTo(ICodedOutputStream output)
{
int serializedSize = this.SerializedSize;
string[] strArray = _thingFieldNames;
if (this.list_.Count > 0)
{
output.WriteMessageArray<thingData>(1, strArray[0], this.list_);
}
}
...
[DebuggerNonUserCode, GeneratedCode("ProtoGen", "2.4.1.473"), CompilerGenerated]
public static class Types
{
// Nested Types
[CompilerGenerated, GeneratedCode("ProtoGen", "2.4.1.473")]
public enum PacketID
{
ID = 19
}
}
}
There are many others like that. I've tried doing something like this with each packet (using protobuf-csharp-port):
Console.WriteLine(Thing.ParseFrom(packet.Buffer).ToString());
I'm expecting to see the actual text data. But I either get nothing, an error about invalid packet tags, or an error about it being a "0".
I've also tried using protobuf-net, but it just gives me random errors about incompatibility, unexpected types, etc.:
Console.WriteLine(ProtoBuf.Serializer.Deserialize<Thing>(ms));
What on Earth am I doing wrong here? Is there a better way to, using all the known types in an assembly, simply decode the Protobuf message and see what's inside? Ideally without having to know beforehand what type of message it is?
Thank you so much if you can figure this out!
Guessing from the failed attempts outlined in the question, I believe that you have some misconceptions about the content of your pcap file.
This line in particular
Console.WriteLine(Thing.ParseFrom(packet.Buffer).ToString());
makes me think that you are working under the wrong assumption that a single pcap packet contains the serialized bytes of one single object. Unfortunately, this is not the case.
As you might know, TCP/IP networks use a layered protocol stack, where each layer adds functionality and isolates upper layer protocols from the details of lower layer protocols (and vice versa). This is done by encapsulating the data sent from the upper layers down to the network and de-encapsulating the data as it travels up the stack on the receiving side.
Now, your pcap file contains the raw data as seen by your network interface, i.e. the serialized payload plus all the data added by the application, transport, internet, and link layer.
Now, if you want to de-serialize the objects contained in your dump, you will need to write some code that removes all the headers of the link layer and internet protocols, (un-)does the work of the transport protocol and reassembles the stream of bytes that was sent over the network.*
Next, you will need to analyze the resulting byte dump and make some sophisticated guesses about the design of the application level protocol. Does it implement a handshake when it starts communicating? Does it send a checksum together with the actual payload? Was the data compressed before it was sent over the network? Does the application encrypt the data prior to sending it? If TCP was used as the transport protocol, how is message framing implemented etc. Of course, if you have access to the source code of the application that generated the data (or at least the application binaries), then you can just read the code (or reverse engineer the binaries) to figure this part out.
Once you are at this point you are in a position to interpret the raw data. All that is left is to write some code that extracts the relevant bytes, feeds it to the protocol-buffer deserializer and voilà, you have your objects back!
(* And there are other minor issues like fragmented IP packets, TCP segments that arrived out of order, and TCP retransmissions, of course.)
To sum it all up:
It is theoretically possible to write a tool that de-serializes objects that were serialized using protocol-buffers from a pcap dump, provided that the dump contains the full communication between both peers, i.e. packets were not truncated by the tool that generated the dump etc.
In practice however, there are multiple hurdles to overcome that are anything but trivial even for a seasoned practitioner of the art, as such a tool must:
be able to deal with all the complex issues of the lower level protocols of TCP/IP to rebuild the data flow between the peers.
be able to understand the application level protocol that was used to transmit the serialized objects.
Note that point 1 above alone results in the requirement to implement the functionality of a TCP/IP stack at least in part. The easiest way to accomplish this would probably consist in reusing code of an open source TCP/IP implementation such as the one found in the Linux or *BSD kernel. Many tools that do similar things, like reconstructing HTTP traffic from capture files, do exactly this. (See e.g. Justsniffer.)
I have inherited a WCF service which acts as a file cache (each file representing the results of a request to a third party API). At the moment if the file doesn't exist the code creates a new request to create the data and it also raises an exception to the client code.
I think the idea is that the clients would come back to request the file again and by then it would be available by them (it takes a couple of seconds to generate the file).
I think there's a code smell here and I should rewrite this part. At the moment the exception is getting caught and bubbled up through a couple of methods. I think I should be establishing at source whether the file exists and pass that information up the call stack.
At the WCF interface I currently have a GetValue() method, though there are two options I think I could use to replace it:
return null if the file does not exist.
Use a bool TryGetValue(string key, out string value) method
Does anyone have any preferences/recommendations?
Thanks
The "TryGet" approach is a little more explicit. With the null-returning approach, you have to document that the method returns null for such and such a reason, and this requires developers to read the documentation. As we all know, some people are allergic to reading documentation.
Another advantage of the "TryGet" approach is that you can use an enum rather than a bool, to give even more information to the caller about why and how the method failed (or succeeded).
Jeffrey Richter’s (CLR in C#) definition of an exception: When an action member cannot complete its task, the member should throw an exception. An exception means that an action member failed to complete the task it was supposed to perform as indicated by its name. My question is should I keep the GetValue method available for the client and raise an error when the data is unavailable or remove it and replace it with TryGetValue()?
Jeffrey Richter's definition is not helpful when you are determining the design of your API, because that includes determining what the tasks of each action member should be.
In your design, you are expecting the value to be unavailable as a matter of course; this means that it is not an exceptional situation for the value to be unavailable. I would therefore use only the TryGet... pattern.
But, truth be told, I would pursue a different approach altogether. Suppose somebody tries this approach:
while (!TryGetValue(key, out value)) {}
or:
SomeType value;
bool flag = false;
while (!flag)
{
try
{
value = GetValue(key);
flag = true;
}
catch {}
}
Your WCF service is going to get a lot of hits. It would probably be better to look into an asynchronous model, so the client is notified through a callback when the result is ready, rather than inviting the client to poll the service continually.
On one hand, I'm told that exceptions in C# are 'expensive', but on the other, I'm stuck on how to implement this.
My problem is this: I'm making a Stream derivitive, that wraps a NetworkStream. Now, the problem I'm facing is this: Read(byte[] buffer, int offset, int count). From the Stream docs for the function:
Returns:
... or zero (0) if the end of the stream has been reached.
The problem is, in the protocol I'm implementing the remote side can send an "end of record" token, or a "please respond" token. Obviously, if this happens at the start of the Read() this causes problems, since I need to return from the function, and I havn't read anything, so I need to return 0, which means the stream is finished, but it isn't... is a EndOfRecordException or similar justified in this case? And in this case, should it aways be thrown when this token is encountered (at the start of the Read() call and make sure these tokens are always at the start by returning early) so that there is some sort of pattern to how these tokens should be handled.
Edit: For what it's worth, these tokens generally come through 3-10 times a second. At the most, I wouldn't expect more than 25 a second.
Exceptions aren't really all that expensive - but they also aren't necessarily the best way to manage expected/normal flow.
To me, it sounds like you aren't actually implementing a Stream - you are encapsulating a stream into a "reader". I might be inclined to write a protocol-specific reader class with suitable methods to detect the end of a record, or Try... methods to get data or return false.
It sounds like you shouldn't really be deriving from Stream if your class is concerned with records. Streams don't generally interpret their data at all - they're just a transport mechanism of data from one place to another.
There have been cases like ZipInputStream in Java which end up being very confusing when a single InputStream effectively has several streams within it, and you can skip between them. Such APIs have been awful to use in my experience. Providing a separate class to implement the "record splitting" which can provide a stream for the data within a record sounds cleaner to me. Then each stream can behave consistently with normal streams. No need for new exceptions.
However, I'm just guessing at your context based on the limited information available. If you could give more details of the bigger picture, that would help.
It's not such a big deal performance-wise, but still... Exceptions are intended for, well, exceptions. Situations that are "unusual". If that is the way the underlying stream behaves, then your stream should be able to handle it. If it can, it should handle it on its own. If not, you can have the user set some callback or something which will get called when you receive a "please respond" token.
I believe that Stream-derived class should deal only with streaming issues and adhere to Stream semantic contract. All higher-level logic (interpreting EOF and EOR tokens) should be placed in some other class.
Maybe you can create an enum that you return, this enum can contain items for EndOfRecord, EndOfStream, ReadOk or whatever you need.
The actual read data can be passed as an out parameter.