I have a large set of XML files of a propriatary schema -the XML files define binary communication protocol (message structure).
I'd like to leverage Google's protocol buffers technology.
I am using existing code to load the XML files into an object model (in memory).
I'd like to generate a .proto file from that object model.
so basically what I am looking for is code/library (in C#/.NET) that represents the .proto file format as an object model and can save that object model into a .proto file.
I took a look at Jon Skeet's dotnet-protobufs, I think I understand what it does (generate c# code based on .proto files)
However, I didn't figure out if I can use it for my project (it probably has the .proto format object model there, but probably only code that can parse this format and not write it out)
protobuf-net (my version of protocol buffers in .NET) has primitive support for generating proto files, but it wouldn't be hard to fill in the blanks. I concentrated on the core engine first, then the generation of C# from proto. Writing an xslt to generate a proto from the object model wouldn't be much different. It would take a few days though... (I have limited time at the moment).
If this would be useful, please let me know.
For info, the protobuf-net engine is compatible with most XmlSerializer classes (and DataContractSerializer, and recently BinaryFormatter) - so if your code currently works as xml, we can probably get it working in protobuf-net. No guarantees, of course...
My code can only serialize and deserialize to binary and text. However, I believe Marc Gravell's project has XML capabilities. In fact, I believe he generates C# code based on loading the binary version of a .proto file (which is itself encoded as a protobuf), writing it out as XML, and then applying XSLT to it...
Related
Assuming all fields of a java class are java primitives, if such an object has been serialized, can it be successfully deserialized by C# into an instance of an "equivalent" C# class?
Is the reverse possible - C# to java?
I realise there are many language agnostic formats, such as XML that could be used to get the job done. I am more interested in whether using the native serialized data is feasible.
The formats of serialized streams are available. I think you can write a class easily to parse the byte stream and create the required class in C#.
An article that specifies the serialized format:
http://www.javaworld.com/community/node/2915
WOX will be helpful to achieve interoperable serialization.
it can serialize/deserialize Java/C# objects into/from standard XML(platform independent)
This is not possible, at least not using the native serialization libraries that both frameworks provide, as stated in this previous SO post.
If you want to achieve cross language serialization/deserialization, you could resort to XML (XSTream for Java, XStream-dot-net for C#) or WOX:
WOX is an XML serializer for Java and C# objects. In other words, WOX
is a library (woxSerializer.jar for Java, and woxSerializer.dll for
C#) to serialize Java and C# objects to XML and back again.
If you're OK with including another dependency, you might consider using an object database such as db4o for the job. I haven't tried this myself, but according the Wikipedia article,
db4o uses a custom feature called "generic reflector" to represent class information, when class definitions are not available, which allows to use it in a mixed Java-.NET environment, for example Java client - .NET server and vice versa.
You can find more information about the above-mentioned reflection API here and here.
In a nutshell, this would lead to a system where you store your Java/C# objects to an (embedded) database (i.e., without client/server architecture, but by loading a single file that contains the whole database) and retrieve C#/Java objects from the database afterwards.
I have used this document with a high amount of success to parse data stored in serialized format on a database:
http://www.jtech.ua.es/j2ee/2005-2006/modulos/rmi/recursos/serial-1.5.0.pdf
The most meaningful info for me was from page 63 to 68.
In my case I had the source code used to serialize the data which was useful to both identify fields and read the data when was written in a non standard way using the ISerializable.WriteObject/ReadObject calls.
I don't know the reason but my serialized data had not "handler" field on any object, it would take 0 bytes. Other than that, everything followed the docs but it gets kind of tricky if you have never done such kind of tasks before
As noted on some comment, this is a good base even if it's written in java:
https://github.com/smartplatf/a-utilities/blob/master/src/main/java/org/anon/utilities/serialize/srdr/SerialStreamReader.java
I have a desktop app that saves out user data (mostly arrays of numbers) to a file by Serializing an object. This file will later be used to reload user data by Deserialising it.
I would also like these files to be user-readable and in some standard structured format. The files will be manually moved around (not transmitted) and will be read by users on both PCs and Unix machines . Just to clarify - my desktop app will not run in Unix but I'd like the Unix users to be able to write a script to parse the data if they needed.
Is there a good reason to use the Soap formatter rather than XML formatter to do this?
Follow the rule you were taut in Grad schools. When it comes to Platform Independence go with XML
Do not use SOAP Formatter. The XML generated is quite bizarre and pretty unusable - I personally believe it is just a binary serialization but in SOAP.
Use XmlSerializer instead.
I'm working a generic protobuf decoder that works as follows:
The user can specify the .proto file at runtime and specify the data file and the program would display the data in the file based on the .proto definition.
To do the above, the most obvious things seems like I would need to interpret the .proto file (or compile it) and then decode the protobuf message using it. Any ideas on how I can proceed on this? Is there a library out there that would help me with this.
As always, any feedback is much appreciated.
Thanks!
I keep meaning to write my own parser, but for now I just use "protoc" to parse the .proto to a protobuf binary. I then deserialize that using my own protobuf library, giving me a populated object model to work with.
I don't know how far along you are, but you might also be interested in some of the runtime support in protobuf-net v2, which allows on-the-fly mapping of protobuf data to types. Alternatively there's also a fairly re-usable reader implementation that might suit your needs.
If you could work from XML, I include a tool in protobuf-net, "protogen", which does code-gen; but pass in a -t:xml and it should transform a .proto into XML for you.
Iirc, "protoc" outputs a protobuf using "descriptor.proto" from the google package.
i am writing a project in C#
i wanna save a class in binary file and then read that file it in C
i wanna know how can i do it without serialize and deserialize
please help me
You are talking about cross-platform serialization.
A few options:
serialize it as text (xml, json); text is still binary, after all - and simple
serialize it manually
use a third party cross-platform serializer
But whatever you do, don't use BinaryFormatter. The reason I stress this is that it is probably the first thing you'll see if you search for C# binary serialization, but is entirely inappropriate for your purposes. The format is proprietary, and includes type information that only makes sense from .NET (not really from unmanaged C).
I'm quite attached to "protocol buffers" as a serialization API, and there are both C# and C versions here.
Saving the state of an object to a file means serializing it.
Reading the state of an object from a file means deserializing it.
You have to use serialization/deserialization to do what you want.
Since you need to do this across different languages, using the built in serializers would probably not be very helpful.
You can use one of the XML serializers for the C# part, but then would have to parse the XML out in c.
Another option is to write your own custom serizlizer to do this. This way you have full control over the file format.
Do you want to save a class? This is not possible since classes are compiled into assemblies (exe,dll) in .net.
I think what you want is to save the state of an object or better suited, a struct to a file.
You can write all fields of the class to a file using the BinaryWriter class. Also you can have a look at this.
I presume you mean you want to have a C# application write a file. Then have a separate C/C++ application read that file? On that assumption, in C# you'll need to look into the System.IO namespace, and specifically the FileStream class.
On a side note, I'd really recommend writing a C# Class Library project that handles this read/write via .NET serialization classes and then invoke it nativly from your C# code, and use COM ([assembly: ComVisible(true)]) to access your .NET code from your C/C++ code.
I have two application that need to talk to each other. App1 needs to be able to serialize an object that App2 can then deserialize. Easily done, right? Here's the problem; App1 is C# based, App2 is Java based. So App1 needs to write out the file in the Java binary file format. How can this be done?
The way I see it, I have two options. The first is figure out some way to serialize a Java object in C#, so that App1 just creates the appropriate file. My other option would be to write a converter in Java that reads in a file and populates the object accordingly and serializes the newly populated object. That way the C# app would only have to write out some sort of formatted text file that the converter then interprets.
I can't make any changes to the Java application.
How should this be done?
Update:
The Java application is already in the hands of customers so changing the serialization scheme would cause the customers existing data to be incompatible. The Java App uses the native java serialization when dealing with this object. Modifications to the Java app can't happen.
The C# app uses protocol buffers to serialize its own data.
The best answer is option 3:
use a language-neutral serialization scheme.
I use JavaScript. Thrift is another option, protocol buffers I believe are more focused on RPC, but should be usable for serialization as well. XML or a custom binary format would be other options.
Edit:
Sorry, didn't notice that you can't make changes to the Java application. That said, the best way to do it would probably be to create your own well defined format, write a java app that can read that format, then output a serialized java object for the legacy app.
"IKVM" might be something you could use. This product allows you to convert compiled java bytecode (.jar, etc.) into a .NET DLL. It's super easy to use, and might give you the interop you need.
Other than this, the easiest way to accomplish this without a binary-level interop is to just use a plain text format, such as a CSV or XML.
Just use XML serialization. Both frameworks have good support, and the simplicity will make it easier to debug / maintain. Write a small program in Java that just imports the XML and writes the binary file.
Your best bet would be to write something that uses Java Native interface. Not fun, but it'll work.
You can do this directly using JNI (not fun but doable) or there may be some tools out there that will generate code for you -- take a look at SWIG: http://www.swig.org/
You would call Java from C# to do the persistence for you.