so I feel like this is a thing that I've seen before, but I can't think what it's called.
I have a java program which loads up an object from a file (I'm going to call this object the catalogue). Defining the file format, and writing the parser for it has taken a fair amount of time. Recently I've had developers that are want to load the file so they can use the catalogue for their own tools, but are working in a different language (current just C#, but potentially others in future).
Most of our tools run out on a server, so to prevent other people having to write their own parser for the file format, I plan on having a web service running which uses my program to load the catalogue, then returns it's contents as a json string. (Encoding the original file as json is not an option). In future I may just return bits of the json string, since the whole contents of the file can be pretty massive.
What I'm wondering is, do I have to write my catalogue object in each language, and write a json parser for it in that language, or is there a tool that will allow me to do something similar? I'm hoping for something that just has a simple format for declaring storage classes, generate corresponding code for various languages, and have a default serialize/deserialize for something like json. Currently I'd need it to support java and C#.
Does such a thing exist?
As long as the target language can make a web call, and parse JSON strings, you should be OK. This covers just about all mainstream languages I think. C# is certainly OK.
Related
Assuming all fields of a java class are java primitives, if such an object has been serialized, can it be successfully deserialized by C# into an instance of an "equivalent" C# class?
Is the reverse possible - C# to java?
I realise there are many language agnostic formats, such as XML that could be used to get the job done. I am more interested in whether using the native serialized data is feasible.
The formats of serialized streams are available. I think you can write a class easily to parse the byte stream and create the required class in C#.
An article that specifies the serialized format:
http://www.javaworld.com/community/node/2915
WOX will be helpful to achieve interoperable serialization.
it can serialize/deserialize Java/C# objects into/from standard XML(platform independent)
This is not possible, at least not using the native serialization libraries that both frameworks provide, as stated in this previous SO post.
If you want to achieve cross language serialization/deserialization, you could resort to XML (XSTream for Java, XStream-dot-net for C#) or WOX:
WOX is an XML serializer for Java and C# objects. In other words, WOX
is a library (woxSerializer.jar for Java, and woxSerializer.dll for
C#) to serialize Java and C# objects to XML and back again.
If you're OK with including another dependency, you might consider using an object database such as db4o for the job. I haven't tried this myself, but according the Wikipedia article,
db4o uses a custom feature called "generic reflector" to represent class information, when class definitions are not available, which allows to use it in a mixed Java-.NET environment, for example Java client - .NET server and vice versa.
You can find more information about the above-mentioned reflection API here and here.
In a nutshell, this would lead to a system where you store your Java/C# objects to an (embedded) database (i.e., without client/server architecture, but by loading a single file that contains the whole database) and retrieve C#/Java objects from the database afterwards.
I have used this document with a high amount of success to parse data stored in serialized format on a database:
http://www.jtech.ua.es/j2ee/2005-2006/modulos/rmi/recursos/serial-1.5.0.pdf
The most meaningful info for me was from page 63 to 68.
In my case I had the source code used to serialize the data which was useful to both identify fields and read the data when was written in a non standard way using the ISerializable.WriteObject/ReadObject calls.
I don't know the reason but my serialized data had not "handler" field on any object, it would take 0 bytes. Other than that, everything followed the docs but it gets kind of tricky if you have never done such kind of tasks before
As noted on some comment, this is a good base even if it's written in java:
https://github.com/smartplatf/a-utilities/blob/master/src/main/java/org/anon/utilities/serialize/srdr/SerialStreamReader.java
I am recieving binary stream from an application I am running in Python.
From the binary stream, I want to create a C# object that is inside the stream in byte array.
How do I deserialise the object and retrieve the object from the binary stream?
We can ignore that it's a python application. I am more interested in how binary streaming works.
You seem to think that all languages automatically use the same serialization scheme.
This is not so.
It is not even theoretically possible, because different programming languages have different notions of what it means to be an object.
If you are specifically interested in how to read a Python serialized stream in C#, then ask that. Otherwise, this question is unanswerable because it is based on a false premise.
FOLLOW UP - Out of curiosity, I did some searching for a Python pickle reader in C#. Nothing in the first 3 pages of search results ... though there was a reference to a pickle reader in C++.
Just to add you a little general info:
In C#/.Net there's a general approach to serialize objects to NOT a binary form, because a binary form needs a lot of protocol-like headers to - note - include the metadata, and this causes the receiver to have to know the .Net/CLR inner structure very well.
Instead, today, the objects are usually serialized to XML (when type information is crucial) or JSON formats (when only data matters), so that any receiver may read them quite easily, and more often - any 3rd party may easily generate new object-like data that our application may "just deserialize", regardless of who generated it and on what platform.
However, binary serialization is still used. XML/JSON data, even if compressed, is still usually larger than the binary image. However, the binary serialization is strictly used when we want the data to not be published to the outside world, or if we somehow magically know that it will be only processed on .Net with use of our assemblies.
C# object
C# does not have objects; it's a .Net object.
Secondly we absolutely CANNOT ignore that it's a Python application, because that implies that it's likely it's not running on .Net and therefore the .Net binary format is not native to your Python runtime. That's not to say that it's not possible for the .Net serialization to be available to you in this case, because if you're running IronPython - the .Net python implementation - then you can simply use the Binary serialization APIs from within that and get the .Net object that was serialized.
If, however, it's Python running on a different platform, then you can decode the information in the binary stream, for that you need to know the format, and for that go straight to the horse's mouth and read through the Binary Format Data Structure spec from MSDN.
This will, of course, require (quite a lot) more work!
If the project you're working on allows you to change the way that the original object is serialized, then I strongly suggest changing over to XML serialization or something similar - that is designed to be portable.
i am writing a project in C#
i wanna save a class in binary file and then read that file it in C
i wanna know how can i do it without serialize and deserialize
please help me
You are talking about cross-platform serialization.
A few options:
serialize it as text (xml, json); text is still binary, after all - and simple
serialize it manually
use a third party cross-platform serializer
But whatever you do, don't use BinaryFormatter. The reason I stress this is that it is probably the first thing you'll see if you search for C# binary serialization, but is entirely inappropriate for your purposes. The format is proprietary, and includes type information that only makes sense from .NET (not really from unmanaged C).
I'm quite attached to "protocol buffers" as a serialization API, and there are both C# and C versions here.
Saving the state of an object to a file means serializing it.
Reading the state of an object from a file means deserializing it.
You have to use serialization/deserialization to do what you want.
Since you need to do this across different languages, using the built in serializers would probably not be very helpful.
You can use one of the XML serializers for the C# part, but then would have to parse the XML out in c.
Another option is to write your own custom serizlizer to do this. This way you have full control over the file format.
Do you want to save a class? This is not possible since classes are compiled into assemblies (exe,dll) in .net.
I think what you want is to save the state of an object or better suited, a struct to a file.
You can write all fields of the class to a file using the BinaryWriter class. Also you can have a look at this.
I presume you mean you want to have a C# application write a file. Then have a separate C/C++ application read that file? On that assumption, in C# you'll need to look into the System.IO namespace, and specifically the FileStream class.
On a side note, I'd really recommend writing a C# Class Library project that handles this read/write via .NET serialization classes and then invoke it nativly from your C# code, and use COM ([assembly: ComVisible(true)]) to access your .NET code from your C/C++ code.
I'm looking at protobuf-net for implementing various messaging formats, and I particularly like the contract-based approach as I don't have to mess with the proto compiler. one thing I couldn't quite find information on is, does this make it difficult to work cross-platform? there are a few C++ apps that would need to be able to parse PB data, and while I understand that protobuf-net serializes to the PB standard format, if I use the contract approach and not a proto file, how does the C++ side parse the data?
can (should?) I write a separate proto file for the (very few) cases where C++ needs to understand the data? and if so, how exactly do I know that the C++ class generated from the proto file is going to match the data from the no-proto-file C# side?
Yes, in theory at least they should match at the binary level, but you might want to limit yourself to types that map simply to ".proto" - so avoid things like DateTime, inheritance ([ProtoInclude]), etc. This also has the advantage that you should be able to use:
string proto = Serializer.GetProto<YourType>();
to get the .proto; it (GetProto) isn't 100%, but it works for basic types. But ultimately, the answer is "testing and tweaking"; perhaps design for interop from the outset - i.e. test this early.
I have two application that need to talk to each other. App1 needs to be able to serialize an object that App2 can then deserialize. Easily done, right? Here's the problem; App1 is C# based, App2 is Java based. So App1 needs to write out the file in the Java binary file format. How can this be done?
The way I see it, I have two options. The first is figure out some way to serialize a Java object in C#, so that App1 just creates the appropriate file. My other option would be to write a converter in Java that reads in a file and populates the object accordingly and serializes the newly populated object. That way the C# app would only have to write out some sort of formatted text file that the converter then interprets.
I can't make any changes to the Java application.
How should this be done?
Update:
The Java application is already in the hands of customers so changing the serialization scheme would cause the customers existing data to be incompatible. The Java App uses the native java serialization when dealing with this object. Modifications to the Java app can't happen.
The C# app uses protocol buffers to serialize its own data.
The best answer is option 3:
use a language-neutral serialization scheme.
I use JavaScript. Thrift is another option, protocol buffers I believe are more focused on RPC, but should be usable for serialization as well. XML or a custom binary format would be other options.
Edit:
Sorry, didn't notice that you can't make changes to the Java application. That said, the best way to do it would probably be to create your own well defined format, write a java app that can read that format, then output a serialized java object for the legacy app.
"IKVM" might be something you could use. This product allows you to convert compiled java bytecode (.jar, etc.) into a .NET DLL. It's super easy to use, and might give you the interop you need.
Other than this, the easiest way to accomplish this without a binary-level interop is to just use a plain text format, such as a CSV or XML.
Just use XML serialization. Both frameworks have good support, and the simplicity will make it easier to debug / maintain. Write a small program in Java that just imports the XML and writes the binary file.
Your best bet would be to write something that uses Java Native interface. Not fun, but it'll work.
You can do this directly using JNI (not fun but doable) or there may be some tools out there that will generate code for you -- take a look at SWIG: http://www.swig.org/
You would call Java from C# to do the persistence for you.