I would like my newer C# 2.0 application to talk to my older Java 1.4 application (can't change versions, sorry). What are my options?
I think that using shared memory would give me better performance, but on the other hand, if I use a network protocol then the architecture would be more flexible. So I'm looking to weigh up both options to see which has the biggest pay off.
I've used XML-RPC implementations that are dog slow, but I assume that was just a bad implementation, and not the actual protocol. Would I be better off going with a lower-level protocol? I've used Google's protobuf before in C++ and Python (over plain old sockets) but I'm not so sure that it's available for Java and C# -- is there anything similar available for the languages I'm using?
I'm looking for the best performance that I can possibly get, but, I'm working with objects and inheritance hierarchies that I'd like to serialize (protobuf is a good example of how this can be done). So, sadly, just sending a simple string over sockets isn't really feasible.
Aha, there's actually C# versions of protobuf!
http://code.google.com/p/protosharp/
http://code.google.com/p/protobuf-csharp-port
... and protobuf has support for Java anyway.
you might also consider JSON as serialization for your objects, much lighter than XML but same capabilities to represent object hierarchies and many libraries are available.
for the communication bus however, i would recommend network since it gives better flexibility.
IMHO,your performance bottleneck is due to serialization/deserialization more than communication bus itself.
Related
Being relatively high-level frameworks, is it possible to implement primitive algorithms like Ricart-Agrawala algorithm and Bully algorithm in Apache Thrift or WCF?
Or, should I use plain old C# with .NET Remoting?
As Jacek pointed out, both Thrift and WFC provide you with the means to build arbitrary complex things on top of it. The relationship is similar to the relationship between a web page (like SO) or a REST service, and the underlying HTTP protocol.
So the answer is clearly yes, but in a way you seem to ask the wrong question.
Is it possible to serialize the class/object in C# and deserialize the same in java. I want to serialize the class and not any XML/JSON data. Please clarify.
Thanks
I see 3 options here. I suggest option 1, Protobufs.
Look into Google's ProtoBufs
Or some equivalent. Here's the java version. Here's a C# port.
Protobufs meant for this sort of language interop. Its binary, small, fast, and language agnostic.
Also it has backwards compatibility, so if you change the serialized objects in the future, you can still read them. This feature is transparent to you too, long as you write code understanding newer variables could be missing when unserialized old objects. This is a huge advantage!
Implement one language's default serialization in the other
You can try implementing the java serialization logic in C#, or the C# serialization routines in Java. I don't suggest this as it will be more difficult, more verbose, almost certainly slower as you're writing new code, and will net you the same result.
Write your serialization routines by hand
This will certainly be fast, but tedious, more error prone, harder to maintain, less flexible...
Here's some benchmarks for libraries like ProtoBufs. This should aide you in selecting the best one for your use case.
We did this a while ago, it worked after lot of tinkering, it really depends on byte encoding, i think JAva uses one and C# uses another (little endian vs. big endian) so you will need to implement a deserializer which takes this affects into account. hope this helps
As others have suggested, your options are going to be external serialization libraries (Google Protobuff, Apache Thrift, etc), or simply using something built-in that's slower/less efficient bandwidth-wise (JSON, XML, etc). You could also write your own, but believe me, it's a maintenance nightmare.
Not using native serialization. The built-in defaults are tied to the binary representation of the data types, which are different for the different VMs. The purpose of XML, JSON, and similar technologies is precisely to provide a format that's generic and can be moved between differing systems. For what it's worth, the overhead in serializing to JSON is usually small, and there's a lot of benefit to being able to read the serialized objects manually, so I'd recommend JSON unless you have a very specific reason why you can't.
Consider OMG's standard CORBA IIOP.
While you many not need the full-on "remote object" support of CORBA, IIOP is the underlying binary protocol for "moving language-neutral objects" (such as an object value parameter) across the wire.
For Java: Java EE EJB's are based on IIOP, there is RMI-IIOP; various support libraries. The IDL-to-Java compiler is delivered with the JDK.
For C# IIOP & integration with Java EE, see IIOP.NET
You can also consider BSON, which is used by MongoDB.
If it is OK for your C#/Java programs to communicate with a mongodb database, you could store your objects there and read it with the appropriate driver.
Regarding BSON itself, see BSON and Data Interchange at the mongoDB blog.
I'm looking for a technology which is targeting on building distributed applications. My friend adviced me to use CORBA (Java & C++ combination) . But I have read it's sort of obsolete stuff. I'm planning to write rather simple distributed application. What solutions would you advice to use? Thanks!
If you want to distribute your code logic to multiple servers and have it managed as a single entity, I would recommend CloudIQ Platform from Appistry. You can deploy Java, .NET and C/C++ code to the framework. From an administrative point of view, the servers work and act as one. When you submit a request for execution, the framework distributes the request to the best available worker, performing load balancing. With this framework, you can have producer/consumer, scatter/gather, and other parallel types of jobs.
The framework also monitors the execution of jobs, so if there is any type of hardware failure, other machines will get allocated the jobs that were running on the failed server.
CORBA is quite old. To choose a library or framework, the questions are: why do you want it to be distributed? (what's the goal? performance / parallelization? scalability? physical constraints on locations of parts of the system?) Which sort of nodes will be running the various parts? What languages would you rather use?
Recommend using ICE(Internet Communications Engine), ICE can support multiple operating system platform (Windows, Linux, Solars, Mac OS, iOS, Android...), multiple developing language (C++, Java, .NET, Python, Ruby, PHP), and it is simpler.
You can use SOAP web services. I'm currently developing distributed testing system on Python & .NET using using SOAP and it is easy to write and deploy.
There are a lot of different SOAP server/client libraries for different languages and platforms.
Yes, CORBA, and technologies like COM and DCOM are all pretty much obsolete... I am not sure exactly what you want to accomplish, but I would look towards .NET remoting to build distributed applications. If your application is really simple, you can even use mailslots or named pipes to pass simple data across a network.
As sinelaw mentioned, there are many questions before a good suggestion can be made, but, you may want to look at REST (http://en.wikipedia.org/wiki/Representational_State_Transfer) as a way to transfer data between applications. REST is nice in that what it can accept and return are flexible, for example, you can upload a file and return a PDF. Though it is used on http, that isn't the only allowed protocol. It is language/platform agnostic.
If you want to go with something that is standardized then SOAP or REST is probably your best bet, if you want to be platform-independent. If you don't mind being restricted to Java/JVM or .NET then there are other options, but that becomes very restricting.
What type of data is being passed? How critical is security? What platforms/languages should be usable? What is the purpose of the program, the goal?
If you want a portable solution that can also be used with different protocols, WCF on Mono might be a good fit
For .Net I suggest you WCF , it's quite simple to implement and very flexible, and about CORBA it's a good choice if your goal is to understand deeply distributed applications, but it's not more recommended for real projects, currently is very difficult to find developers mastering CORBA.
I want to separate modules of my program to communicate with each other. They could be on the same computer, but possibly on different ones.
I was considering 2 methods:
create a class with all details. Send it of to the communication layer. This one serializes it, sends it, the other side deserializes it back to the class and than handles it further.
Create a hashtable (key/value thing). Put all data in it. Send it of to the communicationlayer etc etc
So it boils down to hashtable vs class.
If I think 'loosely coupled', I favor hashtable. It's easy to have one module updated, include new extra params in the hastable, without updating the other side.
Then again with a class I get compile-time type checking, instead of runtime.
Has anyone tackled this previously and has suggestions about this?
Thanks!
edit:
I've awarded points to the answer which was most relevant to my original question, although it isn't the one which was upvoted the most
It sounds like you simply want to incorporate some IPC (Inter-Process Communication) into your system.
The best way of accomplishing this in .NET (3.0 onwards) is with the Windows Communication Foundation (WCF) - a generic framework developed by Microsoft for communication between programs in various different manners (transports) on a common basis.
Although I suspect you will probably want to use named pipes for the purposes of efficiency and robustness, there are a number of other transports available such as TCP and HTTP (see this MSDN article), not to mention a variety of serialisation formats from binary to XML to JSON.
One tends to hit this kind of problem in distributed systems design. It surfaces in Web Service (the WSDL defining the paramers and return types) Messaging systems where the formats of messages might be XML or some other well-defined format. The problem of controlling the coupling of client and server remains in all cases.
What happens with your hash table? Suppose your request contains "NAME" and "PHONE-NUMBER", and suddenly you realise that you need to differentiate "LANDLINE-NUMBER" and "CELL-NUMBER". If you just change the hash table entries to use new values, then your server needs changing at the same time. Suppose at this point you don't just have one client and one server, but are perhaps dealing with some kind of exchange or broker systems, many clients implemented by many teams, many servers implemented by many teams. Asking all of them to upgrade to a new message format at the same time is quite an undertaking.
Hence we tend to seek back-comptible solutions such as additive change, we preserve "PHONE-NUMBER" and add the new fields. The server now tolerates messages containg either old or new format.
Different distribution technologies have different in-built degrees of toleration for back-compatibility. When dealing with serialized classes can you deal with old and new versions? When dealing with WSDL, will the message parsers tolerate additive change.
I would follow the following though process:
1). Will you have a simple relationship between client and server, for example do you code and control both, are free to dictate their release cycles. If "no", then favour flexibility, use hash tables or XML.
2). Even if you are in control look at how easily your serialization framework supports versioning. It's likely that a strongly typed, serialized class interface will be easier to work with, providing you have a clear picture of what it's going to take to make a change to the interface.
You can use Sockets, Remoting, or WCF, eash has pros and cons.
But if the performance is not crucial you can use WCF and serialize and deserialize your classes, and for maximum performance I recommend sockets
What ever happened to the built in support for Remoting?
http://msdn.microsoft.com/en-us/library/aa185916.aspx
It works on TCP/IP or IPC if you want. Its quicker than WCF, and is pretty transparent to your code.
In our experience using WCF extensively over the last few years with various bindings we found WCF not be worth the hassle.
It is just to complicated to correctly use WCF including handling errors on channels correctly while retaining good performance (we gave up on high performance with wcf early on).
For authenticated client scenarios we switched to http rest (without wcf) and do json/protobuf payloading.
For high-speed non-authenticated scenarios (or at least non-kerberos authenticated scenarios) we are using zeromq and protobuf now.
I have used IPC in Win32 code a while ago - critical sections, events, and semaphores.
How is the scene in the .NET environment?
Are there any tutorial explaining all available options and when to use and why?
Most recent Microsoft's stuff in IPC is Windows Communication Foundation. Actually there is nothing new in the lower level (tcp, upd, named pipes etc) But WCF simplifies IPC development greatly.
Useful resource:
Interprocess Communication with WCF on Dr. Dobb's portal
WCF Communication Options in the .NET Framework 3.5
and of course MSDN on WCF
Apart from the obvious (WCF), there is a ZeroMQ binding for C#/CLR which is pretty good:
http://www.zeromq.org/bindings:clr
Does message-oriented IPC, pub/sub and various other strategies with much less code and config than WCF.
It's also at least an order of magnitude faster than anything else and has less latency if you require low latency comms.
With respects to semaphores, locks, mutexes etc. If you share by communicating rather than communicate by sharing, you'll have a whole load less hassle than the traditional paradigm.
I tend to use named pipes or Unix sockets (depending on whether I'm targetting MS.NET or Mono -- I have a class that abstracts it away) since it's easy to use, portable, and allows me to easily interoperate with unmanaged code. That said, if you're only dealing with managed code, go with WCF or remoting -- the latter if you need Mono support, since their WCF support simply isn't there yet.
I would recommend using Memory Mapped Files if you need to use on the machine domain not communication through network. See the following link.
http://techmikael.blogspot.com/2010/02/blazing-fast-ipc-in-net-4-wcf-vs.html
There is also .NET Remoting, which I found quite cool, but I guess they are obsoleting it now that they have WCF.
It sounds as though you're interested in synchronization techniques rather than communication. If so, you might like to start here, or perhaps this more concise overview.