I am working on a PC hosted application which communicates to an embedded device. The application and the embedded device communicates with an UART interface. The communication protocol states that the data payload is formatted in KLV format. The host PC is written in C# and it can receive the hex data from device and put them into a byte array now. But I am not sure how to parse data. Also I am wondering how to compose the KLV data if I need to send some data to device.
If you are using an UART interface between the devices I doubt you receive 'hex'. Hex is just the representation often used for bytes itself. However as you have written it is already converted to bytes (e.g. by BitConverter class - strongly recommend taking a look in it).
Regarding the KLV format - i hope this is referring to the key-length-value protocol. Somewhere should be a document exactly descriping what data you get, because this protocol is often dimplemented different in following terms.
In the basics it is like: [KEY][LENGTH of VALUE][VALUE]. If you convert (Encoding.GetString, e. g. Encoding.ASCII.GetString()) everything to string you should already see some readable data (also I found that some folks used numbers for keys).
So you need the documentation about:
What KEYs are sent (mostly all have got the same length)
What data types you receive in VALUE for each Key
Without this information, you have to listen to the device for some time and afterwards analyse the data.
Related
I want to send a structure in Qt to Unity (C#) by TCP. What confuse me is when I receive the structure at Unity, I get messy code.
How to send and how to receive can I get the correct structure?
To send data over a byte stream based connection you first have to serialize it.
For serializing and, on the receiver side deserializing, data into a byte stream you'll need to make sure both sides expect the data in the same format.
There are a lot of them to chose from, e.g. binary based ones like Google Protocol Buffers, or text based ones like JSON.
I need to send a String data(with out converting to byte[]) to be sent to all system (both android devices) in a network (broadcasting). where ever I am searching for this, in all code they are using byte conversion. and while i am using StreamWritter for sending string data StreamWritter is not supporting broadcasting method. Please help me for this.
You can't.
Sockets are not dealing with strings, so long story short, you have to translate it to string back on the other side.
The Android apps are also only receiving bytes on the transport layer (UDP). As these apps already exist you have to know what kind of encoding they expect (ASCII, UTF-8 ...) and send the string with the needed encoding.
On the Android side these bytes are converted with the encoding to the string that is then used by the app.
If the app uses a specific protocol on top of UDP you could search for an existing implementation in C# for this protocol.
You can't. It's impossible.
The android receiver will have a method that will convert bytes from the stream into a string. Find out how it does it, and reverse the process when you encode.
Generally this'll be the string converted to it's byte[] representation with a int prefixed to it telling you the size of the string.
i'm developing application that is listening to the data coming to the pc and store it in a db
when i'm trying to use any sniffing software it decode the data and i can read it...
but in my code ....i cant read it at all
it come in a format like that
1822262151622341817118815518211616121520941131921572041519912321413018224510453482062312258624219217426213385792952422362282081777270129716688629114817282188771708157542505055171418651781981425595109572128317191993018793431541418175198551682143218916536118562071014546919618158204181231187237183188160147127165111798312311810419822146114761993113815821216617541542372062129733198212250147199288115346102031191275215728146245198190171121209115149107193226253199151253205183146112072202559697791491441131572351381412278441552554817712614110121823714822712523618924690185291182071331471286244143181469018522814822821118012620321315924832238219115405615512392145202385512115735771691111055935782371281492476567165158924021493139815144225143762294713291762001113814720516216041120169912317914878167571392103510118386589521910621319622274158971538465206168139190127867123282255271781242497522124211517622131122113236255230254211206911242051832545515823012124925217318223920523316923122925514321122343602492471242........
can any one tell me what kind of data is that and any code to solve it out??
To see what a real packet sniffer looks like, check out WireShark. There are many different protocols over TCP, and many of them are binary. Those that aren't may be using unicode characters, which are two-byte characters so an ascii display of them would be meaningless.
Anyway, the data you're displaying is pretty meaningless. It looks like decimal data, are you concatenating a bunch of decimal representations of the binary stream interpreted as byte or integer values? That would explain it. You should start by running the stream through System.TextEncoding.ASCII.Decode You'll probably see some recognizable strings. Then try System.TextEncoding.Unicode.Decode, etc.
No, we cannot. And the reason is simple, we don't know what application you are sniffing.
That stream of data could mean anything.
But, I suggest you print the data in hexadecimal. Maybe the data would make more sense.
First of all i am new to networking so i may say dumb thing in here.
Considering a client-server application using sockets(.net with c# if that matters).
The client sends some data, the
server process it and sends back a
string.
The client sends some other data,
the serve process it, queries the db
and sends back several hundreds of
items from the database
The client sends some other type of
data and the server notifies some
other clients
.
My question is how to set the buffer size correctly for reading/writing operation.
Should i do something like this: byte[] buff = new byte[client.ReceiveBufferSize] ?
I am thinking of something like this:
Client sends data to the server(and the server will follow the same pattern)
byte[] bytesToSend=new byte[2048] //2048 to be standard for any command send by the client
bytes 0..1 ->command type
bytes 1..2047 ->command parameters
byte[] bytesToReceive=new byte[8]/byte[64]/byte[8192] //switch(command type)
But..what is happening when a client is notified by the server without sending data?
What is the correct way to accomplish what i am trying to do?
Thanks for reading.
Doesn't matter what is the size of your buffer, it should be just a temporary storage for the data. Read this about reading from binary streams: http://www.yoda.arachsys.com/csharp/readbinary.html
I don't really understand what you are trying to do, but i try to give some advice. Your byte array and the Socket.ReceiveBufferSize property are two different things. The buffer size doesn't matter. You should use a byte array large enough to store the data you want to send or receive. For the size of the byte array, 2048 is probably ok, unless you want to send images, very large text, etc. I've done some simple client server programming and i found it easier if the receiver knows how much data to receive in advance. So for example you send the length of the string first as 1 or 2 bytes, and then the string using utf8 encoding. Use the Encoding.GetByteCount method for getting the size of the string in bytes for the chosen encoding. You should convert numbers to binary using the methods from Bitconverter. I found those helpful and now i wish, i would have discovered them earlier. To send other more complex data types you should decompose them and send them as strings or numbers one at a time.
When designing client/server communication it is good to introduces some a notion of a communication protocol (basically a set of rules, format that will be sent/received) by the client/server.
Here's an example of exchanging messages of variable size over the network
You can simplest think about XML as a protocol of your communication, then you reading socket until null byte was readed. (This is typicaly marker of end of xml document). You can implement this protocol in two ways.
To parse and generate XML document you can use XMLDocument class. You must to it impelemnt some schema. But if you want clean transform data as XML you can use simplest DataSet.
I use the excellent FileHelpers library when I work with text data. It allows me to very easily dump text fields from a file or in-memory string into a class that represents the data.
In working with a big endian microcontroller-based system I need to read a serial data stream. In order to save space on the very limited microcontroller platform I need to write raw binary data which contains field of various multi-byte types (essentially just dumping a struct variable out the serial port).
I like the architecture of FileHelpers. I create a class that represents the data and tag it with attributes that tell the engine how to put data into the class. I can feed the engine a string representing a single record and get an deserialized representation of the data. However, this is different from object serialization in that the raw data is not delimited in any way, it's a simple binary fixed record format.
FileHelpers is probably not suitable for reading such binary data as it cannot handle the nulls that show up and* I suspect that there might be unicode issues (the engine takes input as a string, so I have to read bytes from the serial port and translate them into a unicode string before they go to my data converter classes). As an experiment I have set it up to read the binary stream and as long as I'm careful to not send nulls it works quite well so far. It is easy to set up new converters that read the raw data and account for endian foratting issues and such. It currently fails on nulls and cannot process multiple records (it expect a CRLF between records).
What I want to know is if anyone knows of an open-source library that works similarly to FileHelpers but that is designed to handle binary data.
I'm considering deriving something from FileHelpers to handle this task, but it seems like there ought to be something already available to do this.
*It turns out that it does not complain about nulls in the input stream. I had an unrelated bug in my test program that came up where I expected a problem with the nulls. Should have investigated a little deeper first!
I haven't used filehelpers, so I can't do a direct comparison; however, if you have an object-model that represents your objects, you could try protobuf-net; it is a binary serialization engine for .NET using Google's compact "protocol buffers" wire format. Much more efficient than things like xml, but without the need to write all your own serialization code.
Note that "protocol buffers" does include some very terse markers between fields (typically one byte); this adds a little padding, but greatly improves version tolerance. For "packed" data (i.e. blocks of ints, say, from an array) this can be omitted if desired.
So: if you just want a compact output, it might be good. If you need a specific output, probably less so.
Disclosure: I'm the author, so I'm biased; but it is free.
When I am fiddling with GPS data in the SIRFstarIII binary mode, I use the Python interactive prompt with the serial module to fetch the stream from the USB/serial port and the struct module to convert the bytes as needed (per some format defined by SIRF). Using the interactive prompt is very flexible because I can read the string to a variable, process it, view the results and try again if needed. After the prototyping stage is finished, I have the data format strings that I need to put into the final program.
Your question doesn't mention anything about why you have a C# tag. I understand FileHelpers is a C# library, but I that doesn't tell me what environment you are working in. There is an implementation of Python for .NET called IronPython.
I realize this answer might mean you have to learn a new language, but having an interactive prompt is a very powerful tool for any programmer.