I have documentation on how to use a TCP/IP binary stream API. The attached images show the protocol. I have a working example of this in C#. I want to do this using python instead, as I don't know c# or windows.
I am assuming I would use python sockets, then I have to send the API messages, with payloads looking at this docs.
Could you point me in the right direction to get this going with python?
How would I set it to know this is the authentication message, 0x21 and compress the data etc?
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
s.sendall(u"{'username':'username', 'password':'password'}".encode('utf8'))
in OSI model you are above layer 4 (transport, TCP/IP,...) on at least layer 5 (session). there are some examples of implementations of session layer protocols like http, ftp,... i.e. in http://github.com/python-git/python/blob/master/Lib/ftplib.py or http://github.com/python-git/python/blob/master/Lib/httplib.py
as your protocol includes headers maybe http is the better example
to use the TCP/IP API in http protocol see
http://www.stackoverflow.com/questions/8315209/sending-http-headers-with-python
import socket
sock = socket.socket()
sock.bind(('', 8080))
sock.listen(5)
client, adress = sock.accept()
print "Incoming:", adress
print client.recv(1024)
print
client.send('HTTP/1.0 200 OK\r\n')
client.send("Content-Type: text/html\r\n\r\n")
client.send('<html><body><h1>Hello World</body></html>')
client.close()
print "Answering ..."
print "Finished."
sock.close()
as far as i can see you skipped the headers (version, sequence, type, encoding, ...) in your code completely you have to add them whenever you send a frame
so try
self.socket.send(...headers...)
self.socket.send(u"{'username':'username', 'password':'password'}".encode('utf8')) // has to be send as JSON ???
see also http://www.stackoverflow.com/questions/22083359/send-tetx-http-over-python-socket
ftp example (no headers...)
# Internal: send one line to the server, appending CRLF
def putline(self, line):
line = line + CRLF
if self.debugging > 1: print '*put*', self.sanitize(line)
self.sock.sendall(line)
also see scapy
I try to use tcp protocol once. You can send authorization values(username ,password ) in all requests.And other data with it exp({'username':'username', 'password':'password','data':'value'}).With this there is no any current standart for data. Many of tcp clients send that data like this #A#:username;password;#D:value\n (A -authorization ,D -data), example
Related
I used to filter packets into Wireshark with the simple dtls argument as filter. (Data Transport Layer Security which is some UDP TLS protocol)
Now, i wanted to do the same using C# and PcapDOTNet wrapper that uses WinPcap filters.
Sadly, i can't find anywhere the equivalent, and dtls is not recognised in the C# app, and so doesn't grab any packet anymore. (Simply it crashes the interpreter since the string is not recognised)
using (PacketCommunicator communicator = selectedDevice.Open(65536, PacketDeviceOpenAttributes.None, 1000))
{
using (BerkeleyPacketFilter filter = communicator.CreateFilter("dtls") {
communicator.SetFilter(filter);
communicator.ReceivePackets(1, packetHandler);
}
}
Is there any equivalent, please ?
EDIT : It looks like dtls is only a DISPLAY filter, and not a CAPTURE one. I could only capture filter by using udp port xx (xx being the port) but since the used ports are always randoms, i can't. So i would be glad to find another filtering workaround if you have one! I prefer only capturing the desired packets, rather than capturing everything then filtering the datas...
Wireshark : DTLS
There is only two packets i would like to capture. The one containing The Server Hello Done message or the one containing handshake message (the one with Record Layer) :
EDIT 2 : Ok, i am close to find what i need, but i need your help.
This answer from here must be the solution. tcp[((tcp[12] & 0xf0) >> 2)] = 0x16 is looking for handshake 22, but dtls is udp and not tcp and so the 12 offset might be different. Can anyone help me figure out what would be the correct formula to adapt it for dtls instead of tcp tls ?
I tried to use this on wireshark, but the filter is invalid and i don't really know why. If at least you could make it to work into wireshark, i could experiment differents value myself and come back with a final answer. udp[((udp[12] & 0xf0) >> 2)] = 0x16 is not a valid filter on wireshark.
So, i gave up on the dynamical way of finding the correct position of the data.
But this is what i ended with :
using (PacketCommunicator communicator = selectedDevice.Open(65536, PacketDeviceOpenAttributes.None, 1000))
{
using (BerkeleyPacketFilter filter = communicator.CreateFilter("udp && ((ether[62:4] = 0x16fefd00) || (ether[42:4] = 0x16fefd00))") {
communicator.SetFilter(filter);
communicator.ReceivePackets(1, packetHandler);
}
}
bytes[62:4] is the position of 16fefd00 for ipv6 packets, (42 for ipv4).
The 16 is for Content type handshake protocol 22 and the following fefd is for DTLS version 1.2. The last two zeros are just because using slice of bytes only works for 1,2 or 4, not 3. So i had to take them in consideration.
This is absolutly not perfect, i know, but for now, it works, since i couldn't find any other workaround yet.
I have a (production) WireShark capture file that I need to 'replay' over my (debug) network. I can read and interpret the packets, but I need to tweak a few details before the packets can be sent, like source IP addresses and ports.
The problem, however, is that all data in the PcapDotNet.Packets.Packet is read-only. It just has setters.
So like setting for instance the Ethernet.IpV4.Source will not work.
IpV4Address.TryParse("192.168.1.10", out var newAddress); // for demo sake.
packet.Ethernet.IpV4.Source = newAddress; // Won't work
Is there a simple way to avoid building a new packet from scratch, or is that the only way to create a slightly different packet?
Instead of changing the packet inplace, you should just create a new one based on the old one.
You can use ExtractLayer() on each layer you want to keep from the old packet, and then change the layer properties if necessary.
In this case, you can do:
IpV4Layer ipV4Layer = packet.Ethernet.IpV4.ExtractLayer();
ipV4Layer.Source = newAddress;
Packet newPacket = PacketBuild.Build(DateTime.Now, packet.Ethernet.ExtractLayer(), ipV4Layer, packet.Ethernet.IpV4.Payload.ExtractLayer());
You probably also want to reset the IPv4 checksum, so you should do:
ipV4Layer.HeaderChecksum = null;
And you might need to something similar to the UDP or TCP layers on top of the IPv4 layer, in case you have them.
First linux service listening process is started using the following command:
obexpushd –B[00:15:83:3D:0A:57]:9 –d –o /home/myfolder
On windows the following code is used to perform the obex transfer:
InTheHand.Net.BluetoothAddress address = peerDevice.DeviceAddress;
System.Uri uri = new Uri("obex://" + address.ToString() + "/" + srcfile.Name);
request = new ObexWebRequest(uri);
startcopy = DateTime.Now;
request.ReadFile(file); // this performs the file read from the hard drive
try
{
response = (ObexWebResponse)request.GetResponse(); // here file should be pushed to the listening service
}
catch (System.InvalidOperationException ex)
{
if (response != null) {
response.Close();
}
return;
}
Devices see each other and their obex services are visible as well.
Transfer seems to be successful, but no data is actually transferred.
The code works between windows and windows without a problem.
Obexpushd process ouput shows:
obexpushd 0.10.2 Copyright (C) 2006-2010 Hendrik Sattler
This software comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under certain conditions.
Listening on bluetooth/[00:15:83:3D:0A:57]:9
OBEX_EV_ACCEPTHINT, OBEX_CMD_CONNECT
0: Connection from "bluetooth/[00:09:DD:50:94:0B]:9"
0: OBEX_EV_REQHINT, OBEX_CMD_CONNECT
0: OBEX_EV_REQ, OBEX_CMD_CONNECT
0: Sending response code 0
0: OBEX_EV_REQDONE, OBEX_CMD_CONNECT
0: OBEX_EV_REQHINT, OBEX_CMD_PUT
0.1: OBEX_EV_REQCHECK, OBEX_CMD_PUT
0.1: OBEX_EV_REQDONE, OBEX_CMD_PUT
0.1: OBEX_EV_REQHINT, OBEX_CMD_DISCONNECT
0.1: Sending response code 100
0.1: OBEX_EV_REQ, OBEX_CMD_DISCONNECT
0.1: OBEX_EV_REQDONE, OBEX_CMD_DISCONNECT
I have also tried to disable Authentication in C# code but that did not help.
Does any one have idea how to nail this problem down or where to even start looking?
Ok so it seems that no one is much interested in this topic. :) Fortunately I have found the solution myself.
However, it involved a lot of analysis (of both obex transfer protocol and 32feet library) and a bit of luck.
The difference between Linux obexpushd implementation lies in its interpretation of OBEX transfer packets.
I found the OBEX specification on page: OBEX specification.
After debugging the internals of the 23feet obex transfer I found where the code sends the OBEX PUT command used to send file to the receiver. Obex specification gives the following example for PUT inititialization packet:
PUT Command | length of packet | Name header | Length of Name header | name of object | Length header | Length of object | Object Body chunk header | Length of Body header | bytes of body
The 32feet library sends the first packet without the Body Header which causes error in obexpushd Linux command.
Not shure if it is error in 32feet library, obexpushd or if the OBEX specification is not precise enough, but adding the Body header to the first packet solved the problem. From my experiments it turns out that at least 2 first bytes of the objectt must be sent in the first packet. Moreover, adding the header does not crash anything else and Windows<->Windows transfer still works very well.
I have an Arduino Uno with a Bluesmirf Silver module connected. My Arduino has a temperature sensor which records the temp regularly. The Arduino listens for any string being sent to it over bluetooth and responds with the latest data.
I have written a C# application to fetch this data but I am seeing some strange behaviour. I am using the following code to connect, send a string and get the returned data.
mPort = new SerialPort(mPortName, 115200, Parity.None, 8, StopBits.One);
mPort.Open();
mPort.Write("download");
Thread.Sleep(1000);
while (mPort.BytesToRead > 0)
{
String data = mPort.ReadExisting();
this.BeginInvoke(new Action<String>(AddMessage), data);
}
The data I get back looks like this:
Line added locally within C# application:
Send: download
Lines added based on data received from Arduino:
Read: d???+?
GotData
------
Total Readings, 1069
Num Readings, 360
Lost Readings, 709
Reading Interval, 240000
------
350,19.34
351,19.34
352,19.34
353,20.31
....
All the text looks fine apart from the string which is being echoed back which I sent to the Arduino. Have I done something wrong with the way I sent the data?
FYI - The datasheet for the bluetooth module is here: http://www.sparkfun.com/datasheets/Wireless/Bluetooth/rn-bluetooth-um.pdf
#Jeff - This is the code which I use on my Arduino to receive data: https://github.com/mchr3k/arduino/blob/master/tempsensor/StringReader.cpp
#Jeff - stringDataLen defines the length and I call the overall function from this file: https://github.com/mchr3k/arduino/blob/master/tempsensor/tempsensor.ino
EDIT: Here is the complete source code
Arduino - https://github.com/mchr3k/arduino/tree/master/tempsensor
C# application - https://github.com/mchr3k/arduino/tree/master/serialdownload
The C# code is definitely getting the flow control wrong for some reason. I have switched to use the following code in C# and this gets a string through without corruption.
private void write(SerialPort mPort, string str)
{
foreach (char c in str)
{
mPort.Write(new char[] {c}, 0, 1);
Thread.Sleep(10);
}
}
An incorrect encoding perhaps?
mPort = new SerialPort(mPortName, 115200, Parity.None, 8, StopBits.One);
mPort.Encoding = System.Text.Encoding.ASCII; // Or System.Text.Encoding.UTF8
mPort.Open();
mPort.Write("download");
Read byte-by-byte and check each byte one by one to debug lower level problems. ReadExisting() converts bytes to a String based on the Encoding property.
My issue was caused by me using the SoftwareSerial class to communicate with my Bluetooth module on pins 2 & 3. I was using a baud rate of 115200 which is claimed to be supported on this page: http://arduino.cc/en/Reference/SoftwareSerial
However, this page ( http://arduino.cc/en/Reference/SoftwareSerialBegin ) states that the maximum baud rate supported is actually 9600. I'm not sure whether this is accurate but reducing my baud rate to 9600 has fixed my issue.
I suggest you to decrease communication speed, because there is no reason to use 115200bps (only if your module demand this speed, then it's ok). Also you are sending string "download" which is not good, rather use markers something like "#D" which internally for your Arduino device means, send data to computer. In this way you are sending only two bytes instead eight, and you will decrease probability of error, and Arduino code will be better.
Now, let's try fix the problem. First try use something like this when you are reading data from Arduino device:
ArayList dataReceaved=new ArrayList():
while(serialPort.BytesToRead>0 && serialPort.IsOpen){
dataReceaved.Add(serialPort.ReadByte());
}
So I suggest you to read byte by byte, in this or similar way. Also you shold be careful if you are sending numbers from Arduino device. If you are, then use something like this:
Serial.print(temperatureValue,BYTE);
With this code you explicitly say that data you are sending is byte long. If this not help, please let me know, so we can try something else.
I have to do a Windows application that from times to times access a Gmail account and checks if there is a new email. In case there is, it must read the email body and subject (a simple text email, without images or attachments).
Please, do not use paid libs, and in case of any other libs used, give the download path.
And I need the email body and subject only. So if the long and complex message that comes from Gmail could be parsed and only two strings containing the subject and the body, it would be perfect.
Finally, I only have to get the new messages arrived since the last execution. So the read messages could be marked as "read" and only the new ones (marked as "new") are considered.
The code can be written in Python or C++, but I prefer it in C#.
Related question:
Properly formatted example for Python iMAP email access?
This prints the subject and body of unseen messages, and marks those messages as seen.
import imaplib
import email
def extract_body(payload):
if isinstance(payload,str):
return payload
else:
return '\n'.join([extract_body(part.get_payload()) for part in payload])
conn = imaplib.IMAP4_SSL("imap.gmail.com", 993)
conn.login("user", "password")
conn.select()
typ, data = conn.search(None, 'UNSEEN')
try:
for num in data[0].split():
typ, msg_data = conn.fetch(num, '(RFC822)')
for response_part in msg_data:
if isinstance(response_part, tuple):
msg = email.message_from_string(response_part[1])
subject=msg['subject']
print(subject)
payload=msg.get_payload()
body=extract_body(payload)
print(body)
typ, response = conn.store(num, '+FLAGS', r'(\Seen)')
finally:
try:
conn.close()
except:
pass
conn.logout()
Much of the code above comes from Doug Hellmann's tutorial on imaplib.
Use one of the many C# IMAP libraries.
Note that there are some differences between Gmail-IMAP and IMAPA. For example, due to the fact that Gmail treats folders like labels, the code like the one below doesn't delete message if it's tagged with some other folder:
imap_instance.uid('store', uid, '+FLAGS', '\\Deleted')
imap_instance.expunge()
I know this is an old post but I wanted to add the following link to the Open Source ImapX 2 Library discussion: https://imapx.codeplex.com/ the developers seem to be keeping the project up to date. Great job to those all involved
Google has opened it's Gmail API for accessing your gmail account. You can check a quickstart sample with the basic functionalities at this link:
https://developers.google.com/gmail/api/quickstart/python
from imap_tools import MailBox, Q
# This prints the subject and body of unseen messages, and marks those messages as seen.
with MailBox('imap.mail.com').login('test#mail.com', 'password') as mailbox:
# *mark_seen param = True by default
print([(m.subject, m.html or m.text) for m in mailbox.fetch(Q(seen=False), mark_seen=True)])
imap_tools