python mmap skipping every second byte when writing - python

I have a strange problem with using mmap in python when writing to memory (/dev/mem).
To be clear, reading is done in the same manner and it works OK.
But when it comes to writing, it seems that every second byte is unwritable.
ex.
when I read i get
addr 0x200 val 0x1234
but when I try to write
addr 0x200 val 0x4321
what really is written is
addr 0x200 val 0x0021
When I try to write byte by byte, the same happens.
ex.
write:
addr 0x200 0x43
addr 0x201 0x21
I get
addr 0x200 0x00
addr 0x201 0x21
Code:
class Pydump:
def __init__(self, addr, length = 1, word_size = 4, filename = '/dev/mem'):
if addr < 0 or length < 0: raise ValueError('Address or length invalid')
self._verbose = verbose
self.word_size = word_size
self.mask = ~(self.word_size - 1)
self.base_addr = addr & ~(mmap.PAGESIZE - 1)
self.addr_offset = addr - self.base_addr
self.word_length = length
self.no_of_bytes = self.word_length * self.word_size
# align length to full words
end_addr = addr + self.no_of_bytes
if end_addr % self.mask:
end_addr = (end_addr + self.word_size) & self.mask
self.map_length = end_addr - self.base_addr
self.fname = filename
self.fd = os.open(self.fname, os.O_RDWR | os.O_SYNC)
self.mem = mmap.mmap(self.fd, self.map_length, mmap.MAP_SHARED,
mmap.PROT_READ | mmap.PROT_WRITE,
offset=self.base_addr)
def read(self):
mem = self.mem
virt_base_addr = self.addr_offset & self.mask
mem.seek(virt_base_addr)
data = []
for i in range(self.no_of_bytes):
data.append(struct.unpack('B', mem.read_byte())[0])
abs_addr = self.base_addr + virt_base_addr
return PydumpBuffer(abs_addr, data, self.word_size)
def write(self, data):
mem = self.mem
virt_base_addr = self.addr_offset & self.mask
mem.seek(virt_base_addr)
if self.word_size == 1:
mem.write(struct.pack('B', data))
elif self.word_size == 2:
mem.write(struct.pack('H', data))
elif self.word_size == 4:
mem.write(struct.pack('I', data))
else:
raise ValueError('Invalid word size')
def write_bytes(self, bytes):
if len(bytes) != self.no_of_bytes: raise ValueError('Wrong number of bytes given')
mem = self.mem
virt_base_addr = self.addr_offset & self.mask
mem.seek(virt_base_addr)
for byte in bytes:
mem.write_byte(byte)
Example run (I prepared the memory with other memdump tool [bin] to be 0xEEEEEEEE):
>>> from pydump import Pydump as memdump
>>> memdump(0x18007C20, 1, 4).read()
0xEEEEEEEE
>>> memdump(0x18007C20, 1, 4).write(0x12345678)
>>> memdump(0x18007C20, 1, 4).read()
0x00340078
>>> memdump(0x18007C20, 1, 4).write(0x87654321)
>>> memdump(0x18007C20, 1, 4).read()
0x00650021
Example no 2 (I could not write even 2 bytes at 'first' byte place):
>>> memdump(0x18007C20, 1, 2).write(0xABCD)
>>> memdump(0x18007C20, 1, 4).read()
0x00650021
>>> memdump(0x18007C21, 1, 1).write(0xCD)
>>> memdump(0x18007C20, 1, 4).read()
0x00650021
>>> memdump(0x18007C22, 1, 1).write(0xCD)
>>> memdump(0x18007C20, 1, 4).read()
0x00CDCD00
Any thoughts on what could be the problem ?

I know what is wrong now.
The answer is that it's fault as well of python as my specific hardware/driver.
I looked through the implementation of mmap and it uses memcpy, which as we know copies byte by byte. In C implemention this did not happen, when we needed to write 4 bytes, 4 bytes were written.
So here comes the limitation of my hardware/driver which is that every register must be written in full (I did not know this at the time of stating the question), thus when writing byte by byte I will get weird behaviour.
Some registers are 16bit and some are 32bit.
When I was writing 0xFFFF to 16bit register I ended up having 0x00FF. So memcpy was copying 0xFF twice. My driver got write(0xFF) then write(0xFF), so what I was actually doing (because of the python) was two writes to register of byte 0xFF :)
Probably the same happened with 32bit registers, although it looked different (but also the driver may behave different).
For it to work I would have to change the implementation of python mmap :(

Related

Pico Pi Python - Onewire temperature sensor

I am trying to get temperature sensor to work on pico pi, using PIN 16. I got DS18X20 and onewire library from GitHub. I'm trying to get temperature reading but it shows an error.
I tried running this code:
import time
import machine
import ds18x20
import onewire
# the device is on GPIO12
dat = machine.Pin(22)
# create the onewire object
on = onewire.OneWire(dat)
ds = ds18x20.DS18X20(on)
# scan for devices on the bus
roms = ds.scan()
print('found devices:', roms)
# loop 10 times and print all temperatures
for i in range(10):
print('temperatures:', end=' ')
ds.convert_temp()
time.sleep_ms(750)
for rom in roms:
print(ds.read_temp(rom), end=' ')
print()
This is DS18X0 library:
"""
DS18x20 temperature sensor driver for MicroPython.
This driver uses the OneWire driver to control DS18S20 and DS18B20
temperature sensors. It supports multiple devices on the same 1-wire bus.
The following example assumes the ground of your DS18x20 is connected to
Y11, vcc is connected to Y9 and the data pin is connected to Y10.
>>> from pyb import Pin
>>> gnd = Pin('Y11', Pin.OUT_PP)
>>> gnd.low()
>>> vcc = Pin('Y9', Pin.OUT_PP)
>>> vcc.high()
>>> from ds18x20 import DS18X20
>>> d = DS18X20(Pin('Y10'))
Call read_temps to read all sensors:
>>> result = d.read_temps()
>>> print(result)
[20.875, 20.8125]
Call read_temp to read the temperature of a specific sensor:
>>> result = d.read_temp(d.roms[0])
>>> print(result)
20.25
If only one DS18x20 is attached to the bus, then you don't need to
pass a ROM to read_temp:
>>> result = d.read_temp()
>>> print(result)
20.25
"""
from onewire import OneWire
class DS18X20(object):
def __init__(self, pin):
self.ow = OneWire(pin)
# Scan the 1-wire devices, but only keep those which have the
# correct # first byte in their rom for a DS18x20 device.
self.roms = [rom for rom in self.ow.scan() if rom[0] == 0x10 or rom[0] == 0x28]
def read_temp(self, rom=None):
"""
Read and return the temperature of one DS18x20 device.
Pass the 8-byte bytes object with the ROM of the specific device you want to read.
If only one DS18x20 device is attached to the bus you may omit the rom parameter.
"""
rom = rom or self.roms[0]
ow = self.ow
ow.reset()
ow.select_rom(rom)
ow.write_byte(0x44) # Convert Temp
while True:
if ow.read_bit():
break
ow.reset()
ow.select_rom(rom)
ow.write_byte(0xbe) # Read scratch
data = ow.read_bytes(9)
return self.convert_temp(rom[0], data)
def read_temps(self):
"""
Read and return the temperatures of all attached DS18x20 devices.
"""
temps = []
for rom in self.roms:
temps.append(self.read_temp(rom))
return temps
def convert_temp(self, rom0, data):
"""
Convert the raw temperature data into degrees celsius and return as a float.
"""
temp_lsb = data[0]
temp_msb = data[1]
if rom0 == 0x10:
if temp_msb != 0:
# convert negative number
temp_read = temp_lsb >> 1 | 0x80 # truncate bit 0 by shifting, fill high bit with 1.
temp_read = -((~temp_read + 1) & 0xff) # now convert from two's complement
else:
temp_read = temp_lsb >> 1 # truncate bit 0 by shifting
count_remain = data[6]
count_per_c = data[7]
temp = temp_read - 0.25 + (count_per_c - count_remain) / count_per_c
return temp
elif rom0 == 0x28:
return (temp_msb << 8 | temp_lsb) / 16
else:
assert False
This is onewire library:
#!/usr/bin/env python
#
# Copyright (c) 2019, Pycom Limited.
#
# This software is licensed under the GNU GPL version 3 or any
# later version, with permitted additional terms. For more information
# see the Pycom Licence v1.0 document supplied with this file, or
# available at https://www.pycom.io/opensource/licensing
#
"""
OneWire library for MicroPython
"""
import time
import machine
class OneWire:
CMD_SEARCHROM = const(0xf0)
CMD_READROM = const(0x33)
CMD_MATCHROM = const(0x55)
CMD_SKIPROM = const(0xcc)
def __init__(self, pin):
self.pin = pin
self.pin.init(pin.OPEN_DRAIN, pin.PULL_UP)
print("init")
def init(self, pin):
self.pin = pin
self.pin.init(pin.OPEN_DRAIN, pin.PULL_UP)
print("init")
def reset(self):
"""
Perform the onewire reset function.
Returns True if a device asserted a presence pulse, False otherwise.
"""
sleep_us = time.sleep_us
disable_irq = machine.disable_irq
enable_irq = machine.enable_irq
pin = self.pin
pin(0)
sleep_us(480)
i = disable_irq()
pin(1)
sleep_us(60)
status = not pin()
enable_irq(i)
sleep_us(420)
return status
def read_bit(self):
sleep_us = time.sleep_us
enable_irq = machine.enable_irq
pin = self.pin
pin(1) # half of the devices don't match CRC without this line
i = machine.disable_irq()
pin(0)
sleep_us(1)
pin(1)
sleep_us(1)
value = pin()
enable_irq(i)
sleep_us(40)
return value
def read_byte(self):
value = 0
for i in range(8):
value |= self.read_bit() << i
return value
def read_bytes(self, count):
buf = bytearray(count)
for i in range(count):
buf[i] = self.read_byte()
return buf
def write_bit(self, value):
sleep_us = time.sleep_us
pin = self.pin
i = machine.disable_irq()
pin(0)
sleep_us(1)
pin(value)
sleep_us(60)
pin(1)
sleep_us(1)
machine.enable_irq(i)
def write_byte(self, value):
for i in range(8):
self.write_bit(value & 1)
value >>= 1
def write_bytes(self, buf):
for b in buf:
self.write_byte(b)
def select_rom(self, rom):
"""
Select a specific device to talk to. Pass in rom as a bytearray (8 bytes).
"""
self.reset()
self.write_byte(CMD_MATCHROM)
self.write_bytes(rom)
def crc8(self, data):
"""
Compute CRC
"""
crc = 0
for i in range(len(data)):
byte = data[i]
for b in range(8):
fb_bit = (crc ^ byte) & 0x01
if fb_bit == 0x01:
crc = crc ^ 0x18
crc = (crc >> 1) & 0x7f
if fb_bit == 0x01:
crc = crc | 0x80
byte = byte >> 1
return crc
def scan(self):
"""
Return a list of ROMs for all attached devices.
Each ROM is returned as a bytes object of 8 bytes.
"""
devices = []
diff = 65
rom = False
for i in range(0xff):
rom, diff = self._search_rom(rom, diff)
if rom:
devices += [rom]
if diff == 0:
break
return devices
def _search_rom(self, l_rom, diff):
if not self.reset():
return None, 0
self.write_byte(CMD_SEARCHROM)
if not l_rom:
l_rom = bytearray(8)
rom = bytearray(8)
next_diff = 0
i = 64
for byte in range(8):
r_b = 0
for bit in range(8):
b = self.read_bit()
if self.read_bit():
if b: # there are no devices or there is an error on the bus
return None, 0
else:
if not b: # collision, two devices with different bit meaning
if diff > i or ((l_rom[byte] & (1 << bit)) and diff != i):
b = 1
next_diff = i
self.write_bit(b)
if b:
r_b |= 1 << bit
i -= 1
rom[byte] = r_b
return rom, next_diff
class DS18X20(object):
def __init__(self, onewire):
self.ow = onewire
self.roms = [rom for rom in self.ow.scan() if rom[0] == 0x10 or rom[0] == 0x28]
self.fp = True
try:
1/1
except TypeError:
self.fp = False # floatingpoint not supported
def isbusy(self):
"""
Checks wether one of the DS18x20 devices on the bus is busy
performing a temperature convertion
"""
return not self.ow.read_bit()
def start_conversion(self, rom=None):
"""
Start the temp conversion on one DS18x20 device.
Pass the 8-byte bytes object with the ROM of the specific device you want to read.
If only one DS18x20 device is attached to the bus you may omit the rom parameter.
"""
if (rom==None) and (len(self.roms)>0):
rom=self.roms[0]
if rom!=None:
rom = rom or self.roms[0]
ow = self.ow
ow.reset()
ow.select_rom(rom)
ow.write_byte(0x44) # Convert Temp
def read_temp_async(self, rom=None):
"""
Read the temperature of one DS18x20 device if the convertion is complete,
otherwise return None.
"""
if self.isbusy():
return None
if (rom==None) and (len(self.roms)>0):
rom=self.roms[0]
if rom==None:
return None
else:
ow = self.ow
ow.reset()
ow.select_rom(rom)
ow.write_byte(0xbe) # Read scratch
data = ow.read_bytes(9)
return self.convert_temp(rom[0], data)
def convert_temp(self, rom0, data):
"""
Convert the raw temperature data into degrees celsius and return as a fixed point with 2 decimal places.
"""
temp_lsb = data[0]
temp_msb = data[1]
if rom0 == 0x10:
if temp_msb != 0:
# convert negative number
temp_read = temp_lsb >> 1 | 0x80 # truncate bit 0 by shifting, fill high bit with 1.
temp_read = -((~temp_read + 1) & 0xff) # now convert from two's complement
else:
temp_read = temp_lsb >> 1 # truncate bit 0 by shifting
count_remain = data[6]
count_per_c = data[7]
if self.fp:
return temp_read - 25 + (count_per_c - count_remain) / count_per_c
else:
return 100 * temp_read - 25 + (count_per_c - count_remain) // count_per_c
elif rom0 == 0x28:
temp = None
if self.fp:
temp = (temp_msb << 8 | temp_lsb) / 16
else:
temp = (temp_msb << 8 | temp_lsb) * 100 // 16
if (temp_msb & 0xf8) == 0xf8: # for negative temperature
temp -= 0x1000
return temp
else:
assert False
This is the error I'm getting:
>>> %Run -c $EDITOR_CONTENT
init
Traceback (most recent call last):
File "<stdin>", line 11, in <module>
File "ds18x20.py", line 33, in __init__
File "onewire.py", line 30, in __init__
AttributeError: 'OneWire' object has no attribute 'OPEN_DRAIN'
>>>
I've tried multiple ways to resolve this and watched a bunch of videos on youtube, but not sure what causes this issue.

sending hex data over serial port with checksum

I am new to python and I am trying to write a code to send hex serial data to a radio and receive hex data in response. radio_init_buf variable store the hex data to be sent. The last two bytes with store checksum. radio_init_buf[3] tells the size.
import sys
import glob
import numpy as np
import serial
class serial_communication():
PORT = 'COM2'
# RETURN_VALUE = None
def list_serial_ports(self):
""" Lists serial port names
:raises EnvironmentError:
On unsupported or unknown platforms
:returns:
A list of the serial ports available on the system
"""
if sys.platform.startswith('win'):
ports = ['COM%s' % (i + 1) for i in range(0,10)]
elif sys.platform.startswith('linux') or sys.platform.startswith('cygwin'):
# this excludes your current terminal "/dev/tty"
ports = glob.glob('/dev/tty[A-Za-z]*')
elif sys.platform.startswith('darwin'):
ports = glob.glob('/dev/tty.*')
else:
raise EnvironmentError('Unsupported platform')
result = []
for port in ports:
try:
s = serial.Serial(port)
s.close()
result.append(port)
except (OSError, serial.SerialException):
pass
return result
def serial_open(self):
self.ser = serial.Serial()
self.ser.baudrate = 9600
self.ser.port = sc.PORT
self.ser.parity = serial.PARITY_NONE
self.ser.stopbits = serial.STOPBITS_ONE
self.ser.bytesize = serial.EIGHTBITS
self.ser.writeTimeout = 1000
self.ser.timeout = 1000
try:
self.ser.open()
print("Port OPENED")
self.initialize(self.ser)
except Exception as e:
print("error opening serial port: " + str(e))
exit()
return self.ser
def checksum(self, crc_packet, crc_packet_length):
crc_table= np.array([0x0000, 0xC0C1, 0xC181, 0x0140, 0xC301, 0x03C0, 0x0280, 0xC241,
0xC601, 0x06C0, 0x0780, 0xC741, 0x0500, 0xC5C1, 0xC481, 0x0440,
0xCC01, 0x0CC0, 0x0D80, 0xCD41, 0x0F00, 0xCFC1, 0xCE81, 0x0E40,
0x0A00, 0xCAC1, 0xCB81, 0x0B40, 0xC901, 0x09C0, 0x0880, 0xC841,
0xD801, 0x18C0, 0x1980, 0xD941, 0x1B00, 0xDBC1, 0xDA81, 0x1A40,
0x1E00, 0xDEC1, 0xDF81, 0x1F40, 0xDD01, 0x1DC0, 0x1C80, 0xDC41,
0x1400, 0xD4C1, 0xD581, 0x1540, 0xD701, 0x17C0, 0x1680, 0xD641,
0xD201, 0x12C0, 0x1380, 0xD341, 0x1100, 0xD1C1, 0xD081, 0x1040,
0xF001, 0x30C0, 0x3180, 0xF141, 0x3300, 0xF3C1, 0xF281, 0x3240,
0x3600, 0xF6C1, 0xF781, 0x3740, 0xF501, 0x35C0, 0x3480, 0xF441,
0x3C00, 0xFCC1, 0xFD81, 0x3D40, 0xFF01, 0x3FC0, 0x3E80, 0xFE41,
0xFA01, 0x3AC0, 0x3B80, 0xFB41, 0x3900, 0xF9C1, 0xF881, 0x3840,
0x2800, 0xE8C1, 0xE981, 0x2940, 0xEB01, 0x2BC0, 0x2A80, 0xEA41,
0xEE01, 0x2EC0, 0x2F80, 0xEF41, 0x2D00, 0xEDC1, 0xEC81, 0x2C40,
0xE401, 0x24C0, 0x2580, 0xE541, 0x2700, 0xE7C1, 0xE681, 0x2640,
0x2200, 0xE2C1, 0xE381, 0x2340, 0xE101, 0x21C0, 0x2080, 0xE041,
0xA001, 0x60C0, 0x6180, 0xA141, 0x6300, 0xA3C1, 0xA281, 0x6240,
0x6600, 0xA6C1, 0xA781, 0x6740, 0xA501, 0x65C0, 0x6480, 0xA441,
0x6C00, 0xACC1, 0xAD81, 0x6D40, 0xAF01, 0x6FC0, 0x6E80, 0xAE41,
0xAA01, 0x6AC0, 0x6B80, 0xAB41, 0x6900, 0xA9C1, 0xA881, 0x6840,
0x7800, 0xB8C1, 0xB981, 0x7940, 0xBB01, 0x7BC0, 0x7A80, 0xBA41,
0xBE01, 0x7EC0, 0x7F80, 0xBF41, 0x7D00, 0xBDC1, 0xBC81, 0x7C40,
0xB401, 0x74C0, 0x7580, 0xB541, 0x7700, 0xB7C1, 0xB681, 0x7640,
0x7200, 0xB2C1, 0xB381, 0x7340, 0xB101, 0x71C0, 0x7080, 0xB041,
0x5000, 0x90C1, 0x9181, 0x5140, 0x9301, 0x53C0, 0x5280, 0x9241,
0x9601, 0x56C0, 0x5780, 0x9741, 0x5500, 0x95C1, 0x9481, 0x5440,
0x9C01, 0x5CC0, 0x5D80, 0x9D41, 0x5F00, 0x9FC1, 0x9E81, 0x5E40,
0x5A00, 0x9AC1, 0x9B81, 0x5B40, 0x9901, 0x59C0, 0x5880, 0x9841,
0x8801, 0x48C0, 0x4980, 0x8941, 0x4B00, 0x8BC1, 0x8A81, 0x4A40,
0x4E00, 0x8EC1, 0x8F81, 0x4F40, 0x8D01, 0x4DC0, 0x4C80, 0x8C41,
0x4400, 0x84C1, 0x8581, 0x4540, 0x8701, 0x47C0, 0x4680, 0x8641,
0x8201, 0x42C0, 0x4380, 0x8341, 0x4100, 0x81C1, 0x8081, 0x4040],dtype=np.uint16)
saved_crc_byte1 = crc_packet[crc_packet_length - 1]
saved_crc_byte2 = crc_packet[crc_packet_length - 2]
crc_packet[crc_packet_length - 1] = 0
crc_packet[crc_packet_length - 2] = 0
crc = 0
for crc_loop in range(0,crc_packet_length):
crc = (crc >> 8) ^ crc_table[(crc ^ crc_packet[crc_loop]) & 0xFF]
def initialize(self,serial_port):
ser = serial_port
if ser.isOpen():
print("Initialising...")
try:
ser.flushInput() # flush input buffer, discarding all its contents
ser.flushOutput() # flush output buffer, aborting current output
# and discard all that is in buffer
# write data
#f = open('F:/output.txt', 'wb')
radio_init_buf = np.array([0xAA, 0x00, 0x00, 0x08, 0x09, 0x32, 0x0, 0x0],dtype=np.uint8)
#radio_init_buf="\xAA\x00\x00\x08\x09\x32\x00\x00"
print(radio_init_buf)
self.checksum(radio_init_buf,radio_init_buf[3])
print(radio_init_buf)
ser.write(radio_init_buf)
serial.time.sleep(0.5) # give the serial port sometime to receive the data
#return_value = ser.read(7)
print(return_value)
print("Initialisation Complete")
#comm Link Check
print("Communication Link Checking..")
comm_check_buf = np.array([0xAA, 0x06, 00, 6, 0x0B, 0x70],dtype=np.uint8)
ser.write(comm_check_buf)
print("Link Check Complete")
#clear non-volatile memory
clear_nvm_buf = np.array([0xAA, 0x82, 00, 7, 1, 0, 0],dtype=np.uint8)
self.checksum(clear_nvm_buf, clear_nvm_buf[3])
ser.write(clear_nvm_buf)
#ser.close()
except Exception as e1:
print ("error communicating...: " + str(e1))
ser.close()
else:
print("cannot open serial port ")
sc = serial_communication()
print(sc.list_serial_ports())
sc.serial_open()
When i run the code i get:
['COM1', 'COM2', 'COM3']
Port OPENED
Initialising...
[170 0 0 8 9 50 0 0]
[170 0 0 8 9 50 0 0]
b'\xaa\x90\x00\x12\x01\x0c\x00'
Initialisation Complete
Communication Link Checking..
Link Check Complete
Instead of [170 0 0 8 9 50 0 0], i want the hex data.
Also, it is not returning radio_init_buf with checksum. The result after calling checksum is same.
Displaying in hex:
for n in radio_init_buf:
print("{:#x}".format(n), end='')
print()
{:#x} - a format string, #: adds the 0x prefix, x: presentation will be in hex,
In one line:
print(("{:#x} "*len(radio_init_buf)).format(*radio_init_buf))
It creates a string of length len(radio_init_buf).
In *radio_init_buf , '*' unpacks the list.

How can I send with 'struct.pack' type over Modbus TCP?

I want to send packet over Modbus TCP. I want to use:
But I can not send this way how can I send this packet? (I don't know something will be)
req = struct.pack(
'Something', transaction, identifier, length, unitid, func_code, reg_addr
)
These are my variables:
transaction=0x01
identifier=0x00
length=[0x00,0x06]
unitid=0x01
func_code=0x03
reg_addr=[0x13,0x14,0x15]
At first you can use pymodbus library with very features.
Also struct.pack() not support a list as argument.
0001 0000 0006 11 03 006B 0003 is a standard example of Modbus-TCP packet which contained:
0001: Transaction Identifier
0000: Protocol Identifier
0006: Message Length (6 bytes to follow)
11: The Unit Identifier (17 = 11 hex)
03: The Function Code (read Analog Output Holding Registers)
006B: The Data Address of the first register requested. (40108-40001 = 107 =6B hex)
0003: The total number of registers requested. (read 3 registers 40108 to 40110)
Reference
Thus, you can create a Modbus-TCP packet with the above example:
import struct
transaction = 0x0001
identifier = 0x0000
length = 0x0006
unitid = 0x11
fcode = 0x03 # Holding register fcode.
reg_addr = 0x006B # Register address.
count = 0x0003 # Read three register.
total_pack_string = '0x{:04x}{:04x}{:04x}{:02x}{:02x}{:04x}{:04x}'.format(
transaction, identifier, length, unitid, fcode, reg_addr, count
)
total_pack_hex = hex(int(total_pack_string, 16))
'''Or with using pack method.'''
pack_ = struct.pack(
'>HHHBBHH', transaction, identifier, length, unitid, fcode, reg_addr, count
)
# Then send the pack_ or total_pack_hex using a TCP-Socket.
[NOTE]:
transaction is 2Byte == Short == H
identifier is 2Byte == Short == H
length is 2Byte == Short == H
unitid is 1Byte == B
fcode is 1Byte == B
reg_addr is 2Byte == Short == H
count is 2Byte == Short == H
B is unsigned byte
H is unsigned short
Thus, the format will be like this >HHHBBHH
Using pymodbus equivalent:
from pymodbus.client.sync import ModbusTcpClient
unitid = 0x11
fcode = 0x03 # Holding register fcode.
reg_addr = 0x006B # Register address.
count = 0x0003 # Read three register.
cli = ModbusTcpClient('127.0.0.1', port=502)
if cli.connect():
res = cli.read_holding_registers(reg_addr, count=count, unit=unitid)
if not res.isError():
print(res.registers)
else:
print('There is an error.')
cli.close()
else:
print('Error in connection.')

Using ctypes to call a function that takes char ** because it writes both the buffer and the pointer

I'm trying to call iconv(3) from a Python (3) program, using ctypes. The C type signature of iconv is
size_t iconv(iconv_t cd,
char **inptr, size_t *inbytesleft,
char **outptr, size_t *outbytesleft);
and you are expected to call it like this:
char *inp = "abcdef";
char outbuf[16];
char *outp = outbuf;
size_t ibytes = strlen(inbuf);
size_t obytes = sizeof outbuf;
size_t rv = iconv(cd, &inp, &ibytes, &outp, &obytes);
It will write to outbuf, obviously, and it will also modify all four of the variables inp, outp, ibytes, and obytes to indicate how far it got with the conversion before running into a problem (if any). It guarantees not to write to the input string, despite that not being const.
Now, naively, you reflect that in ctypes like this:
iconv = libc.iconv
iconv.restype = ctypes.c_size_t
iconv.argtypes = [ctypes.c_void_p,
ctypes.POINTER(ctypes.c_char_p),
ctypes.POINTER(ctypes.c_size_t),
ctypes.POINTER(ctypes.c_char_p),
ctypes.POINTER(ctypes.c_size_t)]
(iconv_t is a typedef for void * in the C library I'm testing on) but when I try to call that, I get errors:
>>> obuf = ctypes.create_string_buffer(16)
>>> obuflen = ctypes.c_size_t(16)
>>> iconv(utf8_to_utf16,
... ctypes.byref(ctypes.c_char_p(b"abcdef")),
... ctypes.byref(ctypes.c_size_t(6)),
... ctypes.byref(obuf),
... ctypes.byref(obuflen))
ArgumentError: argument 4: <class 'TypeError'>: expected LP_c_char_p
instance instead of pointer to c_char_Array_16
Trying to explicitly convert obuf to c_char_p doesn't work either:
>>> optr = ctypes.c_char_p(obuf)
TypeError: bytes or integer address expected instead of c_char_Array_16 instance
These type names it's using in the error messages don't appear in the manual, and I'm pretty well stumped. What's the right way to go about this?
(If you are wondering why I would want to do this instead of using Python's built-in encoding converters, the short version is because Python's converters don't support the same set of encodings as [GNU] iconv, nor do they have the //TRANSLIT feature.)
Below is code which I used to find chars which are converted to '"' with translit. Sorry it is not very polished but I hope it could be still usable for somebody. :)
I needed to run it under python2.6 so I tried to wrote it a little more universal. (2.6 has some gotchas so code could be nicer and simpler under python3)
from __future__ import print_function
import sys
import ctypes
libc = ctypes.cdll.LoadLibrary("libc.so.6")
if sys.version_info[0]>2:
def unichr(a):
return chr(a)
LP_c_char2 = ctypes.POINTER( ctypes.c_char_p)
LP_c_char = ctypes.POINTER(ctypes.create_string_buffer(16).__class__)
get_errno_loc = libc.__errno_location
get_errno_loc.restype = ctypes.POINTER(ctypes.c_int)
class MyError(OSError):
def __init__(self, e):
if sys.version_info[0]<=2:
super(MyError, self).__init__(e)
else:
super().__init__(e)
def errcheck(ret, func, args):
if ret == -1 or ret == 2**64-1:
e = get_errno_loc()[0]
raise MyError(e)
return ret
iconv_open = libc.iconv_open
iconv_open.restype = ctypes.c_void_p
ret = iconv_open(
ctypes.c_char_p(b"ISO8859-2//TRANSLIT"),
ctypes.c_char_p(b"UTF-8"))
iconv = libc.iconv
iconv.errcheck = errcheck
iconv.restype = ctypes.c_size_t
obuf = ctypes.create_string_buffer(16)
obuflen = ctypes.c_size_t(16)
optr = LP_c_char(obuf)
inp = b'\xe2\x80\x9c'
iconv.argtypes = [ctypes.c_void_p, LP_c_char2, ctypes.POINTER(ctypes.c_size_t), ctypes.POINTER(LP_c_char), ctypes.POINTER(ctypes.c_size_t)]
r = iconv(ret, LP_c_char2(ctypes.c_char_p(inp)), ctypes.byref(ctypes.c_size_t(len(inp))), ctypes.byref(optr), ctypes.byref(obuflen))
print(obuf.value)
def func(inp = b"bbb"):
assert len(inp)<16 , "too big input"
obuf = ctypes.create_string_buffer(16)
obuflen = ctypes.c_size_t(16)
optr = LP_c_char(obuf)
r = iconv(ret, LP_c_char2(ctypes.c_char_p(inp)), ctypes.byref(ctypes.c_size_t(len(inp))), ctypes.byref(optr), ctypes.byref(obuflen))
return obuf, obuflen, r, obuf.value[:16-obuflen.value]
oo, uu, r, vys = func(b'\xe2\x80\x9c\xe2\x80\x9c')
print(oo.raw, uu, r, vys)
for i in range(sys.maxunicode):
try:
oo, uu, r, vys = func(unichr(i).encode('utf-8'))
if vys==b'"':
print(i, unichr(i))
except UnicodeEncodeError:
pass
except MyError as E:
pass
# print("MyError: {E} , {i}".format(E=E, i=i))

Encrypt data using Objective-C and Decrypt in Python

I have the same issue as this question but unfortunately there was no answer on it.
I have the following objective-c code to encrypt using CCCrypt:
(NSData *)doCrypt:(NSData *)data usingKey:(NSData *)key withInitialVector:(NSData *)iv mode:(int)mode error: (NSError *)error
{
int buffersize = 0;
if(data.length % 16 == 0) { buffersize = data.length + 16; }
else { buffersize = (data.length / 16 + 1) * 16 + 16; }
// int buffersize = (data.length <= 16) ? 16 : data.length;
size_t numBytesEncrypted = 0;
void *buffer = malloc(buffersize * sizeof(uint8_t));
CCCryptorStatus result = CCCrypt(mode, 0x0, 0x1, [key bytes], [key length], [iv bytes], [data bytes], [data length], buffer, buffersize, &numBytesEncrypted);
return [NSData dataWithBytesNoCopy:buffer length:numBytesEncrypted freeWhenDone:YES];
}
I use kCCAlgorithmAES128 with kCCOptionPKCS7Padding as options and call the function with [Cryptor doCrypt:data usingKey:key withInitialVector:nil mode:0x0 error:nil];
Now I would like to decrypt it using python and to do so I have the following code:
def decrypt(self, data, key):
iv = '\x00' * 16
encoder = PKCS7Encoder()
padded_text = encoder.encode(data)
mode = AES.MODE_CBC
cipher = AES.new(key, mode, iv)
decoded = cipher.decrypt(padded_text)
return decoded
The PKCS7Encoder looks like this:
class PKCS7Encoder():
"""
Technique for padding a string as defined in RFC 2315, section 10.3,
note #2
"""
class InvalidBlockSizeError(Exception):
"""Raised for invalid block sizes"""
pass
def __init__(self, block_size=16):
if block_size < 2 or block_size > 255:
raise PKCS7Encoder.InvalidBlockSizeError('The block size must be ' \
'between 2 and 255, inclusive')
self.block_size = block_size
def encode(self, text):
text_length = len(text)
amount_to_pad = self.block_size - (text_length % self.block_size)
if amount_to_pad == 0:
amount_to_pad = self.block_size
pad = chr(amount_to_pad)
return text + pad * amount_to_pad
def decode(self, text):
pad = ord(text[-1])
return text[:-pad]
Yet whenever I call the decrypt() function, it returns garbage. Am I missing something or having a wrong option enabled somewhere?
Example in and output:
NSData *keyData = [[NSData alloc] initWithRandomData:16];
NSLog(#"key: %#", [keyData hex]);
NSString *str = #"abcdefghijklmno";
NSLog(#"str: %#", str);
NSData *encrypted = [Cryptor encrypt:[str dataUsingEncoding:NSUTF8StringEncoding] usingKey:keyData];
NSLog(#"encrypted str: %#", [encrypted hex]);
Gives:
key: 08b6cb24aaec7d0229312195e43ed829
str: a
encrypted str: 52d61265d22a05efee2c8c0c6cd49e9a
And python:
cryptor = Cryptor()
encrypted_hex_string = "52d61265d22a05efee2c8c0c6cd49e9a"
hex_key = "08b6cb24aaec7d0229312195e43ed829"
print cryptor.decrypt(encrypted_hex_string.decode("hex"), hex_key.decode("hex"))
Result:
láz
Which is weird, but if dump the hex I get 610f0f0f0f0f0f0f0f0f0f0f0f0f0f0fb02b09fd58cccf04f042e2c90d6ce17a and 61 = a so I think it just shows wrong.
A bigger input:
key: 08b6cb24aaec7d0229312195e43ed829
str: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
encrypted str: 783fce3eca7ebe60d58b01da3d90105a93bf2d659cfcffc1c2b7f7be7cc0af4016b310551965526ac211f4d6168e3cc5
Result:
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaôNÍ“ƒ˜�Üšw6C%
Here you see that the a's are printed with garbage... so I assume this is a padding error or something like that
The IV is nill at the iOs side and 16x 0's at the Python side (see the code)
Your decryption: aes_decrypt(pkcs7_pad(ciphertext))
Correct decryption: pkcs7_unpad(aes_decrypt(ciphertext))
It has to be done this way, because AES in CBC mode expects plaintexts of a multiple of the block size, but you generally want to encrypt arbitrary plaintexts. Therefore, you need to apply the padding before encryption and remove the padding after decryption.
Keep in mind that a - (b % a) cannot be 0 for any (positive) value of a or b. This means that
if amount_to_pad == 0:
amount_to_pad = self.block_size
is unreachable code and can be removed. Good thing is that a - (b % a) already does what you wanted to do with the if block.
You also should extend the unpad (decode) function to actually check whether every padding byte is the same byte. You should also check that the every padding byte is not zero or larger than the block size.

Categories

Resources