I have the following code in C to calculate a CRC16-USB for some data flows:
uint16_t DRV_CANFDSPI_CalculateCRC16(uint8_t* data, uint16_t size)
{
uint16_t init = CRCBASE;
uint8_t index;
while (size-- != 0) {
index = ((uint8_t*) & init)[1] ^ *data++;
init = (init << 8) ^ crc16_table[index];
}
return init;
}
Where crc16_table is an array of some hex values of 2 bytes (like 0xAFF3) , and data is an array of hex values of 1 byte (like 0xA3) representing the data stream (aqcuired by other means). Size is the length of the data array.
I want to reproduce this piece of code in Python, but I don't know that this statement means:
index = ((uint8_t*) & init)[1] ^ *data++;
I would like to understand that does this statament means and does, so I can reproduce it in Python. I am not an expert in C but have some knowledge, and I mostly undestand the rest of the code, but this line is giving me a headache.
Thanks and have a nice day!
init has type uint16_t, so the expression &init has type "pointer to uint16_t", or uint16_t *. The expression (uint8_t *) &init means "get the address of init, but treat that as the address of a uint8_t object, rather than a uint16_t object".
That address is then used with a subscript operator - ((uint8_t *) &init)[1], which is basically equivalent to "treat init as an array of two uint8_t objects, and give me the value of the second element in that array".
Graphically:
+---+
init: | | <-- ((uint8_t *) &init)[0]
+---+
| | <-- ((uint8_t *) &init)[1]
+---+
So, basically, you're grabbing the lower 8 bits of init, bitwise XORing that with the value of the current byte of the input message, and then advancing data to point to the next byte of the input message.
I found a solution for my problem, with the following code:
def calculateCRC16(data):
init = 0xFFFF
for byte in data:
index = (init >> 8) ^ byte
init = ((init << 8) ^ crc16_table[index]) & 0xFFFF
return init
I think it is fairly simple. I tested this code with the above one in C and the results are the same. The masking in Python in the variable init is necessary as Python does not limit int variables to a fixed bit size. Also, in C, lib should be included for the code to work.
The intent of the index = ((uint8_t*) & init)[1] ^ *data++; statement is to XOR the high eight bits of init with the next byte of data (and to increment `data). Unfortunately, it is written improperly.
In the statement index = ((uint8_t*) & init)[1] ^ *data++;:
& init takes the address of init (which was defined with uint16_t init = CRCBASE;).
(uint8_t*) converts that address to a pointer to uint8_t. Further use of this pointer requires that uint8_t be a character type in the C implementation, which is likely but is not guaranteed by the C standard.
Applying [1] to this pointer fetches the next byte beyond where the pointer points. The fact that the second line uses init << 8, which results in a value solely dependent on the *low** eight bits of init, suggests that the intent in this first line was to fetch the high eight bits of init.
However, the C standard does not require that the bytes of a uint16_t be in any particular order, so it is not assured that using [1] will fetch the desired bits. And it is unnecessary; using init >> 8 in place of ((uint8_t*) & init)[1] would provide the desired bits.
Thus, the code could have been simply:
while (size--)
init = init << 8 ^ crc16_table[init>>8 ^ *data++];
Related
I have a function written in Python that works perfectly for what I need(it wasn't written by me).
I need to convert it to C++ so that it provides the same outcome. I know that it saves that float into 16-bit texture, so I am guessing this is converting 32-bit int into 16-bit float. All I need to to is to make it work in C++. Here is the python function:
def packTextureBits(index):
index = int(index)
index = index +1024
sigh=index&0x8000
sigh=sigh<<16
exptest=index&0x7fff
if exptest==0:
exp=0
else:
exp=index>>10
exp=exp&0x1f
exp=exp-15
exp=exp+127
exp=exp<<23
mant=index&0x3ff
mant=mant<<13
index=sigh|exp|mant
cp = pointer(c_int(index))
fp = cast(cp, POINTER(c_float))
return fp.contents.value
This was my approach in C++, but it returns completely screwed up values:
float PackIntToFloat(int value)
{
value += 1024;
int sign = (value & 0x8000) << 16;
int exp = value & 0x7fff;
if(exp != 0)
{
exp = value >> 10;
exp = exp & 0x1f;
exp = exp - 15 + 127;
exp = exp << 23;
}
int mant = (value & 0x3fff) << 13;
value = sign | exp | mant;
int* cp = new int(value);
float* fp = reinterpret_cast<float*>(cp);
return *fp;
// Also tried return (float)value; but returns other weird values.
}
So I owe you apologize guys. I was being stupid, not doing enough tests before posting here. My C++ solution is 100% working. I tested separate colors of the texture, and as it turned out, I assigned values to the texture the wrong way. I tried pushing floats into the texture, and it was 16 bit texture. I needed to convert these floats into half-precision floats after this conversion above, and then it started working. Texture flag called PF_FloatRGBA led me into believing that floats were the right thing to assign there, and they werent.
I still need to learn a lot. Thanks for all your help!
I am using ctypes (imported as c) in Python 3 to execute a C++ shared library. The library is loaded into python using:
smpLib = c.cdll.LoadLibrary(os.getcwd()+os.sep+'libsmpDyn.so')
One of the functions has the extern 'C' declaration const char* runSmpModel(...). The python function prototype is coded and run as:
proto_SMP = c.CFUNCTYPE(c.c_char_p,...)
runSmpModel = proto_SMP(('runSmpModel',smpLib))
res = runSmpModel(...)
This all works beautifully, but I'm unable to decode the res variable and obtain the string passed out by the C runSmpModel function. The value of res is displayed (I'm using ipython3) as b'\xd0'. The best solution I've found online - res.decode('utf-8') gives me the error:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd0 in position 0: unexpected end of data
The const char* return value from the runSmpModel function comes from
std::string scenID = SMPLib::SMPModel::runModel(...);
return scenID.c_str();
inside runModel, it is ultimately defined as shown here, where scenName is an input string:
auto utcBuffId = newChars(500);
sprintf(utcBuffId, "%s_%u", scenName.c_str(), microSeconds); // catenate scenario name & time
uint64_t scenIdhash = (std::hash < std::string>() (utcBuffId)); // hash it
auto hshCode = newChars(100);
sprintf(hshCode, "%032llX", scenIdhash);
scenId = hshCode;
The value of this specific res should be 0000000000000000BBB00C6CA8B8872E. How can I decode this string?
After a lot of further testing, I've identified the problem as the length of the string passed from the C function. No problems if the string is up to 15 characters in length, but if it's 16 or longer - no dice. For a minimum-working example, the C-code is:
extern "C" {
const char* testMeSO()
{
string scenarioID = "abcdefghijklmnop";
return scenarioID.c_str();
}
}
and python code is (same definition of smpLib as shown above):
proto_TST = c.CFUNCTYPE(c.c_char_p)
testMeSO = proto_TST(('testMeSO',smpLib))
res = testMeSO()
print("Scenario ID: %s"%res.decode('utf-8'))
This gives the decode error, unless any character is removed from the scenarioID variable in the C function. So it seems the question is "how can Python read a C char* longer than 15 characters, using ctypes.
After several days of debugging and testing, I've finally gotten this working, using the second solution posted by #Petesh on this SO post. I don't understand why ctypes is apparently limiting the char * value passed from C to 15 characters (+termination = 256 bits?).
Essentially, the solution is to pass into the C function an extra char * buff buffer that has already been created using ctypes.create_string_buffer(32*16), as well as an unsigned int buffsize of value 32*16. Then, in the C function execute scenarioID.copy(buff,buffsize). The python prototype function is modified in an obvious way.
I'm debugging with python audio, having a hard time with the audio coding.
Here I have a string full of audio data, say, [10, 20, 100].
However the data is stored in a string variable,
data = "����������������"
I want to inspect the values of this string.
Below is the things I tried
Print as int
I tried to use print "%i" % data[0]
ended up with
Traceback (most recent call last):
File "wire.py", line 28, in <module>
print "%i" % data[i]
TypeError: %d format: a number is required, not str
Convert to int
int(data[0]) ended up with
Traceback (most recent call last):
File "wire.py", line 27, in <module>
print int(data[0])
ValueError: invalid literal for int() with base 10: '\xd1'
Any idea on this? I want to print the string in a numerical way since the string is actually an array of sound wave.
EDIT
All your answers turned out to be really helpful.
The string is actually generated from the microphone so I believe it to be raw wave form, or vibration data. Further this should be referred to the audio API document, PortAudio.
After looking into PortAudio, I find this helpful example.
** This routine will be called by the PortAudio engine when audio is needed.
** It may called at interrupt level on some machines so don't do anything
** that could mess up the system like calling malloc() or free().
static int patestCallback( const void *inputBuffer, void *outputBuffer,
unsigned long framesPerBuffer,
const PaStreamCallbackTimeInfo* timeInfo,
PaStreamCallbackFlags statusFlags,
void *userData )
{
paTestData *data = (paTestData*)userData;
float *out = (float*)outputBuffer;
unsigned long i;
(void) timeInfo; /* Prevent unused variable warnings. */
(void) statusFlags;
(void) inputBuffer;
for( i=0; i<framesPerBuffer; i++ )
{
*out++ = data->sine[data->left_phase]; /* left */
*out++ = data->sine[data->right_phase]; /* right */
data->left_phase += 1;
if( data->left_phase >= TABLE_SIZE ) data->left_phase -= TABLE_SIZE;
data->right_phase += 3; /* higher pitch so we can distinguish left and right. */
if( data->right_phase >= TABLE_SIZE ) data->right_phase -= TABLE_SIZE;
}
return paContinue;
}
This indicates that there is some way that I can interpret the data as float
To be clear, your audio data is a byte string. The byte string is a representation of the bytes stored in the audio file. You are not going to simply be able to convert those bytes into meaningful values without knowing what is in the binary first.
As an example, the mp3 specification says that each mp3 contains header frames (described here: http://en.wikipedia.org/wiki/MP3). To read the header you would either need to use something like bitstring, or if you feel comfortable doing the bitwise manipulation yourself then you would just need to unpack an integer (4 bytes) and do some math to figure out the values of the 32 individual bits.
It really all depends on what you are trying to read, and how the data was generated. If you have whole byte numbers, then struct will serve you well.
If you're ok with the \xd1 mentioned above:
for item in data: print repr(item),
Note that for x in data will iterate over each value in the list rather than its location. If you want the location you can use for i in range(len(data)): ...
If you want them in numerical form, replace repr(item) with ord(item).
It is better if you use the new {}.format method:
data = "����������������"
print '{0}'.format(data[3])
You could use ord to map each byte to its numeric value between 0-255:
print map(ord, data)
Or, for Python 3 compatibility, do:
print([ord(c) for c in data])
It will also work with Unicode glyphs, which might not be what you want, so make sure you have a bytearray or an actual str or bytes object in Python 2.
I want to extract data from a file whoose information is stored in big-endian and always unsigned. How does the "cast" from unsigned int to int affect the actual decimal value? Am I correct that the most left bit decides about the whether the value is positive or negative?
I want to parse that file-format with python, and reading and unsigned value is easy:
def toU32(bits):
return ord(bits[0]) << 24 | ord(bits[1]) << 16 | ord(bits[2]) << 8 | ord(bits[3])
but how would the corresponding toS32 function look like?
Thanks for the info about the struct-module. But I am still interested in the solution about my actual question.
I would use struct.
import struct
def toU32(bits):
return struct.unpack_from(">I", bits)[0]
def toS32(bits):
return struct.unpack_from(">i", bits)[0]
The format string, ">I", means read a big endian, ">", unsigned integer, "I", from the string bits. For signed integers you can use ">i".
EDIT
Had to look at another StackOverflow answer to remember how to "convert" a signed integer from an unsigned integer in python. Though it is less of a conversion and more of reinterpreting the bits.
import struct
def toU32(bits):
return ord(bits[0]) << 24 | ord(bits[1]) << 16 | ord(bits[2]) << 8 | ord(bits[3])
def toS32(bits):
candidate = toU32(bits);
if (candidate >> 31): # is the sign bit set?
return (-0x80000000 + (candidate & 0x7fffffff)) # "cast" it to signed
return candidate
for x in range(-5,5):
bits = struct.pack(">i", x)
print toU32(bits)
print toS32(bits)
I would use the struct module's pack and unpack methods.
See Endianness of integers in Python for some examples.
The non-conditional version of toS32(bits) could be something like:
def toS32(bits):
decoded = toU32(bits)
return -(decoded & 0x80000000) + (decoded & 0x7fffffff)
You can pre-compute the mask for any other bit size too of course.
I started on a little toy project in C lately and have been scratching my head over the best way to mimic the strip() functionality that is part of the python string objects.
Reading around for fscanf or sscanf says that the string is processed upto the first whitespace that is encountered.
fgets doesn't help either as I still have newlines sticking around.
I did try a strchr() to search for a whitespace and setting the returned pointer to '\0' explicitly but that doesn't seem to work.
Python strings' strip method removes both trailing and leading whitespace. The two halves of the problem are very different when working on a C "string" (array of char, \0 terminated).
For trailing whitespace: set a pointer (or equivalently index) to the existing trailing \0. Keep decrementing the pointer until it hits against the start-of-string, or any non-white character; set the \0 to right after this terminate-backwards-scan point.
For leading whitespace: set a pointer (or equivalently index) to the start of string; keep incrementing the pointer until it hits a non-white character (possibly the trailing \0); memmove the rest-of-string so that the first non-white goes to the start of string (and similarly for everything following).
There is no standard C implementation for a strip() or trim() function. That said, here's the one included in the Linux kernel:
char *strstrip(char *s)
{
size_t size;
char *end;
size = strlen(s);
if (!size)
return s;
end = s + size - 1;
while (end >= s && isspace(*end))
end--;
*(end + 1) = '\0';
while (*s && isspace(*s))
s++;
return s;
}
If you want to remove, in place, the final newline on a line, you can use this snippet:
size_t s = strlen(buf);
if (s && (buf[s-1] == '\n')) buf[--s] = 0;
To faithfully mimic Python's str.strip([chars]) method (the way I interpreted its workings), you need to allocate space for a new string, fill the new string and return it. After that, when you no longer need the stripped string you need to free the memory it used to have no memory leaks.
Or you can use C pointers and modify the initial string and achieve a similar result.
Suppose your initial string is "____forty two____\n" and you want to strip all underscores and the '\n'
____forty two___\n
^ ptr
If you change ptr to the 'f' and replace the first '_' after two with a '\0' the result is the same as Python's "____forty two____\n".strip("_\n");
____forty two\0___\n
^ptr
Again, this is not the same as Python. The string is modified in place, there's no 2nd string and you cannot revert the changes (the original string is lost).
I wrote C code to implement this function. I also wrote a few trivial tests to make sure my function does sensible things.
This function writes to a buffer you provide, and should never write past the end of the buffer, so it should not be prone to buffer overflow security issues.
Note: only Test() uses stdio.h, so if you just need the function, you only need to include ctype.h (for isspace()) and string.h (for strlen()).
// strstrip.c -- implement white space stripping for a string in C
//
// This code is released into the public domain.
//
// You may use it for any purpose whatsoever, and you don't need to advertise
// where you got it, but you aren't allowed to sue me for giving you free
// code; all the risk of using this is yours.
#include <ctype.h>
#include <stdio.h>
#include <string.h>
// strstrip() -- strip leading and trailing white space from a string
//
// Copies from sIn to sOut, writing at most lenOut characters.
//
// Returns number of characters in returned string, or -1 on an error.
// If you get -1 back, then nothing was written to sOut at all.
int
strstrip(char *sOut, unsigned int lenOut, char const *sIn)
{
char const *pStart, *pEnd;
unsigned int len;
char *pOut;
// if there is no room for any output, or a null pointer, return error!
if (0 == lenOut || !sIn || !sOut)
return -1;
pStart = sIn;
pEnd = sIn + strlen(sIn) - 1;
// skip any leading whitespace
while (*pStart && isspace(*pStart))
++pStart;
// skip any trailing whitespace
while (pEnd >= sIn && isspace(*pEnd))
--pEnd;
pOut = sOut;
len = 0;
// copy into output buffer
while (pStart <= pEnd && len < lenOut - 1)
{
*pOut++ = *pStart++;
++len;
}
// ensure output buffer is properly terminated
*pOut = '\0';
return len;
}
void
Test(const char *s)
{
int len;
char buf[1024];
len = strstrip(buf, sizeof(buf), s);
if (!s)
s = "**null**"; // don't ask printf to print a null string
if (-1 == len)
*buf = '\0'; // don't ask printf to print garbage from buf
printf("Input: \"%s\" Result: \"%s\" (%d chars)\n", s, buf, len);
}
main()
{
Test(NULL);
Test("");
Test(" ");
Test(" ");
Test("x");
Test(" x");
Test(" x ");
Test(" x y z ");
Test("x y z");
}
This potential ‘solution' is by no means as complete or thorough as others have presented. This is for my own toy project in C - a text-based adventure game that I’m working on with my 14-year old son. If you’re using fgets() then strcspn() may just work for you as well. The sample code below is the beginning of an interactive console-based loop.
#include <stdio.h>
#include <string.h> // for strcspn()
int main(void)
{
char input[64];
puts("Press <q> to exit..");
do {
printf("> ");
fgets(input,64,stdin); // fgets() captures '\n'
input[strcspn(input, "\n")] = 0; // replaces '\n' with 0
if (input[0] == '\0') continue;
printf("You entered '%s'\n", input);
} while (strcmp(input,"q")!= 0); // returns 0 (false) when input = "q"
puts("Goodbye!");
return 0;
}