I am working my way through bytes in python and I have to say I have come across numerous difficulties. One of them is int.tobytes() function. Converting integers up to 32 works fine, but after that the output has single characters that I don't understand. For example:
x=90
y=33
z=76
print(x.to_bytes(4, 'big'))
print(y.to_bytes(4, 'big'))
print(z.to_bytes(4, 'big'))
Output:
b'\x00\x00\x00Z'
b'\x00\x00\x00!'
b'\x00\x00\x00L'
I don't understand the single digit byte at the start. For example, I would expect 90 to be:
'\x00\x00\x00\x5A'(Converting it to hexadecimal), but instead I get results like this. I can't seem to find solutions online. What do these single characters represent? Thanks
I am downloading a large data file from the Internet and I want to convert the values '0.00' in one column to a text character 'C'.
i= np.loadtxt('first_simple.txt')
np.set_printoptions(threshold=np.inf)
i=np.asarray(i,str)
np.place(i,i=='0.00','C')
print(i)
I think the data is initially floats.
I got an error message at first to do with the data type, so I converted the data to strings, but now Python just hangs.
I think there must be a better way of doing this task.
Any suggestions?
Thank you!
The code worked after making an alteration:
np.place(i,i=='0.00','C')
to
np.place(i,i=='0.0','C')
When I changed the datatype to a string, it removed a 0 which meant the place command did not find a '0.00' string.
My code works, but it is very slow.
If there's a better solution, can you let me know?
Thank you.
As part of a task I must write a program that copies a number with a number of unknown digits from a file. Python language.
I have created a function which retains into int several digits containing the number.
The function to read from a file:
file.read (1) This way I managed to read one character. I tried to replace the number in my variable file.read (myint) - but the reading does not work. The program ran all the way, but nothing was called.
I tried before to convert myint to int, float, long but to no avail.
Can anyone help?
Thanks .
I do not know where the discussion fits, I'd be happy to guide you.
sorry for my english..
my code part 1
my code part 2
I have a file header which I am reading and planning on writing which contains information about the contents; version information, and other string values.
Writing to the file is not too difficult, it seems pretty straightforward:
outfile.write(struct.pack('<s', "myapp-0.0.1"))
However, when I try reading back the header from the file in another method:
header_version = struct.unpack('<s', infile.read(struct.calcsize('s')))
I have the following error thrown:
struct.error: unpack requires a string argument of length 2
How do I fix this error and what exactly is failing?
Writing to the file is not too difficult, it seems pretty straightforward:
Not quite as straightforward as you think. Try looking at what's in the file, or just printing out what you're writing:
>>> struct.pack('<s', 'myapp-0.0.1')
'm'
As the docs explain:
For the 's' format character, the count is interpreted as the size of the string, not a repeat count like for the other format characters; for example, '10s' means a single 10-byte string, while '10c' means 10 characters. If a count is not given, it defaults to 1.
So, how do you deal with this?
Don't use struct if it's not what you want. The main reason to use struct is to interact with C code that dumps C struct objects directly to/from a buffer/file/socket/whatever, or a binary format spec written in a similar style (e.g. IP headers). It's not meant for general serialization of Python data. As Jon Clements points out in a comment, if all you want to store is a string, just write the string as-is. If you want to store something more complex, consider the json module; if you want something even more flexible and powerful, use pickle.
Use fixed-length strings. If part of your file format spec is that the name must always be 255 characters or less, just write '<255s'. Shorter strings will be padded, longer strings will be truncated (you might want to throw in a check for that to raise an exception instead of silently truncating).
Use some in-band or out-of-band means of passing along the length. The most common is a length prefix. (You may be able to use the 'p' or 'P' formats to help, but it really depends on the C layout/binary format you're trying to match; often you have to do something ugly like struct.pack('<h{}s'.format(len(name)), len(name), name).)
As for why your code is failing, there are multiple reasons. First, read(11) isn't guaranteed to read 11 characters. If there's only 1 character in the file, that's all you'll get. Second, you're not actually calling read(11), you're calling read(1), because struct.calcsize('s') returns 1 (for reasons which should be obvious from the above). Third, either your code isn't exactly what you've shown above, or infile's file pointer isn't at the right place, because that code as written will successfully read in the string 'm' and unpack it as 'm'. (I'm assuming Python 2.x here; 3.x will have more problems, but you wouldn't have even gotten that far.)
For your specific use case ("file header… which contains information about the contents; version information, and other string values"), I'd just use write the strings with newline terminators. (If the strings can have embedded newlines, you could backslash-escape them into \n, use C-style or RFC822-style continuations, quote them, etc.)
This has a number of advantages. For one thing, it makes the format trivially human-readable (and human-editable/-debuggable). And, while sometimes that comes with a space tradeoff, a single-character terminator is at least as efficient, possibly more so, than a length-prefix format would be. And, last but certainly not least, it means the code is dead-simple for both generating and parsing headers.
In a later comment you clarify that you also want to write ints, but that doesn't change anything. A 'i' int value will take 4 bytes, but most apps write a lot of small numbers, which only take 1-2 bytes (+1 for a terminator/separator) if you write them as strings. And if you're not writing small numbers, a Python int can easily be too large to fit in a C int—in which case struct will silently overflow and just write the low 32 bits.
I'm having a problem passing strings that exceed 80 characters in JSON. When I pass a string that's exactly 80 characters long it works like magic. But once I add the 81st letter it craps out. I've tried looking at the json object in firebug and it seems to think the string is an array because it has an expander next to it. Clicking the expander though does nothing. I've tried searching online for caps on JSON string sizes and work arounds but am coming up empty :(. Anybody know anything about this?
edit:
It actually doesn't matter what the string is... using "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz" yields the same results.
Here's my code: (I'm using python)
result = {"test": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
self.response.out.write(simplejson.dumps(result))
would you happen to know the class that encodes strings properly for python? Thanks so much :)
What is the 81st character? Sounds like the string isn't properly escaped, making the json decoder think it is an array. If you could post the string here, or at least the 20 or so characters around 80, I could probably tell you what is wrong. Also, if you could tell how the json string was made. In most languages you can get a class that will make proper json strings out of objects and arrays. For example, php has json_encode();