Related
I have created a variable named 'j' which has some values and I want my code to pick one value at a time and execute.
I tried writing code but it does not work. I'm sharing my code please see when it can be improved.
j = ['0', '1', '3', '4', '6', '7', '9', '10', '11', '12', '13', '14', '15', '16', '17', '18', '19', '20', '21', '22', '23', '24', '25', '26', '27', '28', '29', '30', '31', '32', '33', '34', '35', '36', '37', '38', '39', '40', '41', '42', '43', '44', '45', '46', '47', '48', '49', '50', '51', '52', '53', '54', '55', '56', '57', '58', '59', '60', '61', '62', '63', '64', '65', '66', '67']
for i in j:
labels('i') = mne.read_labels_from_annot('sub-CC721377_T1w', parc='aparc', subjects_dir=subjects_dir)['i']
done
According to the MNE documentation, the function read_labels_from_annot returns a list of labels.
Thus, instead of indexing the result with ...)[0] at the end, you should just capture the entire list:
labels = mne.read_labels_from_annot(...)
This would capture a list of labels, rather than a single label, which would have the effect of 'indexing at the end "[0]" from 0 - 67'.
You asked about adding all the results together into a label_all variable. You didn't specify (and I don't know anything about the MNE package), so it's not clear: do the labels ever repeat? Is is possible that "lab123" will occur in every input file? If so, should the label_all store multiple copies of the same value, or just the unique label names?
I think something like this is what you're after:
import mne
def get_labels_for_subject(sub, *, hemi='both', parc='aparc', **kwargs):
"""Get MNE labels for a given subject. **kwargs allows passing named
parameters like subjects_dir, regexp, and others that default to None."""
labels = mne.read_labels_from_annot(sub, hemi=hemi, parc=parc, **kwargs)
return labels
# List of all the subjects
subjects = [
'sub-CC721377_T1w',
'sub-next???',
]
label_all = []
for s in subjects:
label_all.extend(get_labels_for_subject(s, subjects_dir='.'))
print("Got labels:", label_all)
def calcString( st, crc):
"""Given a binary string and starting CRC, Calc a final CRC-16 """
for ch in st:
crc = (crc >> 8) ^ table[(crc ^ ord(ch)) & 0xFF]
return crc
I am getting different CRC values for the following string values:
051075F9000C1800000000000000000000000000002D0A0C00000000000014
and
\x05\x10\x75\xF9\x00\x0C\x18\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x2D\x0A\x0C\x00\x00\x00\x00\x00\x00\x14
please help me on this.
If i want to convert this
051075F9000C1800000000000000000000000000002D0A0C00000000000014
to
\x05\x10\x75\xF9\x00\x0C\x18\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x2D\x0A\x0C\x00\x00\x00\x00\x00\x00\x14
what should i do?.
With binascii
You could use binascii.unhexlify (thanks #JonClements):
>>> import binascii
>>> binascii.unhexlify(b"051075F9000C1800000000000000000000000000002D0A0C00000000000014")b'\x05\x10u\xf9\x00\x0c\x18\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00-\n\x0c\x00\x00\x00\x00\x00\x00\x14'
With chr and int
If you want to do it yourself, you could simply split your hex string in a list of 2 characters, convert it to an int and get the corresponding character with chr:
>>> import re
>>> crc = "051075F9000C1800000000000000000000000000002D0A0C00000000000014"
>>> if len(crc) % 2: raise ValueError('Incorrect string. Length should be even')
>>> xxs = re.findall('[0-9a-f]{2}', crc, re.I)
>>> xxs
['05', '10', '75', 'F9', '00', '0C', '18', '00', '00', '00', '00', '00', '00', '00', '00', '00', '00', '00', '00', '00', '00', '2D', '0A', '0C', '00', '00', '00', '00', '00', '00', '14']
>>> ''.join(chr(int(xx, 16)) for xx in xxs)
'\x05\x10uù\x00\x0c\x18\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00-\n\x0c\x00\x00\x00\x00\x00\x00\x14'
Note that "\x75" is displayed as "u":
>>> chr(int('75', 16))
'u'
The strings are equal, though:
>>> chr(int('75', 16)) == "\x75"
True
There is a great library for binary and ASCII conversions: binascii
The specific methods you need for converting an ASCII string to a HEX string are binascii.unhexlify(hexstr) or binascii.a2b_hex(hexstr) (these two are actually the same method). From the docs:
Return the binary data represented by the hexadecimal string hexstr. This function is the inverse of b2a_hex(). hexstr must contain an even number of hexadecimal digits (which can be upper or lower case), otherwise a TypeError is raised.
Example:
plain = "051075F9000C1800000000000000000000000000002D0A0C00000000000014"
binary = binascii.unhexlify(plain)
Just for completeness, the opposite operations is:
ascii = binascii.hexlify(binary) # Note: Returns lowercase hex chars
I'm extracting jpeg type bits from mp3 data actually it will be album arts.
I thought about using library called mutagen, but I'd like to try with bits for some practice purpose.
import os
import sys
import re
f = open(sys.argv[1], "rb")
#sys.argv[1] gets mp3 file name ex) test1.mp3
saver = ""
for value in f:
for i in value:
hexval = hex(ord(i))[2:]
if (ord(i) == 0):
saver += "00" #to match with hex form
else:
saver += hexval
header = "ffd8"
tail = "ffd9"
this part of code is to get mp3 as bit form, and then transform it into hex
and find jpeg trailers which starts as "ffd8" and ends with "ffd9"
frontmatch = re.search(header,saver)
endmatch = re.search(tail, saver)
startIndex = frontmatch.start()
endIndex = endmatch.end()
jpgcontents = saver[startIndex:endIndex]
scale = 16 # equals to hexadecimal
numbits = len(jpgcontents) * 4 #log2(scale)
bitcontents = bin(int(jpgcontents, scale))[2:].zfill(numbits)
and here, I get the bits between the header and tail and transform it into
binary form. Which supposed to be the jpg part of the mp3 files.
txtfile = open(sys.argv[1] + "_tr.jpg", "w")
txtfile.write(bitcontents)
and I wrote the bin to the new file with writing type as jpg.
sorry for my wrong naming as txtfile.
But these codes gave the error which is
Error interpreting JPEG image file
(Not a JPEG file: starts with 0x31 0x31)
I'm not sure whether the bits I extracted are wrong or writing to the file
step is wrong. Or there might be other problem in code.
I'm working in linux version with python 2.6. Is there anything wrong with
just writing str type of bin data as JPG?
You are creating a string of ASCII zeroes and ones, i.e. \x30 and \x31, but the JPEG file needs to be proper binary data. So where your file should have a single byte of (for example) \xd8 you instead have these eight bytes: 11011000, or \x31\x31\x30\x31\x31\x30\x30\x30.
You don't need to do all that messy conversion stuff. You can just search directly for the desired byte patterns, writing them using \x hex escape sequences. And you don't even need regex: the simple string .index or .find methods can do this easily and quickly.
with open(fname, 'rb') as f:
data = f.read()
header = "\xff\xd8"
tail = "\xff\xd9"
try:
start = data.index(header)
end = data.index(tail, start) + 2
except ValueError:
print "Can't find JPEG data!"
exit()
print 'Start: %d End: %d Size: %d' % (start, end, end - start)
with open(fname + "_tr.jpg", 'wb') as f:
f.write(data[start:end])
(Tested on Python 2.6.6)
However, extracting embedded JPEG data like this isn't foolproof, since it's possible that those header and tail byte sequences exist in the MP3 sound data.
FWIW, a simpler way to translate binary data to hex strings and back is to use hexlify and unhexlify from the binascii module.
Here are some examples of doing these transformations, both with and without the binascii functions.
from binascii import hexlify, unhexlify
#Create a string of all possible byte values
allbytes = ''.join([chr(i) for i in xrange(256)])
print 'allbytes'
print repr(allbytes)
print '\nhex list'
print [hex(ord(v))[2:].zfill(2) for v in allbytes]
hexstr = hexlify(allbytes)
print '\nhex string'
print hexstr
newbytes = ''.join([chr(int(hexstr[i:i+2], 16)) for i in xrange(0, len(hexstr), 2)])
print '\nNew bytes'
print repr(newbytes)
print '\nUsing unhexlify'
print repr(unhexlify(hexstr))
output
allbytes
'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f !"#$%&\'()*+,-./0123456789:;<=>?#ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~\x7f\x80\x81\x82\x83\x84\x85\x86\x87\x88\x89\x8a\x8b\x8c\x8d\x8e\x8f\x90\x91\x92\x93\x94\x95\x96\x97\x98\x99\x9a\x9b\x9c\x9d\x9e\x9f\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac\xad\xae\xaf\xb0\xb1\xb2\xb3\xb4\xb5\xb6\xb7\xb8\xb9\xba\xbb\xbc\xbd\xbe\xbf\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf\xd0\xd1\xd2\xd3\xd4\xd5\xd6\xd7\xd8\xd9\xda\xdb\xdc\xdd\xde\xdf\xe0\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xeb\xec\xed\xee\xef\xf0\xf1\xf2\xf3\xf4\xf5\xf6\xf7\xf8\xf9\xfa\xfb\xfc\xfd\xfe\xff'
hex list
['00', '01', '02', '03', '04', '05', '06', '07', '08', '09', '0a', '0b', '0c', '0d', '0e', '0f', '10', '11', '12', '13', '14', '15', '16', '17', '18', '19', '1a', '1b', '1c', '1d', '1e', '1f', '20', '21', '22', '23', '24', '25', '26', '27', '28', '29', '2a', '2b', '2c', '2d', '2e', '2f', '30', '31', '32', '33', '34', '35', '36', '37', '38', '39', '3a', '3b', '3c', '3d', '3e', '3f', '40', '41', '42', '43', '44', '45', '46', '47', '48', '49', '4a', '4b', '4c', '4d', '4e', '4f', '50', '51', '52', '53', '54', '55', '56', '57', '58', '59', '5a', '5b', '5c', '5d', '5e', '5f', '60', '61', '62', '63', '64', '65', '66', '67', '68', '69', '6a', '6b', '6c', '6d', '6e', '6f', '70', '71', '72', '73', '74', '75', '76', '77', '78', '79', '7a', '7b', '7c', '7d', '7e', '7f', '80', '81', '82', '83', '84', '85', '86', '87', '88', '89', '8a', '8b', '8c', '8d', '8e', '8f', '90', '91', '92', '93', '94', '95', '96', '97', '98', '99', '9a', '9b', '9c', '9d', '9e', '9f', 'a0', 'a1', 'a2', 'a3', 'a4', 'a5', 'a6', 'a7', 'a8', 'a9', 'aa', 'ab', 'ac', 'ad', 'ae', 'af', 'b0', 'b1', 'b2', 'b3', 'b4', 'b5', 'b6', 'b7', 'b8', 'b9', 'ba', 'bb', 'bc', 'bd', 'be', 'bf', 'c0', 'c1', 'c2', 'c3', 'c4', 'c5', 'c6', 'c7', 'c8', 'c9', 'ca', 'cb', 'cc', 'cd', 'ce', 'cf', 'd0', 'd1', 'd2', 'd3', 'd4', 'd5', 'd6', 'd7', 'd8', 'd9', 'da', 'db', 'dc', 'dd', 'de', 'df', 'e0', 'e1', 'e2', 'e3', 'e4', 'e5', 'e6', 'e7', 'e8', 'e9', 'ea', 'eb', 'ec', 'ed', 'ee', 'ef', 'f0', 'f1', 'f2', 'f3', 'f4', 'f5', 'f6', 'f7', 'f8', 'f9', 'fa', 'fb', 'fc', 'fd', 'fe', 'ff']
hex string
000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f505152535455565758595a5b5c5d5e5f606162636465666768696a6b6c6d6e6f707172737475767778797a7b7c7d7e7f808182838485868788898a8b8c8d8e8f909192939495969798999a9b9c9d9e9fa0a1a2a3a4a5a6a7a8a9aaabacadaeafb0b1b2b3b4b5b6b7b8b9babbbcbdbebfc0c1c2c3c4c5c6c7c8c9cacbcccdcecfd0d1d2d3d4d5d6d7d8d9dadbdcdddedfe0e1e2e3e4e5e6e7e8e9eaebecedeeeff0f1f2f3f4f5f6f7f8f9fafbfcfdfeff
New bytes
'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f !"#$%&\'()*+,-./0123456789:;<=>?#ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~\x7f\x80\x81\x82\x83\x84\x85\x86\x87\x88\x89\x8a\x8b\x8c\x8d\x8e\x8f\x90\x91\x92\x93\x94\x95\x96\x97\x98\x99\x9a\x9b\x9c\x9d\x9e\x9f\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac\xad\xae\xaf\xb0\xb1\xb2\xb3\xb4\xb5\xb6\xb7\xb8\xb9\xba\xbb\xbc\xbd\xbe\xbf\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf\xd0\xd1\xd2\xd3\xd4\xd5\xd6\xd7\xd8\xd9\xda\xdb\xdc\xdd\xde\xdf\xe0\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xeb\xec\xed\xee\xef\xf0\xf1\xf2\xf3\xf4\xf5\xf6\xf7\xf8\xf9\xfa\xfb\xfc\xfd\xfe\xff'
Using unhexlify
'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f !"#$%&\'()*+,-./0123456789:;<=>?#ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~\x7f\x80\x81\x82\x83\x84\x85\x86\x87\x88\x89\x8a\x8b\x8c\x8d\x8e\x8f\x90\x91\x92\x93\x94\x95\x96\x97\x98\x99\x9a\x9b\x9c\x9d\x9e\x9f\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac\xad\xae\xaf\xb0\xb1\xb2\xb3\xb4\xb5\xb6\xb7\xb8\xb9\xba\xbb\xbc\xbd\xbe\xbf\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf\xd0\xd1\xd2\xd3\xd4\xd5\xd6\xd7\xd8\xd9\xda\xdb\xdc\xdd\xde\xdf\xe0\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xeb\xec\xed\xee\xef\xf0\xf1\xf2\xf3\xf4\xf5\xf6\xf7\xf8\xf9\xfa\xfb\xfc\xfd\xfe\xff'
Note that this code needs some modifications to run on Python 3 (apart from converting the print statements to print function calls) because plain Python 3 strings are Unicode strings, not byte strings.
You need to write out as binary
Try:
txtfile = open(sys.argv[1] + "_tr.jpg", "wb")
Oups, you are not doing what you expect. the bin generates a string containing the value in binary form. Let's look at what you have, if the content on the input file was :
saver is a string of hexadecimal characters in textual form something like "313233414243" for an initial string of "132ABC"
jpgcontents has same format and starts with "ffd8" and ends with "ffd9"
you then apply the magic formula bin(int(jpgcontents, scale))[2:].zfill(numbits) that
convert the hexa string to a long integer
convert the long integer to a binary representation string - this part would convert hexa "ff" in integer 255 and end in the string "0b11111111"
remove first characters "0b" and fill the end of buffer if needed
bitcontents is then a string starting with "11111111....". Just rename your file with a .txt extension and open it with a text editor, you will see that it is a large file containing only ASCII characters 0 and 1.
As the header is "ffd8" the file will start with 10 "1". So the error that it starts with 0x31 0x31 because 0x31 is the ascii code of "1".
What you need is convert the hexa string jpgcontents in a binary byte array.
fileimage = ''.join([ jpgcontent[i:i+2] for i in range(0, len(jpgcontent), 2]
You can then safely copy the fileimage buffer to a binary file:
file = open(sys.argv[1] + "_tr.jpg", "wb")
file.write(fileimage)
The easiest method is using the binascii module: https://docs.python.org/2/library/binascii.html.
import binascii
# code in ascii format contained in a list
code = ['00', '01', '02', '03', '04', '05', '06', '07', '08', '09']
bfile = open('bfile.bin', 'w')
for c in code:
# convert the ascii to binary and write it to the file
bfile.write(binascii.unhexlify(c))
bfile.close()
In python, I am looking for python code which I can use to create random data matching any regex. For example, if the regex is
\d{1,100}
I want to have a list of random numbers with a random length between 1 and 100 (equally distributed)
There are some 'regex inverters' available (see here) which compute ALL possible matches, which is not what I want, and which is extremely impracticable. The example above, for example, has more then 10^100 possible matches, which never can be stored in a list. I just need a function to return a match by random.
Maybe there is a package already available which can be used to accomplish this? I need a function that creates a matching string for ANY regex, not just the given one or some other, but maybe 100 different regex. I just cannot code them myself, I want the function extract the pattern to return me a matching string.
If the expressions you match do not have any "advanced" features, like look-ahead or look-behind, then you can parse it yourself and build a proper generator
Treat each part of the regex as a function returning something (e.g., between 1 and 100 digits) and glue them together at the top:
import random
from string import digits, uppercase, letters
def joiner(*items):
# actually should return lambda as the other functions
return ''.join(item() for item in items)
def roll(item, n1, n2=None):
n2 = n2 or n1
return lambda: ''.join(item() for _ in xrange(random.randint(n1, n2)))
def rand(collection):
return lambda: random.choice(collection)
# this is a generator for /\d{1,10}:[A-Z]{5}/
print joiner(roll(rand(digits), 1, 10),
rand(':'),
roll(rand(uppercase), 5))
# [A-C]{2}\d{2,20}#\w{10,1000}
print joiner(roll(rand('ABC'), 2),
roll(rand(digits), 2, 20),
rand('#'),
roll(rand(letters), 10, 1000))
Parsing the regex would be another question. So this solution is not universal, but maybe it's sufficient
Two Python libraries can do this: sre-yield and Hypothesis.
sre-yield
sre-yeld will generate all values matching a given regular expression. It uses SRE, Python's default regular expression engine.
For example,
import sre_yield
list(sre_yield.AllStrings('[a-z]oo$'))
['aoo', 'boo', 'coo', 'doo', 'eoo', 'foo', 'goo', 'hoo', 'ioo', 'joo', 'koo', 'loo', 'moo', 'noo', 'ooo', 'poo', 'qoo', 'roo', 'soo', 'too', 'uoo', 'voo', 'woo', 'xoo', 'yoo', 'zoo']
For decimal numbers,
list(sre_yield.AllStrings('\d{1,2}'))
['0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '00', '01', '02', '03', '04', '05', '06', '07', '08', '09', '10', '11', '12', '13', '14', '15', '16', '17', '18', '19', '20', '21', '22', '23', '24', '25', '26', '27', '28', '29', '30', '31', '32', '33', '34', '35', '36', '37', '38', '39', '40', '41', '42', '43', '44', '45', '46', '47', '48', '49', '50', '51', '52', '53', '54', '55', '56', '57', '58', '59', '60', '61', '62', '63', '64', '65', '66', '67', '68', '69', '70', '71', '72', '73', '74', '75', '76', '77', '78', '79', '80', '81', '82', '83', '84', '85', '86', '87', '88', '89', '90', '91', '92', '93', '94', '95', '96', '97', '98', '99']
Hypothesis
The unit test library Hypothesis will generate random matching examples. It is also built using SRE.
import hypothesis
g=hypothesis.strategies.from_regex(r'^[A-Z][a-z]$')
g.example()
with output such as:
'Gssov', 'Lmsud', 'Ixnoy'
For decimal numbers
d=hypothesis.strategies.from_regex(r'^[0-9]{1,2}$')
will output one or two digit decimal numbers: 65, 7, 67 although not evenly distributed. Using \d yielded unprintable strings.
Note: use begin and end anchors to prevent extraneous characters.
From this answer
You could try using python to call this perl module:
https://metacpan.org/module/String::Random
I need to generate a list of numbers in a specific format. The format is
mylist = [00,01,02,03,04,05,06,07,08,09,10,11,12,13,14,15]
#Numbers between 0-9 are preceded by a zero.
I know how to generate a normal list of numbers using range
>>> for i in range(0,16):
... print i
So, is there any built-in way in python to generate a list of numbers in the specified format.
Python string formatting allows you to specify a precision:
Precision (optional), given as a '.' (dot) followed by the precision.
In this case, you can use it with a value of 2 to get what you want:
>>> ["%.2d" % i for i in range(16)]
['00', '01', '02', '03', '04', '05', '06', '07', '08', '09', '10', '11', '12',
'13', '14', '15']
You could also use the zfill function:
>>> str(3).zfill(2)
'03'
or the string format function:
>>> "{0:02d}".format(3)
'03'
One liner in Python 3.x
[f'{i:>02}' for i in range(1, 16)]
Output:
['01', '02', '03', '04', '05', '06', '07', '08', '09', '10', '11', '12', '13', '14', '15']
in oracle write this query to generate 01 02 03 series
select lpad(rownum+1, decode(length(rownum),1,2, length(rownum)),0) from employees