Develop Reference

Compilation errors in the files included in the PyYAML-3.10 package while calling the module yaml from a python file

I am using the PyYAML-3.10 as part of a Python program on macOS 10, using Python version 2.7.10. I am not able to make sense of these compilation errors. Since PyYAML-3.10 is a stable version of PyYAML, it should give no compilation errors. The errors are listed below. Any suggestions would be appreciated.
File "pyR#TE.py", line 3, in <module>
import yaml
File "/Users/PyR#TE/pyrate-1.0.0/yaml/__init__.py", line 8, in <module>
from .loader import *
File "/Users/PyR#TE/pyrate-1.0.0/yaml/loader.py", line 4, in <module>
from .reader import *
File "/Users/PyR#TE/pyrate-1.0.0/yaml/reader.py", line 45, in <module>
class Reader(object):
File "/Users/PyR#TE/pyrate-1.0.0/yaml/reader.py", line 137, in Reader
NON_PRINTABLE = re.compile('[^\x09\x0A\x0D\x20-\x7E\x85\xA0-\uD7FF\uE000-\uFFFD]')
raise error, v # invalid expression
sre_constants.error: bad character range

It seems that PyYAMP-3.10 is not compatible with Python 2. (Did you mean "PyYAML", by the way? I could not find a reference to a "PyYAMP" package anywhere.) The compilation error you are seeing is from re.compile - when Python is trying to compile a regular expression.
I tried using the line in your error message containing re.compile in Python 2 and Python 3.
Python 2:
>>> import re
>>> re.compile('[^\x09\x0A\x0D\x20-\x7E\x85\xA0-\uD7FF\uE000-\uFFFD]')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/re.py", line 194, in compile
return _compile(pattern, flags)
File "/usr/lib/python2.7/re.py", line 251, in _compile
raise error, v # invalid expression
sre_constants.error: bad character range
>>>
Python 3:
>>> import re
>>> re.compile('[^\x09\x0A\x0D\x20-\x7E\x85\xA0-\uD7FF\uE000-\uFFFD]')
re.compile('[^\t\n\r -~\x85\xa0-\ud7ff\ue000-�]')
>>>
So your options are either to find a package that supports Python 2, or to upgrade your code to Python 3. I recommend upgrading, as Python 2 is no longer supported.

Convert PDF to Image using Python

I am trying to convert a pdf file to image file for this in my ubuntu server i have installed:
python2.7
poppler-utils
pdf2image==1.12.1
My code:
from pdf2image import convert_from_path, convert_from_bytes
images = convert_from_path("/home/user/pdf_file.pdf")
# OR
with open("/home/user/pdf_file.pdf") as pdf:
images = convert_from_bytes(pdf.read())
OUTPUT
When I am using the function "convert_from_path"
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/pdf2image/pdf2image.py", line 143, in convert_from_path
thread_output_file = next(output_file)
TypeError: ThreadSafeGenerator object is not an iterator
When I am using the function "convert_from_bytes"
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "/usr/local/lib/python2.7/dist-packages/pdf2image/pdf2image.py", line 268, in convert_from_bytes
paths_only=paths_only,
File "/usr/local/lib/python2.7/dist-packages/pdf2image/pdf2image.py", line 143, in convert_from_path
thread_output_file = next(output_file)
TypeError: ThreadSafeGenerator object is not an iterator
I have reinstalled all my utilities then i am facing these problems.

If you want to convert PDF to image you can try Python Ghostscript package:
pip install ghostscript
import ghostscript
import locale
def pdf2jpeg(pdf_input_path, jpeg_output_path):
args = ["pef2jpeg", # actual value doesn't matter
"-dNOPAUSE",
"-sDEVICE=jpeg",
"-r144",
"-sOutputFile=" + jpeg_output_path,
pdf_input_path]
encoding = locale.getpreferredencoding()
args = [a.encode(encoding) for a in args]
ghostscript.Ghostscript(*args)
pdf2jpeg(
"...Fixate/ActiveState/pdf/a.pdf",
"...Fixate/ActiveState/pdf/a.jpeg",
)

I failed in python2 too, but succeeded in python3.
There's a same issue happened on an other library:
TypeError: 'threadsafe_iter' object is not an iterator
As they said, it's a python 2 vs 3 issue, caused by next() function.
If modify __next__() -> next() in file/home/***/.local/lib/python2.7/site-packages/pdf2image/generators.py , it will run successful in py2.
BTW, i have create a new issue to pdf2image team.
TypeError: ThreadSafeGenerator object is not an iterator #133
Additional
pdf2image readme said it's a python (3.5+) module.
pdf2image v1.7.1 work on py27. try it by pip install pdf2image==1.7.1

Invalid syntax while running test on Travis

I'm having a problem with Travis on every commit. My tests work on local but on Travis I get this error:
Traceback (most recent call last):
File "/opt/python/3.2.5/lib/python3.2/unittest/case.py", line 370, in _executeTestPart
function()
File "/opt/python/3.2.5/lib/python3.2/unittest/loader.py", line 32, in testFailure
raise exception
ImportError: Failed to import test module: test.test_parser
Traceback (most recent call last):
File "/opt/python/3.2.5/lib/python3.2/unittest/loader.py", line 261, in _find_tests
module = self._get_module_from_name(name)
File "/opt/python/3.2.5/lib/python3.2/unittest/loader.py", line 239, in _get_module_from_name
__import__(name)
File "/home/travis/build/davidmogar/genderator/test/test_parser.py", line 5, in <module>
import genderator
File "/home/travis/build/davidmogar/genderator/genderator/__init__.py", line 3, in <module>
from genderator.parser import Parser
File "/home/travis/build/davidmogar/genderator/genderator/parser.py", line 5, in <module>
from .utils import Normalizer
File "/home/travis/build/davidmogar/genderator/genderator/utils.py", line 63
u'\N{COMBINING TILDE}'
^
SyntaxError: invalid syntax
Here is the code where that line is:
def remove_accent_marks(text):
good_accents = {
u'\N{COMBINING TILDE}',
u'\N{COMBINING CEDILLA}'
}
return ''.join(c for c in unicodedata.normalize('NFKD', text)
if unicodedata.category(c) != 'Mn' or c in good_accents)
I have no idea about what is the problem because as I've said, all test are working in local. Here is my .travis.yml file:
language: python
python:
- "3.2"
- "3.3"
- "3.4"
script: python -m unittest discover
Any idea?

The u'...' syntax in Python 3 is only supported in Python 3.3 and up.
The u prefix is only there to support polyglot Python code (supporting both 2 and 3), and can be safely removed if you don't need to support Python 2.
If you need to support both Python 2 and 3.2, you'll have to use a different approach. You could use a from __future__ import to make all string literals in Python 2 produce unicode string objects; this applies per module:
from __future__ import unicode_literals
def remove_accent_marks(text):
good_accents = {
'\N{COMBINING TILDE}',
'\N{COMBINING CEDILLA}'
}
The strings will be treated as Unicode in both Python 2 and 3.
Or you could create your own polyglot function:
import sys
if sys.version_info[0] < 3:
u = lambda s: unicode(s.replace(r'\\', r'\\\\'), "unicode_escape")
else:
u = lambda s: s
and use that on all your Unicode strings:
def remove_accent_marks(text):
good_accents = {
u('\N{COMBINING TILDE}'),
u('\N{COMBINING CEDILLA}')
}
or you can use the six library to produce that bridge for you:
import six
def remove_accent_marks(text):
good_accents = {
six.u('\N{COMBINING TILDE}'),
six.u('\N{COMBINING CEDILLA}')
}
You may want to read the Python Porting HOWTO.

How to convert hg19.bit file to hg19.fa file using twobitreader?

I have tried this code in pyhthon shell prompt using windows. but I am getting the error as follows
>>> python -m twobitreader hg19.2bit < example.bed
SyntaxError: invalid syntax
I have also tried with the code you have sent
import twobitreader
with open("fas.fa", "w") as output, open('example.bed') as bedfile:
twobit = twobitreader.TwoBitFile('hg19.2bit')
twobitreader.twobit_reader(twobit, input_stream=bedfile, write=output)
When I try to execute the above code I am getting error as
Traceback (most recent call last):
File "D:/genome/sample6.py", line 3, in <module>
with open("fas.fa", "w") as output, open('example.bed') as bedfile:
IOError: [Errno 2] No such file or directory: 'example.bed'`Filed:
I unable to trace the error exactly.

You are trying to execute a shell command in the Python interpreter. The interpreter is not a shell.
Inside the shell you can still execute the same code the command line code would. Both a 2bit file and the input stream is be required for twobit_reader() to do anything. The first line of the function reads:
if input_stream is None: return
The library takes the hg19.2bit file as a TwoBitFile object; the input_stream argument must be a file or other iterable in using the BED format, according to the documentation string for the command line tool:
Regions should be given in BED format on stdin
chrom start(0-based) end(0-based, not included)
To use a BED file of regions, do
python -m twobitreader example.2bit < example.bed
From a Python script, the example.bed input should be an open file passed in as input_stream:
import twobitreader
with open("fas.fa", "w") as output, open('example.bed') as bedfile:
twobit = twobitreader.TwoBitFile('hg19.2bit')
twobitreader.twobit_reader(twobit, input_stream=bedfile, write=output)
The project documentation provides a link for the BED format: http://genome.ucsc.edu/FAQ/FAQformat#format1

PyPDF2 TypeError when trying to run example from lib

I've got PyPDF2 lib from here:
https://github.com/mstamy2/PyPDF2/tree/Python3-3
When trying to run script "Example 1:" from from there see it:
PyPDF2 python versions (2.5 - 3.3) compatibility branch
Traceback (most recent call last):
File "1.py", line 6, in <module>
input1 = PdfFileReader(open("document1.pdf", "rb"))
File "C:\Python33\lib\site-packages\PyPDF2\pdf.py", line 595, in __init__
self.read(stream)
File "C:\Python33\lib\site-packages\PyPDF2\pdf.py", line 1097, in read
streamData = StringIO(xrefstream.getData())
TypeError: initial_value must be str or None, not bytes
What is wrong?

It was a problem related to the compatibility within PyPDF2 and Python 3.
In my case, I have solved it by replacing pdf.py and utils.py with the ones you will find here, where they basically control if you are running Python 3 and, in case you are, receive data as bytes instead of strings.