I have been trying to install Scraperwiki module for Python. However, it generates the error:
""UserWarning: Local Scraperlibs requires pdftohtml, but pdftohtml was not found in the PATH. You probably need to install it".
I looked into poppler as they have pdftohtml file but I don't know how it works - whether there is a python library I need to install or a .exe file. And how do I go about installing it. Running on Windows.
Many Thanks
If you're not intending to use scraperwiki.pdftoxml(), then the warning doesn't apply. It doesn't stop you from installing the scraperwiki package, however.
Also, that function doesn't work on Windows at all as is; it uses NamedTemporaryFiles which behave differently on Windows to Linux.
If you do want to use that function, the simplest way to get an up-to-date version of pdftohtml on Windows is to download Calibre Portable. (The version on Sourceforge is older.)
Install it anywhere; you just need a few files from it. From where you installed it, from the folder containing calibre.exe, you need pdftohtml.exe into your working folder as well as, from the DLLs folder in the Calibre install, freetype.dll, jpeg.dll, libpng12.dll, zlib1.dll.
You'll also need code based on scraperwiki.pdftoxml() instead, like:
def pdftoxml(pdfdata, options):
"""converts pdf file to xml file"""
# lots of hacky Windows fixes c.f. original
with open('input.pdf', 'wb') as f:
f.write(pdfdata)
cmd = 'pdftohtml -xml -nodrm -zoom 1.5 -enc UTF-8 -noframes '
if options:
cmd += options
cmd += 'input.pdf output.xml'
cmd = cmd + " > NUL 2>&1"
os.system(cmd)
with open('output.xml', 'r') as f:
return f.read()
(I was trying to get this working for a user in Windows recently; I'll keep the gist containing this code updated.)
Related
I am practicing some API work in python 3.7 using API star 0.5.X and my python script can't find the .json file that is in the same folder as the python file. I am working on and running the script with Atom editor and I am working in a venv, which is fairly new to me.
I am using a helper function to load in the JSON data using a "with open()" statement. I have tried using the relative and absolute file paths, and in both instances it is unable to locate the file. I have tried launching the file in Atom using terminal and the MacOS finder.
This is what I have so far:
import json
from typing import List
import os
from apistar import App, Route, types, validators
from apistar.http import JSONResponse
print(os.getcwd())
os.chdir('/Users/{myusernamehere}/100days/apistar')
print(os.getcwd())
#helpers
def _load_employee_data():
with open('employees.json') as f:
employees = json.loads(f.read())
return employees
the second print statement prints the correct file path, being the one that 'employees.json' and 'app.py' are located in.
Since the problem is specific to your setup, it's hard to reproduce or provide a solution in code. Your code itself looks to be fine, but there are two things that are likely to be the cause of your issues:
When your script is running, Python needs access to the appropriate source folders and installed packages; you should let something like virtualenv manage this through a virtual environment. From the terminal, you can load the appropriate virtual environment with:/path/to/your/venv/Scripts/activate.sh
If you do, you should expect your script to find the same libraries it did during development in that virtual environment. Make sure you include something like a requirements.txt in your project to allow easy reinstalling of the same modules on a different machine, in a new virtual environment.
Your script, when run by Python, has a 'working directory'. This is the directory that Python is started from and your script not being able to find the file (even though it may be in the same folder as the script itself) is probably due to Python being started from a different directory.
This was a problem due to how the Atom editor works. It was solved by switching to vim.
I only partially understand but apparently this had something to do with Atom having a separate temp directory for working files, or something of that nature. When using vim to edit the script, and then calling it in the terminal the problem was resolved.
Okay, So i had the same issue with i.c.w. VScode :
file = open('file.txt')
print(file.name)
resulted in
FileNotFoundError
file.txt was 100% in the same folder... According to finder on my Mac, ánd the folder column in VS code!
i was pulling out my hair. Switched a lot of interpreters, local python and Conda, to Python 3.8 instead of 3.9, back to python 2.8.
Nothing mattered.
Till I changed :
file = open('file.txt') to: file = open('file.txt', 'a')
It didn't suddenly work, but I saw immediately in the "folder column" of VScode a new file.txt file popping up. In an entirely different folder then where the pythonfile.py was located. So after that; I pushed all local repo's to their remotes; deleted the whole caboodle, and installed them one by one in a new folder through git clone. I opened a new workspace added those project folders and since then it works like a charm.
What exactly went wrong ; im sorry, I have no idea. But for me, a fresh install of the repo's and VScode workspace is what did the trick.
I recently had the same error, on Visual Studio Code, I managed to solve it by instead of clicking the Run Python button, I used the terminal to cd into the project directory and run the python script like that, and no problems!
I am packaging python using pyinstaller 2.1 on OSX Mavericks. I have done this successfully in the past, but this is my first package that uses moviepy and ffmpeg. I use the following import:
from moviepy.video.io import ffmpeg_reader
Without this line in the code, everything works fine and I can launch my final package from its icon in finder. With the moviepy import, it will work if I launch from the terminal like this:
open ./myapp.app
but it will not open if I click on the icon from finder (opens quickly and crashes). I am assuming this has something to do with paths or environment variables that are set in terminal, but are not transferred to my packaged app. I have tried various hidden imports in pyinstaller for moviepy and its dependencies, but nothing seems to work. --debug mode hasn't provided much info to track it down. Any other ideas?
Thanks!
There are a few problems with moviepy and pyinstaller.
First, try writing the error to a text file
try:
from moviepy.video.io import ffmpeg_reader
except Exception as e:
with open('/absolute/path/to/error.txt',mode="w+") as f:
f.write(str(e))
1) You might need to modify these two files to remove the "exec" import statements
moviepy/audio/fx/all/__init__.py,
moviepy/video/fx/all/__init__.py
see here:
https://github.com/pratikone/videoVenom/blob/master/moviepy/audio/fx/all/__init__.py
https://github.com/pratikone/videoVenom/blob/master/moviepy/video/fx/all/__init__.py
2) You might need to have this statement imageio.plugins.ffmpeg.download(), so that ffmpeg is downloaded if not found.
I have a program written in python 3.3 that I'd like to be able to distribute without the need for users to install python or any additional modules. I was able to successfully package this program using cx_Freeze on Windows, but the same script on OS X produced an app that wouldn't launch.
I thought I might have better luck using py2app, but now I'm running into a strange problem. The program opens (it has a GUI built with tkinter) and runs flawlessly when built in Alias mode. When I attempt to construct a final build, however, I get the following message in Terminal:
error: No such file or directory: /Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/setuptools-2.1-py3.3.egg/_markerlib/__init__.pyc
I navigated to that directory and found a .egg file that I'm unable to open or extract. I've tried reinstalling setup tools and well as python itself with no luck. Has anyone experienced this problem?
It looks like the problem is that your setuptools is somehow broken.
To open the egg file, I tried downloading a third party tool, which crashed, and renaming it as a .zip, which failed. If I just double click on it, I get the "choose default application" popup.
Double-clicking it relies on the extension to decide what app to launch.
The best way to check whether something is a valid zip file is to use the unzip tool from the command line. For example:
$ unzip -t setuptools.egg
This will check all of the zip headers, and check the CRC of all files in the archive, and report any errors. Or, if it's not a zip at all, it'll report one error right at the start.
You can also use the file command to do a quick check to see whether it's some well-known type of file. If file /path/to/setuptools-whatever.egg just says "data" instead of "Zip archive data", then it's probably corrupted beyond recognition.
Anyway, assuming your setuptools didn't come with your Python installation (if you're using a python.org binary installer, it didn't), the safest thing to do is uninstall it, then reinstall it cleanly.
The reason it's important to uninstall first is that the current version will, by default, not install a .egg archive, but will instead install a normal unzipped package and egg-info directory, meaning it may not overwrite the old, broken copy.
The documentation covers uninstalling. Just delete the setuptools .egg file, and anything else named setuptools*, from your site-packages (and anywhere else on your sys.path). If you have distribute there as well, kill that too. This will leave a few files sitting around in other places (notably easy_install-3.3 somewhere on your PATH), but they'll get overwritten properly by the installation, so that's OK.
To install, just follow the usual instructions to reinstall it:
$ wget https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py -O - | python
… or, if you don't have write access to site-packages:
$ wget https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py -O - | sudo python
If you use pip, you may want to reinstall it after reinstalling setuptools, and then pip install -U setuptools pip just to make sure you have the latest versions—and to verify that everything is now working.
Distutils offers existing solutions to build a Windows self-extracting EXE. Notably, I can create the package even if I'm on Ubuntu (which is essential given our automated build process).
How do I build an osx installer from an Ubuntu machine?
(This is for python 2.7)
Py2app, cx_freeze, and py2exe are for creating standalone applications, not installers.
You can use Python distutils to create a zip file of your package, and then wrap the zip file in a script that self-extracts and installs the package, like this: http://www.noah.org/wiki/Self-extracting_Python_Script
Or, if you want to create an installer that displays a GUI with a click-through license, then you need OS X PackageMaker to create an mpkg file. I don't think there's a Linux equivalent. If you want to try building an mpkg by hand, try this answer: PackageMaker for creating Mac packages on Windows/Linux
You could do the following, since OS X has a UNIX foundation. (I'm assuming it also has some standard utilities like uudecode and unzip.);
Create a zipfile for your package with the standard Python packaging tools, like distutils.
Use uuencode file.zip file.zip >uu.txt to convert this zip-file to text.
Create a shell-script.
Put the text generated by uuencode as a here-document in that shell-script, that is fed to uudecode, to recreate the zipfile. This should be the second command in the shell-script. The first commands should check for neccesary utilities.
Use whatever Python setup tools you have to install the zipfile.
Do any post-install stuff you want.
Remove the zip-file.
To install the program, run the shell-script. On UNIX, this script should be run as root. Don't know how OS X handles that, though. Below is an example (the included zipfile contains two files with only random noise) ;
#!/bin/sh
# Install script for foo
# Check for special programs that are used in this script.
PROGS="uudecode unzip python"
for P in $PROGS; do
which $P >/dev/null 2>&1
if [ $? -ne 0 ]; then
echo "$(basename $0): The program \"$P\" cannot be found."
exit 1
fi
done
uudecode <<"EOF"
begin 644 test.zip
M4$L#!`H``````(<!K$`````````````````$`!P`9F]O+U54"0`#OHZM3ZB.
MK4]U>`L``03I`P``!.D#``!02P,$"#``````A`&L0$EXM>H``#````(```<`
M'`!F;V\O8F%R550)``.XCJU/N(ZM3W5X"P`!!.D#```$Z0,``)DCNBVSM81_
MA%IQ!+LKC\;\19[/E]F!`J2J*2:9B_8#MC^KM*]]9P1]?1Y1#1+/H%Q"'2J'
M,;$7:C9E+WC]%M(BAUFN]4D\$%F63!^CA8O>'0C(-!YD?$\GQ[;M:?#>=A/[
M61XK<A/]\.?!0$QX<5]T\9<ZP$;_?PRMV-3O(NK)/<MTI,!RYA&OWRK6<8%4
M_1!T/+-'7H^V#C[AK)+U?T>UCU."G]D(+JU[8Z#1EI89#'^10B:8.2$$9Q*#
MY:L4MK'^TVF)A5)9\%"/FF+1T#;]8\,I)$CCHHF#E&,D.GVM1=2N];4J?6WR
M0+YH]DXZ"UQ$GVA^F(`5L![=/<ROX]9RQCRWJ=?+G4Y>56[H=8:!:GTA_V;V
M2V$%U5([0D;T19H]P7+^448+^&M3;[/VJDEJ-SU=Q8U=3,IV^<)A>C)]?#XG
M?-9$B#YZGML`!&`\-CP["]B'FC&K7Z)T6K_&W1K5?M&K8D&^'C^J;H[Q4/ST
M(>QL].68#X)_0#`?9<R3#:4Y#A'X-<NT9C\OM3:4[<)`)LEO#E=*/=0U#]VY
M_R!__:Q'_FP3((*8^6JQ"$_H&BIWDD.6<"3D,'<1^=^.9F^2Z7.:OE"1,SV[
M)M(9!&V(4):?M7^JFR^P"!H4U+(*A;U#Z0QA+]5ZIS]B1;K)&#LJ-Y,L9SQ.
M^Q-"&^##KQG94$L#!`H``````(8!K$#%*"OR``(````"```'`!P`9F]O+V)A
M>E54"0`#O(ZM3[R.K4]U>`L``03I`P``!.D#``!NFMNGLA(C_K!_F=T%?;P3
M#TUC7%4F+!Y#A8<SFK[;E3M.%J`YT>1AVMNJPFOE58Q["7<#AWQFZ!SG_-RW
M*4`##P.7\>+LGTBNTLH-7)CB(LJDMD)COV5'H]O8G_4I&C:PFTVC%4P=+X*B
M%A^I^$>BO+<!D\&8&GS:&VLQHJJ[!NFW0H5FD6+:'OUT2--U1HXQ2R?JF2,9
M(/#>A(/SU#.78MTFADG`ZZTK!6:Z6\;I`2?D6I;FE=_0V?4>_0MC;$0$P?H%
MDHQ]MCK4;,5W=<IZV<)<`7A_B7<_=U,:X[8/$_&/518\NNBDYS,\<',K2C]9
M4M.5UL/R<\'0E>G#$>`I>/[UX$QM.+T\LF4;D^WF6FX3.(L?2V<5B%5)$!5[
ME47K#7\&D*3Y>I)8#45-HL(!F7+$)%7C9,$_YYL]EG='3BN9W[&9!$.<.U?E
M#0L?=6#%J/32)NAMI48M",_)0#JRR!*Y2P:ZE#JWD)5/#UK!!3]*2M47V1GE
M'X0(FN%_*3BX_X'(6X!ONLKN!U/&_ML-L:^FD/24Q)S*-D8)Z>F4Y^+$]\_'
MB\$$;#D__S_RT(Y]MMK?B#%F1&C`>,)'7:12DX=F)T,/!*^(M*[,^N;6E4";
M31K\AG0#[#4L9MB.`1Z!`%#K5G)P<]0!?P\$RFUC/S:Y_Y\0*:\+$U+JEM%"
M9E!+`0(>`PH``````(<!K$`````````````````$`!#`````````$`#M00``
M``!F;V\O550%``.^CJU/=7#+``$$Z0,```3I`P``4$L!`AX#"#``````A`&L
M0$EXM>H``#````(```<`&````````````*2!/#```&9O;R]B87)55`4``[B.
MK4]U>`L``03I`P``!.D#``!02P$"'#,*``````"&`:Q`Q2#K\#`"`````#``
M!P`8````````````I(%_`#``9F]O+V)A>E54!0`#O(ZM3W5X"P`!!.D#```$
:Z0,``%!+!08``````P`#`.0```#`!```````
`
end
EOF
# Unpack your zipfile
unzip test.zip
# Go into the created subdirectory and install
cd foo
python setup.py install
# whatever post-install stuff you want goes here...
cd ..
rm -rf foo test.zip
py2app is the Mac counterpart to py2exe, but I don’t know if you can run it from Ubuntu.
pyinstaller can create OSX executables, but I've only used it on OSX myself, so I don't know if it works from Ubuntu.
I learned about this program reading the documentation for kivy, here: http://kivy.org/docs/guide/packaging-macosx.html
I hope some of this is helpful.
You have similar trouble that I suffered with.
On mac. I bet you want use dmg package instead of Installer unless you really need to install driver or some sort of DRM.
So I'll just assume that you want make your .app directory and packaging it as dmg file so that you can distribute your application easily.
To build a .app directory out of python code, you might wanna use Pyinstaller.
Pros
It's Cross Platform.
Works good on mac, windows, linux
You can control everything on command line.
It's still actively developing.
It packs required dependencies automatically and doesn't create conflict with installed Python on local machine.
Good Documentation.
Cons
bit of inconsistency between new version and old version.
To build .app using Pyinstaller, here's related document link
and to build dmg file, you can use free options out there, but for me, DMG Canvas was pretty useful.
Pros
support command line
support template (Create pretty background and folders using GUI and reuse it on command line)
support license agreement
Cons
Not Free
As far as I know, there's no unified solution from .py to build .app file and to dmg or installer.
So you should write some code to automatize it, but it's pretty straight forward, since both of them support
Command line.
So you can use distutils to create a file, such as
PIL-1.1.6.win32-py2.5.exe
which you can run and use to easily install something. However, the installation requires user input to proceed (you have to click 'OK' three times). I want to create an easily installable windows version that you can just run as a cmd line program, that doesn't require input from the user. Is this possible? Do these .exe files do it already, but you need to pass them a magic cmd line argument to work?
See this post which describes an idea to modify the stub installer like this:
It also mentions another alternative: use setup.py bdist_msi instead, which will produce an msi package, that can be installed unattended
You get the executable by running "setup.py bdist_wininst". You can have something simpler by running "setup.py bdist_dumb". This will produce a .zip file which, unzipped at the root of the drive where Python is installed, provided that it's installed in the same directory as the machine you've build it, will install the library.
Now I don't know if there is an unzip command-line utility under Windows that can be used to do that; I usually have Cygwin installed on all my Windows boxes, but it may prove quite simple to just ship it with the .zip.
I have done this before using a simple batch file to call setuptools' install script passing the egg file path as an argument to it. The only trouble is that you need to ensure that script is in the PATH, which it might not be.
Assuming Python itself is in the PATH you can try something like this in a python script you would distribute with your egg (call it install.py or somesuch).
import sys
from pkg_resources import load_entry_point
def install_egg(egg_file_path):
sys.argv[1] = egg_file_path
easy_install = load_entry_point(
'setuptools==0.6c9',
'console_scripts',
'easy_install'
)
easy_install()
if __name__ == "__main__":
install_egg("PIL-1.1.6.win32-py2.5.egg")
Essentially this does the same as the "easy_install.py" script. You're locating the entrypoint for that script, and setting up sys.argv so that the first argument points at your egg file. It should do the rest for you.
HTH