I'm trying to install the icalendar Python module for use in my Google App Engine project. I have created a Python virtual environment in my project folder, and installed the icalendar package in it. If I run the Python interpreter from the venv, I can import icalendar and use it just fine (help(icalendar) shows it's getting the module from the correct path: venv/lib/python2.7/site-packages/icalendar). I have created a symlink called icalendar pointing to that directory in my project root (same place as app.yaml). The project can deploy and run on GAE, but anytime it tries to use icalendar, I get "Cannot import module 'icalendar'".
I used the exact same steps to install the Twilio library in my app, which worked fine. If I don't include the symlink to the Twilio module, the error message is actually 'No module named twilio', instead of "Cannot import..". Not sure if that's relevant.
I've seen this: https://groups.google.com/forum/?fromgroups=#!topic/google-appengine/FM_NXd9cbus, which is exactly my problem, but there was no solution!
What am I doing wrong?
Edit: I also tried just putting the icalendar module directory in the project root, w/o the symlink. No difference.
You need to check how to use python libraries in GAE.
Pay attention as the docs are a bit confusing, you need to pick one (and only one) solution for a library, you can't mix both (they're both chapters on the above-mentioned page):
Requesting a library - for libs supplied by GAE - for these check notes for what you may need to do for your local development server
Installing a library - for additional libraries you need to include in your app - for these check also the restrictions the library needs to meet
Related
I'm using PythonKit with XCode to call spaCy API from swift code. Installed PythonKit from Home-brew, imported it in my project and it built well; however, at runtime I got the error:
Python library not found. Set the PYTHON_LIBRARY environment variable with the path to a Python library.
So I compiled PythonTools (within PythonKit dir, run swift package generate-xcodeproj and open PythonKit.xcodeproj file ) to see which paths of Python library it would find:
/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python37.zip
/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7
/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload
/Users/fernandabrum/Library/Python/3.7/lib/python/site-packages
/usr/local/lib/python3.7/site-packages
If Python libraries are in my system, why I get an error of Python library not found? What am I doing wrong?
OBS.:I also tried to set PYTHON_LIBRARY with export PYTHON_LIBRARY= and the paths found above, but unsuccessful.
Thanks in advance!
In "Hardened Runtime" under Signing and Capabilities: check "Disable Library Validation"
See the answer here: I can't import Python modules in Xcode 11 using PythonKit
I finally solved this problem, in my case, by setting App Sandbox to NO in my app's .entitlements file. I also have Hardened Runtime and Enable App Sandbox set to NO in build settings, but those settings don't work without modifying the entitlements file... at least not in my case.
My understanding is that I can't distribute an app this way, so maybe kinda useless for some, but this is just a simple dev tool for testing some python code that will eventually be called from Vapor, so not a huge deal for me.
I have same issue. I try to use PythonLibrary.useVersion(3) and get same error as yours.
It looks like the python library does not support iOS yet. It is working fine on OS X.
I'm trying to set up a roguelike Python project, but I can't seem to be able to import libtcod module into my project. This helloworld crashes, and the IDE keeps telling me that there is no module named libtcodpy.
import libtcodpy
def main():
print('Hello World!')
if __name__ == '__main__':
main()
What is the proper way to import modules into Python projects? I'm used to Java, so I was expecting something along the lines of Maven to manage the dependencies. There indeed seems to be something like that in PyCharm as well, this package manager for the venv, which from what I gather serves to isolate the project-specific stuff from the OS- or python-global stuff:
but libtcod simply isn't present in the rather exhaustive list of modules that appears after clicking on the "+" button, just some other module that has something to do with libtcod library (I guess?). Moreover, all the tutorials I found on setting libtcod up advise one to manually copy over files somewhere or run some command that I suppose does the importing somehow and other such solutions, all of which i tried and none of which worked. I don't want to pollute my project structure by using such hodgepodge ways of handling dependencies if I can at all avoid it.
Q: How do I get libtcod to work in my PyCharm project in the most clean and convention-abiding way possible?
Take a look at this github project called tcod: https://github.com/libtcod/python-tcod/blob/master/README.rst#installation
It's a python port of libtcod.
To install using pip, use the following command:
python -m pip install tcod
If you get the error "ImportError: DLL load failed: The specified module could not be found." when trying to import tcod/tdl then you may need the latest Microsoft Visual C runtime.
Blockquote
I'm trying to follow the tutorial at https://www.juliensobczak.com/tell/2016/12/26/anki-scripting.html
And I'm getting the "listcards.py" basic script set up, having cloned anki and installed the virtual environment as well as the requirements from the anki/requirements.txt file.
However, when I run the script from the tutorial entitled "listcards.py", I get a notice that the module 'anki.sched' is not found. ("ModuleNotFoundError: No module named 'anki.sched')
While I could pip install each package, I have a feeling that there must be an underlying reason that these packages are missing- is there a way to have python automatically pull in the named module even if it isn't pre-installed in the manner of how node.js installed referenced dependencies automatically, so that I won't have to manually install every missing package?
I ran into this same problem. anki.sched is a package contained within the anki repository you cloned, so it does exist on your machine. You won't be able to install it using pip.
The solution for me was to write the absolute path of the anki repository you cloned in sys.path.append rather than a relative path. For example, if your script exists in /Users/anki/scripts and your cloned anki repository exists in /Users/anki/anki write this in your script before importing anki modules:
sys.path.append("/Users/anki/anki")
rather than this (which is what is provided in the tutorial):
sys.path.append("../anki")
I'm not 100% sure why the latter fails, but Anki must be looking for the anki.sched module in the wrong place due to the relative reference.
What I did, and I know this is probably not the correct way, is to simply wipe out the root anki folder and copy all the application scripts to it, then the imports worked.
On their Built-in Third-party Libraries page, it says that lxml 3.7.3 is supported but I can't seem to find a way to use that.
If I write in app.yaml:
- name: lxml
version: latest
When I log etree.LXML_VERSION inside the app, I get (2, 3, 0, 0).
Setting the version specifically to "3.7.3" (instead of latest) doesn't seem to work. When I try to deploy, gcloud says:
ERROR: (gcloud.app.deploy) An error occurred while parsing file: [... /app.yaml]
lxml version "3.7.3" is not supported, use one of: "2.3", "2.3.5"
in "... app.yaml", line 17, column 1
I have installed lxml locally inside the ./lib folder, but the folder is ignored on deploy.
Am I doing something wrong?
The problems seemed to happen because of an old google cloud sdk.
The sdk was installed using apt-get and somehow it was not getting updated. This also included an older Python SDK
At this moment the latest Python SDK is 1.9.62, I was using 1.9.52.
Updating the sdk seemed to fix the problem
sudo apt-get update && sudo apt-get install google-cloud-sdk
It wouldn't be the 1st time that inconsistencies exist between the documentation, what's included in the cloud SDK, what's included in the GAE language specific SDK and what's actually available on GAE (in production). See, for example, PyCharm - Can't create App Engine Application using DJango.
The deployment error message you got suggests that the 3.7.3 version doesn't actually exist on GAE, despite being marked as available in the documentation.
The lxml library is on the list of a few special libraries which need extra attention. Most likely because they aren't "pure Python code with no C extensions" and thus they can't be included in the SDKs with the other libraries, so they need to be installed separately on your local system. From Using built-in bundled libraries with the local development server (check that entire section for related info):
Many of the built-in libraries provided by the runtime are
automatically available to the local development server. However, the
following built-in libraries must be installed locally before you can
use them with the local development server:
lxml
matplotlib
mysqldb
numpy
PIL
crcmod
pycrypto
Now, if you really want the 3.7.3 version, you may be out of luck: if indeed it's not "pure Python code with no C extensions" then you also won't be able to vendor it into your app either. Still, it's worth a try. A new try, note that you also need to:
take it out of the app.yaml file's libraries: section - you're not requesting a GAE-provided library anymore
complete the entire vendoring in procedure (you didn't mention creating the appengine_config.py file, for example).
If that doesn't work then you'll have to settle for one of the available versions mentioned in the deployment error messages. You'll need to:
add that version to the app.yaml file's libraries: section
install that version on your local system
drop the vendoring in attempt, if you went for it
I keep getting ImportError: No module named twitter when it's in my lib module
Details:
the twitter module is located in the lib directory.
The lib directory has an empty init.py
I'm getting the error on a page just outside the directory
When I take the twitter module out of the lib directory and just say import twitter, it works
When I put the twitter module back inside the lib and say from lib import twitter, or when I say import twitter, or saying import lib.twitter it says, go to **** and like it
I tried doing this: https://cloud.google.com/appengine/docs/python/tools/libraries27?hl=en , which suggests to:
To tell your app how to find libraries in this directory, create (or modify) a file named appengine_config.py in the root of your project, then add these lines:
from google.appengine.ext import vendor
# Add any libraries installed in the "lib" folder.
vendor.add('lib')
and then tried import twitter, and tried from lib import twitter, and it says, "haha, nice try."
Now what?
tl;dr - the comments are right. 'lib' is not a module, it's a folder containing modules. It needs to be on your Python import path before your import twitter, using the vendor.add('lib'), which should set up your import path correctly when you deploy or use dev_appserver.py. If you place the vendor.add(lib) snippet in appengine_config.py file and run dev_appserver.py or deploy, and the twitter module is inside lib, you should be able to import twitter. If you're running outside of App Engine/devappserver, you'll need to make sure lib is on the PYTHONPATH.
Here is the longer explanation of what the lib folder is; what vendoring is; and how they fit into the bigger picture of managing dependencies on App Engine.
With App Engine, you can specify some dependencies provided by the sandbox in app.yaml. You can se the list of available libraries here. For any other dependencies (which can only be pure Python libraries), the libraries need to be directly included inside your project. Directly including the code for these dependencies as part of your project (rather than just listing them in requirements.txt and expecting them to get installed at deploy time) is commonly called 'vendoring'.
Now, even though we are including dependencies directly, we'd rather just use pip and a requirements.txt, for a few reasons I will mention below. But pip is usually used to install into libraries into the system libraries or a virtualenv, not into the project itself. So a feature was added to pip and a feature was added to App Engine to fix this.
The pip feature install dependencies into a folder rather than your system libraries or a virtualenv. You use the -t flag like this:
pip install -r requirements.txt -t lib
Where 'lib' is the folder to install into. This way, you still specify your dependencies in requirements.txt, and use pip to install them, they just get installed right into the directory specified. As the comments noted, the lib folder is itself not a module - it just contains them. So your lib folder should not have an __init__.py, it should just contain a folder like 'twitter' that does have an __init__.py. Since lib is not a module, from lib import twitter doesn't actually make sense. Also note that lib is an arbitrary name, but the one we usually pick by convention.
There are a few big advantages to vendoring using pip rather than just manually downloading dependencies and adding them to the project. One advantage is you don't need to check the dependencies into source control - just add the requirements.txt, and tell other users to use the pip -t command to vendor the dependencies too. Another advantage is it's just more organized to cleanly separate your code from third-party modules. The point of the vendoring features is to keep these advantages while still following App Engine requirements to include the dependencies in the directory at deploy time.
The App Engine vendor extension which you are using was made to recognize that a folder contains modules that have been 'vendored' and add it to the path. That is the vendor extension you are using in your snippet. It makes sure the lib folder is on your import path so you can import the modules you installed into it.
Once you run the vendor.add command, you should be able to import your modules in the lib folder. As the comment notes, you need to make sure it runs before import twitter. A good practice is to make sure those vendor commands run before anything else. You can accomplish this by putting that code inside a file named appengine_config.py in your directory. That's a special file that runs before anything else. Take a look at an example of doing that here.
A few last notes that might be helpful:
appengine_config.py will run in the GAE environment or when you run dev_appserver, which emulates the GAE environment. However if you're ever running outside of a GAE environment, make sure your PYTHONPATH includes the lib folder you want to import from.
Since you can vendor libraries using pip, you might ask why would you ever include a dependency using app.yaml? The answer is that since only pure Python libraries can be vendored, you still should use app.yaml dependencies for any libraries which need C libraries, such as MySQL.
If you have vendored libraries and app.yaml libaries, you don't want to vendor libraries that will be in the GAE sandbox since the versions might conflict. In that case, it's a good idea to have a separate requirements.txt for dependencies you want to vendor, and dependencies you want included only when running locally but that GAE will provide in the sandbox. Here is an example of that for MySQL.
Create a folder called "libs" (in the home directory of your webapp)
If one library, then just do: pip install -t libs/ your_library_name
if multiple libraries, do:
pip freeze > requirements.txt,
pip install -r requirements.txt -t libs
Now, create a file called appengine_config.py (in the home directory, again)
In the above file, add:
from google.appengine.ext import vendor
vendor.add('libs')
These get uploaded onto your app engine environment and you'll be able to run it!