The solution of just removing all HTML tags is not apprioriate for my App.
From what I have seen so far, I have found two solutions to clean HTML in Python:
bleach (uses html5lib). It works perfectly well on dev server but I could not have it work on production. there is an 'ImportError: No module named html5lib' when I try to import html5lib. Just as if the folder was not there. Maybe a problem with the python path of GAE.
lxml. It is more complicated to have it work on dev server : must install two third party binaries (libxslt and libxml2) to my local Python and then pip install lxml. Then on production, once I declared the lxml library in my app.yaml, it worked just fine.
Are there better solutions than lxml?
Thanks in advance
Related
I am trying to push a flask app (Python 3.5) to Cloud Foundry (CF). The application takes a POST request (text file) and returns a message. It works locally (tested via Postman). However, when attempting to push it to CF, it gives the error -
ImportError: No module named queue
Here is my code which contains queue.
import queue as Queue
self._batch_queue = Queue.Queue(self.BATCH_QUEUE_MAX)
self._example_queue = Queue.Queue(self.BATCH_QUEUE_MAX * self._hps.batch_size)
I've tried the solutions suggested here, but neither of these solve my problem. I think the issue is with the Python in CF not having queue package. (I could be wrong).
Anyone ideas on how to go about solving this will be very appreciated. Thanks in advance!
As mentioned in the comments by #KlausD, it seems like you have the wrong version of Python installed. On Cloud Foundry, you would set the version by including a file called runtime.txt in the root of your project (i.e. the directory from which you're running cf push).
https://docs.cloudfoundry.org/buildpacks/python/index.html#runtime
That file is used to tell the Python buildpack which version of Python to install for you. Suggestions would be python-3.5.x or python-3.6.x which would install the latest 3.5 or 3.6 release. You can specify an exact version like python-3.5.5 but it's not recommended as it's easy to forget to update that file when new versions of Python come out.
You can see which versions of Python are supported by the buildpack here.
https://buildpacks.cloudfoundry.org/#/buildpacks/python/v1.6.17
(Note that link goes to the latest version of the buildpack at the time I wrote this, it will get out of date. In the future, just click the most recent version of the buildpack to see what ships with it).
I am writing a Merchant Data Export API in Python (Pydev plugin for eclipse) and I cannot import requests, is there some option for managing imports?
I have the requests-master folder in my project, downloaded from GitHub. I have read the following articles for downloadingand installing, but nothing works for me.
I am a new python user but would like to learn more.
Thanks for any help!
If you just do a pip install requests, things should work (the issue is that you probably aren't adding the proper folder to the PYTHONPATH for PyDev to recognize the requests module you downloaded... using pip should do the right thing for you without requiring further configurations).
On their Built-in Third-party Libraries page, it says that lxml 3.7.3 is supported but I can't seem to find a way to use that.
If I write in app.yaml:
- name: lxml
version: latest
When I log etree.LXML_VERSION inside the app, I get (2, 3, 0, 0).
Setting the version specifically to "3.7.3" (instead of latest) doesn't seem to work. When I try to deploy, gcloud says:
ERROR: (gcloud.app.deploy) An error occurred while parsing file: [... /app.yaml]
lxml version "3.7.3" is not supported, use one of: "2.3", "2.3.5"
in "... app.yaml", line 17, column 1
I have installed lxml locally inside the ./lib folder, but the folder is ignored on deploy.
Am I doing something wrong?
The problems seemed to happen because of an old google cloud sdk.
The sdk was installed using apt-get and somehow it was not getting updated. This also included an older Python SDK
At this moment the latest Python SDK is 1.9.62, I was using 1.9.52.
Updating the sdk seemed to fix the problem
sudo apt-get update && sudo apt-get install google-cloud-sdk
It wouldn't be the 1st time that inconsistencies exist between the documentation, what's included in the cloud SDK, what's included in the GAE language specific SDK and what's actually available on GAE (in production). See, for example, PyCharm - Can't create App Engine Application using DJango.
The deployment error message you got suggests that the 3.7.3 version doesn't actually exist on GAE, despite being marked as available in the documentation.
The lxml library is on the list of a few special libraries which need extra attention. Most likely because they aren't "pure Python code with no C extensions" and thus they can't be included in the SDKs with the other libraries, so they need to be installed separately on your local system. From Using built-in bundled libraries with the local development server (check that entire section for related info):
Many of the built-in libraries provided by the runtime are
automatically available to the local development server. However, the
following built-in libraries must be installed locally before you can
use them with the local development server:
lxml
matplotlib
mysqldb
numpy
PIL
crcmod
pycrypto
Now, if you really want the 3.7.3 version, you may be out of luck: if indeed it's not "pure Python code with no C extensions" then you also won't be able to vendor it into your app either. Still, it's worth a try. A new try, note that you also need to:
take it out of the app.yaml file's libraries: section - you're not requesting a GAE-provided library anymore
complete the entire vendoring in procedure (you didn't mention creating the appengine_config.py file, for example).
If that doesn't work then you'll have to settle for one of the available versions mentioned in the deployment error messages. You'll need to:
add that version to the app.yaml file's libraries: section
install that version on your local system
drop the vendoring in attempt, if you went for it
I'm using netbeans to write a simple python program which I need the requests module for, I've downloaded requests through terminal and it all seems to be fine there but netbeans can't seem to find it.
This is the error that it's throwing up:
import requests
ImportError: No module named requests
I've tried installing the requests library directly into the python folder but the folder won't let me paste anything into it.
There do seem to be answers on the netbeans forums but their server is down so won't let me on their website to my annoyance!
EDIT
I've tried to run python setup.py install as per other answers on the website but had no luck.
EDIT
have tried completely uninstalling python and requests to make sure it wasn't an installation error but still no luck.
This clearly looks like an error of installation of the request module to some other place than where your netbeans expects when running the code.
In your console run
which python
Check if this gives the same path as the one set in your netbeans. You can set your path by adding new platform using Tools > Python Platforms > New:
I would suggest that you learn bit more about sandboxed environments such as virtualenv. This article shows how you can use a virtualenv to install packages and use the same virtualenv for netbeans so that whatever packages you install in the virtualenv will be available in the netbeans for you to use. For this case, it could be requests.
In the end I gave up with requests, as I was using requests to get json data from an API I decided just to go back to the drawing board and start over rather than attempt to fix something that I couldn't work out. I am now using the urllib import and whilst this may not be the most efficient way it works which is the most important thing.
I'm trying to install the icalendar Python module for use in my Google App Engine project. I have created a Python virtual environment in my project folder, and installed the icalendar package in it. If I run the Python interpreter from the venv, I can import icalendar and use it just fine (help(icalendar) shows it's getting the module from the correct path: venv/lib/python2.7/site-packages/icalendar). I have created a symlink called icalendar pointing to that directory in my project root (same place as app.yaml). The project can deploy and run on GAE, but anytime it tries to use icalendar, I get "Cannot import module 'icalendar'".
I used the exact same steps to install the Twilio library in my app, which worked fine. If I don't include the symlink to the Twilio module, the error message is actually 'No module named twilio', instead of "Cannot import..". Not sure if that's relevant.
I've seen this: https://groups.google.com/forum/?fromgroups=#!topic/google-appengine/FM_NXd9cbus, which is exactly my problem, but there was no solution!
What am I doing wrong?
Edit: I also tried just putting the icalendar module directory in the project root, w/o the symlink. No difference.
You need to check how to use python libraries in GAE.
Pay attention as the docs are a bit confusing, you need to pick one (and only one) solution for a library, you can't mix both (they're both chapters on the above-mentioned page):
Requesting a library - for libs supplied by GAE - for these check notes for what you may need to do for your local development server
Installing a library - for additional libraries you need to include in your app - for these check also the restrictions the library needs to meet