I'm an experienced developer, but not very familiar with Python and Pyramid.
I'm trying to follow some (a bit old and unclear) instructions on deploying a Pyramid web application. My OS is Ubuntu 16.04.
The instructions say to install virtualenv and Pyramid - I do so with apt install virtualenv and apt install python-pyramid. Then they say I should run the app in a virtual environment, so I build that with virtualenv . -ppython3, and activate it with source bin/activate. I install the application from a ready-to-run buildout from GitHub. The buildout includes a "production.ini" file with parameters to pserve.
But Pyramid is not included in the virtual environment built with virtualenv. (There is no "pserve" in the bin directory, e.g.) So I can't run the applications with bin/pserve etc/production.ini, as the instructions say. And if I try with only "pserve", I get errors when trying to access files like "var/waitress-%(process_num)s.sock". Files that the app excepts to find in the virtual environment.
I've looked for flags to tell virtualenv to include Pyramid, but couldn't find any. Am I overlooking something? I'd be most grateful for some help! :-)
/Anders from Sweden
Perhaps you might want to try installing Pyramid in your virtual environment using pip, since apt-installed libraries are installed into /opt, rather than being visible to Python. In the guide, it seems like you're wanting to install Pyramid through the virtual environment so that it can be used by your program, so I think you'd be best using pip rather than apt-get. I did a quick Google search, and it seems like this is the library you need. Here, all you'd have to do is run the installation command once you've already entered the virtual environment with pip install pyramid. This way, you should only have access to it within the virtual environment, as well!
You mentioned it's using buildout - I assume this is zc.buildout. buildout usually manages its own virtualenv and handles installing all of the necessary dependencies. It really depends on how that buildout is configured as there's no standard there for what to do or how to run your app. I would normally expect pserve to be exposed in the bin folder, but maybe another app-specific script is exposed instead.
Related
I'm trying to deploy my Django site on Heroku, and thus, I need a requirements.txt file with the necessary packages that Heroku needs to install for me. I understand Django is a necessary package to be installed. Unfortunately, Django isn't included in the file when I run pip freeze > requirements.txt. Why is this? I'm not sure what to show you so you can tell me what's going wrong. Let me know and I'll add it. FYI the site hosts just fine on my local computer, so Django is definitely installed.
Sounds like you are working in a virtual environment, yet your Django dependency is installed globally. Check which Python packages are installed globally and uninstall Django (you probably don't need it globally). Then install it into your virtual environment. Now the freeze command should output Django as well.
General note: Most packages should be installed into your project virtual environment. There are only few packages where it makes sense to install them globally (eg aws management tools).
I am having issues with flask, and now I am wondering if there's a way to use flask without virtual environment on Python. Why would we need virtual environment with flask?
$ sudo pip install virtualenv
$ sudo apt-get install python-virtualenv
$ virtualenv venv
$ . venv/bin/activate
$ venv\Scripts\activate
I was searching on Google and couldn't find a good answer for that! If there's a way to use flask without virtual environment could you please show me how?
Yes, you can. You can use any python library without virtualenv. What virtualenv does is create a sandbox environment for you so you can install whatever python libraries you want without affecting anything else on your computer. When you delete that virtual environment, all those libraries go away like it never happened.
That way you can have one project that uses version 1 or Flask and another project that uses version 2 and they won't step on each other in any way. It lets you segregate python projects so you don't have to worry about them interfering with each other.
It's generally recommended that you use it. In addition to the benefits already mentioned, it helps eliminate environmental issues between your development environment and other environments like production. Otherwise you can get into a situation where things work fine on your box but when you go to deploy it, there are problems. Usually that's because you were using the wrong version of a library without realizing it. The virtualenv system helps prevent that from happening by making sure your app only has access to the versions of the libraries you want it to. When you move your app to production the entire virtualenv sandbox goes with it so it's pretty likely it will work the same way as it did on your dev box.
I've been working on a Python project for a while which uses quite some third party libraries. I want to deploy my project to another server, but I don't know by heart which projects I use and digging through every line of source code would be too much work.
Is there a way to generate a list of third party modules in my project so I can use this with PIP's installer? Thank you for your help
pip install -r dependencies.txt # <-- I need to generate this file from a project
Provided that you're using a virtual environment to keep your dependencies separate from the globally installed pip packages, you should be able to use pip's freeze command, like so:
pip freeze > dependencies.txt
If you haven't been using a virtual environment, then you probably need to peruse the source code to find the modules. A virtual environment is a means of keeping your python project isolated from the global environment, meaning that you can only import modules that are installed within that environment, and it should only contain modules relevant to its corresponding python project. I recommend that you read up on Virtual Environments, they are very useful for larger projects.
I ended up writing a python module which does this instead as I couldn't find it. The source code is available on GitHub. You can install it like so:
$ pip install pip-module-scanner
Using it is pretty simple, for full usage examples check the GitHub repo.
$ pip-module-scanner
foo==0.0.1
bar==2.0.0
I run Vagrant on Mac OS X. I am coding inside a virtual machine with CentOS 6, and I have the same versions of Python and Ruby in my development and production environment. I have these restrictions:
I cannot manually install. Everything must come through RPM.
I cannot use pip install and gem install to install the libraries I want as the system is managed through Puppet, and everything I add will be removed.
yum has old packages. I usually cannot find the latest versions of the libraries.
I would like to put my libraries locally in a lib directory near my scripts, and create an RPM that includes those frozen versions of dependencies. I cannot find an easy way to bundle my libraries for my scripts and push everything into my production server. I would like to know the easiest way to gather my dependencies in Python and Ruby.
I tried:
virtualenv (with --relocatable option)
PYTHONPATH
sys.path.append("lib path")
I don't know which is the right way to go. Also for ruby, is there any way to solve my problems with bundler? I see that bundler is for rails. Does it work for custom small scripts?
I like the approach in Node.JS and NPM; all packages are stored locally in node_modules. I have nodejs rpm installed, and I deploy a folder with my application on the production server. I would like to do it this way in Ruby and Python.
I don't know Node, but what you describe for NPM seems to be exactly what a virtualenv is. Once the virtualenv is activated, pip installs only within that virtualenv - so puppet won't interfere. You can write out your current list of packages to a requirements.txt file with pip freeze, and recreate the whole thing again with pip install -r requirements.txt. Ideally you would then deploy with puppet, and the deploy step would involve creating or updating the virtualenv, activating it, then running that pip command.
Maybe take a look at Docker?
With Docker you could create a image of your specific environment and deploy that.
https://www.docker.com/whatisdocker/
Being new to the python games I seem to have missed out on some knowledge on how you can develop on a program but also keep it in your live environment.
Programs like gpodder can be run directly from the source checkout which is really handy however others want to be "installed" to run.
A lot of programs are distributed with a setup.py with instructions to run "python ./setup.py install" as root which will put stuff somewhere in your file-system. There are even install commands like "develop" which seem to hold the promise of what I want. So I tried:
export PYTHONPATH=/home/alex/python
python ./setup.py develop --install-dir=/home/alex/python
Which downloaded a bunch of stuff locally and seems magically ensure the application I'm hacking on is still being run out of the src tree. So I guess my roundabout question is is this the correct way of developing python code? How do things like easy_install and pip factor into this?
So I tried the following:
python /usr/share/pyshared/virtualenv.py /home/alex/src/goobook
cd /home/alex/src/goobook/googbook.git
/home/alex/src/goobook/bin/python ./setup.py develop
And finally linked the program in question to my ~/bin
cd /home/alex/src/goobook
linkbin.pl bin/goobook
However invocation throws up a load of extra chatter which seems to imply it's wrong:
17:17 alex#socrates/i686 [goobook] >goobook --help
/home/alex/bin/goobook:5: UserWarning: Module pkg_resources was already imported from /home/alex/src/goobook/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/pkg_resources.py, but /home/alex/src/goobook/lib/python2.5/site-packages/distribute-0.6.10-py2.5.egg is being added to sys.path
from pkg_resources import load_entry_point
/home/alex/bin/goobook:5: UserWarning: Module site was already imported from /home/alex/src/goobook/lib/python2.5/site.pyc, but /home/alex/src/goobook/lib/python2.5/site-packages/distribute-0.6.10-py2.5.egg is being added to sys.path
from pkg_resources import load_entry_point
Install:
http://pypi.python.org/pypi/virtualenv
to set up a localized virtual environment for your libraries, and:
http://pypi.python.org/pypi/setuptools
i.e. "easy_install" to install new things.
Virtualenv allows you to work in completely independent and isolated Python environments. It will let you easily create multiple environments which have different Python packages installed or different versions of a same package. Virtualenv also lets you easily switch between your different environments.
As of 2012, the de facto preferred tool for package management in Python is pip rather than setuptools. Pip is able to handle dependencies and to install/uninstall globally or inside a virtual environment. Pip even comes out-of-the-box with virtualenv.
Python 3
Also worth mentioning is the fact that virtual environments are becoming a part of Python itself in release 3.3, with the implementation of PEP 405.
The Python Packaging User Guide, which "aims to be the authoritative resource on how to package, publish and install Python distributions using current tools", recommends using pip to install in "development mode":
pip install -e <path>
Thus in the root directory of your package you can simply
pip install -e .
See installing from a local source tree.
The best way to develop Python apps with dependencies is to:
Download desired version of the python interpreter.
Install and use buildout (http://www.buildout.org/).
Buildout is something like Maven for Java (will fetch all needed packages automatically).
This way your Python interpreter will not be polluted by third party packages (this is important if you will be running developed application on other machines). Additionally you can integrate buildout with virtualenv package (this allows you to create virtual python interpreters for each project).