I have downloaded pyOpt from its website and installed it on python in ubuntu, using the instructions on the website.
Still, I cannot import and use it in my pycharm projects.
I had the same problem, but I did not dare to try Akin's solution for my research project (I think it is a good solution if anyone wants to stick to PyOpt package with python3).
I used Pyomo instead.
By the way, Laurent's answer actually points to another package. PyOpt and pyopt are two different packages.
I have been trying to figure this out too. It seems that the code for the pyOpt package is written in Python 2; when attempting to import anything once all system requirements have been met, the errors I received all point towards this. I will update this answer once I have figured out a way round this.
It should be feasible to simply have the script code converted to python 3 but I have not do this before.
Edit
You can use the package 2to3 to convert all the scripts to be compatible with Python 3.
Navigate to the correct folder through the command line in which the pyOpt files are.
Then use 2to3[pyOpt-1.2.0]-wand this will rewrite all the scripts to be compatible with Python 3. Note that you don't need the [] in the command. See 1 for further details.
1
Then you should be able to run python setup.py install and all should be golden.
PyOpt correctly works with Python 3. I have tried with Python 3.6.
I recommend you to use a virtualenv. You can consult the user guide online.
For instance, you can do:
python -m venv .venv
source .venv/bin/activate
pip install pyopt
If you use pew, you can do:
pew new -p python3.6 pyOpt -i pyOpt
This will create a new virtualenv for you and install pyopt.
To test pyopt, you can follow the documentation. You can try:
>>> import pyopt
>>> expose = pyopt.Exposer()
>>> #expose.args
... def regular_function(arg1:str, arg2:int):
... """Your help - the docstring"""
... print(repr(arg1), repr(arg2))
...
>>> expose.run()
Usage: arg1 arg2
To work with PyCharm, you can consult the article “Virtual environment”.
In short: Open the Preferences and search Add Python Interpreter. Clic on And Interpreter... and select Existing environment. That way, you can select the python executable which is in .env/bin directory.
IMHO you should use Click instead of PyOpt, really.
Related
On a Linux server, I have some Python scripts using the built-in sqlite3 module (+ some Sqlite extensions built from source, as detailed in Upgrade Python's sqlite3 on Debian).
For another Python script, I need a newer version of the Sqlite shared library than the one I already have on the system. Reason: I need Sqlite higher than 3.25.0 for Window Functions.
If I install it from source here and do make and make install, it will probably overwrite previous versions of this library on the server, and could potentially break other OS tools using it.
How do you handle the general problem of having multiple versions of the Sqlite shared library?
I don't think Python virtual environments can be used for this context, or would it be possible?
Note: pip3 install --upgrade sqlite3 does not exist: we cannot upgrade Python's built-in sqlite3 package like this. And by the way we probably should not, since it could break some OS tools using Python + sqlite3.
This is very tricky and will need a little code change in your scripts.
What to do:
First, check the sqlite3 library version included with python just in case:
python -c "import sqlite3; print(sqlite3.connect(':memory:').execute('SELECT sqlite_version();').fetchall())
In my computer (python 3.8, windows) the output is [('3.35.5',)] which means python has the sqlite 3.35.5 library. I have no sqlite installed in my system: this is the library that comes with python3.8.
IF your python sqlite3 library is not the one you need :-( you have an alternative: you can use the pysqlite3 instead of the sqlite3 standard library. In this case:
You'll need to build the pysqlite3 library by yourself using the Sqlite3 'amalgamation' that matches the version you want to use (more on later).
You'll need to install the library, and...
You will need to change your python script imports
import pysqlite3 as sqlite3 # instead of sqlite3
Ok, what is the 'amalgamation` and how to build pysqlite3?
The amalgamation is the whole sqlite3 library in just one .c file (with the sqlite3.h file). You can get it from the sqlite3 download page: sqlite3.36 amalgamation.
Once you have the amalgamation, follow the instructions to build statically pysqlite3, and install the package.
Now you can use pysqlite3 in your code.
If you want 2 different version of sqlite3 (python3) on 2 different environments, you can do that.
Since you mentioned that sqlite3 is part of the std library, it seems like you can try the pysqlite3 package instead.
If you can't run pip, run the following command first.
sudo apt install python3-pip
Then,
pip install virtualenv
python3 -m venv sqlitev1 #(whatever name you want)
source sqlitev1/bin/activate
pip install pysqlite3==0.4.4 #(this can be whatever version u want)
source deactivate
python3 -m venv sqlitev2 #(whatever name you want)
source sqlitev2/bin/activate
pip install pysqlite3==0.4.4 #(this can be whatever version u want)
source deactivate
Now you have 2 python environments, sqlitev1 and sqlitev2, with 2 different version of sqlite3.
It might be super hacky but you can make the new version of sqlite and then make sure that the path pointing to the new version is on the pythonpath environment before the built in one. Python will scan the python path from first to last to find an import, so the new version first in the python path for the processes that want the new version and then exclude that path with the old processes that need the built in one. You can accomplish this with a bash script that loads the env and then runs the python process for the new services.
Again this is super hacky so last resort.
If you want a different version of Sqlite than that installed with your distro, and a Python that uses that version, then you could
Compile sqlite to an alternative location
Compile Python to a different location, and point it to the custom Sqlite installation.
The "pointing" is covered in the accepted answer to this question. The question body itself shows how you might compile sqlite to a custom location.
The other answer to that question proposes setting the LD_LIBRARY_PATH environment variable to the directory containing the custom sqlite build to avoid having to compile Python. This might work with a virtualenv (it could be set in the preactive hook, for example).
See also
What is LD_LIBRARY_PATH and how to use it?
Set LD_LIBRARY_PATH before importing in python
Another approach would be to compile pysqlite3 in a virtualenv over the custom sqlite build. You can read about this in this blog post (I won't copy the details as it isn't clear what licence is used by the blog).
I have my own package in python and I am using it very often. what is the most elegant or conventional directory where i should put my package so it is going to be imported without playing with PYTHONPATH or sys.path?
What about site-packages for example?
/usr/lib/python2.7/site-packages.
Is it common in python to copy and paste the package there ?
I usually put the stuff i want to have ready to import in the user site directory:
~/.local/lib/pythonX.X/site-packages
To show the right directory for your platform, you can use python -m site --user-site
edit: it will show up in sys.path once you create it:
mkdir -p "`python -m site --user-site`"
So if your a novice like myself and your directories are not very well organized you may want to try this method.
Open your python terminal. Import a module that you know works such as numpy in my case and do the following.
Import numpy
numpy.__file__
which results in
'/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site- packages/numpy/__init__.py'
The result of numpy.__file__ is the location you should put the python file with your module (excluding the numpy/__init__.py) so for me that would be
/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site- packages
To do this just go to your terminal and type
mv "location of your module" "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site- packages"
Now you should be able to import your module.
This is something that works for me (I have to frequently create python packages that are uploaded to a private pip repository). elaborating on the comment by #joran on the question.
create a "build directory" which is used as a workspace to create packages. any directory of your choice will do
Copy your python package dir there, and create a setup.py file. this should help in creating the setup.py correctly.
create a virtualenv for the project you are working on. virtualenvs have a bunch of other benefits, I am not going into the details here.
create a local dist package python setup.py sdist --format=tar. the package created should ideally be in the dist folder.
Install the package on your virtualenv (after activating it). pip install <yourpackage>.tar
you can use pip install --force-reinstall if you need to play around with the libraries more and re-create the dist packages.
I've found that this method works great for me. If you do not need to package the modules for use of other systems instead of just your local, this method might be an overkill
Happy hacking.
If you're developing a module I would recommend to follow this.
import sys
sys.path.append("/home/mylinux/python-packages")
Now any module you keep in python-packages is importable by Python-interpreter.
On my Mac, I did a sudo find / -name "site-packages". That gave me a few paths like /Library/Python/2.6/site-packages, /Library/Python/2.7/site-packages, and /opt/X11/lib/python2.6/site-packages.
So, I knew where to put my modules if I was using v2.7 or v2.6.
Hope it helps.
import folders could be extracted by adding following source code:
import sys
print sys.path
automatic symlink generation example would be:
ln -s \`pip show em | grep "Location"| cut -d " " -f2\` \`python -m site --user-site\`
instead of "em" you may use other package you've "just installed but the python can't see it"
below I'll explain in more details as being requested in the comment.
suppose you've installed python module em or pyserial with the following command (examples are for ubuntu):
sudo pip install pyserial
and the output is like this:
Collecting pyserial
Downloading pyserial-3.3-py2.py3-none-any.whl (189kB)
100% |████████████████████████████████| 194kB 2.3MB/s
Installing collected packages: pyserial
Successfully installed pyserial-3.3
the question would be following - python can't see the module pyserial, why?
because the location where the module has been installed isn't the one python is looking at for your particular user account.
solution - we have to create symlink from the path where pyserial arrived to the path where your python is looking for.
symlink creation command would be:
ln -s <what_to_link> <where_to_link>
instead of typing exact location we are asking pip to tell us where it stored modules by executing command:
pip show pyserial | grep "Location"| cut -d " " -f2
instead of typing exact location we are asking python to tell us where it looks for the modules being installed by executing command:
python -m site --user-site
both commands has to be escaped with "`" character (usually on the left of your 1 button for the US keyboards)
in result following command will be provided for ln and the missing symlink would be created:
ln -s /usr/local/lib/python2.7/dist-packages /home/<your_username>/.local/lib/python2.7/site-packages
or something similar, depending on your distro and python/pip defaults.
After installing the Nest Neural Simulator, I keep getting the following error when trying to run any of the example python files that came in the installation. I've tried re-installing Nest, Python, and using Anaconda, but no go.
Python error:
ImportError: No module named nest
Suggestions?
At https://nest-simulator.org/documentation you now find many different install instructions and how to solve the "ImportError: no module named nest" depends on the way you installed NEST.
System Python
The problem with the nest python module not being found is usually, that NEST is installed for a specific Python version and you can not load it from another. So while many OS still use Python 2.7 you may need to explicitly run
$ python3
>>> import nest
Additionally, if you have multiple Python 3.x versions installed, modules may still be installed for a different version and you have to explicitly start python with python3.6 or python3.8, etc.
Conda package
As #nosratullah-mohammadi already mentioned, if you have a Conda flavour installed, using the pre-built package is a very quick solution. The link in his post is unfortunately broken; this one should work, then go to "Installation" in the side bar.
$ conda create --name nest -c conda-forge python3 nest-simulator
$ conda activate nest
$ python # this should load the Python from the conda env
>>> import nest # this loads nest which is installed explicitly for that Python
From Source
For every install from source, be sure to have Python and other prerequisites installed before building NEST. Then you can create your temporary build directory (can be deleted afterwards) and configure with the flags you need.
cd somewhere
mkdir nest-build
cd nest-build
cmake -DCMAKE_INSTALL_PREFIX:PATH=/install/path -Dwith-python=3 .../sources/of/nest-simulator
Replace somewhere, /install/path and .../sources/of/nest-simulator with the paths correct for your setup. (A popular choice when compiling from source in conjunction with Conda environments, for example, is to use -CMAKE_INSTALL_PREFIX=$CONDA_PREFIX, which installs NEST directly into the active environment. Conda is however in no way necessary for NEST.)
Add more -D... flags as you prefer. Possible flags you see with cmake -LA .../sources/of/nest-simulator, as pointed out here. You are probably interested in many of the with-xyz at the end. Check the aforementioned documentation for deatils.
Check that the paths and libraries reported in the Configuration Summary make sense (you may need to scroll up a bit to see). It could for example look something like this:
--------------------------------------------------------------------------------
NEST Configuration Summary
--------------------------------------------------------------------------------
[...]
Python bindings : Yes (Python 3.6.8: /home/yourname/miniconda3/envs/nest/bin/python3)
Includes : /home/yourname/miniconda3/envs/nest/include/python3.6m
Libraries : /home/yourname/miniconda3/envs/nest/lib/libpython3.6m.so
Cython bindings : Yes (Cython 0.27.3: /home/yourname/miniconda3/envs/nest/bin/cython)
[...]
--------------------------------------------------------------------------------
[...]
PyNEST will be installed to:
/home/yourname/miniconda3/envs/nest/lib/python3.6/site-packages
--------------------------------------------------------------------------------
In this example CMake configured everything for Python3.6 from my conda environment.
If you are satisfied with your settings and all the found Python versions match, run the usual
$ make # optionally with -j$(nproc)
$ make install
$ make installcheck
In case that works out fine you are done and can delete the build directory to free the space. Congratulations!
Also if things get too mixed up and it doesn't seem to do what you expect, it is sometimes useful to delete the build directory and start off clean.
there is a new method of installation added to other methods, wich is installing nest with conda package and it's in its beta version. but it works and it's really simple.
you can find the installation from here!
simply after install mini conda package run your terminal and type this :
conda create --name ENVNAME -c conda-forge nest-simulator python
then type :
conda activate ENVNAME
and you're good to go!
NEST now provide the solution to that problem and similar ones, by providing a script which automatically sets the relevant system variables:
If your operating system does not find the nest executable or Python
does not find the nest module, your path variables may not be set
correctly. This may also be the case if Python cannot load the nest
module due to missing or incompatible libraries. In this case, please
run
source </path/to/nest_install_dir>/bin/nest_vars.sh
to set the necessary environment variables. You may want to include
this line in your .bashrc file, so that the environment variables are
set automatically.
https://nest-simulator.readthedocs.io/en/latest/installation/linux_install.html
Turns out I needed to move the directory where I installed nest (Users/name/opt/nest) into a nest folder in the following directory in anaconda. Specifically, I moved the contents of the folder (from the nest installation):
/Users/name/opt/nest/lib/python2.7/site-packages/nest
Into this one:
/anaconda/lib/python2.7/site-packages/nest
Disclaimer: I could very well run into problems for not having copied all the contents of the Nest installation, but this little hack is helping me run example files now.
I have recently tried to use pylearn2, a deep machin learning package for Python developed at University of Montreal.
I've just installed it and tried to run a simple example, but it did not work.
I have been using a pc with an Ubuntu 13.10 system, on which I found ipython installed.
I have installed Theano and later pylearn2, by following this webpage instructions:
http://deeplearning.net/software/pylearn2/
I have also modified the .bashrc file, as suggested
I thought that everything went well, and then I tried this Quick start example:
http://deeplearning.net/software/pylearn2/tutorial/index.html
I stopped at the first command:
python make_dataset.py
My terminal states:
Traceback (most recent call last): File "make_dataset.py", line 14,
in
Do you have any ideas on why it is not working?
Do you why these errors occur?
Thanks a lot
EDIT: the 14 line is the first non-commented line of the file. It states
from pylearn2.utils import serial
Without more information, I can only guess, but my first guess is…
You haven't actually installed pylearn2, because if you follow the linked docs to grab the git repo and add a PYLEARN2_DATA_PATH variable, nothing gets installed into site-packages (or dist-packages or anywhere else on sys.path).
This means that pylearn2 will only work when you start Python from within the top-level directory of the pylearn2 repo.
So, if you run a script like this:
$ cd /path/to/pylearn2
$ cd scripts/tutorials/grbm_smd/
$ python make_dataset.py
… it won't actually work.
It looks like there is a setup.py file in the repository. Does it work? I have no idea. Even though the docs don't mention using it, you might want to try. Either this:
$ pip install .
… or, if you don't have pip or it doesn't work on this package:
$ python setup.py install
Either way, of course, you may need sudo or a flag to install to your user site-packages instead of system, etc., as with any other Python package.
If that doesn't work, you might be able to just add /path/to/pylearn2 to your sys.path in some way. The most obvious way is by doing an export PYTHONPATH=/path/to/pylearn2:$PYTHONPATH in your ~/.bashrc.
Also, you will need to either source ~/.bashrc or create a new shell to get any effects of modifying the file.
If you're wondering why the instructions and the tutorial together don't give you enough information to make this work without a lot of hassle, I think that's covered in the very top of the documentation:
Pylearn2 is still undergoing rapid development. Don’t expect a clean road without bumps!
And the very fact that there is no PyPI download yet implies that this really is not ready for novices to use. If you don't know enough about using Python packages (and bash basics) to muddle through on your own, there's a good chance you won't be able to use this package.
The question is related to the answer to "Unable to install Python without sudo access".
I need to install python-setuptools to install python modules.
I have extracted the installation package.
I get the following error when configuring
[~/wepapps/pythonModules/setuptools-0.6c9]# ./configure --prefix=/home/masi/.local
-bash: ./configure: No such file or directory
I did not find the solution at the program's homepage.
How can I resolve this error?
As Noah states, setuptools isn't an automake package so doesn't use ‘./configure’. Instead it's a pure-Python-style ‘setup.py’ (distutils) script.
You shouldn't normally need to play with .pydistutils.cfg, as long as you run it with the right version of Python. So if you haven't added the .local/bin folder to PATH, you'd have to say explicitly:
/home/masi/.local/bin/python setup.py install
AIUI this should Just Work.
I did not find the solution at the program's homepage.
Yeah, they want you to install it from a shell script egg which uses the default version of Python. Which you don't want.
(Another approach if you can't get setuptools to work is to skip it and install each module and dependency manually. Personally I have a bit of an aversion to setuptools/egg, as it contains far too much “clever” magic for my tastes and makes a mess of my filesystem. But I'm an old curmudgeon like that. Most Python modules can be obtained as simple Python files or plain old distutils scripts, but unfortunately there are some that demand eggs.)
You might want to check http://peak.telecommunity.com/DevCenter/EasyInstall#custom-installation-locations.
EasyInstall is a python module with some shell scripts (or some shell scripts with a python module?) and does not use the unix make tool that gets configured with the "./configure" command. It looks like your best bet is to try editing ~/.pydistutils.cfg to include:
[install]
install_lib = /home/masi/.local/lib/python/site-packages/
install_scripts = /home/masi/.local/bin
You'll also presumably have made the ~/.local/bin/ folder part of your PATH so you can run the easy_install script. (I'm not sure exactly where the site-packages directory will be under .local, but it shouldn't be hard to find.)
Hope this helps.