I have an R script that can be executed in a terminal with Rscript app/myapp.R. The R script is stored in an R package project so that I can benefit from the documentation, check and unit tests tools.
Just for the sake of user-friendlyness, I was wondering if there is a method to mimic python's behaviour of a __main__.py inside a module.
EDIT 6/1/2020 : I actually need other users, that are not accustumed to R, to use the script. So once the package is installed, finding the full path of the script is not really an option for them.
In a python project, when I have the following package structure :
mypackage
├── mymodule
│ ├── __init__.py
│ └── __main__.py
└── setup.py
I can do python -m mymodule from any folder in the terminal provided the package was indeed installed in my python library. The command will execute mypackage/mymodule/__main__.py.
I'd like to have the same behaviour for an R package. Let's assume that the R package mypackage is already installed in my R user's library. I would like to be able to run mypackage/app/myapp.R from anywhere in the terminal (actually, I'd like others to be able to install the package and run the app without having to clone the repo).
I know that I can do
Rscript app/myapp.R
but that will only work if I cd into path/to/mypackage. I'd like to be able to do something like below from anywhere in the terminal provided the package is installed in the R user's library.
Rscript -m myapp
demo() seems to be made for interactive sessions and I need a non-interactive session.
As it turns out there was a really simple solution using R command and not Rscript. Just wrap up the script within a function from the package, then use :
R -e "mypackage::myfunc()"
Related
I want to import business code from my test folder.
I installed my code using python setup.py install, which copy the code into the anaconda site-package folder. But I want to test my code in my dev. directory, as I don't want to constantly install my code to test it after any small change.
I am using Anaconda and Spyder IDE.
Here is my dir structure:
dev
└── myproject
├── setup.py
├── myproject
│ ├── __init__.py
│ └── myproject.py
└── test
├── __init__.py
└── test_importprojecthere.py
I took it from here: Running unittest with typical test directory structure
For now, I'm simply trying to import a function.
# dev: projectfile.py
def hello():
print('Hello world!')
Here is where I call it from.
# dev: test_importprojecthere.py
from myproject.projectfile import hello # Use installed package.
hello()
more information:
Python relative paths for unit tests
Import a module from a relative path Not ideal!
relative path not working even with __init__.py
When you are creating packages, you do not want to install them in your site-packages.
Do not use
python setup.py install
Instead, use
python setup.py develop or...
pip install -e .
Those commands install your setup using symlinks. Instead of creating a copy, it links to your actual project. So when you modify it, it's being modified instantly everywhere.
Thanks to that, you will not have to update/install or do anything to run your unit tests.
-- "But now, every time I modify my package, everything using it might break. I don't want to break my production code!"
Do not worry! This is the reason why we use virtual environments. When you create a package, make sure you create a virtual environment, that you might name, by example "myproject_dev".
In the dev. env. use python setup.py develop so that you can develop on it.
In the prod. env. use python setup.py install, so that you use a copy, that will not suddenly change. You won't modify it, because in prod, you consume your package, you don't develop on it! Keeps things simple an safe.
I have created a project called NumerMP a few days ago which will provide multi-threaded numerical methods (using OpenMP library), and will provide python API for end users (using wrapper functons-f2py). I had to use f2py smart way, due to customizing fortran signature files (.pyf) before generating extension modules. To do so I ran commands below:
python -m numpy.f2py trial_file.f90 -m module_name -h trial_file.pyf
python -m numpy.f2py -c --f90flags=’-fopenmp’ -lgomp trial_file.pyf trial_file.f90
Then I tested generated extension modules (.so files) inside package itself and they worked correctly, Then I built whole project and uploaded to test pypi and installed it using pip, But I got an ImportError.
The structure of the package is as follows:
When I run each sample wrapper function inside package itself, as you can see they run correctly.
But I got error below when I used it inside other project:
Update
You have some sibling submodules:
module
|- first
| |- functions.py
|- second
| |- wrapper.py
Then in second/wrappdr.py:
from ..first import functions
functions.function1()
I am developing a Python application, and using Conda to distribute it and manage dependencies.
I have successfully run conda-build to wrap up my code into a .tar.bz2 file. I created a new conda environment, and tested installing the package. Everything works, and I am able to run my code this way.
What I am wondering is, how do I structure my package and/or Python code, so that I can run my script by simply typing with my conda environment active, rather than navigating to the folder where I developed the code?
For example, in this conda cheatsheet: https://conda.io/docs/_downloads/conda-cheatsheet.pdf, the fourth line from the top, "Run a package after install, example Spyder" the command to run is simply "spyder".
My Python program is launched from a single .py file, launch.py, but how do I let conda know which script to call? When I do now, it simply says "command not found".
This section in the metadata specification seems relevant: https://conda.io/docs/user-guide/tasks/build-packages/define-metadata.html#entry-point
I tried setting that to "entry: python launch.py" in my meta.yaml, rebuilding and reinstalling, but that did not work and I am not sure where to go from here.
Any help would be tremendously appreciated, thank you.
Edit: After more digging, I may have found the answer, it seems I want to create an entry point as described here: https://setuptools.readthedocs.io/en/latest/setuptools.html#dynamic-discovery-of-services-and-plugins Still trying to put it all together though.
Let's say your package structure is something like
my_package/
├── my_package/
│ ├── __init__.py
| ├── __main__.py
├── conda.recipe/
│ ├── meta.yaml
├── setup.py
The conda recipe in meta.yaml doesn't need to go in a conda.recipe folder, it's just convenient if you want to store the recipe with the rest of the code. Now, in __main__.py, you have some code that you want to call (this code could be in any module, __main__.py happens to be a convenient example because code in there is also executed if you run python -m my_package). The code in __main__.py might be like
import sys
# Import whatever else from your package or otherwise to run the code
def main(argv):
# Process command line arguments, etc
if __name__ == '__main__':
sys.exit(main())
Now, in your meta.yaml, you'll have something like:
package:
name: my_package
version: '1.0.0'
source:
path: ..
build:
number: 0
script: python setup.py install --single-version-externally-managed --record=record.txt
entry_points:
- my_package_script = my_package.__main__:main
requirements:
- ...
The key line is my_package_script = my_package.__main__:main. The syntax here is the same as for setup.py entry points:
script name = package name.module name:function name
So, in our example, we're using the main function from the __main__.py module in the my_package package.
For a complete example, you can take a look at my package here: https://github.com/bryanwweber/CanSen
The following is my directory structure.
ankur
├── ankur1
│ ├── __init__.py
│ └── util.py
├── ankur2
│ └── main.py
└── __init__.py
In main.py, I am importing the following.
import ankur.ankur1.util
When I execute the code in Windows, it works perfectly fine. But in Linux, I get the following error.
ImportError: No module named ankur.ankur1.util
I also read the official python doc on Modules and Packages.
Your package structure is OK. Your import statement is OK. The only thing missing is for the package to be visible in sys.path, a list of locations where import statements can be resolved.
Usually we do this by "installing" the package locally with pip, which copies your code into site-packages†. This directory is one of the entries in sys.path, so when your code is installed in site-packages, the import statements can now be resolved as usual.
However, to install your code you'll need an installer (setup.py script) or a build system (pyproject.toml file) defined for the package. Your project doesn't appear to have any installer or build system, so you'll need to create one (see the Python Packaging User Guide for details about that) and then install the package with pip. If you don't want to learn Python packaging just yet, you'll need to find another way around.
It is possible to modify sys.path directly in main.py, which is subsequently enabling the statement import ankur.ankur1.util to be resolved. This is hacky and I recommend against that. It would add the restriction that executing main.py is the only entry point to the rest of the package, and so any other code wanting to import ankur will first need to know the path to main.py on the filesystem. That's a messy approach and should be avoided.
Another way around is to use the environment - there is an environment variable PYTHONPATH which can be used to augment the default search path for module files. In your shell:
export PYTHONPATH=/path/to/parent # linux/macOS
SET PYTHONPATH=C:/path/to/parent # Windows
Where parent is the directory containing ankur subdirectory.
† The exact location of site-packages depends on your OS/platform, but you can check with import sysconfig; sysconfig.get_paths()["purelib"]
Python newbie here. I'm trying to package a console app following this doc. To this end I created the following directory structure:
.
├── bin
│ └── txts
├── setup.py
└── txtstyle
├── __init__.py
├── ...
└── [snip]
My app has one executable script which I placed under bin. I could successfully run
python setup.py sdist
and create a tar.gz. However I can't execute the script under bin due to import errors.
So my question is how can the script access the main module from under bin?
You need to install the package. This puts all modules in the global module path, and thus allows you to import them. For development, use python setup.py develop which links the modules into the module python instead of copying them. This way you don't need to reinstall the package each time you changed a module.
There is a tool called virtualenv which creates virtual python environments. You can install modules into such environments without touching the global Python interpreter.