I am an avid python user. I have been programming and performing a lot of my statistics using R. Recently, I tried to go into one of my notebooks to perform some statistical analysis. I have written over 5000 lines of code. Now, I have used R functions scattered everywhere throughout my program. Unfortunately, I am unable to even use any of the functions i have written before.
This is what i have done before:
%load_ext rmagic
import rpy2.robjects as R
import pandas.rpy.common as com
from rpy2.robjects.packages import importr
import scipy.stats as sp
stats=importr('stats')
TSA = importr('TSA')
forecast = importr('forecast')
fUnitRoots = importr('fUnitRoots')
tseries = importr('tseries')
urca = importr('urca')
VARS = importr('vars')
zoo = importr('zoo')
aod = importr('aod')
Now, I can't even run any of this any more as i get an import error "r_magic extension has been moved".
Also, i have called R functions by doing the following:
%R acf(x)
Above statement no longer works.
But if i do....
R.r('acf(x)')
it works. This seems like an annoying change i have to incorporate in my large program. Is there a workaround towards this solution?
Thanks
The rmagic is now in rpy2. Do:
%load_ext rpy2.ipython
Related
There is a function called dea(x, y, *args) in library(Benchmarking) which returns useful objects. I've described 3 key ones below:
crs = dea(mydata_matrix_x, my_data_matrix_y, RTS="IN", ORIENTATION= "in") # both matrixes have N rows
efficiency(crs) # a 'numeric' type object which looks like a 1xN vector
peers(crs) # A matrix: Nx2 (looks to me like a pandas dataframe when run in .ipynb file with R kernel)
lambda(crs) # A matrix: Nx2 of type dbl (also looks like a dataframe)
Now I would like to programatically vary my_data_matrix_x. This matrix represents my inputs. At first it will be a Nx10 matrix. However I intend to drop each column sequentially and run dea() on the Nx9 matrix, then graph the efficiency(crs) scores that come out. The issue is I have no idea how to achieve this in R (amongst other things) and would rather circumvent the issue by writing all my code in Python and importing this dea() function somehow from an R script
I believe the best solution available to me will be to read and write from files:
from Benchmarking_script.r import dea
def test_inputs(data, input):
INPUTS = ['input 1', 'input2', 'input3', 'input4,' 'input5']
OUTPUTS = ['output1', 'output2']
data_inputs = data.drop(f"{input}", axis=1)
data_outputs = data[OUTPUTS]
data_inputs.to_csv("my_inputs.csv")
data_outputs.to_csv("my_outputs.csv")
run Benchmarking.dea(data_inputs, data_outputs, RTS="crs", ORIENTATION="in")
clearly this last line won't work: I am interested to hear flexible (and simple!) ways to run this dea() function idiomatically as if it was a native Python function
Related SO questions
The closest answer on SO I've found has been Importing any function from an R package into python
When adapting the code I've written
import pandas as pd
data = pd.read_csv("path/to_data.csv")
import rpy2
import rpy2.robjects as robjects
import rpy2.robjects.packages as rpackages
from rpy2.robjects.vectors import StrVector
from rpy2.robjects.packages import importr
utils = rpackages.importr('utils')
utils.chooseCRANmirror(ind=1)
packnames = ('Benchmarking')
utils.install_packages(StrVector(packnames))
Benchmarking = importr('Benchmarking')
crs = Benchmarking.dea(data['Age'], data['CO2'], RTS='crs', ORIENTATION='in')
--------------------------------------------------------------
NotImplementedError: Conversion 'py2rpy' not defined for objects of type '<class 'pandas.core.series.Series'>'
So importing the function natively as a Python file hasn't worked
The second approach is the way to go. You need to use a converter context so python and r variables would be converted automatically. Specifically, try pandas2ri submodule shipped with rpy2. Something like this:
from rpy2.robjects import pandas2ri
with pandas2ri:
crs = Benchmarking.dea(data['Age'], data['CO2'], RTS='crs', ORIENTATION='in')
If this doesn't work, update your post with the error.
This might be a pretty niche problem but I've come across it and found a solution which maybe will help someone eventually. Basically, I'm trying to call the GUDHI C++ library through R using rpy2 using the following code:
import gudhi
import rpy2.robjects.packages as rpackages
import rpy2.robjects.vectors as rvectors
import rpy2.robjects as robjects
import rpy2.robjects.numpy2ri
import numpy as np
point_cloud = np.random.uniform(0, 10, size=(30, 3))
# tda.point_cloud_to_pl(point_cloud, method="GUDHI_R")
rpy2.robjects.numpy2ri.activate()
# import R's "base" package
base = rpackages.importr('base')
# import R's "utils" package
utils = rpackages.importr('utils')
# select a mirror for R packages
utils.chooseCRANmirror(ind=1) # select the first mirror in the list
# R package names
packnames = ('TDA', 'deldir', 'Matrix', 'SparseM')
names_to_install = [x for x in packnames if not rpackages.isinstalled(x)]
if len(names_to_install) > 0:
utils.install_packages(rvectors.StrVector(names_to_install))
# Importing packages
rpackages.importr('TDA')
rpackages.importr('deldir')
rpackages.importr('Matrix')
rpackages.importr('SparseM')
X = robjects.r.assign("X", point_cloud)
alpha_complex_diag = robjects.r['alphaComplexDiag']
ph_output = alpha_complex_diag(X)
print(ph_output)
But keep getting a segmentation fault error saying
*** caught segfault ***
R[write to console]: address 0x7fa6ac2c6090, cause 'invalid permissions'
The solution is to remove the import gudhi at the top of the python script. For some reason that causes R to not be able to use it? Perhaps because they are trying to call the same library?
It may indeed be a conflict between dynamically loaded libraries used by both Python and R. One such example is scipy doing what seems odd things with BLAS, and this causes a crash: https://github.com/rpy2/rpy2/issues/505
I do not know GUDHI at all unfortunately. If this is a blocker you'd need to run this through a debugger to find what it is happening in the C library.
There is a package in R that I need to use on my data. All my data preprocessing has already been done in python and all the modelling as well. The package in R is 'PMA'. I have used r2py before using Rs PLS package as follows
import numpy as np
from rpy2.robjects.numpy2ri import numpy2ri
import rpy2.robjects as ro
def Rpcr(X_train,Y_train,X_test):
ro.r('''source('R_pls.R')''')
r_pls=ro.globalenv['R_pls']
r_x_train=numpy2ri(X_train)
r_y_train=numpy2ri(Y_train)
r_x_test=numpy2ri(X_test)
p_res=r_pls(r_x_train,r_y_train,r_x_test)
yp_test=np.array(p_res[0])
yp_test=yp_test.reshape((yp_test.size,))
yp_train=np.array(p_res[1])
yp_train=yp_train.reshape((yp_train.size,))
ncomps=np.array(p_res[2])
ncomps=ncomps.reshape((ncomps.size,))
return yp_test,yp_train,ncomps
when I followed this format is gave an error that function numpy2ri does not exist.
So I have been working off of rpy2 manual and have tried a number of things with no success. The package I am working with in R is implemented like so:
library('PMA')
cspa=CCA(X,Z,typex="standard", typez="standard", K=1, penaltyx=0.25, penaltyz=0.25)
# X and Z are dataframes with dimension ppm and pXq
# cspa returns an R object which I need two attributes u and v
U<-cspa$u
V<-cspa$v
So trying to implement something like I was seeing on the rpy2 tried to load the module in python and use it in python like so
import rpy2.robjects as ro
from rpy2.robjects.packages import SignatureTranslatedAnonymousPackage as STAP
from rpy2.robjects import numpy2ri
from rpy2.robjects.packages import importr
base=importr('base'
scca=importr('PMA')
numpy2ri.activate() # To turn NumPy arrays X1 and X2 to r objects
out=scca.CCA(X1,X2,typex="standard",typez="standard", K=1, penaltyz=0.25,penaltyz=0.25)
and got the following error
OMP: Error #15: Initializing libomp.dylib, but found libiomp5.dylib already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://openmp.llvm.org/
Abort trap: 6
I also tried using R code directly using an example they had
string<-'''SCCA<-function(X,Z,K,alpha){
library("PMA")
scca<-CCA(X,Z,typex="standard",typez="standard",K=K penaltyx=alpha,penaltyz=alpha)
u<-scca$u
v<-scca$v
out<-list(U=u,V=v)
return(out)}'''
scca=STAP(string,"scca")
which as I understand can be used like an r function directly
numpy2ri.activate()
scca(X,Z,1,0.25)
this results in the same error as above.
So I do not know exactly how to fix it and have been unable to find anything similar.
The error for some reason is a mac-os issue. https://stackoverflow.com/a/53014308/1628393
Thus all you have to do
is modify it with this command and it works well
os.environ['KMP_DUPLICATE_LIB_OK']='True'
string<-'''SCCA<-function(X,Z,K,alpha){
library("PMA")
scca<-CCA(X,Z,typex="standard",typez="standard",K=Kpenaltyx=alpha,penaltyz=alpha)
u<-scca$u
v<-scca$v
out<-list(U=u,V=v)
return(out)}'''
scca=STAP(string,"scca")
then the function is called by
scca.SCCA(X,Z,1,0.25)
When calling a function in the survival package in R from within python with the rpy2 interface I get the following error:
RRuntimeError: Error in formula[[2]] : subscript out of bounds
Any pointer to solve the issue please?
Thanks
Code:
import pandas as pd
import rpy2.robjects.packages as rpackages
from rpy2.robjects.vectors import StrVector
from rpy2.robjects.packages import importr
import rpy2.robjects as ro
R = ro.r
from rpy2.robjects import pandas2ri
pandas2ri.activate()
## install the survival package
utils = rpackages.importr('utils')
utils.chooseCRANmirror(ind=1) # select the first mirror in the list
utils.install_packages(StrVector('survival'))
#Load the library and example data set
survival=importr('survival')
infert = R('infert')
## Linear model works fine
reslm=R.lm('case~spontaneous+induced',data=infert)
#Run the example clogit function, which fails
rescl=R.clogit('case~spontaneous+induced+strata(stratum)',data=infert)
After trying around, I found out, there is a difference, whether you offer the R instance of rpy2 the full R-code string to execute, or not.
Thus, you can make your function run, by giving as much as possible as R code:
#Run the example clogit function, which fails
rescl=R.clogit('case~spontaneous+induced+strata(stratum)',data=infert)
#But give the R code to be executed as one complete string - this works:
rescl=R('clogit(case ~ spontaneous + induced + strata(stratum), data = infert)')
If you capture the return value to a variable within R, you can inspect the data and get out the critical information of the model
by the usual functions in R.
E.g.
R('rescl.in.R <- clogit(case ~ spontaneous + induced + strata(stratum), data = infert)')
R('str(rescl.in.R)')
# or:
R('coef(rescl.in.R)')
## array([1.98587552, 1.40901163])
R('names(rescl.in.R)')
## array(['coefficients', 'var', 'loglik', 'score', 'iter',
## 'linear.predictors', 'residuals', 'means', 'method', 'n', 'nevent',
## 'terms', 'assign', 'wald.test', 'y', 'formula', 'xlevels', 'call',
## 'userCall'], dtype='<U17')
It helps a lot - at least in this first phase of using rpy2 (for me, too), to have your r instance open and trying the code in parallel which you do, since the output in R is far more readable and you know and see what you are doing and what you could address.
In Python, the output is stripped off of important informations (like the name etc) - and in addition, it is not pretty-printed.
This fails when including the strata() function within the formula because it's not evaluated in the right environment. In R, formulas are special language constructs and so they need to be treated separately by rpy2.
So, for your example, this would look like:
rescl = R.clogit(ro.Formula('case ~ spontaneous + induced + strata(stratum)'),
data = infert)
See the documentation for rpy2.robjects.Formula for more details. That documentation also discusses the pros & cons of this approach vs that provided by #Gwang-jin-kim
Through rpy2 in jupyter, you may plot your data directly from python using R objects. How can you set par(mfrow=c(1,2) in python?
For instance, I want to automatically feed a matrix with variable size from python and plot it (among other statistical analyses) using rpy2. But instead of plotting a single boxplot, I want all of them to be output.
Here's some sample code
import rpy2.ipython
import rpy2.robjects as ro
import scipy as sp
import re #python for regex
from rpy2.robjects.packages import importr
rpy2.robjects.numpy2ri.activate()
%load_ext rpy2.ipython
%R
test=[[1,3,2],[6,5,7,8,9]]
def funtoanalyze(grouparray):
a={}
data=numpy.array(test)
for ig in range(len(grouparray)):
key=grouparray[ig]
value=data[ig]
a[key]=value
next
rbox=ro.r('boxplot')
for gro in a:
datar=a[gro]
ro.r('dev.new()')
rbox(ro.FloatVector(datar[:]),xlab="",main=gro)
return
funtoanalyze(["group33","group2"]) #only plots last group
Your use of %load_ext rpy2.ipython suggests that you want to have your figure in the jupyter notebook.
R is using "graphical devices" to output figures, and calling par(mfrow=c(...)) will either put the setting in an open graphical device or open a new default device and set the parameter.
The "magic" %%R is scanning if figures were generated on default devices and display them in the notebook. The following should work:
%%R
par(mfrow=c(1,2))
plot(0, 0)
plot(0, 0)
If you do not want to use the R magic, there are other utilities for the jupyter notebook in rpy2. For plotting there is a context manager (see https://bitbucket.org/rpy2/rpy2/issues/330/ipython-plotting-wrapper - I don't remember if there is more documentation), but the most advanced utilities are tailored for ggplot2. Check for example this slides and the following ones:
https://lgautier.github.io/odsc-ppda-slides/#/5/13
The full notebook is here:
https://github.com/lgautier/odsc-ppda-slides/blob/master/notebooks/slides.ipynb
There is a docker container shipping with everything needed to run the notebook:
https://github.com/lgautier/pragmatic-polyglot-data-analysis