I am writing a Network File System, and I have started by trying to understand the xmp.py provided with python-fuse.
Well basically I'm sending all the calls over the network, executing it on the server-side and sending the result to the client and returning it to the user. These parts are working perfectly fine.
The problem I am facing is that, the xmp.py is not able to create a file and hence the problem is showing up in my Network File System too.
The steps I have followed are:
Installed fuse-python by:
sudo apt-get install python-fuse
Then I go into the directory where xmp.py is located:
cd /usr/share/doc/python-fuse/examples/
I ran the xmp.py program with the following command-line arguments
python xmp.py -o root=/home/user /tmp/fuse
where /home/user will be replicated or mounted on /tmp/fuse
Then I cd into the /tmp/fuse directory and try to make a file with cat, like:
cd /tmp/fuse
cat > temp.txt
I get the error
bash: temp.txt: Invalid argument
After a lot of poking around, I believe that this is due to the function
def getattr(self, path):
return os.lstat("." + path)
When I do a cat > temp.txt, os.lstat("./temp.txt") is called and an error is thrown as the file /home/user/temp.txt is not found. After this the program gets stuck for a while and slows down my computer.
Please help me figure out what I'm doing wrong.
Thanks.
I think i have figured it out, the problem that i saw was once os.lstat("./temp.txt") fails, the file class's __init__ method is called and then immediately truncate is called and the program returns.
One solution that i followed was to not use the file class at all, and to implement the Fuse class' open/create to make the file.
This approach worked for me. If anyone has a better suggestion please let me know. I would like to use the object-oriented approach and use the file class.
Related
I am aware that similar questions have been asked before, but I either don't understand the answers, or there aren't any; so I decided to describe my problem in as much detail as possible.
Problem:
RStudio reticulate package uses Python from this path:
"/usr/bin/python"
but I want it to use python from this path - always, as a default:
"/Library/Frameworks/Python.framework/Versions/3.7/bin/python3"
How do I know it happens?
I open RStudio, and create a new python script. A new file with an extension .py is generated. I type something in:
import pandas as pd
and execute (by clicking cmd+enter). I then see what happens in the console - the reticulate package is called:
reticulate::repl_python()
Python 2.7.10 (/usr/bin/python)
Reticulate 1.12 REPL -- A Python interpreter in R.
I would like to permanently change where the reticulate package looks for Python.
From the Terminal I know:
$ python --version
Python 3.7.3
which python3
/Library/Frameworks/Python.framework/Versions/3.7/bin/python3
So, I would like to tell RStudio to always look in this path to find Python 3.7. I have tried to use the following command, run from an R script:
use_python("/Library/Frameworks/Python.framework/Versions/3.7/bin/python3")
but it doesn't do anything - my naive understanding is that this command is useful within an R markdown file, i.e. when I have code that combines R and Python, in separate chunks. I would like to change the path that is used when a Python script is run from within RStudio. Is there some kind of a config file I could edit?
I hope this makes sense. I am not a regular Python user, only started learning now, and I am also not very good with paths, so I would appreciate step-by-step answers.
OK, so I have posted too early - after some more googling I can solve this problem myself, but I think it is worth posting an answer here for people like me (i.e. not path-proficient or python-proficient).
There is something like a config file for R, called .Renviron. In order to access it, use Terminal to go to your home directory (i.e. the one that you go to when you type 'cd'). If you have never used this file before, it might not exist, in which case you need to create it.
Once in your home directory, type:
ls -a
then check on the list of files that appears whether .Renviron is there. Below are instructions what if you already have .Renviron (IF YES), and what if you don't (IF NO).
IF NO, type:
touch .Renviron
which creates the file.
IF YES, just proceed as below (without using the touch command).
Write:
nano .Renviron
the .Renviron file will open. In it, add a line that says:
RETICULATE_PYTHON="enter your desired path here"
so, in my case, this is:
RETICULATE_PYTHON="/Library/Frameworks/Python.framework/Versions/3.7/bin/python3"
now save the file by exiting nano (ctrl+x) and clicking 'y' when it asks whether to save changes (press 'y' then press enter).
restart you RStudio. It should work now. I hope this is useful!
I recently tried to run Regularized Greedy Forest algorithm (rgf Classifier) from this package https://pypi.python.org/pypi/rgf_python.
I did run pip install rgf_python, but when I tried to run the example this error pops out in spyder:
File "C:\Users\me\Anaconda3\lib\site-packages\rgf\sklearn.py", line 111, in <module>
"config flag 'exe_location' to RGF execution file.".format(_EXE_PATH))
Exception: C:\Users\me\rgf.exe is not executable file. Please set config flag 'exe_location' to RGF execution file.
What should I change in order for config the flag 'exe_location'? May sound beginner but can't find it anywhere.
Official doccument is here.
https://github.com/fukatani/rgf_python#installation
For accurate support, I want to know your OS and python version.
In brief, you should do the following two things.
First, you should build rgf execution file.
$ git clone https://github.com/fukatani/rgf_python.git
$ cd rgf_python/include/rgf/build
$ make
If succeeded, you can find rgf_python/include/rgf/bin/rgf if you use linux.
Second, you should register exe_location.
There are several methods, but I recommend the method editing ~/.rgfrc.
Open ~/.rgfrc by your text editer, and add line.
exe_location=/path/to/rgf_python/include/rgf/bin/rgf.exe
exe_location should be full path, please do not use relative path.
Thanks.
I am not sure what I am supposed to be searching, because everything I have been trying to look up has not really given me the answers that I need.
If anyone can point me in the right direction of exactly what I should be looking at, I would greatly appreciate it. Basically, I want to know the best way to go about making a python program which I can distribute (not publicly).
Once someone has the file they run a script, or run the make command, and the result of this would be that the user can invoke the program from within any directory on the computer by just typing the name (and passing args) in the terminal.
I only know of one way to do this, which I believe is the wrong way. My idea was something as follows:
1.) A main folder, let's call it Ghoul. It would contain 3 files:
setup.sh
ghoul.sh
ghoul.py
2.) The user would have to run setup.sh, which would make ghoul.sh an executable and it would change the name to ghoul and then I would move it to /bin
3.) Within the ghoul file which is now in /bin, the file itself would contain "python /home/user/Ghoul/ghoul.py $#" And this would allow the program to run from any directory just by typing "ghoul"
I hope it's clear what I am trying to accomplish, I know this can't be the appropriate way of doing this. What should I be looking up and reading on to accomplish this? I would like to know how I can do this on Ubuntu and if also ..Windows.
I am also interested if I can limit the "ghoul" command to work only in the Ghoul directory, but of course in any directory that is within the Ghoul directory instead of from within any directory on the system.
Thank you.
On Unix like systems
#! /usr/bin/env python
in the very first line of your chmod +x script file will make it executable from the current directory using ./filename
If you want completely straightforward execution from anywhere you can put it on the path somewhere however you choose.
On windows its more complicated, if you want commandline execution the default install should give you the ability to execute
myscript.py
If its in your path. to drop the .py you need to get .py into the PATHEXT variable. Alternatively if it neds to be distributed to someone without any python install check out www.py2exe.org
I am trying to run the basic helloworld code described here https://cloud.google.com/appengine/docs/python/. However, whenever I try the dev_appserver.py helloworld/ command, I get a usage error for the dev_appserver.py command.
I have installed Python 2.7 and also have Python 2.7 Anaconda installed on my system. Could the Anaconda Python be the cause of the issue?
The file structure of my code is as follows:
Project
helloworld
app.yaml
helloworld.py
README.md
I have tried executing the dev_appserver.py helloworld/ command from inside the 'Project' folder and the 'helloworld' folder. But I get the same error in both cases.
Any help would be greatly appreciated!
Thanks!
I think I have found the "real" problem behind this error.
Tried putting some print statements in the dev_appserver.py file and found that what ever argument we are giving is not being passed and hence the error. On googling came across this SO post which explains the problem. On doing that change the following command worked flawlessly :D
dev_appserver.py app.yaml
Quoting that SO answer for easier reference:
I think I solved this. For some reason there is a SECOND place in the registry (besides that shown by the file associations stored in HKEY_CLASSES_ROOT\Python.File\shell\open\command):
[HKEY_CLASSES_ROOT\Applications\python.exe\shell\open\command]
#="\"C:\\Python25\\python.exe\" \"%1\" %*"
This seems to be the controlling setting on my system. The registry setting above adds the "%*" to pass all arguments to python.exe (it was missing in my registry for some reason).
You just need to execute the dev_appserver.py with path to app.yaml in the command itself, as suggested in comments.
I think OP could've identified a potential issue when describing multiple Python installations.
If I don't specify which Python installation (I thought I only had one...), then, it fails:
C:\Python27>"C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\bin\dev_appserver.py" "C:\users\jessmine\documents\ttbtamer\app.yaml"
usage: dev_appserver.py [-h] [-A APP_ID] [--host HOST] [--port PORT]
...etc...
dev_appserver.py: error: too few arguments
But if I specify which Python to use by invoking Python first, # C:\Python27\python.exe, then it works:
C:\Python27>"C:\Python27\python.exe" "C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\bin\dev_appserver.py" "C:\users\jessmine\documents\ttbtamer\app.yaml"
INFO 2016-11-18 10:09:14,299 devappserver2.py:769] Skipping SDK update check.
Anyway since I don't think I have other Python installations on my computer, then I may be misinterpreting the difference here. But for me, the fix is to explicitly invoke python.exe.
(And to be clear, I know about putting the Python.exe location into %PATH%, but I expected that if it wasn't there, that the error would've been something like "'dev_appserver.py' is not recognized as an internal or external command", rather than executing and printing a Python error....)
EDIT
After changing the "associate a file type or protocol with a specific program" (example here) for *.py to explicitly use C:\Python27\python.exe , then I no longer needed to manually invoke C:\Python27\python.exe in my cmd; i.e. my first example worked correctly :
C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\bin>dev_appserver.py "C:\users\jessmine\documents\ttbtamer\app.yaml"
INFO 2016-11-18 10:33:50,269 devappserver2.py:769] Skipping SDK update check.
I had the same problem. I even installed everything on a fresh machine and I was getting the same error again.
So I "debugged" dev_appserver.py and I discovered that the argument passed to it (i.e. 'app.yaml' or '.' or '\hello_world') was not passed down to the following code file:
...\google-cloud-sdk\platform\google_appengine\google\appengine\tools\devappserver2\devappserver2.py
In this file the arguments were checked, producing the infamous error "too few arguments".
Out of desperation for the time lost, I made a quick (and for sure ugly) patching.
I commented out at lines 302 and 303 the validation rule:
parser.add_argument(
'config_paths', metavar=arg_name, nargs='+', help=arg_help)
At line 758 I replaced:
options.config_paths, options.app_id)
with a static file name:
{'app.yaml'}, options.app_id)
At least now I am able again to start the server and develop my application. Now that I can work again, I will try to understand better how to correct the problem.
I'm not a Python nor a Google App Engine expert, so I hope someone will propose a better correction and a better explanation of the problem, because I cannot believe that Google can release such bugged code!
I suggest cd into your helloworld folder and and run dev_appserver.py . don't forget the ending dot '.' sign.
Hope it works
In my case I was able to resolve this issue by peforming following two actions:
1). added "python" in the start of the command
2). provided full path to the "dev_appserver.py" file.
So in Google docs you will find to execute the following:-
> dev_appserver.py ./ --php_executable_path=/path/to/php-cgi
(note that I was trying to run the php example and on windows environment here...)
Instead running the following command worked:
> python "c:\<path to the directory containing the dev_appserver.py script>\dev_appserver.py" ./ --php_executable_path=/path/to/php-cgi
I have created a little pre-commit hook in python. This hook works like a charm under Linux, but in Windows it keeps telling me:
error: cannot spawn .git/hooks/pre-commit: No such file or directory
I know there have been similar questions here about the same issue and the conclusion seams to be the shebang. My script has this on the very first line:
#!F:\PortableApps\PortablePython3.2\App\python.exe
It's also interesting to note that executing the script simply by writing .git/hooks/pre-commit works wonderful, but as soon as I try to commit, git spits out the above message.
Another interesting thing is, when I convert the encoding from ANSI to UTF-8 (using Notepad++), I get the following error when trying to execute the script:
.git/hooks/pre-commit: Cannot execute binary file
I'm using the following tools:
PortablePython 3.2.1.1
msysgit 1.7.6 (Portable)
I used the proxy-approach to make the python script work under windows (with msysgit). The complete script (with description on how I did it) might be found here: https://gist.github.com/1839424
Here is the important part about making it work under Windows
If you're working with Windows (and "msysgit"), it's a little more complicated. Since "msysgit" seems to have a problem handling the SHEBANG, you'll have to use a little trick to make the script executable (further information on this problem can be found here).
In order to make the script work, you'll want to remove the SHEBANG from the Python script ("pre-commit.py") and use a wrapper bash-script to call the interpreter. This script should look something like this:
#!/bin/sh
python .git/hooks/pre-commit.py
Store this script as a file called "pre-commit" (no file-ending). This assumes that you have Python in your PATH. If you don't, you can also specify the full path to your interpreter-executable.
This script will be called by "git commit" and call the python-script to check for the huge files. The path after the SHEBANG should not be changed, as "msysgit" will remap it automatically. You must specify a path relative to the repo-root for the Python script to be executed (because thats from where the script is called).
Afterwards you'll want to copy both the wrapper-file ("pre-commit") and the Python-script ("pre-commit.py") to your repos ".git/hooks"-directory, personalize the Python-script ("max_file_size" and "git_binary_path") and mark the "pre-commit"-file executable.