Develop Reference

How to change the default python version/path in Rstudio

I am aware that similar questions have been asked before, but I either don't understand the answers, or there aren't any; so I decided to describe my problem in as much detail as possible.
Problem:
RStudio reticulate package uses Python from this path:
"/usr/bin/python"
but I want it to use python from this path - always, as a default:
"/Library/Frameworks/Python.framework/Versions/3.7/bin/python3"
How do I know it happens?
I open RStudio, and create a new python script. A new file with an extension .py is generated. I type something in:
import pandas as pd
and execute (by clicking cmd+enter). I then see what happens in the console - the reticulate package is called:
reticulate::repl_python()
Python 2.7.10 (/usr/bin/python)
Reticulate 1.12 REPL -- A Python interpreter in R.
I would like to permanently change where the reticulate package looks for Python.
From the Terminal I know:
$ python --version
Python 3.7.3
which python3
/Library/Frameworks/Python.framework/Versions/3.7/bin/python3
So, I would like to tell RStudio to always look in this path to find Python 3.7. I have tried to use the following command, run from an R script:
use_python("/Library/Frameworks/Python.framework/Versions/3.7/bin/python3")
but it doesn't do anything - my naive understanding is that this command is useful within an R markdown file, i.e. when I have code that combines R and Python, in separate chunks. I would like to change the path that is used when a Python script is run from within RStudio. Is there some kind of a config file I could edit?
I hope this makes sense. I am not a regular Python user, only started learning now, and I am also not very good with paths, so I would appreciate step-by-step answers.

OK, so I have posted too early - after some more googling I can solve this problem myself, but I think it is worth posting an answer here for people like me (i.e. not path-proficient or python-proficient).
There is something like a config file for R, called .Renviron. In order to access it, use Terminal to go to your home directory (i.e. the one that you go to when you type 'cd'). If you have never used this file before, it might not exist, in which case you need to create it.
Once in your home directory, type:
ls -a
then check on the list of files that appears whether .Renviron is there. Below are instructions what if you already have .Renviron (IF YES), and what if you don't (IF NO).
IF NO, type:
touch .Renviron
which creates the file.
IF YES, just proceed as below (without using the touch command).
Write:
nano .Renviron
the .Renviron file will open. In it, add a line that says:
RETICULATE_PYTHON="enter your desired path here"
so, in my case, this is:
RETICULATE_PYTHON="/Library/Frameworks/Python.framework/Versions/3.7/bin/python3"
now save the file by exiting nano (ctrl+x) and clicking 'y' when it asks whether to save changes (press 'y' then press enter).
restart you RStudio. It should work now. I hope this is useful!

How do I run external programs in Python without using the entire path?

Now, I'm working on a voice controlling assistant application.
When I say "open Google Chrome", it should open Chrome. I do not have any problems with speech recognition but I do have a problem with starting the program.
For example, the following code:
import os
os.system("chrome.exe")
Works fine for the notepad.exe or cmd.exe programs but when I give external programs like chrome, I need to give the entire path.
However, in C# we can only give the direct name like chrome.exe to run the program.
So, my problem is that, is there any ways to start external programs without giving the entire path like in C#?
Giving path to start the program would be a serious problem. Because when we move the program to another computer, we will face many code errors.

Try os.path.abspath
os.path.abspath("chrome.exe")
returns the following:
C:\Users\Yourname\chrome.exe

The PATH system variable governs this inside Python just as outside. If you want to modify it from Python, it's os.environ.
As an aside, a better solution is probably to use subprocess instead, as suggested in the os.system documentation.
The PATH manipulation will be the same regardless.
import os
import subprocess
os.environ['PATH'] = os.environ['PATH'] + r';c:\google\chrome\'
subprocess.run(['chrome.exe'])
... where obviously you need to add the directory which contains chrome.exe to your PATH.
Probably you will want to make sure the PATH is correct even before running Python in your scenario, though.

The first part of this answer is OS specific and doesn't solve it in code or python
Add the path where Chrome exists to your PATH variable (you'll likely need a restart), and then when you execute a command such as chrome.exe it will try to run it from that path location
UPDATE:
If you want to implement a search for the absolute path here are some good resources on path listing How do I list all files of a directory?
But again you'll likely need some educated guesses and will need to do some OS specific things to find it.
The Chrome binary is often installed in the same set of locations

I solved this problem by using C#. Like I already mentioned, we can directly call programs like "chrome.exe" in C#. I created a console application in C# which open Google Chrome via using bellow code. Then I complied that console application as .exe, after that I link that exe file with my python program.
C#
Process ExternalProcess = new Process();
ExternalProcess.StartInfo.FileName = "chrome.exe";
ExternalProcess.StartInfo.WindowStyle = ProcessWindowStyle.Maximized;
ExternalProcess.Start();
ExternalProcess.WaitForExit();
Python
import os
os.system('console_application.exe')

How do you I add a module into my PYTHONPATH?

I am attempting to download a code from github which contains the library "ee" - Google Earth Engine. GitBash is giving me an error:
ModuleNotFoundError: No module named 'fcntl'
fcntl is a module within the library Google Earth Engine. I have Windows and it seems Linux is required. I was directed to add the module (fcntl) to the PYTHONPATH. Any other suggestions for this error would be helpful as well! The code I intend to add in PYTHONPATH is below.
def fcntl(fd, op, arg=0):
return 0
def ioctl(fd, op, arg=0, mutable_flag=True):
if mutable_flag:
return 0
else:
return ""
def flock(fd, op):
return
def lockf(fd, operation, length=0, start=0, whence=0):
return

First, this is probably not going to work for you.
You can't turn Windows to Linux just by adding modules to your Python library. The reason you don't have the fcntl module on your path is that fcntl isn't included on Windows. And the reason it isn't included on Windows is that the Windows OS doesn't support the syscalls that module wraps, or anything close enough to reasonably emulate those syscalls.
If you have code that requires fcntl, that code cannot run on Windows (unless you do some significant work to port it to not require fcntl in the first place).
If you have code that doesn't require fcntl but uses it anyway, or if you just need something for temporary development purposes so you can catch and fix file sharing errors while porting the code to not require fcntl, then you can use msoliman's dummy code, which I'll explain how to do below. But you seem to be expecting it to do magic, and it won't do that.
You may not be sure. Maybe you're using code that uses other code that uses other code that uses fcntl in some scenarios but not others, it may not actually need fcntl to do any of the things you're actually trying to do with it.
If you want to test that, you can take msoliman's dummy code, and change each function body to this:
raise RuntimeError('Oops, using fcntl!')
Then run the program and see if it fails with that error. If not, you don't actually need fcntl after all. (Or at least you don't need it for any of the things you tested—it's always possible that some other thing you need to do with the app that you didn't think to test will need it.)
If your code actually needs fcntl, and you don't want to/can't port that code to Windows code that uses Win32 API calls (or a cross-platform library like portalocker), then what you probably need to do is install Linux and run the program there.
There are multiple ways to run Linux on top of Windows, rather than instead of Windows. For example, you could install Docker for Windows and then build a linux docker container with the app. Or you could use VMWare Player to, in effect, run a Linux image as an application under Windows, and then do your work inside that image. And so on.
Finally, msoliman's "Place this module in your PYTHONPATH" is a little misleading.
What you actually need to do is get it into your sys.path. PYTHONPATH is just one way of doing that, and probably not the one you want here.
The options are:
Just put it in the same directory as your script. As the docs say, "As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter."
Put it in your user or system site packages, or some other directory that's already on your default sys.path. You can import sys; print(sys.path) to get a list of these directories. If you see something inside your home directory, that's a good place to put it; if not, look for something with site-packages in the name.
Put it in some other directory somewhere else, and set the PYTHONPATH environment variable to be the full path to directory. You can set an environment variable in the Windows cmd command prompt by writing SET PYTHONPATH C:\Path\To\Directory. This will only persist as long as the current command prompt window. If you want to set it permanently, there's a setting somewhere in Control Panel (it changes with each Windows version; Super User should have good up-to-date answers for each version) where you can set System and User environment variables. Any User environment variable will take effect in every new command prompt window you open from now on.

Hidden .db files using Python and SQLite3

I am using sqlite3 and python for a new website I am creating. The problem is, the "files_storage.db" file I am trying to create will not appear in any Windows 10 Folder Window, PyCharm's Directory View, nor will it appear via the command line interface.
The catch to this is, if I execute my python script multiple times, I get an error that states the database file already exists... So this file is somewhere, I guess it is a game of cat and mouse to find it.
I have ran into this problem before, but I have ALWAYS been able to find the file via the command line. Usually, I wouldn't both yall with such a question but, this is really irking me and I am going to run into serious issues when it comes time to put this baby on a server. :(
thanks in advance, and here's some screenshots I suppose.

You are using a relative file path.
Relative paths are converted to absolute ones by something like os.path.join(os.getcwd(), <relative path>). So they depend on the current working directory of the process.
Try to open it with an absolute path (starting with drive letter) to avoid any ambiguities.

If you use just a filename without a path, the file will be saved in whatever the current working directory of the Python interpreter is.
To see where the current working directory is, add the following code to the beginning of your program:
import os
print(os.getcwd())
You should then see the working directory in the output.
There is a setting for the current working directory in your IDE somewhere. See e.g. the answers to this question.

You can also do something like:
import os
path = os.path.expanduser("~") + '/Documents'
print(path)
This will allow you to access the directories for the current user. For me, this prints:
'/Users/thomasweeks/Documents'

Default working directory for Python IDLE?

Is there a configuration file where I can set its default working directory? It currently defaults to my home directory, but I want to set it to another directory when it starts. I know I can do "import os" followed by "os.chdir("")" but that's kind of troublesome. It'd be great if there is a conf file that I can edit and change that setting, but I am unable to find it.
In particular, I've looked into my OS (Ubuntu)'s desktop entry '/usr/share/applications/idle-python3.2.desktop', which doesn't contain a conf file, but points to '/usr/lib/python3.2/idlelib/PyShell.py', which points to config-*.def conf files under the same folder, with 'config-main.def' being the most likely candidate. However I am unable to find where the default path is specified or how it can be changed.
It seems that the path is hard-coded in PyShell.py, though I could be wrong with my limited knowledge on Python. I will keep looking, but would appreciate it if somebody knows the answer on top of his or her head. Thanks in advance.

I actually just discovered the easiest answer, if you use the shortcut link labeled "IDLE (Python GUI)". This is in Windows Vista, so I don't know if it'll work in other OS's.
1) Right-click "Properties".
2) Select "Shortcut" tab.
3) In "Start In", write file path (e.g. "C:\Users...").
This is also my answer here: Default save path for Python IDLE? Let me know if this works!

I've found a solution after looking into PyShell.py:
Create a python file under a preferred directory, in my case '~/.idlerc/init.py', and copy/paste the following lines:
import os
os.chdir('<your preferred directory>')
Pass "-r '~/.idlerc/init.py' " argument to the IDLE startup command, like the following (your exec location and name may vary depending on OS, etc):
/usr/bin/idle-python3.2 -n -r ~/.idlerc/init.py

Just use a shell script such as:
#!/bin/bash
cd /Users/pu/Projects/L-Python
/usr/bin/idle
and run that instead of stock idle. The example is on OS X, adapt to your system.

I'm new to python and learning from 'Dive into Python' by mark Pilgrim (can be found online free)
the answer is in chapter 2.4 - hope he doesn't mind me pasting it here as its also plugging his book and is in the GPL
Before you go any further, I want to briefly mention the library
search path. Python looks in several places when you try to import a
module. Specifically, it looks in all the directories defined in
sys.path. This is just a list, and you can easily view it or modify it
with standard list methods. (You'll learn more about lists later in
this chapter.)
Example 2.4. Import Search Path
import sys
sys.path
sys.path.append('/my/new/path')
It's a good book I am a programmer - usually I find learning from books sends me quickly to sleep - not the case here ....

All I had to do here (Linux Mint 18.2 Xfce) ...
Just add path in line "working directory" = "Arbeitsverzeichnis"

It can change depending on where you installed Python. Open up IDLE, import os, then call os.getcwd() and that should tell you exactly where your IDLE is working on.

One default path is specified in idlelib.IOBinding.IOBinding.dirname or idlelib.IOBinding.IOBinding.filename
Ubuntu
So my idle-python3.desktop
file in /usr/share/applications looks like this:
[Desktop Entry]
Name=IDLE (using Python-3)
Comment=Integrated Development Environment for Python (using Python-3)
Exec=python3 -c "import idlelib.IOBinding, os; idlelib.IOBinding.IOBinding.dirname='/DEFAULT/DIRECTORY';import idlelib.idle"
Icon=/usr/share/pixmaps/python3.xpm
Terminal=false
Type=Application
Categories=Application;Development;
StartupNotify=true
To use it you need to set /DEFAULT/DIRECTORY to your desired directory, copy it with root rights into /usr/share/applications. You can also use it for Python 2 but then you need to replace the 3s with 2s.
ConfigFiles
There are also extensions that can be loaded. These must be modules and you specify them by module name. The config files for IDLE are located in HOME/.idlerc and parsed with a configparser. I did not get further with this.

Here's a way to reset IDLE's default working directory for MacOS if you launch Idle as an application by double-clicking it. You need a different solution if you launch Idle from a command line in Terminal. This solution is a permanent fix. You don't have to rechange the directory everytime you launch IDLE. I wish it were easier.
The idea is to edit a resource file inside of the IDLE package in Applications.
Start by finding the the file. In Finder, go to IDLE in Applications (in the Python folder) as if you wanted to open it. Right click and select "show package contents". Open Contents, then open Resources. In Resources, you'll see a file called idlemain.py. This file executes when you launch idle and sets, among other things, the working directory. We're going to edit that.
But before you can edit it, you need to give yourself permission to write to it. To do that, right click on the idlemain.py and select get info. Scroll to the bottom of the getinfo window and you'll see the Sharing & Permissions section. On the bottom right there's a lock icom. Click the lock and follow the prompts to unlock it. Once it's unlocked, look to the left for the + (under the list of users with permissions). Click it. That will bring up a window with a list of users you can add. Select yourself (probably the name of your computer or your user account) and click Select. You'll see yourself added to the list of names with permissions. Click where is says "Read only" next to your name and change it to "Read & Write". Be careful not to change anything else. When you're done, click the lock again to lock the changes.
Now go back to idlemain.py and open it with any text editor (you could use Idle, TextEdit, or anything. Right under the import statement at the top is the code to change the default working directory. Read the comment if you like, then replace the single line of code under the comment with
os.chdir('path of your desired working directory')
Mine looks like this:
os.chdir('/Users/MyName/Documents/Python')
Save your changes (which should work because you gave yourself permission). Next time you start Idle, you should be in your desired working directory. You can check with the following commands:
import os
os.getcwd()

This ought to be the number one answer. I have been playing around this for an hour or more and nothing worked. Paul explains this perfectly. It's just like the PATH statement in Windows. I successfully imported a module by appending my personal "PythonModules" path/dir on my Mac (starting at "/users/etc") using a simple
import xxxx command in Idle.