Develop Reference

Is there a way to be able to use a variable path using os

The goal is to run through a half stable and half variable path.
I am trying to run through a path (go to lowest folder which is called Archive) and fill a list with files that have a certain ending. This works quite well for a stable path such as this.
fileInPath='\\server123456789\provider\COUNTRY\CATEGORY\Archive
My code runs through the path (recursive) and lists all files that have a certain ending. This works well. For simplicity I will just print the file name in the following code.
import csv
import os
fileInPath='\\\\server123456789\\provider\\COUNTRY\\CATEGORY\\Archive
fileOutPath=some path
csvSeparator=';'
fileList = []
for subdir, dirs, files in os.walk(fileInPath):
for file in files:
if file[-3:].upper()=='PAR':
print (file)
The problem is that I can manage to have country and category to be variable e.g. by using *

The standard library module pathlib provides a simple way to do this.
Your file list can be obtained with
from pathlib import Path
list(Path("//server123456789/provider/".glob("*/*/Archive/*.PAR"))
Note I'm using / instead of \\ pathlib handles the conversion for you on windows.

Rename Single .txt File In Multiple Subdirectories

I have ~60 subdirectories in a single directory. Each of these contain thousands of files, but they all contain a file named test_all_results.txt.
What I would like to do is to rename each test_all_results.txt file so that it now has the name:
foldername_all_results.txt
What is the best way to do this?

Easily accomplished using Python os interface.
Assuming you are currently in the main directory:
import os
#get a list of all sub directories
subdir = os.listdir()
for dir in subdir:
if os.path.isdir(dir): #check if directory
os.chdir(dir) #move to sub directory
os.rename('test_all_results.txt', 'foldername_all_results.txt')
os.chdir('..') #return to main directory

Using python in Linux, make this:
import os
os.system("mv old_name.txt new_name.txt")
You can automatize with a loop, renaming all filenames.

You can do:
(change your code accordingly)
import os
# current directory is the target
direct = "."
for path, dirs, files in os.walk(direct):
for f in files:
if os.path.splitext(f)[0] == "test_all_results.txt":
os.rename(os.path.join(path, f), os.path.join(path, "foldername_all_results.txt"))

There's an answer that tells you to use the os.system() method, if you do decide to call Linux commands from Python, I'd advise that you use the subprocess module instead.
Here's how you'd run the mv command with two arguments using subprocess.call:
import subprocess
subprocess.call(["mv", "filename.txt", "new-name.txt"])
INFO: here's an old (but relevant) article that explains why it's dangerous to use these methods.
Good luck.

How to get the path of a program in python?

I'm doing a program in which Chimera needs to be opened, I'm opening it with:
def generate_files_bat(filename):
f = open(filename, 'w')
text = """echo off SET PATH=%PATH%;"C:\\Program Files (x86)\\Chimera 1.6.1\\bin" chimera colpeps.cmd"""
print >>f, text
f.close()
But I need to find Chimera apart from the computer the python program is running. Is there any way the path can be searched by the python program in any computer?

Generally speaking, I don't think it is such a good idea to search the path for a program. Imagine, for example that two different versions were installed on the machine. Are-you sure to find the right one? Maybe a configuraition file parsed with the standard module ConfigParser would be a better option?
Anyway, to go back to your question, in order to find a file or directory, you could try to use os.walk which recursively walks trough a directory tree.
Here is an example invoking os.walk from a generator, allowing you to collect either the first or all matching file names. Please note that the generator result is only based on file name. If you require more advanced filtering (say, to only keep executable files), you will probably use something like os.stat() to extend the test.
import os
def fileInPath(name, root):
for base, dirs, files in os.walk(root):
if name in files:
yield os.path.join(base, name)
print("Search for only one result:")
print(next(fileInPath("python", "/home/sylvain")))
print("Display all matching files:")
print([i for i in fileInPath("python", "/home/sylvain")])

There is which for Linux and where for Windows. They both give you the path to the executable, provided it lies in a directory that is 'searched' by the console (so it has to be in %PATH% in case of Windows)

There is a package called Unipath, that does elegant, clean path calculations.
Have look here for the AbstractPath constructor
Example:
from unipath import Path
prom_dir = Path(__file__)

Python operating on files in a folder - 'for file in folder'

I know a folder's path, and for every file in the folder I would like to do some operations. So essentially what I'm looking for is a for file in folder type of code that gives me access to the files in variables.
What is the Python way of doing this?
Thanks
EDIT - example: my folder will contain a bunch of XML files, and I have a python routine already to parse them into variables I need.

This will allow you to access and print all the file names in your current directory:
import os
for filename in os.listdir('.'):
print filename
The os module contains much more information about the various functions available. The os.listdir() function can also take any other paths you want to specify.

Does the glob library look helpful?
It will perform some pattern matching, and accepts both absolute and relative addresses.
>>> import glob
>>> for file in glob.glob("*.xml"): # only loops over XML documents
print file

For people coming at this from a python version 3.5 or later, we now have the superior os.scandir() which has tremendous performance improvements over os.listdir()
For more information about the improvements/benefits, check out https://benhoyt.com/writings/scandir/

How can I find path to given file?

I have a file, for example "something.exe" and I want to find path to this file
How can I do this in python?

Perhaps os.path.abspath() would do it:
import os
print os.path.abspath("something.exe")
If your something.exe is not in the current directory, you can pass any relative path and abspath() will resolve it.

use os.path.abspath to get a normalized absolutized version of the pathname
use os.walk to get it's location
import os
exe = 'something.exe'
#if the exe just in current dir
print os.path.abspath(exe)
# output
# D:\python\note\something.exe
#if we need find it first
for root, dirs, files in os.walk(r'D:\python'):
for name in files:
if name == exe:
print os.path.abspath(os.path.join(root, name))
# output
# D:\python\note\something.exe

if you absolutely do not know where it is, the only way is to find it starting from root c:\
import os
for r,d,f in os.walk("c:\\"):
for files in f:
if files == "something.exe":
print os.path.join(r,files)
else, if you know that there are only few places you store you exe, like your system32, then start finding it from there. you can also make use of os.environ["PATH"] if you always put your .exe in one of those directories in your PATH variable.
for p in os.environ["PATH"].split(";"):
for r,d,f in os.walk(p):
for files in f:
if files == "something.exe":
print os.path.join(r,files)

Just to mention, another option to achieve this task could be the subprocess module, to help us execute a command in terminal, like this:
import subprocess
command = "find"
directory = "/Possible/path/"
flag = "-iname"
file = "something.foo"
args = [command, directory, flag, file]
process = subprocess.run(args, stdout=subprocess.PIPE)
path = process.stdout.decode().strip("\n")
print(path)
With this we emulate passing the following command to the Terminal:
find /Posible/path -iname "something.foo".
After that, given that the attribute stdout is binary string, we need to decode it, and remove the trailing "\n" character.
I tested it with the %timeit magic in spyder, and the performance is 0.3 seconds slower than the os.walk() option.
I noted that you are in Windows, so you may search for a command that behaves similar to find in Unix.
Finally, if you have several files with the same name in different directories, the resulting string will contain all of them. In consequence, you need to deal with that appropriately, maybe using regular expressions.

This is really old thread, but might be useful to someone who stumbles across this. In python 3, there is a module called "glob" which takes "egrep" style search strings and returns system appropriate pathing (i.e. Unix\Linux and Windows).
https://docs.python.org/3/library/glob.html
Example usage would be:
results = glob.glob('./**/FILE_NAME')
Then you get a list of matches in the result variable.

Uh... This question is a bit unclear.
What do you mean "have"? Do you have the name of the file? Have you opened it? Is it a file object? Is it a file descriptor? What???
If it's a name, what do you mean with "find"? Do you want to search for the file in a bunch of directories? Or do you know which directory it's in?
If it is a file object, then you must have opened it, reasonably, and then you know the path already, although you can get the filename from fileob.name too.