I have a script 'preprocessing.py' containing the function for text preprocessing:
def preprocess():
#...some code here
with open('stopwords.txt') as sw:
for line in sw.readlines():
stop_words.add(something)
#...some more code than doesn't matter
return stop_words
Now I want to use this function in another Python script.
So, I do the following:
import sys
sys.path.insert(0, '/path/to/first/script')
from preprocessing import preprocess
x = preprocess(my_text)
Finally, I end up with the issue:
IOError: [Errno 2] No such file or directory: 'stopwords.txt'
The problem is surely that the 'stopwords.txt' file is located next to the first script, not the second.
Is there any way to specify the path to this file, not making any changes to the script 'preprocessing.py'?
Thank you.
Since you're running on a *nix like system, it seems, why not use that marvellous environment to glue your stuff together?
cat stopwords.txt | python preprocess.py | python process.py
Of course, your scripts should just use the standard input, and produce just standard output. See! Remove code and get functionality for free!
The simplest, and possibly most sensible way is to pass in the fully pathed filename:
def preprocess(filename):
#...some code here
with open(filename) as sw:
for line in sw.readlines():
stop_words.add(something)
#...some more code than doesn't matter
return stop_words
Then you can call it appropriately.
Looks like you can put
import os
os.chdir('path/to/first/script')
in your second script. Please try.
import os
def preprocess():
#...some code here
# get path in same dir
path = os.path.splitext(__file__)
# join them with file name
file_id = os.path.join(path, "stopwords.txt")
with open(file_id) as sw:
for line in sw.readlines():
stop_words.add(something)
#...some more code than doesn't matter
return stop_words
Related
I have script1, in there the String "path" gets set. Then the script1 runs blender as a subprocess, together with script2.
Now script2 needs to have access to the String "path", how can I do this?
Currently im Saving the string to a text file and then accesing it from script2 but I know this solution is very very ugly.
maybe someone has an idea ? :)
script1:
path=("/example/ex/")
subprocess.run([blenderpath, "--background", blenderscene, "--python", scriptpath])
script2 (atm just reading out the txt file with the right path, but that's not how I want it to be):
file=open("Blabla")
file_name = fiel.readline()
mat_path = file_name
def prepscene(mat_path)
It works right now with the text file, but If I try to import the variable into the second script it won't work, if I try to somehow start the blender script with it it also won't work.
import sys # to get command line args
argv = sys.argv
argv will be array
If you just want to access the varible you can use importing.
ScriptA:
...
path = "/example/ex/"
...
ScriptB:
from .ScriptA import path
(this only works if both scripts are in the same directory)
Let us have a look at the Directory structure of my Python Package
packman
weights.py
functions:
weigh()
force()
relatives.py
functions:
roll()
torque()
__init__.py
data
work.txt
rastor.txt
Now I have two questions.
Firstly suppose I want to access work.txt, from the function weigh() inside weights.py how would I address it?
I initially tried with this
f = open("data/work.txt")
While this method does succesfully work when the code is run inside main. However it fails to find the file when it is used as a package and it raises the issue
FileNotFoundError: [Errno 2] No such file or directory: 'data/work.txt'
How should I write the address of work.txt to make it more universal?
My other question is when I want to call the function weigh() of weights.py from the function roll() inside relatives.py, how would I do it?
I usually have a main.py or similar single entry point for my applications. Then I can do something like this to get the path for my application:
import os
app_location = os.path.dirname(os.path.abspath(__file__))
Now you can pass this location to your other modules or perhaps even use the same idea in them to get their location. Since you now have this location, you can easily do something like this:
data_location = os.path.join(app_location, 'data', 'work.txt')
with open(data_location) as f:
# do something with the file object
for line in f:
print(line)
As for your second question, just import weights into your relatives.py script and call weights.weigh()
Taking an online class on python. typed in from what i can tell the exact some code as the prof. but when i run it, i dont get anything back. not even a error. it just starts $ again. nothing happens at all.
import os
def rfile():
file_list = os.listdir(r"C:/home/zorba/Downloads/prank(2)/prank")
print(file_list)
it is a simple code but i cant get it to do anything. not even an error. i run it and nothing happens. im using linux so maybe im using lisdir() wrong. i dont know but i can continue with this lesson without this code working.
the location of the file is home/zorba/Downloads/prank (2) and the file under prank (2) is called prank.
You don't just define functions, you have to "call" them:
import os
def rfile():
file_list = os.listdir(r"C:/home/zorba/Downloads/prank(2)/prank")
print(file_list)
rfile() # Actually call the function!!
If you are using Linux then most likely the path
C:/home/zorba/Downloads/prank(2)/prank
does not exist
Replace it with Linux path like
/home/someuser/somedir
glob allows to use * as in Unix, so you can search, for example, for files in a directory staring with pr, like this: pr*.
Try this:
import glob
file_list= glob.glob(r'C:/home/zorba/Downloads/prank(2)/*')
I am working on project euler and wanted to time all of my code. What I have is directory of files in the form 'problemxxx.py' where xxx is the problem number. Each of these files has a main() function that returns the answer. So I have created a file called run.py, located in the same directory as the problem files. I am able to get the name of the file through command prompt. But when I try to import the problem file, I continue to get ImportError: No module named problem. Below is the code for run.py so far, along with the command prompt used.
# run.py
import sys
problem = sys.argv[1]
import problem # I have also tired 'from problem import main' w/ same result
# will add timeit functions later, but trying to get this to run first
problem.main()
The command prompts that I have tried are the following: (both of which give the ImportError stated above)
python run.py problem001
python run.py problem001.py
How can I import the function main() from the file problem001.py? Does importing not work with the file name stored as a variable? Is there a better solution than trying to get the file name through command prompt? Let me know if I need to add more information, and thank you for any help!
You can do this by using the __import__() function.
# run.py
import sys
problem = __import__(sys.argv[1], fromlist=["main"]) # I have also tired 'from problem import main' w/ same result
problem.main()
Then if you have problem001.py like this:
def main():
print "In sub_main"
Calling python run.py problem001 prints:
In sub_main
A cleaner way to do this (instead of the __import__ way) is to use the importlib module. Your run.py needs to changes:
import importlib
problem = importlib.import_module(sys.argv[1])
Alternatives are mentioned in this question.
For sure! You can use __ import_ built-in function like __import__(problem). However this is not recommended to use, because it is not nice in terms of coding-style. I think if you are using this for testing purposes then you should use unittest module, either way try to avoid these constructions.
Regards
You can use exec() trick:
import sys
problem = sys.argv[1]
exec('import %s' % problem)
exec('%s.main()' % problem)
I have a very old fortran file, which is too complex for me to convert into python. So I have to compile the file and run it via python.
For the fortran file to work it requires 3 input values on 3 lines from the mobcal.run file. They are as follows:
line 1 - Name of file to run
line 2 - Name of output file
line 3 - random seed number
I change the input values per worker in the run() function.
When I run my script (see below), only 2 output file were created, but all 32 processers were running which i found out via the top command
I'm guessing the issue is here, is that there was not enough time to change the mobcal.run file for each worker.
The only solution I have come up with so far is to put a time.sleep(random.randint(1,100)) at the beginning of the run() function. But I dont find this solution very elegant and may not always work as two workers may have the same random.randint, is there a more pythonic way to solve this?
def run(mfj_file):
import shutil
import random
import subprocess
#shutil.copy('./mfj_files/%s' % mfj_file, './')
print 'Calculating cross sections for: %s' % mfj_file[:-4]
with open('mobcal.run', 'w') as outf:
outf.write(mfj_file+'\n'+mfj_file[:-4]+'.out\n'+str(random.randint(5000000,6000000)))
ccs = subprocess.Popen(['./a.out'])
ccs.wait()
shutil.move('./'+mfj_file[:-4]+'.out', './results/%s.out' % mfj_file[:-4])
def mobcal_multi_cpu():
from multiprocessing import Pool
import os
import shutil
mfj_list = os.listdir('./mfj_files/')
for f in mfj_list:
shutil.copy('./mfj_files/'+f, './')
if __name__ == '__main__':
pool = Pool(processes=32)
pool.map(run,mfj_list)
mobcal_multi_cpu()
I assume your a.out looks in the current working directory for its mobcal.run. If you run each instance in it's own directory then each process can have it's own mobcal.run without clobbering the others. This isn't necessarily the most pythonic way but it's the most unixy.
import tempfile
import os
def run(mjf_file):
dir = tempfile.mkdtemp(dir=".")
os.chdir(dir)
# rest of function here
# create mobcal.run in current directory
# while changing references to other files from "./" to "../"
Create several directories, with one mobcal.run each, and run your fortran program into them instead.
If you need a sleep() in multiprocessing you are doing it wrong.