Using command line args from different files in python - python

I recently discovered (Much to my surprised) you can call command line args in files other than the one that is explicitly called when you enter it.
So, you can run python file1.py abc in command line, and use sys.argv[1] to get the string 'abc' from within file2.py or file3.py.
I still feel like this shouldn't work, but I'm glad it does, since it saved me a lot of trouble.
But now I'd really appreciate an answer as to why/how this works. I had assumed that sys.argv[1] would be local to each file.

As for the how/why, sys is only imported once (when python starts up). When sys is imported, it's argv member gets populated with the commandline arguements. Subsequent import statements return the same sys module object so no matter where you import sys from, you'll always get the same object and therefore sys.argv will always be the same list no matter where you reference it in your application.
Whether you should be doing commandline parsing in more than one place is a different question. Generally, my answer would be "NO" unless you are only hacking together a script to work for the next 2 or 3 days. Anything that you expect to last should do all it's parsing up front (probably with a robust argument parser like argparse) and pass the data necessary for the various functions/classes to them from it's entry point.

Related

Python's help function executes another program "string.py"

I've been messing around with python a little bit and have created a program saved as string.py where I tested out some string functions.
In another program function.py in the same directory I wrote this code
def say_hi(first = 'John', last = 'Doe'):
"""Say hello."""
print('Hi {} {}!'.format(first, last))
help(say_hi)
which, however, executed the string.py program. I found out after some testing that renaming string.py to anything else solves the problem and the function.py program is executed as intended, but I'd like to understand why the help function executed the other program in the first place.
You are shadowing the name string, which is a built-in module: https://docs.python.org/3/library/string.html
string is a commonly used module. Lots of built-in functions in python run scripts that have import string in them, meaning it'll import your string.py and not the built-in one.
This is just yet another example of why it's a bad idea to name scripts or variables with names that already exist in standard Python.

Get Path To File of Caller From Within Library

I want to be able to get the path to the file which is importing my python library. How can I do that?
For example:
The user creates a file at C:\Users\Bob\bobsproject\main.py. From within the library, I want to be able to get the path to the file, and read it as a txt. How can I do that?
If you want to get the name of the driver script that is (possibly indirectly) loading your library, you can use the fact that python runs a script under the name __main__. You can get it from sys.modules just like any other module and access its __file__ attribute if it exists:
import sys
try:
print(sys.modules['__main__'].__file__)
except KeyError:
print('libray not loaded from script')
except AttributeError:
print('script not loaded from file')
The KeyError is unlikely to ever occur (not even if you run the script with python -m), but it's useful to be safe. The AttributeError is much more likely, and can easily be demonstrated with something like python -c.
If you want something more complex, like the file containing the code that actually called your library function, you will likely have to use the inspect module or similar. This will be even less robust as a matter of course, but may still suit your needs:
import inspect
module = inspect.getmodule(inspect.stack()[1][0])
try:
print(module.__file__)
except AttributeError:
print(f'module "{module.__name__}" not loaded from file')
Notice that inspect.getmodule explicitly uses the word "guess" in its official documentation, while inspect.stack can be a fidgety beast sometimes.
Code for second part referenced from here: https://stackoverflow.com/a/1095621/2988730.
Remember that there are two options here. If you place this code directly in your library module, it will be executed exactly once, when the module is first imported. If you place it in a function that the user can call directly, you will see the printouts every time. If you place the second snippet it in a utility function that you then call from your public module functions, don't forget to increment the frame index to reflect that:
module = inspect.getmodule(inspect.stack()[2][0])

How should I organize my scripts which are mostly the same?

So I'm new to Python and I need some help on how to improve my life. I learned Python for work and need to cut my workload a little. I have three different scripts which I run around 5 copies of at the same time all the time, they read XML data and add in information etc... However, when I make a change to a script I have to change the 5 other files too, which is annoying after a while. I can't just run the same script 5 times because each file needs some different parameters which I store as variables at the start in every script (different filepaths...).
But I'm sure theres a much better way out there?
A very small example:
script1.py
xml.open('c:\file1.xls')
while True:
do script...
script2.py
xml.open('c:\file2.xls')
while True:
do exactley the same script...
etc...
You'll want to learn about Python functions and modules.
A function is the solution to your problem: it bundles some functionality and allows you to call it to run it, with only minor differences passed as a parameter:
def do_something_with_my_sheet(name):
xml.open(name)
while True:
do script...
Elsewhere in your script, you can just call the function:
do_something_with_my_sheet('c:\file1.xls')
Now, if you want to use the same function from multiple other scripts, you can put the function in a module and import it from both scripts. For example:
This is my_module.py:
def do_something_with_my_sheet(name):
xml.open(name)
while True:
do script...
This is script1.py:
import my_module
my_module.do_something_with_my_sheet('c:\file1.xls')
And this could be script2.py (showing a different style of import):
from my_module import do_something_with_my_sheet
do_something_with_my_sheet('c:\file2.xls')
Note that the examples above assume you have everything sitting in a single folder, all the scripts in one place. You can separate stuff for easier reuse by putting your module in a package, but that's beyond the scope of this answer - look into it if you're curious.
You only need one script, that takes the name of the file as an argument:
import sys
xml.open(sys.argv[1])
while True:
do script...
Then run the script. Other variables can be passed as additional arguments, accessed via sys.argv[2], etc.
If there are many such parameters, it may be easier to save them in a configuration file, the pass the name of the configuration file as the single argument. Your script would then parse the file for all the information it needs.
For example, you might have a JSON file with contents like
{
"filename": "c:\file1.xls",
"some_param": 6,
"some_other_param": True
}
and your script would look like
import json
import sys
with open(sys.argv[1]) as f:
config = json.load(f)
xml.open(config['filename'])
while True:
do stuff using config['some_param'] and config['some_other_param']

Nested function causing troubles

I've got a Python script.
I've had several functions in this script which I decided to move to a 'package' folder beside the main script.
In this folder, I created a *.py file where I put all my functions.
I've placed an empty init.py near this file within the 'package' folder.
When starting the code of my main script with:
from package_folder.my_functions import *
the script works well when calling every functions from that file.
But when trying to import it directly:
import package_folder.my_functions
it doesn't seems to work as well as with the above technique.
The cause seems to be the fact that in the file wellmy_functions.py, I have a function that needs an other one, declared previously in that file.
I had this obscure error on that function that needs an other one:
TypeError: 'NoneType' object is not callable
Is this permissible and if not, how to manage this case?
It's generally not a good idea to use from module import *. Wildcard importing leads to namespace pollution; you imported more names than you need and if you accidentally refer to an imported name you may not get the NameError you wanted.
Also, if a future version of the library added additional names, you could end up masking other names, leading to strange bugs still:
Example
from my_mod1 import func1
from my_mod2 import *
If you upgrade my_mod2 and it now includes a my_mod2.func1 it'll replace the my_mod1.func1 import in the 1st line.

Pass variable between python scripts

I'm sure this is very simple but I've been unable to get it working correctly. I need to have my main python script call another python script and pass variables from the original script to the script that I've called
So for a simplistic example my first script is,
first.py
x = 5
import second
and my second script is,
second.py
print x
and I would expect it to print x but I get
NameError: name 'x' is not defined
I'm not sure if import is right way to achieve this, but if someone could shed light on it in a simple way that would be great!
thanks,
EDIT
After reading the comments I thought I would expand on my question. Aswin Murugesh answer fixes the import problem I was having, however the solution does not have the desired outcome as I can not seem to pass items in a list this way.
In first.py I have a list which I process as follows
for insert, (list) in enumerate(list, start =1):
'call second.py passing current list item'
I wanted to pass each item in the list to a second python file for further processing (web scraping), I didn't want to do this in first.py as this is meant to be the main 'scan' program which then calls other programs. I hope this now make more sense.
Thanks for the comments thus far.
When you call a script, the calling script can access the namespace of the called script. (In your case, first can access the namespace of second.) However, what you are asking for is the other way around. Your variable is defined in the calling script, and you want the called script to access the caller's namespace.
An answer is already stated in this SO post, in the question itself:
Access namespace of calling module
But I will just explain it here in your context.
To get what you want in your case, start off the called script with the following line:
from __main__ import *
This allows it to access the namespace (all variables and functions) of the caller script.
So now your calling script is, as before:
x=5
import second
and the called script is:
from __main__ import *
print x
This should work fine.
use the following script:
first.py:
x=5
second.py
import first
print first.x
this will print the x value. Always imported script data should be referenced with the script name, like in first.x
To avoid namespace pollution, import the variables you want individually: from __main__ import x, and so on. Otherwise you'll end up with naming conflicts you weren't aware of.
Try use exec
Python3.5:
first.py
x=5
exec(open('second.py').read())
second.py
print(x)
You can also pass x by using:
x=5
myVars = {'x':x}
exec(open('second.py').read(), myVars)
Not sure if this is a good way.
Finally,
I created a package for Python to solve this problem.
Install Guli from PIP.
$ pip install guli
Guli doesn't require installing any additional PIP package.
With the package you can
Guli can be used to pass between different Python scripts, between many processes or at the same script.
pass variables between main Process and another (Multiprocess) Process.
Pass variables between different Python scripts.
Pass variables between 'Main Process' and another (Multiprocess) Process.
Use variables at the same script.
Create / Delete / Edit - GuliVariables.
Example
import guli
import multiprocessing
string = guli.GuliVariable("hello").get()
print(string) # returns empty string ""
def my_function():
''' change the value from another process '''
guli.GuliVariable("hello").setValue(4)
multiprocessing.Process(target=my_function).start()
import time
time.sleep(0.01) # delay after process to catch the update
string = guli.GuliVariable("hello").get()
print(string) # returns "success!!!"
Hope I solved the problem for many people!

Categories

Resources