Develop Reference

How do I specify the path to the library to run a script from another script?

I need to check the change of one parameter, if it has changed - I need to restart the script.
code:
import subprocess, os.path, time
from engine.db_manager import DbManager
DB = DbManager(os.path.abspath(os.path.join('../webserver/db.sqlite3')))
tmbot = None
telegram_config = DB.get_config('telegram')
old_telegram_token = ''
vkbot = None
vk_config = DB.get_config('vk')
old_vk_token = ''
while True:
telegram_config = DB.get_config('telegram')
if old_telegram_token != telegram_config['token']:
if vkbot != None:
tmbot.terminate()
tmbot = subprocess.Popen(['python', 'tm/tm_bot.py'])
old_telegram_token = telegram_config['token']
print('telegram token was updated')
vk_config = DB.get_config('vk')
if old_vk_token != vk_config['token']:
if vkbot != None:
vkbot.terminate()
vkbot = subprocess.Popen(['python', 'vk/vk_bot.py'])
old_vk_token = vk_config['token']
print('vk token was updated')
time.sleep(30)
I get errors:
enter image description here

While there might be subtle differences between unix and windows, the straight-up answer is: you can use PYTHONPATH environment variable to let python know where to look for libraries.
However, if you use venv, I'd recommend activating it first, or calling the relevant binary instead of setting the environment variable.
Consider this scenario: you have a venv at /tmp/so_demo/venv, and you try to run this file:
$ cat /tmp/so_demo/myscript.py
import requests
print("great success!")
Running the system python interpreter will not find the requests module, and will yield the following error:
$ python3 /tmp/so_demo/myscript.py
Traceback (most recent call last):
File "/tmp/so_demo/myscript.py", line 1, in <module>
import requests
ModuleNotFoundError: No module named 'requests'
As I have installed requests in the venv, if I provide the path to python, it will know where to look:
$ PYTHONPATH='/tmp/so_demo/venv/lib/python3.8/site-packages/' python3 /tmp/so_demo/myscript.py
great success!
but using the binary inside the venv is better, as you don't need to be familiar with the internal venv directory paths, and is much more portable (notice that the path I provided earlier depicts the minor version):
$ /tmp/so_demo/venv/bin/python /tmp/so_demo/myscript.py
great success!

How to compile multiple subprocess python files into single .exe file using pyinstaller

I have a similar question to this one:Similar Question.
I have a GUI and where the user can input information and the other scripts use some of that information to run.I have 4 different scripts for each button. I run them as a subprocess so that the main gui doesn’t act up or say that it’s not responding. This is an example of what I have since the code is really long since I used PAGE to generate the gui.
###Main.py#####
import subprocess
def resource_path(relative_path):
#I got this from another post to include images but I'm also using it to include the scripts"
try:
# PyInstaller creates a temp folder and stores path in _MEIPASS
base_path = sys._MEIPASS
except Exception:
base_path = os.path.abspath(".")
return os.path.join(base_path, relative_path)
Class aclass:
def get_info(self):
global ModelNumber, Serial,SpecFile,dateprint,Oper,outputfolder
ModelNumber=self.Model.get()
Serial=self.SerialNumber.get()
outputfolder=self.TEntry2.get()
SpecFile= self.Spec_File.get()
return ModelNumber,Serial,SpecFile,outputfolder
def First(self):
aclass.get_info(self) #Where I use the resource path function
First_proc = subprocess.Popen([sys.executable, resource_path('first.py'),str(ModelNumber),str(Serial),str(path),str(outputfolder)])
First_proc.wait()
#####First.py#####
import numpy as np
import scipy
from main import aclass
ModelNumber = sys.argv[1]
Serial = sys.argv[2]
path = sys.argv[3]
path_save = sys.argv[4]
and this goes on for my second,third, and fourth scripts.
In my spec file, I added:
a.datas +=[('first.py','C\\path\\to\\script\\first.py','DATA')]
a.datas +=[('main.py','C\\path\\to\\script\\main.py','DATA')]
this compiles and it works, but when I try to convert it to an .exe, it crashes because it can't import first.py properly and its own libraries (numpy,scipy....etc). I've tried adding it to the a.datas, and runtime_hooks=['first.py'] in the spec file...and I can't get it to work. Any ideas? I'm not sure if it's giving me this error because it is a subprocess.

Assuming you can't restructure your app so this isn't necessary (e.g., by using multiprocessing instead of subprocess), there are three solutions:
Ensure that the .exe contains the scripts as an (executable) zipfile—or just use pkg_resources—and copy the script out to a temporary directory so you can run it from there.
Write a multi-entrypoint wrapper script that can be run as your main program, and also run as each script—because, while you can't run a script out of the packed exe, you can import a module out of it.
Using pkg_resources again, write a wrapper that runs the script by loading it as a string and running it with exec instead.
The second one is probably the cleanest, but it is a bit of work. And, while we could rely on setuptools entrypoints to some of the work, trying to explain how to do this is much harder than explaining how to do it manually,1 so I'm going to do the latter.
Let's say your code looked like this:
# main.py
import subprocess
import sys
spam, eggs = sys.argv[1], sys.argv[2]
subprocess.run([sys.executable, 'vikings.py', spam])
subprocess.run([sys.executable, 'waitress.py', spam, eggs])
# vikings.py
import sys
print(' '.join(['spam'] * int(sys.argv[1])))
# waitress.py
import sys
import time
spam, eggs = int(sys.argv[1]), int(sys.argv[2]))
if eggs > spam:
print("You can't have more eggs than spam!")
sys.exit(2)
print("Frying...")
time.sleep(2)
raise Exception("This sketch is getting too silly!")
So, you run it like this:
$ python3 main.py 3 4
spam spam spam
You can't have more eggs than spam!
We want to reorganize it so there's a script that looks at the command-line arguments to decide what to import. Here's the smallest change to do that:
# main.py
import subprocess
import sys
if sys.argv[1][:2] == '--':
script = sys.argv[1][2:]
if script == 'vikings':
import vikings
vikings.run(*sys.argv[2:])
elif script == 'waitress':
import waitress
waitress.run(*sys.argv[2:])
else:
raise Exception(f'Unknown script {script}')
else:
spam, eggs = sys.argv[1], sys.argv[2]
subprocess.run([sys.executable, __file__, '--vikings', spam])
subprocess.run([sys.executable, __file__, '--waitress', spam, eggs])
# vikings.py
def run(spam):
print(' '.join(['spam'] * int(spam)))
# waitress.py
import sys
import time
def run(spam, eggs):
spam, eggs = int(spam), int(eggs)
if eggs > spam:
print("You can't have more eggs than spam!")
sys.exit(2)
print("Frying...")
time.sleep(2)
raise Exception("This sketch is getting too silly!")
And now:
$ python3 main.py 3 4
spam spam spam
You can't have more eggs than spam!
A few changes you might want to consider in real life:
DRY: We have the same three lines of code copied and pasted for each script, and we have to type each script name three times. You can just use something like __import__(sys.argv[1][2:]).run(sys.argv[2:]) with appropriate error handling.
Use argparse instead of this hacky special casing for the first argument. If you're already sending non-trivial arguments to the scripts, you're probably already using argparse or an alternative anyway.
Add an if __name__ == '__main__': block to each script that just calls run(sys.argv[1:]), so that during development you can still run the scripts directly to test them.
I didn't do any of these because they'd obscure the idea for this trivial example.
1 The documentation is great as a refresher if you've already done it, but as a tutorial and explanatory rationale, not so much. And trying to write the tutorial that the brilliant PyPA guys haven't been able to come up with for years… that's probably beyond the scope of an SO answer.

Get the list of all environment names of Elastic beanstalk using Python

I am using below python code to get the ApplicationName and EnvironmentName of the AWS Elastic Beanstalk. Can anyone please let me know how to print/get all the environment names by using for loop or some other way. Thanks
#!/usr/bin/env python3
import boto3
import json
def get_info():
try:
eb = boto3.client('elasticbeanstalk',"us-east-1")
response = eb.describe_environments()
app_name=(response['Environments'][0]['ApplicationName'])
env_name=(response['Environments'][0]['EnvironmentName'])
print app_name
print env_name
except:
raise
if __name__ == '__main__':
get_info()

I am not sure about the code, as I have no way to test it presently, however, assuming that response['Environments'] is a list, the following should work. Please note that, if it is not a list then it will not work or you may have to change the code a bit to get the right result.
#!/usr/bin/env python3
import boto3
import json
def get_info():
try:
eb = boto3.client('elasticbeanstalk',"us-east-1")
response = eb.describe_environments()
for item in response['Environments']:
app_name = response['Environments'][item]['ApplicationName']
env_name = response['Environments'][item]['EnvironmentName']
print app_name
print env_name
except:
raise
if __name__ == '__main__':
get_info()
Also, given the fact that you are using a try-except block, I will say that it is always wiser to catch the particular exception you want to catch. I am not sure which that will be, but a all-catch except is generally not a good idea.

Thanks #SRC; I made below changes to your script to get it working for me.
app_name = item['ApplicationName']
env_name = item['EnvironmentName']

Running a Python unittest test on multiple files

I want to add unit testing to the assessment of my high school programming class.
If I have twenty submissions of files that look like this:
def calculateReturn(principle, rate, freq, time):
final = principle * (1 + (rate/freq)) ** (freq * time)
return final
Can I use a test case like this?
import unittest
class test(unittest.TestCase):
def test1(self):
value = calculateReturn(5000, 0.05, 12, 11)
self.assertAlmostEqual(value, 8235.05, 2)
if __name__ == '__main__':
unittest.main()
How do I run this one simple test on twenty modules?
FURTHER INFORMATION
For testing I have created three "submissions" all of which show different ways of calculating x^y.
submission1.py:
from math import pow
def powerFunction(base, power):
result = pow(base, power)
return result
submission2.py:
def powerFunction(base, power):
result = base ** power
return result
submission3.py:
def powerFunction(base, power):
result = 1
for i in range(power):
result = result * base
return result
The test code is:
import unittest
import importlib
class MyTest(unittest.TestCase):
def setUp(self):
pass
def test_power_3_4(self):
self.assertEqual(module.powerFunction(2, 3), 8)
files = ['submission1', 'submission2', 'submission3']
for file in files:
module = importlib.import_module(file)
print module
unittest.main()
if the test code is run the console output shows only submission1 being tested:
/System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7
/Users/staff/PycharmProjects/UnitTest/powerTest.py
<module 'submission1' from '/Users/staff/PycharmProjects/UnitTest/
submission1.pyc'>
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
Process finished with exit code 0
Interestingly if I don't use unit testing I can correctly import and test using this approach:
import importlib
files = ['submission1', 'submission2', 'submission3']
for file in files:
module = importlib.import_module(file)
print module
print module.powerFunction(2,3)
The console output here is:
/System/Library/Frameworks/Python.framework/Versions/2.7/bin/
python2.7 /Users/staff/PycharmProjects/UnitTest/importlib1.py
<module 'submission1' from '/Users/staff/PycharmProjects/UnitTest/
submission1.pyc'>
8.0
<module 'submission2' from '/Users/staff/PycharmProjects/UnitTest/
submission2.pyc'>
8
<module 'submission3' from '/Users/staff/PycharmProjects/UnitTest/
submission3.pyc'>
8
Process finished with exit code 0
It may well be that the unittest module is not the best approach here but I'm still interested on how to implement it.

You can use importlib to load Python modules from specific files, then run the test cases on each one.
glob may be helpful to create the list of files.

Given that this has been active for a month with no answers I have come the the realisation that it is because I'm asking for the wrong thing.
From what I can gather, unittest is for running a suite of tests on a single application. It is not designed to run a single test on a suite of applications.
John's suggestion to investigate importlib helped set me on the path to success. Thanks John.
The code posted in the original post update seems to be the most appropriate solution to my problem.

python the hard way ex 50: "permission denied"

Per the book, I successfully installed lpthw.web, then created the module named app.py. First I typed it exactly, then tried cutting and pasting from the website, to be 100% sure.
When I run app.py on my OS Yosemite Mac, I get the message
Permission denied.
I think it has to do with the command import web. I tested this by commenting out all of the lines except this one and I got the same error. However, I made a simple file, put it in bin and was able to run it. Not sure what’s going on. How do I change this?
Thanks for your help.
For reference, here is the code for bin/app.py:
import web
urls = (
'/', 'index'
)
app = web.application(urls, globals())
class index:
def GET(self):
greeting = "hello world"
return greeting
if __name__ == "__main__":
app.run()