Develop Reference

exec() python command stops the whole execution

I am trying to run a script that sequentially changes some parameters in a config file (MET_config_EEv40.cfg) and runs a script ('IS_MET_EEv40_RAW.py') that retrieves these new config parameters:
config_filename = os.getcwd() + '/MET_config_EEv40.cfg'
import sys
parser = configparser.ConfigParser()
parser.read('MET_config_EEv40.cfg')
parser.set('RAW', 'product', 'ERA')
parser.set('RAW', 'gee_product', 'ECMWF/ERA5_LAND/HOURLY')
parser.set('RAW', 'indicator', 'PRCP')
parser.set('RAW', 'resolution', '11110')
with open('MET_config_EEv40.cfg', 'w') as configfile:
parser.write(configfile)
## execute file
import sys
os.system(exec(open('IS_MET_EEv40_RAW.py').read()))
#exec(open('IS_MET_EEv40_RAW.py').read())
print('I am here')
After this execution, I get the output of my script as expected:
Period of Reference: 2005 - 2019
Area of Interest: /InfoSequia/GIS/ink/shp_basin_wgs84.shp
Raw data is up to date. No new dates available in raw data
Press any key to continue . . .
But it never prints the end line: I am here, so that means that after the execution of the script, the algorithm is terminated. That is not what I want it to do, as I would like to be able to change some other config parameters and run the script again.
That output is showed because of this line of the code:
if (delta.days<=1):
sys.exit('Raw data is up to date. No new dates available in raw data')
So could be that sys.exit is ending both processes? Any ideas to replace sys.exit() inside the code to avoid this?
Im executing this file from a .bat file that contains the following:
#echo OFF
docker exec container python MET/PRCPmain.py
pause

exec(source, globals=None, locals=None, /) does
Execute the given source in the context of globals and locals.
So
import sys
exec("sys.exit(0)")
print("after")
is same as writing
import sys
sys.exit(0)
print("after")
which obviously terminate and does not print after.
exec has optional argument globals which you can use to provide your alternative to sys for example
class MySys:
def exit(self, *args):
pass
exec("sys.exit(0)",{"sys":MySys()})
print("after")
which does output
after
as it does use exit from MySys instance. If your codes make use of other things from sys and want it to work normally you would need method mimicking sys function in MySys class

trying to parse ldif file using python-ldap ldifparser

I have ldif file as below, I want to extract only the dn and changetype here using the ldif parser of python-ldap package.
dn: cn=abc, cn=def, cn="dc=grid,dc=mycompany,dc=com", cn=tree, cn=config
changetype: add
objectClass: top
cn: abc
description: myserver
I have written parser code as below:
from ldif import LDIFParser, LDIFRecordList
parser = LDIFRecordList(open("cluster1.ldif", "r"))
parser.parse()
for dn, entry in parser.all_records:
print(dn)
print(entry)
but this reads everything and skips the changetype key am not sure what is causing that. Is there a better way to parse the ldif file?
Adding output of requested commands in comment:
python -c 'import sys; import ldap; print("\n".join([sys.version, ldap.__version__]))'
2.7.5 (default, May 31 2018, 09:41:32)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-28)]
2.4.15

Update
You appear to be using a very old version of the python-ldap module. Version 2.4.15 was released over seven years ago. The current release is 3.3.1, which is what you get if you pip install python-ldap.
Python 2 itself went end-of-life in Janurary 2020, and version 2.7.5 was released back in 2013.
The software you're working with is very old and has bugs that were fixed in more recent versions. You should upgrade.
As I mentioned in comments, I'm not able to reproduce the behavior you've described. If I put your sample LDIF content into cluster1.ldif, I see:
>>> from ldif import LDIFParser, LDIFRecordList
>>> parser = LDIFRecordList(open("cluster1.ldif", "r"))
>>> parser.parse()
>>> for dn, entry in parser.all_records:
... print(dn)
... print(entry)
...
cn=abc, cn=def, cn="dc=grid,dc=mycompany,dc=com", cn=tree, cn=config
{'changetype': [b'add'], 'objectClass': [b'top '], 'cn': [b'abc'], 'description': [b'myserver']}
>>>
You asked about an alternative to use LDIFRecordList. You can of course write your own handler by subclassing LDIFParser, but the underlying LDIF parsing is still going to be the same. That would look something like:
from ldif import LDIFParser, LDIFRecordList
class MyParser(LDIFParser):
def __init__(self, *args, **kwargs):
self.records = []
super().__init__(*args, **kwargs)
def handle(self, dn, entry):
self.records.append((dn, entry))
parser = MyParser(open("cluster1.ldif", "r"))
parser.parse()
for dn, entry in parser.records:
print(dn)
print(entry)
...but that's really just re-implementing LDIFRecordList, so I don't think you obtain any benefit from doing this.
I'm using python 3.9.6 and python-ldap 3.3.1. If you continue to see different behavior, would you update your question to include the output of python -c 'import sys; import ldap; print("\n".join([sys.version, lda p.__version__]))'

If I understand right, you want something like this:
#!/usr/bin/python
from ldif import LDIFParser
from sys import argv
class myParser( LDIFParser ):
def handle( self, dn, entry ):
print( "DN:", dn )
print( "CT:", entry['changetype'], "\n" )
parser = myParser( open( argv[1] ) )
parser.parse()
LDIFParser.handle() method is invoked for every DN of your LDIF and in it you have direct access to DN (dn variable) and AVPs (entry variable), latter being a dictionary of attribute names with associated values as byte arrays.
Does that help?

import python module when launching from VBA

It is the first time I write as I really didn't find any solution to my issue.
I want to allow my user to launch some Python program from Excel.
So i have this VBA code at some point:
lg_ErrorCode = wsh.Run(str_PythonPath & " " & str_PythonArg, 1, bl_StopVBwhilePython)
If lg_ErrorCode <> 0 Then
MsgBox "Couldn't run python script! " _
+ vbCrLf + CStr(lg_ErrorCode)
Run_Python = False
End If
str_PythonPath = "C:\Python34\python.exe C:\Users\XXXX\Documents\4_Python\Scan_FTP\test.py"
str_PythonArg = "arg1 arg2"
After multiple testing, the row in error in Python is when I try to import another module (I precise that this VBA code is working without the below row in Python):
import fct_Ftp as ftp
The architecture of the module is as follow:
4_Python
-folder: Scan_FTP
- file: test.py (The one launch from VBA)
-file: fct_Ftp.py
(For information, I change the architecture of the file, and try to copy the file at some other position just to test without success)
The import has no problem when I launch Test.py directly with:
import sys, os
sys.path.append('../')
But from VBA, this import is not working.
So I figured out this more generic solution, that dont work as well from Excel/VBA
import sys, os
def m_importingFunction():
str_absPath = os.path.abspath('')
str_absPathDad = os.path.dirname(str_absPath)
l_absPathSons = [os.path.abspath(x[0]) for x in os.walk('.')]
l_Dir = l_absPathSons + [str_absPathDad]
l_DirPy = [Dir for Dir in l_Dir if 'Python' in Dir]
for Dir in l_DirPy:
sys.path.append(Dir)
print(Dir)
m_importingFunction()
try:
import fct_Ftp as ftp
# ftp = __import__ ("fct_Ftp")
write += 'YAAAA' # write a file YAAAA from Python
except:
write += 'NOOOOOO' # write a file NOOOOO from VBA
f= open(write + ".txt","w+")
f.close()
Can you please help me as it is a very tricky questions ?
Many thanks to you guys.

You are able to start your program from the command line?
Why not create a batch file with excel which you then start in a shell?

How to compile multiple subprocess python files into single .exe file using pyinstaller

I have a similar question to this one:Similar Question.
I have a GUI and where the user can input information and the other scripts use some of that information to run.I have 4 different scripts for each button. I run them as a subprocess so that the main gui doesn’t act up or say that it’s not responding. This is an example of what I have since the code is really long since I used PAGE to generate the gui.
###Main.py#####
import subprocess
def resource_path(relative_path):
#I got this from another post to include images but I'm also using it to include the scripts"
try:
# PyInstaller creates a temp folder and stores path in _MEIPASS
base_path = sys._MEIPASS
except Exception:
base_path = os.path.abspath(".")
return os.path.join(base_path, relative_path)
Class aclass:
def get_info(self):
global ModelNumber, Serial,SpecFile,dateprint,Oper,outputfolder
ModelNumber=self.Model.get()
Serial=self.SerialNumber.get()
outputfolder=self.TEntry2.get()
SpecFile= self.Spec_File.get()
return ModelNumber,Serial,SpecFile,outputfolder
def First(self):
aclass.get_info(self) #Where I use the resource path function
First_proc = subprocess.Popen([sys.executable, resource_path('first.py'),str(ModelNumber),str(Serial),str(path),str(outputfolder)])
First_proc.wait()
#####First.py#####
import numpy as np
import scipy
from main import aclass
ModelNumber = sys.argv[1]
Serial = sys.argv[2]
path = sys.argv[3]
path_save = sys.argv[4]
and this goes on for my second,third, and fourth scripts.
In my spec file, I added:
a.datas +=[('first.py','C\\path\\to\\script\\first.py','DATA')]
a.datas +=[('main.py','C\\path\\to\\script\\main.py','DATA')]
this compiles and it works, but when I try to convert it to an .exe, it crashes because it can't import first.py properly and its own libraries (numpy,scipy....etc). I've tried adding it to the a.datas, and runtime_hooks=['first.py'] in the spec file...and I can't get it to work. Any ideas? I'm not sure if it's giving me this error because it is a subprocess.

Assuming you can't restructure your app so this isn't necessary (e.g., by using multiprocessing instead of subprocess), there are three solutions:
Ensure that the .exe contains the scripts as an (executable) zipfile—or just use pkg_resources—and copy the script out to a temporary directory so you can run it from there.
Write a multi-entrypoint wrapper script that can be run as your main program, and also run as each script—because, while you can't run a script out of the packed exe, you can import a module out of it.
Using pkg_resources again, write a wrapper that runs the script by loading it as a string and running it with exec instead.
The second one is probably the cleanest, but it is a bit of work. And, while we could rely on setuptools entrypoints to some of the work, trying to explain how to do this is much harder than explaining how to do it manually,1 so I'm going to do the latter.
Let's say your code looked like this:
# main.py
import subprocess
import sys
spam, eggs = sys.argv[1], sys.argv[2]
subprocess.run([sys.executable, 'vikings.py', spam])
subprocess.run([sys.executable, 'waitress.py', spam, eggs])
# vikings.py
import sys
print(' '.join(['spam'] * int(sys.argv[1])))
# waitress.py
import sys
import time
spam, eggs = int(sys.argv[1]), int(sys.argv[2]))
if eggs > spam:
print("You can't have more eggs than spam!")
sys.exit(2)
print("Frying...")
time.sleep(2)
raise Exception("This sketch is getting too silly!")
So, you run it like this:
$ python3 main.py 3 4
spam spam spam
You can't have more eggs than spam!
We want to reorganize it so there's a script that looks at the command-line arguments to decide what to import. Here's the smallest change to do that:
# main.py
import subprocess
import sys
if sys.argv[1][:2] == '--':
script = sys.argv[1][2:]
if script == 'vikings':
import vikings
vikings.run(*sys.argv[2:])
elif script == 'waitress':
import waitress
waitress.run(*sys.argv[2:])
else:
raise Exception(f'Unknown script {script}')
else:
spam, eggs = sys.argv[1], sys.argv[2]
subprocess.run([sys.executable, __file__, '--vikings', spam])
subprocess.run([sys.executable, __file__, '--waitress', spam, eggs])
# vikings.py
def run(spam):
print(' '.join(['spam'] * int(spam)))
# waitress.py
import sys
import time
def run(spam, eggs):
spam, eggs = int(spam), int(eggs)
if eggs > spam:
print("You can't have more eggs than spam!")
sys.exit(2)
print("Frying...")
time.sleep(2)
raise Exception("This sketch is getting too silly!")
And now:
$ python3 main.py 3 4
spam spam spam
You can't have more eggs than spam!
A few changes you might want to consider in real life:
DRY: We have the same three lines of code copied and pasted for each script, and we have to type each script name three times. You can just use something like __import__(sys.argv[1][2:]).run(sys.argv[2:]) with appropriate error handling.
Use argparse instead of this hacky special casing for the first argument. If you're already sending non-trivial arguments to the scripts, you're probably already using argparse or an alternative anyway.
Add an if __name__ == '__main__': block to each script that just calls run(sys.argv[1:]), so that during development you can still run the scripts directly to test them.
I didn't do any of these because they'd obscure the idea for this trivial example.
1 The documentation is great as a refresher if you've already done it, but as a tutorial and explanatory rationale, not so much. And trying to write the tutorial that the brilliant PyPA guys haven't been able to come up with for years… that's probably beyond the scope of an SO answer.

Python-zxing decode returns empty data

I've trying to decode qr or aztec code data by Python-zxing. Everytime I get empty data without any error in python Shell. What I do wrong?
import zxing
image = "aztec.png"
rd = zxing.BarCodeReader()
rs = rd.decode(image)
print rs.data
print rs
Output:
''
<zxing.BarCode instance at 0x0312A260>
Python ver. 2.7.11 (Windows)
P.S.
When I run script from cmd I've message:
Exception in thread "main" java.lang.NoClassDefFoundError: com/google/zxing/client/j2se/CommandLineRunner

Assuming the mvn installation of Zxing is correct,add the path of the Zxing folder while creating an instance of reader (in this case 'rd')
rd = zxing.BarCodeReader("/path/to/zxing")
FYI: Am running it on Raspbian, not windows, but had the same error.

You forgot class inheritance. See below. Answer made compatible for python 3; but seriously... this is not the pep-way to do it. For long-term compatibility you should check by versioning and use an if-statement.
image = "aztec.png"
zxing = zxing() # notice zxhing()
rd = zxing.BarCodeReader()
rs = rd.decode(image)
try:
print (rs.data)
print (rs)
except:
print (rs.data)
print (rs)

print(rs.raw) # This returns the decoded text.
You can also use rs.parsed.
print(rs.format) # This returns the Format like the detected one is DataMatrix. QR Code etc.
print(rs.points) # This returns the boundary of points where its detected.