Python - How do I split output? [duplicate] - python

This question already has answers here:
How to duplicate sys.stdout to a log file?
(19 answers)
Closed 9 years ago.
Newb python question. I've been reading up on tee() and different ways of splitting output. But i cant find a good example for splitting output to the terminal and to a log file. I've been playing around with some options and this is what I have so far:
def logname():
env.warn_only = True
timestamp = time.strftime("%d_%b_%Y")
return "%s_%s" % (env.host_string, timestamp)
sys.stdout = open('/home/path/to/my/log/directory/%s' % logname(), 'w')
The above will log to a file with the host name_datestamp but won't display anything on screen. Then when I want to stop logging i do:
sys.stdout = sys.__stdout__
How can I log to my file with the definiton above and display to the terminal at the same time? Am I on the right path with tee()?


Like this?
[user#machine ~]$ python
Python 2.7.3 (default, Aug 9 2012, 17:23:57)
[GCC 4.7.1 20120720 (Red Hat 4.7.1-5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>>
>>> class Tee(object):
... def __init__(self, logfile, stdio = sys.__stdout__):
... self.logf = open(logfile, 'w')
... self.stdio = stdio
... def write(self, data):
... self.logf.write(data)
... self.stdio.write(data)
... def flush(self):
... self.logf.flush()
... self.stdio.flush()
... def __getattr__(self, k):
... return getattr(self.stdio, k)
... def __dir__(self):
... return dir(self.stdio)
...
>>> sys.stdout = Tee('/tmp/output')
>>> print 'some test output'
some test output
>>>
[user#machine ~]$ cat /tmp/output
some test output
[user#machine ~]$

Related

Python Json Output From AWS Not Removing Single Quotes

I am kind of new to Python and playing around with classes. I created a class to access aws ec2 instances to stop and start along with current state. Don't know how else to explain this other than showing the issue.
#!/usr/bin/python
import boto3
import json
import datetime
from botocore.exceptions import ClientError
class Ec2(object):
def __init__(self, instance_id, region):
self.instances = [instance_id]
self.region = 'us-gov-west-1'
self.state = ''
if region is not None:
self.region = region
self.ec2 = boto3.client('ec2', region_name=region)
self._setState()
def _setState(self):
def convert_timestamp(datetime_key):
if isinstance(datetime_key, (datetime.date, datetime.datetime)):
return datetime_key.timestamp()
status = self.ec2.describe_instances(InstanceIds=self.instances)
# Find the current status of the instance
dump = json.dumps(status, default=convert_timestamp)
body = json.loads(dump)
self.state = body["Reservations"][0]["Instances"][0]["State"]["Name"]
def getState(self):
self._setState()
return self.state
...
So if in the Python3 interpretor I run this:
from ec2 import Ec2
ssm_1 = Ec2('i-12345abcdef','us-gov-west-1')
ssm_1.getState()
Python 3.8.0 (default, Oct 28 2019, 16:14:01)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ec2 import Ec2
>>> ssm_1 = Ec2('i-12345abcdef','us-gov-west-1')
>>> ssm_1.getState()
'stopped'
>>>
If I change the method _setState in the class to:
def _setState(self):
def convert_timestamp(datetime_key):
if isinstance(datetime_key, (datetime.date, datetime.datetime)):
return datetime_key.timestamp()
status = self.ec2.describe_instances(InstanceIds=self.instances)
# Find the current status of the instance
dump = json.dumps(status, default=convert_timestamp)
body = json.loads(dump)
self.state = body["Reservations"][0]["Instances"][0]["State"]["Name"]
self.state = self.state.replace("'", "")
then rerun above the single quotes do not get removed! It appears that these quotes are not part of the actual string, validated by changing the last line to:
self.state = self.state[1:-1]
>>> ssm_1.getState()
'toppe'
so it removed the first and last letters not the single quotes, s & d
BUT if I change _setState to:
def _setState(self):
def convert_timestamp(datetime_key):
if isinstance(datetime_key, (datetime.date, datetime.datetime)):
return datetime_key.timestamp()
status = self.ec2.describe_instances(InstanceIds=self.instances)
# Find the current status of the instance
dump = json.dumps(status, default=convert_timestamp)
body = json.loads(dump)
self.state = body["Reservations"][0]["Instances"][0]["State"]["Name"]
Now run:
>>> from ec2 import Ec2
>>> ssm_1 = Ec2('i-12345abcdef','us-gov-west-1')
>>> ssm_1.getState().replace("'", "")
stopped
>>>
the single quotes are now gone!
So it appears in the class the string does not actually include the single quotes but outside the class it does. Why? Thanks in advance.

Python3 get process base-address from PID

I am trying to get the base-address of a process in Windows (64-bit), with Python3, assuming to know the PID.
I looked over all questions here on stack, but the solutions are old/not working.
I assume to have the PID of the process in a variable called pid.
One of the many pieces of code I tried is
PROCESS_ALL_ACCESS = 0x1F0FFF
processHandle = win32api.OpenProcess(PROCESS_ALL_ACCESS, False, pid)
modules = win32process.EnumProcessModules(processHandle)
fileName = win32process.GetModuleFileNameEx(processHandle, modules[0])
base_address = win32api.GetModuleHandle(fileName)
processHandle.close()
But I get error on GetModuleHandle: 'Impossible to find the specified module'.
Thank you for the help.

According to [MS.Docs]: GetModuleHandleW function (emphasis is mine):
Retrieves a module handle for the specified module. The module must have been loaded by the calling process.
That means that it will work fine for the current process, but for any other one you'd get Undefined Behavior, because you try retrieving:
The .dll (or .exe) name from the other process (GetModuleFileNameEx call)
The handle for the name at previous step (GetModuleHandle call) but in the current process (if loaded), which makes no sense
Although there's no clear documentation on this topic (or at least I couldn't find any), the handle is the base address. This is a principle that you also rely on (calling GetModuleHandle), but you can use the values returned by EnumProcessModules directly (look at the example below, the values are the same).
If you want to be rigorous, you could use [MS.Docs]: GetModuleInformation function. Unfortunately, that's not exported by PyWin32, and an alternative is using [Python 3.Docs]: ctypes - A foreign function library for Python.
code00.py:
#!/usr/bin/env python3
import sys
import win32api as wapi
import win32process as wproc
import win32con as wcon
import ctypes as ct
from ctypes import wintypes as wt
import traceback as tb
class MODULEINFO(ct.Structure):
_fields_ = [
("lpBaseOfDll", ct.c_void_p),
("SizeOfImage", wt.DWORD),
("EntryPoint", ct.c_void_p),
]
get_module_information_func_name = "GetModuleInformation"
GetModuleInformation = getattr(ct.WinDLL("kernel32"), get_module_information_func_name, getattr(ct.WinDLL("psapi"), get_module_information_func_name))
GetModuleInformation.argtypes = [wt.HANDLE, wt.HMODULE, ct.POINTER(MODULEINFO)]
GetModuleInformation.restype = wt.BOOL
def get_base_address_original(process_handle, module_handle):
module_file_name = wproc.GetModuleFileNameEx(process_handle, module_handle)
print(" File for module {0:d}: {1:s}".format(module_handle, module_file_name))
module_base_address = wapi.GetModuleHandle(module_file_name)
return module_base_address
def get_base_address_new(process_handle, module_handle):
module_info = MODULEINFO()
res = GetModuleInformation(process_handle.handle, module_handle, ct.byref(module_info))
print(" Result: {0:}, Base: {1:d}, Size: {2:d}".format(res, module_info.lpBaseOfDll, module_info.SizeOfImage))
if not res:
print(" {0:s} failed: {1:d}".format(get_module_information_func_name, getattr(ct.WinDLL("kernel32"), "GetLastError")()))
return module_info.lpBaseOfDll
def main(*argv):
pid = int(argv[0]) if argv and argv[0].isdecimal() else wapi.GetCurrentProcessId()
print("Working on pid {0:d}".format(pid))
process_handle = wapi.OpenProcess(wcon.PROCESS_ALL_ACCESS, False, pid)
print("Process handle: {0:d}".format(process_handle.handle))
module_handles = wproc.EnumProcessModules(process_handle)
print("Loaded modules: {0:}".format(module_handles))
module_index = 0 # 0 - the executable itself
module_handle = module_handles[module_index]
get_base_address_funcs = [
#get_base_address_original, # Original behavior moved in a function
get_base_address_new,
]
for get_base_address in get_base_address_funcs:
print("\nAttempting {0:s}".format(get_base_address.__name__))
try:
module_base_address = get_base_address(process_handle, module_handle)
print(" Base address: 0x{0:016X} ({1:d})".format(module_base_address, module_base_address))
except:
tb.print_exc()
process_handle.close()
#input("\nPress ENTER to exit> ")
if __name__ == "__main__":
print("Python {0:s} {1:d}bit on {2:s}\n".format(" ".join(item.strip() for item in sys.version.split("\n")), 64 if sys.maxsize > 0x100000000 else 32, sys.platform))
main(*sys.argv[1:])
print("\nDone.")
Output:
e:\Work\Dev\StackOverflow\q059610466>"e:\Work\Dev\VEnvs\py_pc064_03.07.06_test0\Scripts\python.exe" code00.py
Python 3.7.6 (tags/v3.7.6:43364a7ae0, Dec 19 2019, 00:42:30) [MSC v.1916 64 bit (AMD64)] 64bit on win32
Working on pid 59608
Process handle: 452
Loaded modules: (140696816713728, 140714582343680, 140714572513280, 140714535354368, 140714547544064, 140713592946688, 140714443341824, 140714557898752, 140714556325888, 140714550362112, 140714414964736, 140714562486272, 140714532798464, 140714555473920, 140714548592640, 140714533322752, 140714531946496, 140714553769984, 140714555670528, 140714558750720, 140714581426176, 140714556129280, 140714546036736, 140714518052864, 140714532601856, 140714524737536, 140714210361344, 1797128192, 140714574151680, 140714535026688, 140714557046784, 140714538172416, 140714531291136, 140714530963456, 140714530766848, 140714530832384, 1796931584, 140714561044480, 140714573299712, 140714215014400, 140714529849344, 1798438912, 140714559995904, 140714167042048)
Attempting get_base_address_new
Result: 1, Base: 140696816713728, Size: 110592
Base address: 0x00007FF687C80000 (140696816713728)
Done.
e:\Work\Dev\StackOverflow\q059610466>:: Attempting to run with Task Manager pid
e:\Work\Dev\StackOverflow\q059610466>"e:\Work\Dev\VEnvs\py_pc064_03.07.06_test0\Scripts\python.exe" code00.py 22784
Python 3.7.6 (tags/v3.7.6:43364a7ae0, Dec 19 2019, 00:42:30) [MSC v.1916 64 bit (AMD64)] 64bit on win32
Working on pid 22784
Process handle: 480
Loaded modules: (140699900903424, 140714582343680, 140714572513280, 140714535354368, 140714547544064, 140714573299712, 140714531946496, 140714550362112, 140714562486272, 140714532798464, 140714530963456, 140714530766848, 140714556981248, 140714557898752, 140714556325888, 140714555473920, 140714365222912, 140714548592640, 140714496753664, 140714533322752, 140714553769984, 140714574151680, 140714535026688, 140714557046784, 140714538172416, 140714581426176, 140714558750720, 140714531291136, 140714530832384, 140714546036736, 140714444521472, 140714567467008, 140714532601856, 140714468966400, 140714452385792, 140714267115520, 140714510843904, 140714478731264, 140713698263040, 140714510254080, 140714556129280, 140714565435392, 140714110091264, 140714491379712, 140714455007232, 140714514382848, 140714459529216, 140714281140224, 140714370859008, 140714471260160, 140714566746112, 140713839362048, 140714555670528, 140714171695104, 140714508615680, 140714514841600, 140714029154304, 140714036625408, 140714329636864, 140714447011840, 140714434691072, 140714470866944, 140714561044480, 140714520870912, 140714469883904, 140714494787584, 140714293592064, 140713999335424, 140714400743424, 140714497605632, 140714502193152, 140714197254144, 140714415030272, 140714035576832, 140714065854464, 140714513006592, 140714529652736, 140714512809984, 140714495049728, 140714038657024, 140714371448832, 140714421911552, 140714325966848, 140714196074496, 140714057924608, 140714058317824, 140714064281600, 140714058121216, 140714519756800, 140714327539712, 140714311614464, 140714501079040, 140714546167808, 140714531422208, 140714531553280, 140714557767680, 140714518052864, 140714524737536, 140714167631872, 140714528669696, 140714331865088, 140714310369280, 140714310238208, 140714520018944, 140714458939392, 2018133999616, 140714401988608, 2018141863936, 140714514644992, 140714454810624, 140714294640640)
Attempting get_base_address_new
Result: 1, Base: 140699900903424, Size: 1105920
Base address: 0x00007FF73F9D0000 (140699900903424)
Done.
Update #0
According to [MS.Docs]: MODULEINFO structure (Remarks section, emphasis still mine):
The load address of a module is the same as the HMODULE value.
So, things seem to be pretty straightforward.
code01.py:
#!/usr/bin/env python3
import sys
import win32api as wapi
import win32process as wproc
import win32con as wcon
def main(*argv):
pid = int(argv[0]) if argv and argv[0].isdecimal() else wapi.GetCurrentProcessId()
print("Working on pid {0:d}".format(pid))
process_handle = wapi.OpenProcess(wcon.PROCESS_ALL_ACCESS, False, pid)
print(" Process handle: {0:d}".format(process_handle.handle))
module_handles = wproc.EnumProcessModules(process_handle)
module_handles_count = len(module_handles)
print(" Loaded modules count: {0:d}".format(module_handles_count))
module_index = 0 # 0 - the executable itself
if module_index > module_handles_count:
module_index = 0
module_handle = module_handles[module_index]
module_file_name = wproc.GetModuleFileNameEx(process_handle, module_handle)
print(" File [{0:s}] (index {1:d}) is loaded at address 0x{2:016X} ({3:d})".format(module_file_name, module_index, module_handle, module_handle))
process_handle.close()
if __name__ == "__main__":
print("Python {0:s} {1:d}bit on {2:s}\n".format(" ".join(item.strip() for item in sys.version.split("\n")), 64 if sys.maxsize > 0x100000000 else 32, sys.platform))
main(*sys.argv[1:])
print("\nDone.")
Output:
e:\Work\Dev\StackOverflow\q059610466>"e:\Work\Dev\VEnvs\py_pc064_03.07.06_test0\Scripts\python.exe" code01.py
Python 3.7.6 (tags/v3.7.6:43364a7ae0, Dec 19 2019, 00:42:30) [MSC v.1916 64 bit (AMD64)] 64bit on win32
Working on pid 7184
Process handle: 456
Loaded modules count: 43
File [e:\Work\Dev\VEnvs\py_pc064_03.07.06_test0\Scripts\python.exe] (index 0) is loaded at address 0x00007FF687C80000 (140696816713728)
Done.
e:\Work\Dev\StackOverflow\q059610466>:: Attempting to run with Task Manager pid
e:\Work\Dev\StackOverflow\q059610466>"e:\Work\Dev\VEnvs\py_pc064_03.07.06_test0\Scripts\python.exe" code01.py 22784
Python 3.7.6 (tags/v3.7.6:43364a7ae0, Dec 19 2019, 00:42:30) [MSC v.1916 64 bit (AMD64)] 64bit on win32
Working on pid 22784
Process handle: 624
Loaded modules count: 111
File [C:\WINDOWS\system32\taskmgr.exe] (index 0) is loaded at address 0x00007FF73F9D0000 (140699900903424)
Done.

Compare two IPv6 addresses for equality

I need to compare two IP version6, if there are equal.
It is not a simple string comparison because the same address can be written in multiple ways.
I can't use third-party packages.
For example:
2041:0000:140F:0000:0000:0000:875B:131B
2041:0000:140F::875B:131B
2041:0:140F::875B:131B
and from windows ip config can be: 2041:0:140F::875B:131B%11

If you are on Python 3.3+ you can use the standard library module ipaddress:
Python 3.7.5 (default, Dec 15 2019, 17:54:26)
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ipaddress
>>> a=ipaddress.ip_address('2041:0000:140F:0000:0000:0000:875B:131B')
>>> b=ipaddress.ip_address('2041:0000:140F::875B:131B')
>>> c=ipaddress.ip_address('2041:0:140F::875B:131B')
>>> a==b==c
True

you can try with socket library :
import socket
ip1 = "2041:0000:140F:0000:0000:0000:875B:131B"
ip2 = "2041:0000:140F::875B:131B"
ip3 = "2041:0000:140F::875B:131B"
if socket.inet_pton(socket.AF_INET6, ip1) == socket.inet_pton(socket.AF_INET6, ip2) == socket.inet_pton(socket.AF_INET6, ip3):
print ("match")
Output:

For example:
import sys
def extend_ipv6(src):
parts = src.split(":")
if '%' in parts[-1]:
parts[-1], _ = parts[-1].split('%', 2)
n_parts = len(parts)
out = list()
for part in parts:
if len(part) == 0:
for i in range(8-n_parts+1):
out.append("0000")
else:
out.append("0"*(4-len(part))+part)
return ":".join(out)
def main():
for arg in sys.argv[1:]:
print(extend_ipv6(arg))
if __name__ == "__main__":
main()
python3 do.py 2041:0000:140F:0000:0000:0000:875B:131B 2041:0000:140F::875B:131B 2041:0:140F::875B:131B 2041:0:140F::875B:131B%11
2041:0000:140F:0000:0000:0000:875B:131B
2041:0000:140F:0000:0000:0000:875B:131B
2041:0000:140F:0000:0000:0000:875B:131B
2041:0000:140F:0000:0000:0000:875B:131B
will be this solution enough for your task?

I need help in python2 to making a counter that can count up and store the count data into a file

This is my code, I have to add a counter into my code so that every time the button is pressed it will count up once and i would like to to keep on counting up every time the button is pressed. I searched online and found out that I have to store the counter data to a file but i do not know how to write a code to do the count and store the count to a file. please help
from time import sleep
import RPi.GPIO as GPIO
import os
import sys
import webbrowser
GPIO.setmode(GPIO.BOARD)
button=40
button1=11
car=("/home/pi/Desktop/htb.mp4")
car2=("/home/pi/Desktop/htb2.mp4")
GPIO.setup(button,GPIO.IN)
GPIO.setup(button1,GPIO.IN)
quit_video=True
player=False
while(1):
if GPIO.input(button)==0 and GPIO.input(button1)==1:
print " thru beam "
os.system('pkill omxplayer')
os.system('omxplayer -r htb.mp4')
sleep(.5)
if GPIO.input(button1)==0 and GPIO.input(button)==1:
print " that other sensor "
os.system('pkill omxplayer')
os.system('omxplayer -r htb2.mp4')
sleep(.5)
else:
print " home "
webbrowser.open('https://www.google.com.sg/')
sleep(.5)

Such a program will help you:
import os
if not os.path.isfile("counter.txt"):
counter = 1
f = open("counter.txt","w+")
f.write(str(counter))
f.close()
print (counter)
else:
f = open("counter.txt","r+")
counter = int(f.readline())
counter += 1
f.seek(0)
f.write(str(counter))
f.close()
print (counter)
As you see below, it shares a counter between different runs:
Python 2.7.10 (default, May 23 2015, 09:44:00) [MSC v.1500 64 bit (AMD64)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> ================================ RESTART ================================
>>>
1
>>> ================================ RESTART ================================
>>>
2
>>> ================================ RESTART ================================
>>>
3
>>> ================================ RESTART ================================
>>>
4
>>> ================================ RESTART ================================
>>>
5
>>>

Keyboard shortcuts broken running interactive Python Console from a script

You can start an interactive console from inside a script with following code:
import code
# do something here
vars = globals()
vars.update(locals())
shell = code.InteractiveConsole(vars)
shell.interact()
When I run the script like so:
$ python my_script.py
an interactive console opens:
Python 2.7.2+ (default, Jul 20 2012, 22:12:53)
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>>
The console has all globals and locals loaded which is great since I can test stuff easily.
The problem here is that arrows don't work as they normally do when starting an Python console. They simply display escaped characters to the console:
>>> ^[[A^[[B^[[C^[[D
This means that I can't recall previous commands using the up/down arrow keys and I can't edit the lines with the left/right arrow keys either.
Does anyone know why is that and/or how to avoid that?

Check out readline and rlcompleter:
import code
import readline
import rlcompleter
# do something here
vars = globals()
vars.update(locals())
readline.set_completer(rlcompleter.Completer(vars).complete)
readline.parse_and_bind("tab: complete")
shell = code.InteractiveConsole(vars)
shell.interact()

This is the one I use:
def debug_breakpoint():
"""
Python debug breakpoint.
"""
from code import InteractiveConsole
from inspect import currentframe
try:
import readline # noqa
except ImportError:
pass
caller = currentframe().f_back
env = {}
env.update(caller.f_globals)
env.update(caller.f_locals)
shell = InteractiveConsole(env)
shell.interact(
'* Break: {} ::: Line {}\n'
'* Continue with Ctrl+D...'.format(
caller.f_code.co_filename, caller.f_lineno
)
)
For example, consider the following script:
a = 10
b = 20
c = 'Hello'
debug_breakpoint()
a = 20
b = c
c = a
mylist = [a, b, c]
debug_breakpoint()
def bar():
a = '1_one'
b = '2+2'
debug_breakpoint()
bar()
When executed, this file shows to following behavior:
$ python test_debug.py
* Break: test_debug.py ::: Line 24
* Continue with Ctrl+D...
>>> a
10
>>>
* Break: test_debug.py ::: Line 32
* Continue with Ctrl+D...
>>> b
'Hello'
>>> mylist
[20, 'Hello', 20]
>>> mylist.append(a)
>>>
* Break: test_debug.py ::: Line 38
* Continue with Ctrl+D...
>>> a
'1_one'
>>> mylist
[20, 'Hello', 20, 20]

Categories

Resources