I have a text file with data such as
b'\x00\x09\x00\xfe'
This was piped into a text file from a TCP socket stream. Call this text file 'stream.txt'. I opened this file with the following code:
f = open("stream.txt", "rb")
bytes_read = f.read()
When I open this file within another Python script, I get a '\' for each '\' in the original file. On top of this, I cannot access the bytes array as such, since it appears to have become a string. That is, 'bytes_read' is now
'b"\\x00\\x09\\x00\\xfe"'
How can I recover this string as a bytes array?
The client code I used to capture this data is the following script:
from socket import *
clientsock = socket(AF_INET, SOCK_STREAM)
clientsock.connect(('1.2.3.4', 2000)) # Open the TCP socket
clientsock.sendall(b'myCommand') # Send a command to the server
data = clientsock.recv(16) # Wait for the response
print(data) # For piping to 'stream.txt'
clientsock.close()
As the data was printed to the terminal, I redirected it to a file:
$ python3 client.py > stream.txt
My goal is to bypass the redirect into text file and pipe directly into a plotter... But first I wanted to get this to work.
Was able to solve this by writing directly to a file. So rather than using 'print(data)', and redirecting to a file, I tried this:
file = open("rawData", "wb")
...
file.write(data)
...
file.close()
Was able to process "rawData" as expected.
Related
I want to download config file from my router via web scraping. The procedure I want to achieve is this:
Save the config file into disk
Send a factory reset
Load the config file previously downloaded.
So far, I have this code:
with requests.Session() as s: # To login into the modem
pagePostBackUp = 'https://192.168.1.1/goform/BackUp'
s.post(urlLogin, data=loginCredentials, verify=False, timeout=5)
dataBackUp = {'dir': 'admin/','file': 'cmconfig.cfg'}
resultBackUp = s.post(pagePostBackUp, data=dataBackUp, verify=False, timeout=10)
print(resultBackUp.text)
The last line is what I want to save into a file. But, when I try to do it with this code:
f = open('/Users/user/Desktop/file.cfg', 'w')
Throws an error that ascii codec can't encode character. If I save the file with, for example, encode='utf16', differs from what I originally download manually.
So, the question is, How can I save this file with the same encoding the router gives me via web? (As unicode). The content of the file looks like this:
�����g���m��� ������Z������ofpqJ
U\V,.o /����zf��v���~W3=,�D};y�tL�cJ
Change the last line of your code to the following:
with open('/Users/user/Desktop/file.cfg', 'wb') as f:
f.write(resultBackUp.content)
This will treat the payload as data (bytes), not text: the file is opened in binary mode, and the content is taken as-is.
There's no encoding/decoding happening.
I want to read data from Arduino and store in a text file in pc through serial port using pyserial whenever I try to execute the Python code it gives this message I tried many things but didn't work out.
Code:
import io
import serial
from datetime import datetime
from serial import SerialException
connected=False
outfile='C:\Users\Yassine\hello.txt'
ser = serial.Serial(port="COM12", baudrate=9600,timeout=None,bytesize=serial.EIGHTBITS,parity=serial.PARITY_NONE,stopbits=serial.STOPBITS_ONE)
sio = io.TextIOWrapper(io.BufferedRWPair(ser, ser, 1), encoding='ascii', newline ='\r')
with open(outfile,'a') as f:
while ser.isOpen():
try:
datastring=ser.readline()
except serial.SerialException:
pass
print datastring
print datetime.now()
f.write(datetime.now().isoformat() +'\t'+ datastring +'\n' )
f.flush()
while not ser.isOpen():
pass
ser.close()
Check that COM12 is actually the Arduino by looking in device manager.
Or you can execute this in command line to get a list of available serial ports:
python -m serial.tools.list_ports
You may also have something else trying to access the Arduino serial port. Make sure the Serial Monitor in Arduino IDE is closed.
I want to open txt file read the last value of it & write it in other txt file,it's working but the reading value(arg) written to the next file jump the line which is not good for me i want it at the same line with other variables
with open(outfile,'a') as f:
with open (inputfile,'r') as f1:
arg =f1.readline() // that variable i read from the txt
print (arg )
f.write(datetime.now().strftime("%Y-%m-%d ; %H:%M:%S")+'\n'+valueRead+ '\n' +arg+ '\n') // the file i write to
f.flush()
f1.close()
f.close()
TTTs (this is my arg variable that i read from txt file this is what i get)
2017-05-12 ; 15:48:23 TTS (this is how i want it )
Thanks for helping guys
SOLVED
It looks like I needed to close the sys.stdout file (which was open for writing) with a
sys.stdout.close()
and then re-open it, but in read mode, with
sys.stdout = open ('/home/myuser/verify_yslog_conf/remote_hostname/hi', 'r')
and then it uploaded correctly. I used a
time.sleep(60)
to cat the file mid-run for troubleshooting. It was empty until the def had finished, which by default closed the "hi" file. So I realized that I needed to close it to fully write the file. thanks.
Code below is meant to check if syslog is running, and if so, comment out any lines with "#" (but NOT #192.168.x.xx or 192.168.x.xxx) from the pulled remote syslog.conf file. After commenting those lines locally, it is supposed to upload the new file to the remote server in the old location. However, all that happens is that my /etc/syslog.conf file gets erased. The file isn't removed, it is just empty.
I know the issue is from the put statement
fabric.operations.put('/home/myuser/verify_yslog_conf/remote_hostname/hi', '/etc/syslog.conf', use_sudo=True)
but nothing looks wrong with it.
def verify():
#lines below are good; keep them
ip1 = '192.168.x.xx'
ip2 = '92.168.x.xxx'
#check if syslog is running
output = sudo("/sbin/service syslog status")
if 'is running...' in output:
#pull the syslog file from the remote server
fabric.operations.get('/etc/syslog.conf')
#read thru the file
fh = open('/home/myuser/verify_yslog_conf/remote_hostname/syslog.conf', 'r')
f = fh.read()
fh.close()
#if the file needs commenting ...
if re.search(r'(#(?!({0}|{1})))'.format(ip1, ip2), f):
#get the file again -- maybe the problem? was advised to do this
fabric.operations.get('/etc/syslog.conf')
#save the file locally for reading and then for writing (r+ failed)
conf = open ('/home/myuser/verify_yslog_conf/remote_hostname/syslog.conf', 'r')
sys.stdout = open ('/home/myuser/verify_yslog_conf/remote_hostname/hi', 'w')
#for every line in the old /etc/syslog.conf
for line in conf:
#if it has an # but not followed by the permitted IPs
if re.search(r'(#(?!({0}|{1})))'.format(ip1, ip2), line):
#comment out the line then print it (does it twice, dont care)
line = "#" + line
sys.stdout.write(line)
sys.stdout.write(line)
conf.close()
#upload the newly commented file to the remote host
fabric.operations.put('/home/myuser/verify_yslog_conf/remote_hostname/hi', '/etc/syslog.conf', use_sudo=True)
else:
print GOOD
else:
print RSYSLOG
Try to sys.stdout.close() immediately prior to the fabric.operations.put command.
It's likely that before the put operation, the data isn't flushed to the file, so it's sending an empty file, but when the script finishes, it automatically flushes (writes) the data which is why the file appears normal on the local system.
I'm appending values to a log file every 6th second. Every 30 sec I'm transferring this log to an FTP server as a file. But instead of transfering the whole file, I just want to append the collected data to the file on my server. I haven't been able to figure out how to open the server file and then append the values.
My code so far:
session = ftplib.FTP(authData[0],authData[1],authData[2])
session.cwd("//"+serverCatalog()+"//") # open server catalog
file = open(fileName(),'rb')
with open(fileName(), 'rb') as f:
f = f.readlines()
for line in f:
collected = line
# In some way open server file, write lines to it
session.storbinary('STOR ' + fileName(), open(fileName(), 'a'), 1024)
file.close()
session.quit()
Instead, do I have to download the server file open and append, then send it back?
Above was my question, the full solution is below:
session.cwd("//"+serverCatalog()+"//") # open server catalog
localfile = open("logfile.txt",'rb')
session.storbinary('APPE serverfile.txt', localfile)
localfile.close()
Use APPE instead of STOR.
Source: http://www.gossamer-threads.com/lists/python/python/22960 (link to web.archive.org)
I have this python code:
import sys
import dpkt
f = file("pcaop.Pcap")
pcap = dpkt.pcap.Reader(f)
i = 0
for ts, buf in pcap:
print "Ya"
dpkt throws NeedData on the 52nd packet. The same one every time - I've checked packet 52 and it is the same as everyone else on wireshark.
What causes this?
Solution is provided here: Python stops reading file using read
I had the same problem when dpkt.pcap was working fine under Linux but failed instantly when run in Windows.
The problem is that when a file is opened in text mode open("filename", "r") the file is read until EOF is encountered. Thus, open("filename", "rb")