Python inserting variable string as file name - python

I'm trying to create a file with a unique file name for every time my script runs. I am only intending to do this to every week or month. so I chose to use the date for the file name.
f = open('%s.csv', 'wb') %name
is where I'm getting this error.
Traceback (most recent call last):
File "C:\Users\User\workspace\new3\stjohnsinvoices\BabblevoiceInvoiceswpath.py", line 143, in <module>
f = open('%s.csv', 'ab') %name
TypeError: unsupported operand type(s) for %: 'file' and 'str'
it works if I use a static filename, is there an issue with the open function, that means you can't pass a string like this?
name is a string and has values such as :
31/1/2013BVI
Many thanks for any help.


You need to put % name straight after the string:
f = open('%s.csv' % name, 'wb')
The reason your code doesn't work is because you are trying to % a file, which isn't string formatting, and is also invalid.


you can do something like
filename = "%s.csv" % name
f = open(filename , 'wb')
or f = open('%s.csv' % name, 'wb')


Very similar to peixe.
You don't have to mention the number if the variables you add as parameters are in order of appearance
f = open('{}.csv'.format(name), 'wb')
Another option - the f-string formatting (ref):
f = open(f"{name}.csv", 'wb')


And with the new string formatting method...
f = open('{0}.csv'.format(name), 'wb')


Even better are f-strings in python 3!
f = open(f'{name}.csv', 'wb')


import hashlib
filename = file_for_download
with open(filename, "rb") as f:
bytes = f.read() # read entire file as bytes
msg_hash = hashlib.sha256(bytes).hexdigest();
print(f"MSG_HASH = {msg_hash}")

Related

error Passing a String variable as filename to open a file

i have this problem: i'm tryna list all filenames from a directory and then print them with numbers on the left to let users select the file to use. Numbers are there because they match with the index of the position of filenames in the list. When i select a specific string filename in the list and pass it to
`f = open (filename, "r")
data = json.loads(f.read())`
i get:
Traceback (most recent call last):
File "test.py", line 170, in
data = json.loads(f.read())
AttributeError: 'str' object has no attribute 'loads'
full code:
jsons=glob.glob("*.json")
print(type(jsons))
print(type(jsons[0]))
n=0
if(jsons):
for json in jsons:
print(str(n) + ' - ' + json)
n=n+1
jsonselection=int(input('select your settings file: '))
filename=(" ".join(jsons[jsonselection].split()))
print()
print('your selection: ' + filename)
else:
sys.exit(colored('no json files available.','red'))
f = open (filename, "r")
data = json.loads(f.read())
actually if i pass to the method a random variable defined by me like name='file' it works.. i just can't understand why. Thanks in advance for the help

That would most likely be because of this line of code.
jsons=glob.glob("*.json")
and then you have
for json in jsons:

Python read in file: ERROR: line contains NULL byte

I would like to parse an .ubx File(=my input file). This file contains many different NMEA sentences as well as raw receiver data. The output file should just contain informations out of GGA sentences. This works fine as far as the .ubx File does not contain any raw messages. However if it contains raw data
I get the following error:
Traceback (most recent call last):
File "C:...myParser.py", line 25, in
for row in reader:
Error: line contains NULL byte
My code looks like this:
import csv
from datetime import datetime
import math
# adapt this to your file
INPUT_FILENAME = 'Rover.ubx'
OUTPUT_FILENAME = 'out2.csv'
# open the input file in read mode
with open(INPUT_FILENAME, 'r') as input_file:
# open the output file in write mode
with open(OUTPUT_FILENAME, 'wt') as output_file:
# create a csv reader object from the input file (nmea files are basically csv)
reader = csv.reader(input_file)
# create a csv writer object for the output file
writer = csv.writer(output_file, delimiter=',', lineterminator='\n')
# write the header line to the csv file
writer.writerow(['Time','Longitude','Latitude','Altitude','Quality','Number of Sat.','HDOP','Geoid seperation','diffAge'])
# iterate over all the rows in the nmea file
for row in reader:
if row[0].startswith('$GNGGA'):
time = row[1]
# merge the time and date columns into one Python datetime object (usually more convenient than having both separately)
date_and_time = datetime.strptime(time, '%H%M%S.%f')
date_and_time = date_and_time.strftime('%H:%M:%S.%f')[:-6] #
writer.writerow([date_and_time])
My .ubx file looks like this:
$GNGSA,A,3,16,25,29,20,31,26,05,21,,,,,1.30,0.70,1.10*10
$GNGSA,A,3,88,79,78,81,82,80,72,,,,,,1.30,0.70,1.10*16
$GPGSV,4,1,13,02,08,040,17,04,,,47,05,18,071,44,09,02,348,24*49
$GPGSV,4,2,13,12,03,118,24,16,12,298,36,20,15,118,30,21,44,179,51*74
$GPGSV,4,3,13,23,06,324,35,25,37,121,47,26,40,299,48,29,60,061,49*73
$GPGSV,4,4,13,31,52,239,51*42
$GLGSV,3,1,10,65,07,076,24,70,01,085,,71,04,342,34,72,13,029,35*64
$GLGSV,3,2,10,78,35,164,41,79,75,214,48,80,34,322,46,81,79,269,49*64
$GLGSV,3,3,10,82,28,235,52,88,39,043,43*6D
$GNGLL,4951.69412,N,00839.03672,E,124610.00,A,D*71
$GNGST,124610.00,12,,,,0.010,0.010,0.010*4B
$GNZDA,124610.00,03,07,2016,00,00*79
µb<  ¸½¸Abð½ . SB éF é v.¥ # 1 f =•Iè ,
Ïÿÿ£Ëÿÿd¡ ¬M 0+ùÿÿ³øÿÿµj #ª ² -K*
,¨ , éºJU /) ++ f 5 .lG NL C8G /{; „> é óK 3 — Bòl . "¿ 2 bm¡
4âH ÐM X cRˆ 35 »7 Óo‡ž "*ßÿÿØÜÿÿUhQ`
3ŒðÿÿÂïÿÿþþûù ÂÈÿÿñÅÿÿJX ES
$²I uM N:w (YÃÿÿV¿ÿÿ> =ìî 1¥éÿÿèÿÿmk³m /?ÔÿÿÒÿÿšz+Ú ­Ïÿÿ6ÍÿÿêwÇ\ ? ]? ˜B Aÿƒ y µbÐD‹lçtæ#p3,}ßœŒ-vAh
¿M"A‚UE ôû JQý
'wA´üát¸jžAÀ‚"Å
)DÂï–ŽtAöÙüñÅ›A|$Å ôû/ Ìcd§ÇørA†áãì˜AØY–Ä ôû1 /Áƒ´zsAc5+_’ô™AìéNÅ ôû( ¶y(,wvAFøÈV§ƒA˜ÝwE ôû$ _S R‰wAhÙ]‘ÑëžAÇ9Å vwAòܧsAŒöƒd§Ò™AÜOÄ ôû3 kœÕ}vA;D.ž‡žAÒûàÄ #ˆ" ϬŸ ntAfˆÞ3ךA~Y2E ôû3 :GVtAæ93l)ÆšAß yE ôû4 Uþy.TwA<âƒ' ¦žAhmëC ôû" ¯4Çï ›wAþ‰Ì½6ŸAŠû¶D ~~xI]tA<ÞÿrÁšAmHE ôû/ ÖÆ#ÈgŸsAXnþ‚†4šA'0tE ôû. ·ÈO:’
sA¢B†i™Aë%
E ôû/ >Þ,À8vA°‚9êœA>ÇD ôû, ø(¼+çŠuAÆOÁ לAÈΆD
ôû# ¨Ä-_c¯qAuÓ?]> —AÐкà ôû0 ÆUV¨ØZsA]ðÛñß™AÛ'Å ôû, ™mv7žqAYÐ:›Ä‘—AdWxD ôû1 ûö>%vA}„
ëV˜A.êbE
AÝ$GNRMC,124611.00,A,4951.69413,N,00839.03672,E,0.009,,030716,,,D*62
$GNVTG,,T,,M,0.009,N,0.016,K,D*36
$GNGNS,124611.00,4951.69413,N,00839.03672,E,RR,15,0.70,162.5,47.6,1.0,0000*42
$GNGGA,124611.00,4951.69413,N,00839.03672,E,4,12,0.70,162.5,M,47.6,M,1.0,0000*6A
$GNGSA,A,3,16,25,29,20,31,26,05,21,,,,,1.31,0.70,1.10*11
$GNGSA,A,3,88,79,78,81,82,80,72,,,,,,1.31,0.70,1.10*17
$GPGSV,4,1,13,02,08,040,18,04,,,47,05,18,071,44,09,02,348,21*43
$GPGSV,4,2,13,12,03,118,24,16,
I already searched for similar problems. However I was not able to find a solution which workes for me.
I ended up with code like that:
import csv
def unfussy_reader(csv_reader):
while True:
try:
yield next(csv_reader)
except csv.Error:
# log the problem or whatever
print("Problem with some row")
continue
if __name__ == '__main__':
#
# Generate malformed csv file for
# demonstration purposes
#
with open("temp.csv", "w") as fout:
fout.write("abc,def\nghi\x00,klm\n123,456")
#
# Open the malformed file for reading, fire up a
# conventional CSV reader over it, wrap that reader
# in our "unfussy" generator and enumerate over that
# generator.
#
with open("Rover.ubx") as fin:
reader = unfussy_reader(csv.reader(fin))
for n, row in enumerate(reader):
fout.write(row[0])
However I was not able to simply write a file containing just all the rows read in with the unfuss_reader wrapper using the above code.
Would be glad if you could help me.
Here is an Image of how the .ubx file looks in notepad++image
Thanks!

I am not quite sure but your file looks pretty binary. You should try to open it as such
with open(INPUT_FILENAME, 'rb') as input_file:

It seems like you did not open the file with correct coding format.
So the raw message cannot be read correctly.
If it is encoded as UTF8, you need to open the file with coding option:
with open(INPUT_FILENAME, 'r', newline='', encoding='utf8') as input_file

Hey if anyone else has this proglem to read in NMEA sentences of uBlox .ubx files
this pyhton code worked for me:
def read_in():
with open('GNGGA.txt', 'w') as GNGGA:
with open('GNRMC.txt','w') as GNRMC:
with open('rover.ubx', 'rb') as f:
for line in f:
#print line
if line.startswith('$GNGGA'):
#print line
GNGGA.write(line)
if line.startswith('$GNRMC'):
GNRMC.write(line)
read_in()

You could also use the gnssdump command line utility which is installed with the PyGPSClient and pygnssutils Python packages.
e.g.
gnssdump filename=Rover.ubx msgfilter=GNGGA
See gnssdump -h for help.
Alternatively if you want a simple Python script you could use the pyubx2 Python package, e.g.
from pyubx2 import UBXReader
with open("Rover.ubx", "rb") as stream:
ubr = UBXReader(stream)
for (_, parsed_data) in ubr.iterate():
if parsed_data.identity in ("GNGGA", "GNRMC"):
print(parsed_data)

IOError: [Errno 22] invalid mode ('w') or filename

I am getting this error thrown when trying to make a file. It is being designed to take a created .csv file and put it into a plain text file.
I would like it to create a new file after it has been run with the date and time stamp but I seem to get the Errno 22 when trying to generate the file.
Any ideas?
import csv
import time
f = open(raw_input('Enter file name: '),"r")
saveFile = open ('Bursarcodes_'+time.strftime("%x")+ '_'+time.strftime("%X")+
'.txt', 'w+')
csv_f = csv.reader(f)
for row in csv_f:
saveFile.write( 'insert into bursarcode_lookup(bursarcode, note_id)' +
' values (\'' + row[0] + '\', ' + row[1] + ')\n')
f.close()
saveFile.close()

You cannot have slashes (/) and colons (:, but allowed in Unix) in your file name, but they are exactly what strftime generates in its output.
Python tries to help you, it says:
No such file or directory: 'Bursarcodes_01/09/15_19:59:24.txt'
Replace time.strftime("%x") with this:
time.strftime("%x").replace('/', '.')
...and time.strftime("%X") with this:
time.strftime("%X").replace(':', '_')

A cleaned-up and extended version:
import csv
import sys
import time
def make_output_fname():
# Thanks to #Andrew:
return time.strftime("Bursarcodes_%x_%X.txt").replace("/", "-").replace(":", "-")
def main(csv_fname=None, outfname=None, *args):
if not csv_fname:
# first arg not given - prompt for filename
csv_fname = raw_input("Enter .csv file name: ")
if not outfname:
# second arg not given - use serialized filename
outfname = make_output_fname()
with open(csv_fname) as inf, open(outfname, "w") as outf:
incsv = csv.reader(inf)
for row in incsv:
outf.write(
"insert into bursarcode_lookup(bursarcode, note_id) values ('{0}', '{1}')\n"
.format(*row)
)
if __name__=="__main__":
# pass any command-line arguments to main()
main(*sys.argv[1:])
You can now run it from the command-line as well.
Note that if any data items in your csv file contain unescaped single-quotes (') you will get invalid sql.

Why does my file show up blank in python?

I'm trying to get my python file to save numbers into a text file, but it always goes blank when I try it. I've done this many times before but it refuses to work this time.
openfile = 'example'
total = 0.5 #another example
totalstr = str(total)
file = open("%s.txt" % (openfile), "w")
file.write(totalstr)
file.close

"file" is a standard Python type. You want to rename things a bit. I'm also assuming "openfile" should be the string filename you want to use. Both answers so far are correct but putting them together gives:
my_file_name = "myfile"
total = 0.5
my_file_handle = open("%s.txt" %(my_file_name), "w")
my_file_handle.write(str(total))
my_file_handle.close()

file is a keyword in python. So,
print '%s' %(file)
prints
<type 'file'>
You should use:
openfile = 'file'

This works for me:
openfile = "file"
total = 0.5
totalstr = str(total)
file = open("%s.txt" % (openfile), "w")
file.write(totalstr)
file.close()
See if you can spot the changes.

csv2json.py error

I am trying to run the script csv2json.py in the Command Prompt, but I get this error:
C:\Users\A\Documents\PROJECTS\Django\sw2>csv2json.py csvtest1.csv wkw1.Lawyer
Converting C:\Users\A\Documents\PROJECTS\Django\sw2csvtest1.csv from CSV to JSON as C:\Users\A\Documents\PROJECTS\Django\sw2csvtest1.csv.json
Traceback (most recent call last):
File "C:\Users\A\Documents\PROJECTS\Django\sw2\csv2json.py", line 37, in <module>
f = open(in_file, 'r' )
IOError: [Errno 2] No such file or directory: 'C:\\Users\\A\\Documents\\PROJECTS\\Django\\sw2csvtest1.csv'
Here are the relevant lines from the snippet:
31 in_file = dirname(__file__) + input_file_name
32 out_file = dirname(__file__) + input_file_name + ".json"
34 print "Converting %s from CSV to JSON as %s" % (in_file, out_file)
36 f = open(in_file, 'r' )
37 fo = open(out_file, 'w')
It seems that the directory name and file name are combined. How can I make this script run?
Thanks.
Edit:
Altering lines 31 and 32 as answered by Denis Otkidach worked fine. But I realized that the first column name needs to be pk and each row needs to start with an integer:
for row in reader:
if not header_row:
header_row = row
continue
pk = row[0]
model = model_name
fields = {}
for i in range(len(row)-1):
active_field = row[i+1]
So my csv row now looks like this (including the header row):
pk, firm_url, firm_name, first, last, school, year_graduated
1, http://www.graychase.com/aabbas, Gray & Chase, Amr A, Babas, The George Washington University Law School, 2005
Is this a requirement of the django fixture or json format? If so, I need to find a way to add the pk numbers to each row. Can I delete this pk column? Any suggestions?
Edit 2
I keep getting this ValidationError: "This value must be an integer". There is only one integer field and that's the pk. Is there a way to find out from the traceback what the line numbers refer to?
Problem installing fixture 'C:\Users\A\Documents\Projects\Django\sw2\wkw2\fixtures\csvtest1.csv.json': Traceback (most recent call last):
File "C:\Python26\Lib\site-packages\django\core\management\commands\loaddata.py", line 150, in handle
for obj in objects:
File "C:\Python26\lib\site-packages\django\core\serializers\json.py", line 41, in Deserializer
for obj in PythonDeserializer(simplejson.load(stream)):
File "C:\Python26\lib\site-packages\django\core\serializers\python.py", line 95, in Deserializer
data[field.attname] = field.rel.to._meta.get_field(field.rel.field_name).to_python(field_value)
File "C:\Python26\lib\site-packages\django\db\models\fields\__init__.py", line 356, in to_python
_("This value must be an integer."))
ValidationError: This value must be an integer.

+ is used incorrectly here, the proper way to combine directory name and file name is using os.path.join(). But there is no need to combine directory where script is located with file name, since it's common to pass relative path to current working directory. So, change lines 31-32 to the following:
in_file = input_file_name
out_file = in_file + '.json'

from os import path
in_file = path.join(dirname(__file__), input_file_name )
out_file = path.join(dirname(__file__), input_file_name + ".json" )
[...]

You should be using os.path.join rather than just concatenating dirname() and filenames.
import os.path
in_file = os.path.join(dirname(__file__), input_file_name)
out_file = os.path.join(dirname(__file__), input_file_name + ".json")
will fix your problem, though depending on what exactly you're doing, there's probably a more elegant way to do it.

Categories

Resources