I have a program that creates a 2d array in Python but how do I save it as a csv file, it is
value_a = int(input("Type in a value for a: "))
value_b = int(input("Now a value for b: "))
value_c = int(input("And a value for c: "))
d = value_a + value_b + value_c
result = [[value_a, value_b, value_c, d]] # put the initial values into the array
number_of_loops = int(input("type in the number of loops the program must execute: "))
def loops(a, b, c, n):
global result
for i in range(n):
one_loop = [] # assign an empty array for the result of one loop
temp_a = a
a = ((a + 1) * 2) # This adds 1 to a and then multiplies by 2
one_loop.append(str(a))
b = b * 2
one_loop.append(b)
c = (temp_a + b)
one_loop.append(c)
d = a + b + c
one_loop.append(d)
result.append(one_loop)
print(result)
loops(value_a, value_b, value_c, number_of_loops)
print(result)
It prints ok but how do I save the array as a csv file
Use csvwriter.writerows,
import csv
with open(filename, 'w') as f:
writer = csv.writer(f)
writer.writerows(result)
If you're able to use third-party libraries and you're going to be working with 2d (or more) arrays in Python, I'd recommend you use a library like numpy or pandas. Numpy includes a method to write out arrays as csv files called savetxt. Good luck!
Python comes with CSV writing and reading functionality. See The Python Standard Library » 13.1csv — CSV File Reading and Writing for fuller documentation, but here is a quick example taken from that page and adapted to your problem:
import csv
with open('eggs.csv', 'wb') as csvfile:
spamwriter = csv.writer(csvfile, delimiter=' ',
quotechar='|', quoting=csv.QUOTE_MINIMAL)
for row in results:
spamwriter.writerow(row)
Related
The code uses the matrix and arrpow functions to calculate the fibonacci numbers for the elements in my list, num. Oddly, right after a.append(float(row[0])) is completed, the error I get is
IndexError: list index out of range
Which is obviously coming from b.append.
Here's the file I want to pull from
import time
import math
import csv
import matplotlib.pyplot as plt
def arrpow(arr, n):
yarr=arr
if n<1:
pass
if n==1:
return arr
yarr = arrpow(arr, n//2)
yarr = [[yarr[0][0]*yarr[0][0]+yarr[0][1]*yarr[1][0],yarr[0][0]*yarr[0][1]+yarr[0][1]*yarr[1][1]],
[yarr[1][0]*yarr[0][0]+yarr[1][1]*yarr[1][0],yarr[1][0]*yarr[0][1]+yarr[1][1]*yarr[1][1]]]
if n%2:
yarr=[[yarr[0][0]*arr[0][0]+yarr[0][1]*arr[1][0],yarr[0][0]*arr[0][1]+yarr[0][1]*arr[1][1]],
[yarr[1][0]*arr[0][0]+yarr[1][1]*arr[1][0],yarr[1][0]*arr[0][1]+yarr[1][1]*arr[1][1]]]
return yarr
def matrix(n):
arr= [[1,1],[1,0]]
f=arrpow(arr,n-1)[0][0]
return f
num = [10,100,1000,10000,100000,1000000]
with open('matrix.dat', 'w') as h:
for i in num:
start_time = 0
start_time = time.time()
run = matrix(i)
h.write(str(math.log10(i)))
h.write('\n')
h.write((str(math.log10(time.time()-start_time))))
h.write('\n')
a = []
b = []
with open('matrix.dat','r+') as csvfile:
plots = csv.reader(csvfile, delimiter=',')
for row in plots:
a.append(float(row[0]))
b.append(float(row[1]))
plt.plot(a,b,label = " ")
row = ['1.0']
So row is a list with 1 value. row[1] is trying to access the second index of a list with 1 value. That is why you are getting an error.
When you are constructing matrix.dat, you do not add a comma for the CSV reader to separate the data. So when it tries to read the file, the whole thing is converted into a 1-element array. Attempting to access the second element throws an error because it doesn't exist.
Solution: Replace \n on line 34 with a comma (,).
I'm trying to read a specifically formatted file (namely, the Butcher tableau) in python 3.5.
The file looks like this(tab separated):
S
a1 b11 b12 ... b1S
a2 b21 b22 ... b2S
...
aS bS1 bS2 ... bSS
0.0 c1 c2 ... cS
[tolerance]
for example, (tab separated)
2
0.0 0.0 0.0
1.0 0.5 0.5
0.0 0.5 0.5
0.0001
So my code looks like i'm writing in C. Is there a more pythonic approach to parsing this file? Maybe there are numpy methods that could be used here?
#the data from .dat file
S = 0 #method order, first char in .dat file
a = [] #S-dim left column of buther tableau
b = [] #S-dim matrix
c = [] #S-dim lower row
tolerance = 0 # for implicit methods
def parse_method(file_name):
'read the file_name, process lines, produce a Method object'
try:
with open('methods\\' + file_name) as file:
global S
S = int(next(file))
temp = []
for line in file:
temp.append([float(x) for x in line.replace('\n', '').split('\t')])
for i in range(S):
a.append(temp[i].pop(0))
b.append(temp[i])
global c
c = temp[S][1:]
global tolerance
tolerance = temp[-1][0] if len(temp)>S+1 else 0
except OSError as ioerror:
print('File Error: ' + str(ioerror))
My suggestion using Numpy:
import numpy as np
def read_butcher(filename):
with open(filename, 'rb') as fh:
S = int(fh.readline())
array = np.fromfile(fh, float, (S+1)**2, '\t')
rest = fh.read().strip()
array.shape = (S+1, S+1)
a = array[:-1, 0]
b = array[:-1, 1:]
c = array[ -1, 1:]
tolerance = float(rest) if rest else 0.0
return a, b, c, tolerance
Although I'm not entirely sure how consistently numpy.fromfile advances the file pointer... There are no guarantees in the documentation.
Handling of file exceptions should probably be done outside of the parsing method.
Code -
from collections import namedtuple
def parse_file(file_name):
with open('a.txt', 'r') as f:
file_content = f.readlines()
file_content = [line.strip('\n') for line in file_content]
s = int(file_content[0])
a = [float(file_content[i].split()[0]) for i in range(1, s + 1)]
b = [list(map(float, file_content[i].split()[1:]))
for i in range(1, s + 1)]
c = list(map(float, file_content[-2].split()))
tolerance = float(file_content[-1])
ButcherTableau = namedtuple('ButcherTableau', 's a b c tolerance')
bt = ButcherTableau(s, a, b, c, tolerance)
return bt
p = parse_file('a.txt')
print('S :', p.s)
print('a :', p.a)
print('b :', p.b)
print('c :', p.c)
print('tolerance :', p.tolerance)
Output -
S : 2
a : [0.0, 1.0]
b : [[0.0, 0.0], [0.5, 0.5]]
c : [0.0, 0.5, 0.5]
tolerance : 0.0001
Here's a bunch of suggestions you should consider:
from collections import namedtuple
import csv
def parse_method(file_name):
# for conveniency create a namedtuple
bt = namedtuple('ButcherTableau', dict(a=[], b=[], c=[], order=0, tolerance=0))
line = None
# advice ①: do not assume file path in a function, make assumptions as close to your main function as possible (to make it easier to parameterize later on)
# advice ②: do not call your file "file" so you're not shadowing the class "file" that's loaded globally at runtime
with open(file_name, 'r') as f:
# read the first line alone to setup your "method order" value before reading all the tab separated values
bt.order = int(f.readline())
# create a csv reader with cell separator as tabs
# and create an enumerator to have indexes for each line
for idx, line in enumerate(csv.reader(f, delimiter='\t')))
# instead of iterating again, you can just check the index
# and build your a and b values
if idx < bt.order:
bt.a.append(line.pop(0))
bt.b.append(line)
# if line is None (as set before the for), it means we did not iterate, meaning that we need to make it an error
if not line:
raise Exception("File is empty. Could not parse {}".format(file_name))
# finally you can build your c (and tolerance) values with the last line, which conveniently is still available once the for is finished
bt.c = line[1:]
bt.tolerance = line[0] if idx > S+1 else 0
# avoid the globals, return the namedtuple instead and use the results in the caller function
return bt
This code is untested (just rework of your code as I read it), so it might not work as is, but you might want take the good ideas and make them your own.
I want to do the equivalent to adding elements in a python list recursively in Numpy, As in the following code
matrix = open('workfile', 'w')
A = []
for row in matrix:
A.append(row)
print A
I have tried the following:
matrix = open('workfile', 'w')
A = np.array([])
for row in matrix:
A = numpy.append(row)
print A
It does not return the desired output, as in the list.
Edit this is the sample code:
mat = scipy.io.loadmat('file.mat')
var1 = mat['data1']
A = np.array([])
for row in var1:
np.append(A, row)
print A
This is just the simplest case of what I want to do, but there is more data processing in the loop, I am putting it this way so the example is clear.
You need to pass the array, A, to Numpy.
matrix = open('workfile', 'w')
A = np.array([])
for row in matrix:
A = numpy.append(A, row)
print A
However, loading from the files directly is probably a nicer solution.
I have 3000000 ints' long array which I want to output to a file. How can I do that?
Also, is this
for i in range(1000):
for k in range(1000):
(r, g, b) = rgb_im.getpixel((i, k))
rr.append(r)
gg.append(g)
bb.append(b)
d.extend(rr)
d.extend(gg)
d.extend(bb)
a good practice to join array together?
All of the arrays are declared like this d = array('B')
EDIT:
Managed to output all int`s delimited by ' ' with this
from PIL import Image
import array
side = 500
for j in range(1000):
im = Image.open(r'C:\Users\Ivars\Desktop\RS\Shape\%02d.jpg' % (j))
rgb_im = im.convert('RGB')
d = array.array('B')
rr = array.array('B')
gg = array.array('B')
bb = array.array('B')
f = open(r'C:\Users\Ivars\Desktop\RS\ShapeData\%02d.txt' % (j), 'w')
for i in range(side):
for k in range(side):
(r, g, b) = rgb_im.getpixel((i, k))
rr.append(r)
gg.append(g)
bb.append(b)
d.extend(rr)
d.extend(gg)
d.extend(bb)
o = ' '.join(str(t) for t in d)
print('#', j, ' - ', len(o))
f.write(o)
f.close()
if you're using python >= 2.6 then you can use print functions from the future!
from __future__ import print_function
#your code
# This will print out a string representation of list to the file.
# If you need it formatted differently, then you'll have to construct the string yourself
print(d, file=open('/path/to/file.txt','w')
#you can join the list items with an empty string to get only the numbers
print("".join(d),file=('/path/to/file.txt','w'))
This has the side effect of turning print from a statement into a function, so you'll have to wrap whatever you want printed in ()
You want tofile(), which requires you to open a file object. See https://docs.python.org/2/library/array.html and https://docs.python.org/2/library/stdtypes.html#bltin-file-objects. Also, have you considered using NumPy?
import array
a = array.array('B')
b = array.array('B')
a.append(3)
a.append(4)
print a
print b
with open('c:/test.dat', 'w') as f:
a.tofile(f)
with open('c:/test.dat', 'r') as f:
b.fromfile(f, 2)
print b
EDIT: Based on your edit, you can use numpy with PIL and generate the array in a line or two, without looping. See, e.g., Conversion between Pillow Image object and numpy array changes dimension for example code.
I am trying to write out a line to a new file based on input from a csv file, with elements from different rows and different columns for example
test.csv:
name1, value1, integer1, integer1a
name2, value2, integer2, integer2a
name3, value3, integer3, integer3a
desired output:
command integer1:integer1a moretext integer2:integer2a
command integer2:integer2a moretext integer3:integer3a
I realize this will probably some type of loop, I am just getting lost in the references for loop interation and python maps
Assuming python 2, you want the output in a text file, and you have the command and moretext saved as variables earlier in the code.
from csv import reader
f = reader(open('test.csv'))
data = [str(r[2]) +':' + str(r[3]) for r in f]
out = open('out.txt', 'w')
for i in range(len(data)-1):
print >> out, command + data[i] + moretext + data[i+1]
out.close()
Easiest to adapt to your need would be to build a list of tuples from your file :
data = []
for l in open('file.csv'):
data.append( l.strip().split() )
at this point data is a list of quadruples. So you can do your example like this:
for i in range(len(data)/2):
_,_,i1,i2 = data[2*i]
_,_,j1,j2 = data[2*i+1]
print('command {}:{} moretext {}:{}'.format( i1,i2,j1,j2 ))
Here I use _ to say that I don't care about the two first variables of the quadruple. Hence, I don't even named them. It's a nice feature to write clear code.
You can also do this in a single loop:
f = open('file.csv')
while True:
l1 = f.readline()
l2 = f.readline()
if not l1 or not l2: break # file ended
_,_,i1,i2 = l1.strip().split()
_,_,j1,j2 = l2.strip().split()
print('command {}:{} moretext {}:{}'.format( i1,i2,j1,j2 ))
Here's a simple and general function that takes any iterable and generates all sequential pairs (e.g., 1, 2, 3 becomes (1, 2), (2, 3)):
def pairwise(iterable):
it = iter(iterable)
a = next(it)
for b in it:
yield a, b
a = b
This can then be used to solve your particular problem as follows:
with open('outputfile', 'w') as out:
for (_, _, a1, a2), (_, _, b1, b2) in pairwise(
[w.strip() for w in l.split(',')] for l in open('test.csv')):
out.write('command %s:%s moretext %s:%s\n' % (a1, a2, b1, b2))
One advantage of doing it this way is that you don't read the whole input into memory before starting the output, thus it will work well for streaming and arbitrarily large files.