Develop Reference

Why is my file not getting overwritten from the start in Python and why can't I read entire lines?

So, I have a program which gets a first random number and then two other numbers stored in file, in the form of two variables. My problems start from the moment I try to get the variables from the file.
import random
tries = random.randint(0, 20)
output = open("Results", "r+")
average = int(output.readline(1))
coef = int(output.readline(2))
first_line = (average * coef + tries)
first_line = str(first_line)
second_line = coef + 1
second_line = str(second_line)
output_text = first_line + "\n" + second_line
output.write(output_text)
output.close()
I already have a Results file with previous results which looks like :
11
1
After running the program once and getting tries = 13, the Results file looks like that :
11
114
2
Where I would like to have :
24
2
I think that the problem comes partially from the fact that when I try to read the past results, turns out that average = 1 and coef = 1 where average is suppose to be equal to 11. And that the other part comes from the moment when I try to write them back onto my file.
P.S. : I am using Python 3.9.6 and Spyder from the Anaconda distribution.
P.S.S. : What's wrong with my post ? I just want to know the reason to make my future posts better.

So thanks, to #ThePhoton and some googling I managed to get it all working so this is my new code:
import random
tries = random.randint(0, 20)
output = open("Results", "r")
somme = int(output.readline())
coef = int(output.readline())
output.close()
output = open("Results", "w")
first_line = somme + tries
second_line = coef + 1
first_line = str(first_line)
second_line = str(second_line)
output_text = [first_line, "\n", second_line]
output.writelines(output_text)
output.close()

Python reading rows from csv, operating and organizing rows of numbers

I am a non-programmer geographer, heard some programming concepts but very newby :-)
I am to read six rows of environmental data. 1000 lines at the most, each time.
Each row housing two digit numbers (0 to 99) a summer issue, only positive numbers.
Once I read them I am to display the numbers 0 to 99 vertically with the number of occurrences for the reading for each of the six rows:
0 = 230.....0 = 3........0 = 230......0 = 123......0 = 223......0 = 334
1 = 67......1 = 657......1 = 627......1 = 767......1 = 467......1 = 337
2 = 762.....2 = 328......2 = 987......2 = 326......2 = 32.......2 = 123
.
.
99 = 3.....99 = 34.......99 = 1.......99 = 89......99 = 78......99 = 123
If I can get this far I will feel great. Once I learn how to do this and I can look at the data I can decide what makes sense to run next; excel, graphs, statistics, statistics in R, get the numbers into a matrix to manipulate from there, etc. First time so I am figuring this out as I go.
Any help will be much appreciated,
Adolfo
I am working in the research for the restoration of Quebrada Verde watershed in Valparaiso, Chile.

from array import array
import sys
if len(sys.argv) > 1:
count = array('H', [0]*100)
file = open(sys.argv[1], 'r')
if file:
for line in file:
count[int(line)]+=1
file.close()
for a in range (100):
print(a, count[a], sep='\t')
else:
print('unable to open the file')
else:
print('usage: python', sys.argv[0], ' file')

Convert file to binary code in Python

I am looking to convert a file to binary for a project, preferably using Python as I am most comfortable with it, though if walked-through, I could probably use another language.
Basically, I need this for a project I am working on where we want to store data using a DNA strand and thus need to store files in binary ('A's and 'T's = 0, 'G's and 'C's = 1)
Any idea how I could proceed? I did find that use could encode in base64, then decode it, but it seems a bit inefficient, and the code that I have doesn't seem to work...
import base64
import tkinter as tk
from tkinter import filedialog
root = tk.Tk()
root.withdraw()
file_path = filedialog.askopenfilename()
print(file_path)
with open(file_path) as f:
encoded = base64.b64encode(f.readlines())
print(encoded)
Also, I already have a program to do that simply with text. Any tips on how to improve it would also be appreciated!
import binascii
t = bytearray(str(input("Texte?")), 'utf8')
h = binascii.hexlify(t)
b = bin(int(h, 16)).replace('b','')
#removing the b that appears in the end for some reason
g = b.replace('1','G').replace('0','A')
print(g)
For example, if I input test:
ok so for the text to DNA:
I input 'test' and expect the DNA sequence that comes from the binary
the binary being: 01110100011001010111001101110100 (Also I asked to print every conversion in the example so that it is more comprehensible)
>>>Texte?test #Asks the text
>>>b'74657374' #converts to hex
>>>01110100011001010111001101110100 #converts to binary
>>>AGGGAGAAAGGAAGAGAGGGAAGGAGGGAGAA #converts 0 to A and 1 to G

So, thanks to #jonrshape and Sergey Vturin, I finally was able to achieve what I wanted!
My program asks for a file, turns it into binary, which then gives me its equivalent in "DNA code" using pairs of binary numbers (00 = A, 01 = T, 10 = G, 11 = C)
import binascii
from tkinter import filedialog
file_path = filedialog.askopenfilename()
x = ""
with open(file_path, 'rb') as f:
for chunk in iter(lambda: f.read(32), b''):
x += str(binascii.hexlify(chunk)).replace("b","").replace("'","")
b = bin(int(x, 16)).replace('b','')
g = [b[i:i+2] for i in range(0, len(b), 2)]
dna = ""
for i in g:
if i == "00":
dna += "A"
elif i == "01":
dna += "T"
elif i == "10":
dna += "G"
elif i == "11":
dna += "C"
print(x) #hexdump
print(b) #converted to binary
print(dna) #converted to "DNA"

Of course, it is inefficient!
base64 is designed to store binary in a text. It makes a bigger size block after conversion.
btw: what efficiency do you want? compactness?
if so: second sample is much nearer to what you want
btw: in your task you loose information! Are you aware of this?
Here is a sample how to store and restore.
It stores data in an easy to understand Hex-In-Text format -- just for the sake of a demo. If you want compactness - you can easily modify the code so as to store in binary file or if you want 00011001 view - modification will be easy too.
import math
#"make a long test string"
import numpy as np
s=''.join((str(x) for x in np.random.randint(4,size=33)))\
.replace('0','A').replace('1','T').replace('2','G').replace('3','C')
def store_(s):
size=len(s) #size will changed to fit 8*integer so remember true value of it and store with data
s2=s.replace('A','0').replace('T','0').replace('G','1').replace('C','1')\
.ljust( int(math.ceil(size/8.)*8),'0') #add '0' to 8xInt to the right
a=(hex( eval('0b'+s2[i*8:i*8+8]) )[2:].rjust(2,'0') for i in xrange(len(s2)/8))
return ''.join(a),size
yourDataAsHexInText,sizeToStore=store_(s)
print yourDataAsHexInText,sizeToStore
def restore_(s,size=None):
if size==None: size=len(s)/2
a=( bin(eval('0x'+s[i*2:i*2+2]))[2:].rjust(8,'0') for i in xrange(len(s)/2))
#you loose information, remember?, so it`s only A or G
return (''.join(a).replace('1','G').replace('0','A') )[:size]
restore_(yourDataAsHexInText,sizeToStore)
print "so check it"
print s ,"(input)"
print store_(s)
print s.replace('C','G').replace('T','A') ,"to compare with information loss"
print restore_(*store_(s)),"restored"
print s.replace('C','G').replace('T','A') == restore_(*store_(s))
result in my test:
63c9308a00 33
so check it
AGCAATGCCGATGTTCATCGTATACTTTGACTA (input)
('63c9308a00', 33)
AGGAAAGGGGAAGAAGAAGGAAAAGAAAGAGAA to compare with information loss
AGGAAAGGGGAAGAAGAAGGAAAAGAAAGAGAA restored
True

python csv delimiter doesn't work properly

I try to write a python code to extract DVDL values from the input. Here is the truncated input.
A V E R A G E S O V E R 50000 S T E P S
NSTEP = 50000 TIME(PS) = 300.000 TEMP(K) = 300.05 PRESS = -70.0
Etot = -89575.9555 EKtot = 23331.1725 EPtot = -112907.1281
BOND = 759.8213 ANGLE = 2120.6039 DIHED = 4231.4019
1-4 NB = 940.8403 1-4 EEL = 12588.1950 VDWAALS = 13690.9435
EELEC = -147238.9339 EHBOND = 0.0000 RESTRAINT = 0.0000
DV/DL = 13.0462
EKCMT = 10212.3016 VIRIAL = 10891.5181 VOLUME = 416404.8626
Density = 0.9411
Ewald error estimate: 0.6036E-04
R M S F L U C T U A T I O N S
NSTEP = 50000 TIME(PS) = 300.000 TEMP(K) = 1.49 PRESS = 129.9
Etot = 727.7890 EKtot = 115.7534 EPtot = 718.8344
BOND = 23.1328 ANGLE = 36.1180 DIHED = 19.9971
1-4 NB = 12.7636 1-4 EEL = 37.3848 VDWAALS = 145.7213
EELEC = 739.4128 EHBOND = 0.0000 RESTRAINT = 0.0000
DV/DL = 3.7510
EKCMT = 76.6138 VIRIAL = 1195.5824 VOLUME = 43181.7604
Density = 0.0891
Ewald error estimate: 0.4462E-04
Here is the script. Basically we have a lot of DVDL in the input (not in the above truncated input) and we only want the last two. So we read all of them into a list and only get the last two. Finally, we write the last two DVDL in the list into a csv file. The desire output is
13.0462, 3.7510
However, the following script (python 2.7) will bring the output like this. Could any guru enlighten? Thanks.
13.0462""3.7510""
Here is the script:
import os
import csv
DVDL=[]
filename="input.out"
file=open(filename,'r')
with open("out.csv",'wb') as outfile: # define output name
line=file.readlines()
for a in line:
if ' DV/DL =' in a:
DVDL.append(line[line.index(a)].split(' ')[1]) # Extract DVDL number
print DVDL[-2:] # We only need the last two DVDL
yeeha="".join(str(a) for a in DVDL[-2:])
print yeeha
writer = csv.writer(outfile, delimiter=',',lineterminator='\n')#Output the list into a csv file called "outfile"
writer.writerows(yeeha)

As the commenter who proposed an approach has not had the chance to outline some code for this, here's how I'd suggest doing it (edited to allow optionally signed floating point numbers with optional exponents, as suggested by an answer to Python regular expression that matches floating point numbers):
import re,sys
pat = re.compile("DV/DL += +([+-]?(\d+(\.\d*)?|\.\d+)([eE][+-]?\d+)?)")
values = []
for line in open("input.out","r"):
m = pat.search(line)
if m:
values.append(m.group(1))
outfile = open("out.csv","w")
outfile.write(",".join(values[-2:]))
Having run this script:
$ cat out.csv
13.0462,3.7510
I haven't used the csv module in this case because it isn't really necessary for a simple output file like this. However, adding the following lines to the script will use csv to write the same data into out1.csv:
import csv
writer = csv.writer(open("out1.csv","w"))
writer.writerow(values[-2:])

Python read data from file and convert to double precision

I've been reading an ASCII data file using python. Then I covert the data into a numpy array.
However, I've noticed that the numbers are being rounded.
E.g. My original value from the file is: 2368999.932089
which python has rounded to: 2368999.93209
here is an example of my code:
import numpy as np
datafil = open("test.txt",'r')
tempvar = []
header = datafil.readline()
for line in datafil:
word = line.split()
char = word[0] # take the first element word[0] of the list
word.pop() # remove the last element from the list "word"
if char[0:3] >= '224' and char[0:3] < '225':
tempvar.append(word)
strvar = np.array(tempvar,dtype = np.longdouble) # Here I want to read all data as double
print(strvar.shape)
var = strvar[:,0:23]
print(var[0,22]) # here it prints 2368999.93209 but the actual value is 2368999.932089
Any ideas guys?
Abedin

I think this is not a problem of your code. It's the usual floating point representation in Python. See
https://docs.python.org/2/tutorial/floatingpoint.html
I think when you print it, print already formatted your number to str
In [1]: a=2368999.932089
In [2]: print a
2368999.93209
In [3]: str(a)
Out[3]: '2368999.93209'
In [4]: repr(a)
Out[4]: '2368999.932089'
In [5]: a-2368999.93209
Out[5]: -9.997747838497162e-07

I'm not totally sure what you're trying to do, but simplified with test.txt containing only
asdf
2368999.932089
and then the code:
import numpy as np
datafil = open("test.txt",'r')
tempvar = []
header = datafil.readline()
for line in datafil:
tempvar.append(line)
print(tempvar)
strvar = np.array(tempvar, dtype=np.float)
print(strvar.shape)
print(strvar)
I get the following output:
$ python3 so.py
['2368999.932089']
(1,)
[ 2368999.932089]
which seems to be working fine.
Edit: Updated with your provided line, so test.txt is
asdf
t JD a e incl lasc aper truean rdnnode RA Dec RArate Decrate metdr1 metddr1 metra1 metdec1 metbeta1 metdv1 metsl1 metarrJD1 beta JDej name 223.187263 2450520.619348 3.12966 0.61835 70.7196 282.97 171.324 -96.2738 1.19968 325.317 35.8075 0.662368 0.364967 0.215336 3.21729 -133.586 46.4884 59.7421 37.7195 282.821 2450681.900221 0 2368999.932089 EH2003
and the code
import numpy as np
datafil = open("test.txt",'r')
tempvar = []
header = datafil.readline()
for line in datafil:
tempvar.append(line.split(' '))
print(tempvar)
strvar = np.array(tempvar[0][-2], dtype=np.float)
print(strvar)
the last print still outputs 2368999.932089 for me. So I'm guessing this is a platform issue? What happens if you force dtype=np.float64 or dtype=np.float128? Some other sanity checks: have you tried spitting out the text before it is converted to a float? And what do you get from doing something like:
>>> np.array('2368999.932089')
array('2368999.932089',
dtype='<U14')
>>> float('2368999.932089')
2368999.932089