Develop Reference

How to delete several lines in txt with conditional

everyone! I have just started learning python
I have a problem with some txt file
I want to delete all data with KMAG less than 5.5
but I have no idea any suggestions?
code below just what I could
file = open("experiment.txt", "r")
for line in file:
if 'KMAG' in line:
print(line)
file.close()
enter image description here

You need to do two things. First, it appears that this file has multiline records delimited by a line with a single decimal number. Use that to read the file a record at a time:
import re
from decimal import Decimal
def get_records(fileobj):
record = []
for line in fileobj:
if re.match(r"\s*\d+\s*$", line):
# got new record, emit old
if record:
yield record
record = [line]
else:
record.append(line)
if record:
yield record
return
Now you can peek into each record to see if its data you want to keep. I'm using the decimal module because because the python binary float does not exactly represent decimal floats.
min_val = Decimal("5.5")
with open("experiment.txt") as infile, open("foo.txt", "w") as outfile:
for record in get_records(infile):
# we got record number\nheader\ndata with kmag\n...
kmag = re.split(r"\s+", record[2].strip())[-1]
if Decimal(kmag) >= min_val:
outfile.writelines(record)

Write a program that writes 100 random integers created randomly into a file. Python File i/o

import random
afile = open("Random_intger.txt", "w")
for i in range(input("The 100 random integers written are: ")):
line = str(random.randint(1,100))
afile.write(line)
print(line)
afile.close()
print("\nReading the file now." )
afile = open("Random_integer.txt", "r")
print(afile.read())
afile.close()
When I run it:
It says TypeError: 'str' object cannot be interpreted as an integer
It creates the file labeled Random_intger.txt but there are no integers.
also, I'm using a MacBook Air, is that part of the problem?

Make the below changes in your code.
for i in range(**int(input("The 100 random integers written are: "))**):
you need to convert the data from stdin to integer, the default type from the input function is a string.
I hope this will solve your issue.

There are multiple problems in your code. The first one is that random.randint(1,100) is not giving you 100 random numbers but a single random value between 1 (inclusive) and 100 (inclusive) and that your for loop is a bit buggy (don't use input here, or do you want to read something from stdin?).
Next thing: You are opening the file "Random_intger.txt" to write your numbers to. But you read from the file "Random_integer.txt" ...
Fixed code:
import random
filename = "Random_integer.txt"
# use a with statement. Like this you don't need to
# remember to close the stream ...
with open(filename, "w") as afile:
print("The 100 random integers written are: ")
for i in range(100):
line = str(random.randint(1,100))
afile.write(line)
afile.write("\n")
print(line)
print("\nReading the file now." )
with open(filename, "r") as afile:
print(afile.read())

import random
out_file = "Random_integer.txt"
afile = open(out_file, "w")
for i in range(100):
line = str(random.randint(1,100)) + '\n'
afile.write(line)
print(line)
afile.close()
print("Reading the file now." )
afile = open(out_file, "r")
print(afile.read())
afile.close()

Python name files to its corresponding value in a csv

Let's say I have a csv file with two columns:
HEX;TITLE
0xAB;BN4
0xAC;ZF4
0xAD;ET6
I have a python script that reads the csv and transforms each hex to a photo, and writes it to a jpeg. This is the script so you can see what it does:
import binascii
import os
count = 0
with open('photos.txt', 'r') as f:
for i in f:
count = count + 1
photo = i[2:]
cleaned = photo.strip("\r\n")
transform = binascii.a2b_hex(cleaned)
with open("{}.jpg".format(count), 'wb') as output:
output.write(transform)
f.close()
The count in my script gives the file a name and it is incremental. In the folder I will see: 1.jpeg, 2.jpeg, 3.jpeg and so on.
But my question is: How do I name those files to its corresponding TITLE-value in the csv? So that the first file will be BN4.jpeg, and the second file ZF4.jpeg?
FYI: the input file has currently one column containing the hex for the script above.

use the csv module to get 2 separate fields. No need to strip, just unpack the data while reading:
with open('photos.csv', 'r') as f:
cr = csv.reader(f,delimiter=";")
for photo,filename in cr:
transform = binascii.a2b_hex(photo[2:])
with open("{}.jpg".format(filename), 'wb') as output:
output.write(transform)

Convert a text CSV binary file and get a random line from it in Python without reading it to memory

I have some CSV text files in the format:
1.3, 0, 1.0
20.0, 3.2, 0
30.5, 5.0, 5.2
The files are about 3.5Gb in size and I cannot read any of them in to memory in Pandas in a useful amount of time.
But I don't need to read the all file, because what I want to do, is to choose some random lines from the file and read the values there, and I know it's theoretically possible to do it if the file is formatted in a way that all the fields have the same size - for instance, float16 in a binary file.
Now, I think I can just convert it, using the NumPy method specified in the answer to question:
How to output list of floats to a binary file in Python
But, how do I go about picking a random line from it after the conversion is done?
In a normal text file, I could just do:
import random
offset = random.randrange(filesize)
f = open('really_big_file')
f.seek(offset) #go to random position
f.readline() # discard - bound to be partial line
random_line = f.readline() # bingo!
But I can't find a way for this to work in a binary file made from NumPy.

I'd use struct to convert to binary:
import struct
with open('input.txt') as fin, open('output.txt','wb') as fout:
for line in fin:
#You could also use `csv` if you're not lazy like me ...
out_line = struct.pack('3f',*(float(x) for x in line.split(',')))
fout.write(out_line)
This writes everything as standard 4-byte floats on most systems.
Now, to read the data again:
with open('output.txt','rb') as fin:
line_size = 12 #each line is 12 bytes long (3 floats, 4 bytes each)
offset = random.randrange(filesize//line_size) #pick n'th line randomly
f.seek(offset*line_size) #seek to position of n'th line
three_floats_bytes = f.read(line_size)
three_floats = struct.unpack('3f',three_floats_bytes)
If you're concerned about disk space and want to compress the data down using np.float16 (2 byte floats), you can do that too using the basic skeleton above, just substitute np.fromstring for struct.unpack and ndarray.tostring in place of struct.pack (with the appropriate data-type ndarray of course -- and line_size would drop to 6 ...).

You'd have to play around with offsets depending on storage size, but:
import csv
import struct
import random
count = 0
with open('input.csv') as fin, open('input.dat', 'wb') as fout:
csvin = csv.reader(fin)
for row in csvin:
for col in map(float, row):
fout.write(struct.pack('f', col))
count += 1
with open('input.dat', 'rb') as fin:
i = random.randrange(count)
fin.seek(i * 4)
print struct.unpack('f', fin.read(4))

So, using the example provided by the helpfull answers, I found a way to do it with NumPy if someone is interested:
# this converts the file from text CSV to bin
with zipfile.ZipFile("input.zip", 'r') as inputZipFile:
inputCSVFile = inputZipFile.open(inputZipFile.namelist()[0], 'r') # it's 1 file only zip
with open("output.bin", 'wb') as outFile:
outCSVFile = csv.writer(outFile, dialect='excel')
for line in inputCSVFile:
lineParsed = ast.literal_eval(line)
lineOut = numpy.array(lineParsed,'float16')
lineOut.tofile(outFile)
outFile.close()
inputCSVFile.close()
inputZipFile.close()
# this reads random lines from the binary file
with open("output.bin", 'wb') as file:
file.seek(0)
lineSize = 20 # float16 has 2 bytes and there are 10 values:
fileSize = os.path.getsize("output.bin")
offset = random.randrange(fileSize//lineSize)
file.seek(offset * lineSize)
random_line = file.read(lineSize)
randomArr = numpy.fromstring(random_line, dtype='float16')

Reading binary file (.chn) in Python

In python, how do I read a binary file (here I need to read a .chn file) and show the result in binary format?

Assuming that values are separated by a space:
with open('myfile.chn', 'rb') as f:
data = []
for line in f: # a file supports direct iteration
data.extend(hex(int(x, 2)) for x in line.split())
In Python is better to use open() over file(), documentation says it explicitly:
When opening a file, it’s preferable to use open() instead of invoking
the file constructor directly.
rb mode will open the file in binary mode.
Reference:
http://docs.python.org/library/functions.html#open

try this:
with open('myfile.chn') as f:
data=f.read()
data=[bin(ord(x)).strip('0b') for x in data]
print ''.join(data)
and if you want only the binary data it will be in the list.
with open('myfile.chn') as f:
data=f.read()
data=[bin(ord(x)).strip('0b') for x in data]
print data
In data now you will have the list of binary numbers. you can take this and convert to hexadecimal number

with file('myfile.chn') as f:
data = f.read() # read all strings at once and return as a list of strings
data = [hex(int(x, 2)) for x in data] # convert to a list of hex strings (by interim getting the decimal value)