I'm trying to write a code that takes a file name as an input. Finds this file on my computer and then changes the name of the file according to the text on the 2 first lines of the file.
import os
filename = input("Enter your file name: ")
def info(filename):
with open(filename, 'r') as filehandle:
current_line = 1
for line in filehandle:
if current_line <=2:
yield(line)
current_line += 1
info = list(info(filename))
print(info)
path = r'C:\Users\marku\Desktop\INF100'
date = str(info[1])
place = str(info[0])
finalname = date + '_' + place + '.txt'
old = os.path.join(path, filename)
new = os.path.join(path, finalname)
os.rename(old, new)
However, I get WinError 123 when trying to run this code.
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect:
'C:\\Users\\marku\\Desktop\\INF100\\qwerty.txt' -> 'C:\\Users\\marku\\Desktop\\INF100\\2019-06-01\n_Oslo\n.txt'
Related
in the error output message I have the .json file in a streamed tweets folder. The data exists im not sure what is wrong with my path
import json
import os, os.path
# counter keeps track of the unique tweet ids and is used later when
# prepending the ElasticSearch compatibility
counter = 1
# find all files in folderPath, does not include subdirectories or directories
folderPath = r"c:/Users/Katherine/Downloads/Final Project-.zip/Final
Project/122_finalproject_part2/streamed_tweets";
files = next(os.walk(folderPath))[2]
# filecount is keeps track of how many files are in the streamedtweets folder
filecount = len(files) - 1
print(filecount)
ctr = 0
for ctr in range(filecount):
inFileName = open('streamedtweets/tweet_data_' + str(filecount) + '.json', 'r')
outFileName = open('elastic_data/elastictwitter_data_' + str(filecount) + '.json', 'w')
ERROR
File "add_elastic.py", line 16, in <module>
inFileName = open('streamedtweets/tweet_data_' + str(filecount) + '.json', 'r')
IOError: [Errno 2] No such file or directory: 'streamedtweets/tweet_data_32.json'
You expected tweet_data_ to be appended with 0, 1, 2, 3... but in actuality you're always appending 32 which doesn't seem to exist. A better way would be to loop directly through the files you have instead of guessing its name.
Try doing this instead:
folderPath = r"c:/Users/Katherine/Downloads/Final Project-.zip/Final
Project/122_finalproject_part2/streamed_tweets"
files = next(os.walk(folderPath))[2]
for i, file in enumerate(files):
if file.startswith('tweet_data_') and file.endswith('.json'):
inFileName = open(file, 'r')
outFileName = open('elastic_data/elastictwitter_data_' + str(i) + '.json', 'w')
And even better way would be to use the with statement to manage your files opening and closing:
for i, file in enumerate(files):
if file.startswith('tweet_data_') and file.endswith('.json'):
with open(file, 'r') as inFileName, open('elastic_data/elastictwitter_data_' + str(i) + '.json', 'w') as outFileName:
# do something with inFileName / outFileName
I am trying to read all fasta files from test folder and put the name of file in all headers of individual file. The code working for first file and dont proceed to second file and return error. Could you help me find bug in my code or edit it. Thanks
import sys, glob, os, string
header = ''
check = 0
path = "./test/"
dirs = os.listdir(path)
for file in dirs:
fp = open(file, "r")
fpx = open('%s_output.txt' % file, 'w')
for line in fp:
if line.startswith('>'):
line = line.rstrip()
check = check + 1
if check >= 1:
header = line
fpx.write(header + '_' + file + '\n')
else:
line = line.rstrip()
fpx.write(line + '\n')
It would be good to provide the error message you are getting! I think this must fail with "File not found" because you try to open the file by name instead of path. Try fp = open(os.path.join(path, file), "r"):
import sys, glob, os, string
header = ''
check = 0
path = "./test/"
dirs = os.listdir(path)
for file in dirs:
fp = open(os.path.join(path, file), "r")
fpx = open('%s_output.txt' % file, 'w')
for line in fp:
if line.startswith('>'):
line = line.rstrip()
check = check + 1
if check >= 1:
header = line
fpx.write(header + '_' + file + '\n')
else:
line = line.rstrip()
fpx.write(line + '\n')
So I'm writing a script to take large csv files and divide them into chunks. These files each have lines formatted accordingly:
01/07/2003,1545,12.47,12.48,12.43,12.44,137423
Where the first field is the date. The next field to the right is a time value. These data points are at minute granularity. My goal is to fill files with 8 days worth of data, so I want to write all the lines from a file for 8 days worth into a new file.
Right now, I'm only seeing the program write one line per "chunk," rather than all the lines. Code shown below and screenshots included showing how the chunk directories are made and the file as well as its contents.
For reference, day 8 shown and 1559 means it stored the last line right before the mod operator became true. So I'm thinking that everything is getting overwritten somehow since only the last values are being stored.
import os
import time
CWD = os.getcwd()
WRITEDIR = CWD+"/Divided Data/"
if not os.path.exists(WRITEDIR):
os.makedirs(WRITEDIR)
FILEDIR = CWD+"/SP500"
os.chdir(FILEDIR)
valid_files = []
filelist = open("filelist.txt", 'r')
for file in filelist:
cur_file = open(file.rstrip()+".csv", 'r')
cur_file.readline() #skip first line
prev_day = ""
count = 0
chunk_count = 1
for line in cur_file:
day = line[3:5]
WDIR = WRITEDIR + "Chunk"
cur_dir = os.getcwd()
path = WDIR + " "+ str(chunk_count)
if not os.path.exists(path):
os.makedirs(path)
if(day != prev_day):
# print(day)
prev_day = day
count += 1
#Create new directory
if(count % 8 == 0):
chunk_count += 1
PATH = WDIR + " " + str(chunk_count)
if not os.path.exists(PATH):
os.makedirs(PATH)
print("Chunk count: " + str(chunk_count))
print("Global count: " + str(count))
temp_path = WDIR +" "+str(chunk_count)
os.chdir(temp_path)
fname = file.rstrip()+str(chunk_count)+".csv"
with open(fname, 'w') as f:
try:
f.write(line + '\n')
except:
print("Could not write to file. \n")
os.chdir(cur_dir)
if(chunk_count >= 406):
continue
cur_file.close()
# count += 1
The answer is in the comment but let me give it here so that your question is answered.
You're opening your file in 'w' mode which overwrites all the previously written content. You need to open it in the 'a' (append) mode:
fname = file.rstrip()+str(chunk_count)+".csv"
with open(fname, 'a') as f:
See more on open function and modes in Python documentation. It specifically mentions about 'w' mode:
note that 'w+' truncates the file
I'm trying to find a string in files contained within a directory. I have a string like banana that I know that exists in a few of the files.
import os
import sys
user_input = input("What is the name of you directory?")
directory = os.listdir(user_input)
searchString = input("What word are you trying to find?")
for fname in directory: # change directory as needed
if searchString in fname:
f = open(fname,'r')
print('found string in file %s') %fname
else:
print('string not found')
When the program runs, it just outputs string not found for every file. There are three files that contain the word banana, so the program isn't working as it should. Why isn't it finding the string in the files?
You are trying to search for string in filename, use open(filename, 'r').read():
import os
user_input = input('What is the name of your directory')
directory = os.listdir(user_input)
searchstring = input('What word are you trying to find?')
for fname in directory:
if os.path.isfile(user_input + os.sep + fname):
# Full path
f = open(user_input + os.sep + fname, 'r')
if searchstring in f.read():
print('found string in file %s' % fname)
else:
print('string not found')
f.close()
We use user_input + os.sep + fname to get full path.
os.listdir gives files and directories names, so we use os.path.isfile to check for files.
Here is another version using the Path module from pathlib instead of os.
def search_in_file(path,searchstring):
with open(path, 'r') as file:
if searchstring in file.read():
print(f' found string in file {path.name}')
else:
print('string not found')
from pathlib import Path
user_input = input('What is the name of your directory')
searchstring = input('What word are you trying to find?')
dir_content = sorted(Path(user_input).iterdir())
for path in dir_content:
if not path.is_dir():
search_in_file(path, searchstring)
This is my solution for the problem. It comes with the feature of also checking in sub-directories, as well as being able to handle multiple file types. It is also quite easy to add support for other ones. The downside is of course that it's quite chunky code. But let me know what you think.
import os
import docx2txt
from pptx import Presentation
import pdfplumber
def findFiles(strings, dir, subDirs, fileContent, fileExtensions):
# Finds all the files in 'dir' that contain one string from 'strings'.
# Additional parameters:
# 'subDirs': True/False : Look in sub-directories of your folder
# 'fileContent': True/False :Also look for the strings in the file content of every file
# 'fileExtensions': True/False : Look for a specific file extension -> 'fileContent' is ignored
filesInDir = []
foundFiles = []
filesFound = 0
if not subDirs:
for filename in os.listdir(dir):
if os.path.isfile(os.path.join(dir, filename).replace("\\", "/")):
filesInDir.append(os.path.join(dir, filename).replace("\\", "/"))
else:
for root, subdirs, files in os.walk(dir):
for f in files:
if not os.path.isdir(os.path.join(root, f).replace("\\", "/")):
filesInDir.append(os.path.join(root, f).replace("\\", "/"))
print(filesInDir)
# Find files that contain the keyword
if filesInDir:
for file in filesInDir:
print("Current file: "+file)
# Define what is to be searched in
filename, extension = os.path.splitext(file)
if fileExtensions:
fileText = extension
else:
fileText = os.path.basename(filename).lower()
if fileContent:
fileText += getFileContent(file).lower()
# Check for translations
for string in strings:
print(string)
if string in fileText:
foundFiles.append(file)
filesFound += 1
break
return foundFiles
def getFileContent(filename):
'''Returns the content of a file of a supported type (list: supportedTypes)'''
if filename.partition(".")[2] in supportedTypes:
if filename.endswith(".pdf"):
content = ""
with pdfplumber.open(filename) as pdf:
for x in range(0, len(pdf.pages)):
page = pdf.pages[x]
content = content + page.extract_text()
return content
elif filename.endswith(".txt"):
with open(filename, 'r') as f:
content = ""
lines = f.readlines()
for x in lines:
content = content + x
f.close()
return content
elif filename.endswith(".docx"):
content = docx2txt.process(filename)
return content
elif filename.endswith(".pptx"):
content = ""
prs = Presentation(filename)
for slide in prs.slides:
for shape in slide.shapes:
if hasattr(shape, "text"):
content = content+shape.text
return content
else:
return ""
supportedTypes = ["txt", "docx", "pdf", "pptx"]
print(findFiles(strings=["buch"], dir="C:/Users/User/Desktop/", subDirs=True, fileContent=True, fileExtensions=False))
Here is the most simple answer I can give you. You don't need the colors, they are just cool and you may find that you can learn more than one thing in my code :)
import os
from time import sleep
#The colours of the things
class bcolors:
HEADER = '\033[95m'
OKBLUE = '\033[94m'
OKGREEN = '\033[92m'
WARNING = '\033[93m'
FAIL = '\033[91m'
ENDC = '\033[0m'
BOLD = '\033[1m'
UNDERLINE = '\033[4m'
# Ask the user to enter string to search
search_path = input("Enter directory path to search : ")
file_type = input("File Type : ")
search_str = input("Enter the search string : ")
# Append a directory separator if not already present
if not (search_path.endswith("/") or search_path.endswith("\\") ):
search_path = search_path + "/"
# If path does not exist, set search path to current directory
if not os.path.exists(search_path):
search_path ="."
# Repeat for each file in the directory
for fname in os.listdir(path=search_path):
# Apply file type filter
if fname.endswith(file_type):
# Open file for reading
fo = open(search_path + fname, 'r')
# Read the first line from the file
line = fo.read()
# Initialize counter for line number
line_no = 1
# Loop until EOF
if line != '' :
# Search for string in line
index = line.find(search_str)
if ( index != -1) :
print(bcolors.OKGREEN + '[+]' + bcolors.ENDC + ' ', fname, sep="")
print(' ')
sleep(0.01)
else:
print(bcolors.FAIL + '[-]' + bcolors.ENDC + ' ', fname, ' ', 'does not contain', ' ', search_str, sep="")
print(" ")
sleep(0.01)
line = fo.readline()
# Increment line counter
line_no += 1
# Close the files
fo.close()
That is it!
I was trying with the following code for this kind of problem, please have a look.
import os,sys
search_path=input("Put the directory here:")
search_str = input("Enter your string")
# Append a directory separator if not already present
if not (search_path.endswith("/") or search_path.endswith("\\") ):
search_path = search_path + "/"
# If path does not exist, set search path to current directory
if not os.path.exists(search_path):
search_path ="."
# Repeat for each file in the directory
for fname in os.listdir(path=search_path):
# Apply file type filter
if fname.endswith(file_type):
# Open file for reading
fo = open(search_path + fname)
# Read the first line from the file
line = fo.readline()
# Initialize counter for line number
line_no = 1
# Loop until EOF
while line != '' :
# Search for string in line
index = line.find(search_str)
if ( index != -1) :
print(fname, "[", line_no, ",", index, "] ", line, sep="")
# Read next line
line = fo.readline()
# Increment line counter
line_no += 1
# Close the files
fo.close()
I'm trying to write a script that allows a user to create a folder with any name they want, and then create a file with any name they want. Once they do they, the program asks them for 3 names and writes them into the file. I then want to allow the user to input a number from 1 to 3 and display the number of lines they want. I'm getting an error right now when trying to read the file saying something along the lines of
TypeError: invalid file: <_io.TextIOWrapper name='C:blah blah ' mode='a' encoding='cp1252'>
The code is below:
import os, sys
folder = input("What would you like your folder name to be?")
path = r'C:\Users\Administrator\Desktop\%s' %(folder)
if not os.path.exists(path): os.makedirs(path)
file = input("What name would you like for the file in this folder?")
file = file + ".txt"
completePath = os.path.join(path, file)
newFile = open(completePath, 'w')
newFile.close()
count = 0
while count < 3:
newFile = open(completePath, 'a')
write = input("Input the first and last name of someone: ")
newFile.write(write + '\n')
newFile.close()
count += 1
infile = open(newFile, 'r')
display = int(input("How many names from 1 to 10 would you like to display? "))
print (infile.readlines(5))
You have newFile defiled as an opened file. Then you open it within a while loop, and it is a file, again.
And when you try then to open a file using the newFile variable, Python tries to open a file with a name, contained in a newFile variable. But it is not a file name - it is a file!
This makes Python sad...
Try this one:
import os, sys
folder = input("What would you like your folder name to be?")
path = r'C:\Users\Administrator\Desktop\%s' %(folder)
if not os.path.exists(path): os.makedirs(path)
file = input("What name would you like for the file in this folder?")
file = file + ".txt"
completePath = os.path.join(path, file) # completePath is a string
newFile = open(completePath, 'w') # here, newFile is a file handle
newFile.close()
count = 0
while count < 3:
newFile = open(completePath, 'a') # again, newFile is a file handle
write = input("Input the first and last name of someone: ")
newFile.write(write + '\n')
newFile.close()
count += 1
infile = open(completePath, 'r') # opening file with its path, not its handle
infile.readlines(2)