Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I use python in opensuse, my problem is i need to execute large data in my folder.
for example
python myprogram.py 20140101.txt
i still need to run for a lot data with like that naming (20140101.txt) ex 20140204 etc..
my question is how to make my program running automatically for all data together.
use bash like this:
for file in /dir/*.txt
do
python myprogram.py $file
done
For a pure python solution, have a look at fileinput.
It is part of the python standard library and let's you loop over files given via standard input or via a list of files, e.g.:
import fileinput
for line in fileinput.input():
process(line)
So you could do:
./python myprogram 2014*.txt
The glob module is useful for processing multiple files with different names.
https://docs.python.org/2/library/glob.html
A python solution would be to use "glob". It helps you creating lists of files' name based on a certain pattern. You can then loop through those filenames to execute your commands on. See example below.
import glob
txt_files = glob.glob("201401*.txt")
for txt in txt_files:
print txt
my_txt_file = open(txt, "r")
For further reference:
https://docs.python.org/3/library/glob.htm
Related
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 3 years ago.
Improve this question
I want to find the current working directory (cwd) with os library and write .txt file on it.
Something like this:
import os
data=["somedatahere"]
#get_the_current_directory
#if this_is_the_current_directory:
new_file=open("a_data.txt", "a")
new_file.write(data)
new_file.close()
It can be done using the os library, but the new pathlib is more convenient if you are using Python 3.4 or later:
import pathlib
data_filename = pathlib.Path(__file__).with_name('a_data.txt')
with open(data_filename, 'a') as file_handle:
file_handle.write('Hello, world\n')
Basically, the with_name function said, "same dir with the script, but with this name".
import os
You're halfway there!
The rest of it is:
print(os.getcwd())
Of course, you don't need to know that value,
as a_data.txt or ./a_data.txt suffices.
As a side note, you'd be better off closing with a with handler:
with open('a_data.txt', 'a') as new_file:
new_file.write(data)
Habitually using a Resource Manager means
never having to say, "sorry, forgot to close it!"
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 5 years ago.
Improve this question
I want to cat txt files from a folder and cat results should be shown in terminal (Obviously). I have tried using listdir() it but it doesn't work. Required some help!
a simple implementation uses glob to generate absolute paths to files with .txt extension, in a loop which reads the file and prints it on standard output:
import glob,sys
for filepath in sorted(glob.glob("path/to/directory/*.txt")):
with open(filepath) as f:
sys.stdout.write(f.read())
using fileinput allows to read all the files, and line by line, probably less memory intensive and shorter:
import glob,sys,fileinput
for f in fileinput.input(sorted(glob.glob("path/to/directory/*.txt"))):
sys.stdout.write(f)
note that sorted is better to ensure that the files are processed in deterministic order (some filesystems don't respect that order)
sys.stdout.write(f) still writes line by line, but as suggested by comments you could improve performance and not use so much memory by using shutil.copyfileobj(f,sys.stdout)
Just use the command in terminal "cat *"
Or if you want to do it in python:
import os
allTextFileContents = os.popen("cat *").read();
print(allTextFileContents);
You can also do other things with the contents since its stored as a variable!
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I have 732 .txt files in a folder.
I want to make a unique Pandas dataframe for each of them, but without doing it manually one by one.
Is there a way to open all the files in Python and to use a for loop to create dateframes for each one? Could someone provide me a starting code example?
Thank you!
It sounds like you're wanting to use Pandas to read a bunch of CSVs. If they're all in the same directory and nothing else with a .txt extension is in there, you can use
import glob
files = glob.glob('./*.txt')
to get the list of relevant files.
Then you can use a list comprehension to get a list of dataframes:
import pandas as pd
dfs = [pd.read_csv(f) for f in files]
with whatever params you need for read_csv in there.
The following pseudocode should open all the .txt files in any given directory and help you build a dataframe for each. It does not use glob because glob is slow:
import os
dir = '/where/your/txts/are/'
for filename in os.listdir(dir):
if filename.endswith('.txt'):
content = open(dir+filename, 'r').read()
dataframe = build_your_dataframe(content)
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I'm trying to search a file in my directory files. I saw an example and couldn't understand:
import os
import glob
file_glob = './rpts/folderA/*_fileA.txt'
magic_check = re.compile('[*?[]')
if(magic_check.search(file_glob))
file_list = glob.glob(os.path.expanduser(file_glob))
What does the ./ part mean? I understand that ../ is switch to previous dir.
What I think it does:
expand the wild card to get a list of files that matches the regex
The files are stored in a list called file_list
Magic check regex, [*?[]: What is the [ inside [ ] for?
As Martijn said, this is UNIX shell notation for the current location (cwd or pwd). The reason it's in the command is to be more robust. If the user's environment doesn't have "./" in the search path ($PATH variable), then the shell won't find the file rpts/folderA/*_fileA.txt. With "./" at the front, this script is independent of $PATH.
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 7 years ago.
Improve this question
I have an assignment which needs me to convert 4 separate py files into a zip folder. Do you guys know how to do this?
This is the part of the instruction which tells me to compress it
Programs: Name your programs, q1.py, q2.py, q3.py, q4.py and submit all as a zip file(named A1_my_upi.zip) to the assignment drop box before the deadline (No late submission
accepted).
I have read on the internet that I have to import zipfile. Can some one please clairfy?
Let's say that your files are in /users/matthew/documents/school/assignments/a1. Here are a couple of ways in which you can do this:
Python:
import os
from zipfile import Zipfile
dirpath = '/users/matthew/documents/school/assignments/a1'
with open("A1_my_upi.zip", 'w') as outfile:
for fname in os.listdir(dirpath):
if not fname.endswith('.py'):
continue
outfile.write(os.path.join(dirpath, fname))
Now, you have a A1_my_upi.zip that you can submit
If you're on a linux/mac computer, you can do this:
$ cd /users/matthew/documents/school/assignments/a1
$ cd ..
$ zip A1_my_upi.zip a1/*.py
Now, you have a A1_my_upi.zip that you can submit. It exists at /users/matthew/documents/school/assignments/A1_my_upi.zip