I want to upload one large .ndjson to Python. Is there a way to add a progress bar so that I know how much of the file is uploaded?
import json
import pandas as pd
df = map(json.loads, open('dump.ndjson'))
df = pd.DataFrame.from_records(records)
This is the way to upload the file. Code is good because when I split the file into 100 pieces I can upload one by one. But is there a way to add a progress bar so that I can upload the file at once and to see the progress of uploading?
PS. Im not thinking on gui, I have head for tqdm progress bar. I was thinking something like that, so that I can see progress in my console
If the question is about a GUI, there's PySimpleGUI that allows easy using of progress bars that rely on Tk, Wx or Qt frameworks.
This works on Linux / Windows.
Example from their cookbook:
import PySimpleGUI as sg
# layout the window
layout = [[sg.Text('Uploading...')],
[sg.ProgressBar(100, orientation='h', size=(20, 20), key='progressbar')],
[sg.Cancel()]]
# create the window`
window = sg.Window('My Program Upload', layout)
progress_bar = window['progressbar']
# loop that would normally do something useful
for i in range(1000):
# check to see if the cancel button was clicked and exit loop if clicked
event, values = window.read(timeout=10)
if event == 'Cancel' or event == sg.WIN_CLOSED:
break
# update bar with loop value +1 so that bar eventually reaches the maximum
progress_bar.UpdateBar(i + 1)
# TODO: Insert your upload code here
# done with loop... need to destroy the window as it's still open
window.close()
I use that kind of progress bar and keep my upload function in a separate thread so the UI is non blocking.
Related
I am looking for a simple way to display changing real time data in a GUI in python. I am connected to 2 devices and want to display data constantly (like 20 different values), and when I press a button I want to control the one device.
Unfortunately I fail already with the display of the data. For this I have looked at some tkinter tutorials and explanations.
My idea was to implement it with a config function and to overwrite the label continuously. As example how I wanted to display one value:
import tkinter as tk
from pydualsense import pydualsense
# connect to the device
dualsense = pydualsense()
dualsense.init()
# create a window
window = tk.Tk()
# function for updating data
def show_data():
global dualsense
data_label_output.config(text=dualsense.state.LX)
# showing the data as a lable
data_label_output = tk.Label(window)
data_label_output.grid(row=1, column=1)
show_data()
#### or different solution
# showing the data as a lable
data_label_output = tk.Label(window, comand=show_data)
data_label_output.grid(row=1, column=1)
window.mainloop()
Unfortunately, the value is displayed only once at the beginning and nothing changes after that.
Another problem:
When I press the button, I want to be able to control the one device. For this I have a while True loop that permanently checks if a button is pressed and then executes actions. As a separate program no problem, but how do I integrate this into the tkinter GUI? When I start this PyCharm always crashes.
I use PyCharm and Python 3.8
About simple and functional ideas I would be happy, also to other tools/modules etc., as long as you can easily and quickly implement the idea. It's only for a research project and the programming is only a means to an end.
You can use the after method in tkinter to run something after a short delay. The following code will run show_data once the GUI is ready and then again every 1000 milliseconds.
import tkinter as tk
from pydualsense import pydualsense
# connect to the device
dualsense = pydualsense()
dualsense.init()
# create a window
window = tk.Tk()
# function for updating data
def show_data():
global dualsense
data_label_output.config(text=dualsense.state.LX)
window.after(1000,show_data)
# showing the data as a lable
data_label_output = tk.Label(window)
data_label_output.grid(row=1, column=1)
window.after_idle(show_data)
window.mainloop()
This resolves the updating issue, I'm not sure what behaviour you want when you press the button but if you elaborate and explain, I might be able to help and update this answer.
I am looking for a way to use a PYSimpleGUI progress bar... without a loop
I have looked for several days on the internet with no luck to find an example.
seems everybody does their example with a loop, or on a timer.
I'd like to do something more like a definition that I Can call to to update
I dont know what to change to make it a manually updated item...
I want to be able to tell it i=0 at teh beginning of my script
and periodically place update marks thru the script(i=i+4)
so that I can update it as each Major Step in my script is done
This is the PySimpleGUI script, plus some lines showing what I want to do
This currently auto iterates...and I do not know how to change it
I'm just trying to learn and cant find any examples online to do what I want to do.
import PySimpleGUI as sg
import time
from time import sleep
import PySimpleGUI as sg
def prog():
layout = [[sg.Text('Completed Tasks')],
[sg.ProgressBar(100, orientation='h', size=(50, 20), key='progressbar')],
[sg.Cancel()]]
window = sg.Window('Progress').Layout(layout)
progress_bar = window.FindElement('progressbar')
for i in range(100):
event, values = window.Read(timeout=0)
progress_bar.UpdateBar(i + 4)
time.sleep(2)
window.Close()
prog()
time.sleep(2)
#______________________________________________________________
#I'd like to be able to do this
#i=0 at this point
prog()
#do Scripty Stuff
#Update Progress Bar Manually
#i=4 at this point
#do more scriptic writings
#Update Progress bar Manually
#i=8 at this point
#and so forth and so on until I reach 100
Figured it out
just leave everything as a single line, not a definition
heres an example to help others
I just did real numbers in the Update bar section
but you can use variables (i=0) and then update it in the script with an i=i+1
and then use i as your number in the update bar function
i=0
progress_bar.UpdateBar(i, 5)
#i returns a value of 0
i=i+1
progress_bar.UpdateBar(i, 5)
#i now returns a vlaue of 1
#repeat until you reach your maximum value
#this Script will create a Progress Bar
#The Progress will be Manually Updated using the format listed below
#progress_bar.UpdateBar(Value of Bar, Maximum Bar Value)
#the Window uses a .Finalize Function to make the window Persistent
#Import the PySimpleGUI Library
import PySimpleGUI as sg
#Import the Time Library for use in this script
import time
#this is for the Layout Design of the Window
layout = [[sg.Text('Custom Text')],
[sg.ProgressBar(1, orientation='h', size=(20, 20), key='progress')],
]
#This Creates the Physical Window
window = sg.Window('Window Title', layout).Finalize()
progress_bar = window.FindElement('progress')
#This Updates the Window
#progress_bar.UpdateBar(Current Value to show, Maximum Value to show)
progress_bar.UpdateBar(0, 5)
#adding time.sleep(length in Seconds) has been used to Simulate adding your script in between Bar Updates
time.sleep(.5)
progress_bar.UpdateBar(1, 5)
time.sleep(.5)
progress_bar.UpdateBar(2, 5)
time.sleep(.5)
progress_bar.UpdateBar(3, 5)
time.sleep(.5)
progress_bar.UpdateBar(4, 5)
time.sleep(.5)
progress_bar.UpdateBar(5, 5)
time.sleep(.5)
#I paused for 3 seconds at the end to give you time to see it has completed before closing the window
time.sleep(3)
#This will Close The Window
window.Close()
Just a quick note for everyone that is finding this answer in 2021 or later:
The window.FindElement() is deprecated, one uses window.find_element() nowadays :)
#soundtechscott: Thanks for your answer, I also encountered that issue today and your solution worked out for me.
I have a general problem, which I sketch here as the details will be too involved to post here. I know the problem statement, is a bit fuzzy and I may need to go back and forth with an expert in this site. (unfortunately, it is difficult to put everything up here based on the type of the problem. I will really appreciate any help).
I am trying to create a GUI application using Tkinter. The way I am doing is as follows. I have a background script (say back.py) which has the data loaded, calculations done and graph plotted.
Now the way I want to do is that I have a GUI script which uses Tkinter and calls the back.py ( using import). Now I have the window created, with a button. This is where I am stuck. I want to click the button to trigger the background script and generate the plots ( which the background script generates).
After this I want to close the plot and want my GUI to pop up some buttons to input me some parameters. These parameters will be input to the next part of the back.py code ( I chose the parameters based on the plot). When I again click the button ( with the parameters selected), I want to start running the background code again which will output me a file.
How can I do this?. A general rough idea will be helpful.
Let me put an example as much as I can:( at least the skeleton of the code)
I have a background file say (a.py) and gui file ( say g.py)
a.py
import ...
def progA():
# reading a file
# doing something and generating a plot from the file
# Once the GUI's first part is done generating the plot, I need to close that
# plot (or button) and then click the button 2 to run the next function
def progB(y1, y2,y3):
#run a code... and generate an output file
g.py
from tkinter import *
from tkinter.ttk import *
class GUI ():
def create widgets(self):
#....
def create panel(self):
#create buttons
panel1 = ...
btn1 = Button(panel1, text="yyyyy", command=progA)
btn1.pack()
def create_panel1(self):
#create buttons
panel1 = ...
btn1 = Button(panel1, text="yyyyy", command=progA)
btn1.pack()
def create_panel2(self):
#create buttons
panel2 = ...
btn2 = Button(panel1, text="yyyyy", command=progB)
btn2.pack()
All_Entries = []
window = Tk()
D=GUI(window)
window.mainloop()
import a
runprogram1 = a.progA()
runprogram2 = a.probB(x, y, z)
My question is now, does the above makes sense? So I have a couple of questions:
How will I ensure that when I close the plots (from the output of progA), that the second button will show up?
Where can I input the there values of the parameters in the second button?
I'm having a problem, where I wish to run several command line functions from a python program using a GUI. I don't know if my problem is specific to PyQt4 or if it has to do with my bad use of python code.
What I wish to do is have a label on my GUI change its text value to inform the user which command is being executed. My problem however, arises when I run several commands using a for loop. I would like the label to update itself with every loop, however, the program is not updating the GUI label with every loop, instead, it only updates itself once the entire for loop is completed, and displays only the last command that was executed.
I am using PyQt4 for my GUI environment. And I have established that the text variable for the label is indeed being updated with every loop, but, it is not actually showing up visually in the GUI.
Is there a way for me to force the label to update itself? I have tried the update() and repaint() methods within the loop, but they don't make any difference.
I would really appreciate any help.
Thank you.
Ronny.
Here is the code I am using:
# -*- coding: utf-8 -*-
import sys, os
from PyQt4 import QtGui, QtCore
Gui = QtGui
Core = QtCore
# ================================================== CREATE WINDOW OBJECT CLASS
class Win(Gui.QWidget):
def __init__(self, parent = None):
Gui.QWidget.__init__(self, parent)
# --------------------------------------------------- SETUP PLAY BUTTON
self.but1 = Gui.QPushButton("Run Commands",self)
self.but1.setGeometry(10,10, 200, 100)
# -------------------------------------------------------- SETUP LABELS
self.label1 = Gui.QLabel("No Commands running", self)
self.label1.move(10, 120)
# ------------------------------------------------------- SETUP ACTIONS
self.connect(self.but1, Core.SIGNAL("clicked()"), runCommands)
# ======================================================= RUN COMMAND FUNCTION
def runCommands():
for i in commands:
win.label1.setText(i) # Make label display the command being run
print win.label1.text() # This shows that the value is actually
# changing with every loop, but its just not
# being reflected in the GUI label
os.system(i)
# ======================================================================== MAIN
# ------------------------------------------------------ THE TERMINAL COMMANDS
com1 = "espeak 'senntence 1'"
com2 = "espeak 'senntence 2'"
com3 = "espeak 'senntence 3'"
com4 = "espeak 'senntence 4'"
com5 = "espeak 'senntence 5'"
commands = (com1, com2, com3, com4, com5)
# --------------------------------------------------- SETUP THE GUI ENVIRONMENT
app = Gui.QApplication(sys.argv)
win = Win()
win.show()
sys.exit(app.exec_())
The label gets updated all right, but the GUI isn't redrawn before the end of your loop.
Here's what you can do about it:
Move your long-running loop to a secondary thread, drawing the GUI is happening in the main thread.
Call app.processEvents() in your loop. This gives Qt the chance to process events and redraw the GUI.
Break up your loop and let it run using a QTimer with a timeout of 0.
Using a thread is the best option, but involves quite a bit more work than just calling processEvents. Doing it with a timer is the old fashioned way and is not recommanded anymore. (see the documentation)
You have a basic misunderstanding of how such a GUI works. A Qt GUI has to run in an event loop of its own. Your loop runs instead, and the GUI can't do its work between the executions of your loop. That is, while your for loop is running the GUI code doesn't get CPU time and won't update.
You can set up a timer with an event, and execute your code in handlers of this event a set amount of time - this will solve your problem.
Or you can just call repaint() it update the GUI instantly.
I want a progress bar that shows the user the download progress. When updating the GUI and downloading at the same time the progress bar freezes, and I understand why but I don't know how to solve it. I tried multithreading using this post:
Tkinter: How to use threads to preventing main event loop from “freezing” and using The Basics of Python Multithreading and Queues as a guid to help me fit it to my needs. The problem is that which way I try to achieve my goal, I always seem to make a mistake when changing it to do what I need it to do.
The most basic version of my code (without multithreading):
from Tkinter import *
import ttk
from urllib import URLopener # Downloading files
# Make frame to tell user what file is getting downloaded
self.Progressmsg = Label(self, text="TempValue")
self.Progressmsg.pack(pady=(10,0))
# Make progress bar to show user download progress
self.Progressbar = ttk.Progressbar(self, mode="determinate", orient='horizontal', lengt=280, maximum=len(self.AllClasses))
self.Progressbar.pack(padx=10, pady=10)
self.Progressbar["value"] = 0
def DownloadFile(Class):
# Update progress message
self.Progressmsg["text"] = "Downloading {0}.ics...".format(Class)
# Download each file from saxion website
CalFile = URLopener()
CalFile.retrieve("http://[school website]/ical/group/{0}.ics".format(Class), "Data/{0}.ics".format(Class))
# Update progress bar
self.Progressbar["value"] += 1
for Study in self.Parameters["Classes"]:
for Class in Study:
DownloadFile(Class)
Notes: In this code AllClasses is a list of different classes from which a calendar file has to be downloaded.
The code itself is part of a fairly large class which I didn't include. This is why I am using self.[variablename]
When this code runs the progressbar doesn't load or update, all the files download properly and when they are downloaded the progress bar updates everything at once. My question is: how do I solve this problem in my case?
Try this:
# Update progress bar
self.Progressbar["value"] += 1
self.Progressbar.update_idletasks()
If it does not work then use self.Progressbar.update() instead.
The GUI won't reflect your changes if there is something else to do (like downloading the next file) unless you call update_idletasks() or update().