Develop Reference

How to classify the image data from the folders?

It is our folder tree.
dataset/
|
|
|-----train/
| |
| |--dog/
| | |
| | |--German Shepherd/
| | | |
| | | |-- image1.png
| | | |-- image2.png
| | | |-- ...
| | |
| | |--Bulldog/
| | |--Labrador Retriever/
| | |--Poodle/
| | |--Chihuahua/
| | |--...
| |
| |--cat/
| | |
| | |--Siamese cat/
| | |--British Shorthair/
| | |--Maine Coon/
| | |--Persian cat/
| | |--Ragdoll/
| | |--...
|
|-----test/
|
|--...
how can be classify it with tensorflow-python ?
I am not using any pre-trained model then how can be classify it ?

Updating values onto an ASCII art without changing the format [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 months ago.
Improve this question
I am exploring ways on bash as well as on Python to print out values onto an ASCII art for improving the readability.
The one difficulty is to update values without changing the format of the art.
The ascii art looks something like this:
========================================================
| | | | | |
| | | | | |
| CPU | | GPU | | HDD |
| | | | | |
| | | | | |
| ${CPU_W}W | | | | |
| ${CPU_Freq}MHz| | | |Avail Mem${size}G|
| | |${GPU_W}W | | Used Mem${size}G|
| | |${GPU}Mhz | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
========================================================
So far, I was able to prevent the format from changing on Bash. But doing this, is not allowing me to change the values.
cat << "EOF"
========================================================
| | | | | |
| | | | | |
| CPU | | GPU | | HDD |
| | | | | |
| | | | | |
| ${CPU_W}W | | | | |
| ${CPU_Freq}MHz| | | |Avail Mem${size}G|
| | |${GPU_W}W | | Used Mem${size}G|
| | |${GPU}Mhz | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
========================================================
EOF
Without cat << "EOF" ...ascii art.... EOF, I can update the values but the format keeps changing .
Is there anyway to keep the same format even with the values changing? Thanks in advance.

I'm not exactly sure what you're trying to achieve... so?
f = """
========================================================
| | | | | |
| | | | | |
| CPU | | GPU | | HDD |
| | | | | |
| | | | | |
|{cpuW:>15}| | | | |
|{cpuF:>15}| | | |Avail Mem{ramA:>8}|
| | |{gpuW:>10}| | Used Mem{ramU:>8}|
| | |{gpuF:>10}| | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
========================================================
"""
def main():
cpuW = 12.2
cpuF = 2354
gpuW = 15.2
gpuF = 789
ramA = 16.1
ramU = 12.2
d = {
'cpuW': f'{cpuW:.1f} W ',
'cpuF': f'{cpuF:d} MHz ',
'gpuW': f'{gpuW:.1f} W ',
'gpuF': f'{gpuF:d} MHz ',
'ramA': f' {ramA:.1f} GB',
'ramU': f' {ramU:.1f} GB',
}
out = f.format_map(d)
print(out)
if __name__ == '__main__':
main()
Output is:
========================================================
| | | | | |
| | | | | |
| CPU | | GPU | | HDD |
| | | | | |
| | | | | |
| 12.2 W | | | | |
| 2354 MHz | | | |Avail Mem 16.1 GB|
| | | 15.2 W | | Used Mem 12.2 GB|
| | | 789 MHz | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
========================================================

In python, you can use the format and str.ljust method.
format(value[, format_spec])
Convert a value to a “formatted” representation, as controlled by format_spec. The interpretation of format_spec will depend on the type of the value argument; however, there is a standard formatting syntax that is used by most built-in types: Format Specification Mini-Language.
It depends on what you want the notation to be (e.g how many zeros, ecc...).
here is an example:
>>> characters = 10
>>> format(32,".2E").zfill(characters)
'003.20E+01'
>>> #first number is the minimum number of characters
>>> format(32,"{0}.2E".format(characters))
' 3.20E+01'
or, with fstrings:
>>> f"{32:.2E}"
'3.20E+01'
Example on how to use it in ascii art:
>>> def create_ascii_art(CPU_W,CPU_freq,GPU_W,
... GPU,Free_Mem,Used_Mem):
... CPU_W,CPU_freq,GPU_W,GPU,Free_Mem,Used_Mem = map(float,(CPU_W,CPU_freq,GPU_W,GPU,Free_Mem,Used_Mem))
... return f'''
...========================================================
...| | | | | |
...| | | | | |
...| CPU | | GPU | | HDD |
...| | | | | |
...| | | | | |
...|{CPU_W:12.2E}W | | | | |
...|{CPU_freq:12.2E}MHz| | | |Avail Mem{Free_Mem:7.3}G|
...| | |{GPU_W:8.2E}W | | Used Mem{Used_Mem:7.3}G|
...| | |{GPU:6.0E}Mhz | | |
...| | | | | |
...| | | | | |
...| | | | | |
...| | | | | |
...========================================================'''
>>> print(create_ascii_art(1,1,1,1,1,1))
output:
========================================================
| | | | | |
| | | | | |
| CPU | | GPU | | HDD |
| | | | | |
| | | | | |
| 1.00E+00W | | | | |
| 1.00E+00MHz| | | |Avail Mem 1.0G|
| | |1.00E+00W | | Used Mem 1.0G|
| | | 1E+00Mhz | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
========================================================

The "format" does not change; the display width of the strings in the here document depends on the values of the variables. You'll want to make sure the values are padded to the expected width (or truncated, if necessary).
A slightly tortured way to accomplish this is to pad the actual values:
#!/bin/bash
:
printf -v CPU_W '%8s' "$CPU_W"
printf -v CPU_Freq '%11s' "$CPU_Freq"
printf -v GPU_W '%8s' "$GPU_W"
printf -v GPU '%6s' "$GPU"
printf -v size '%7s' "$size"
cat <<EOF
========================================================
| | | | | |
| | | | | |
| CPU | | GPU | | HDD |
| | | | | |
| | | | | |
| ${CPU_W}W | | | | |
| ${CPU_Freq}MHz| | | |Avail Mem${size}G|
| | |${GPU_W}W | | Used Mem${size}G|
| | |${GPU}Mhz | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
========================================================
EOF
Probably a better solution altogether is to use printf directly to format the output. (Personally, I would perceive a notable reduction in my blood pressure if you removed at least half of the repetitive "ASCII art", which would somewhat tidy up the necessary code, too.)
We can't know what values these variables contain; if they are integers, maybe experiment with i instead of s for the printf format code, or correspondingly f or perhaps g for floating-point values.
If you need truncation, try %8.8 instead of %8, etc.
An altogether nicer approach is to use a library for producing this format. I can't recommend any particular one for Bash; for Python, look at tabulate.

Control Windows app (HMA VPN) using PyWinAuto

I am trying to write Python code to request my vpn software to connect / disconnect upon certain conditions. I use HMA VPN and am running on Windows 10. Python 3.9.5.
The PyWinAuto module would appear to have the required functionality, but I cannot get it to work correctly. I'm sure it's my coding, not the module itself... ;)
I'm fairly sure I'm referring to the control incorrectly when I call it with the click() method, but can anyone suggest the correct approach, given the dump_tree that I have included? (I have tried tens of variations based based on the code snippets in the Getting Started guide, but no joy).
I'm looking to be able to toggle Button4 in GroupBox2, which is named either "Connect" or "Disconnect", depending on the current state of the VPN. I also want to be able to click Button5, "Change IP Address", but once I figure out one button, the other should be easy.
Any thoughts where I am going wrong?
My code snippet is as follows:
from pywinauto import Desktop, Application
Desktop(backend='uia').HMAVPN.dump_tree()
app = Application().start('C:\Program Files\Privax\HMA VPN\Vpn.exe', timeout=5)
dlg_spec = app.window(title='HMA VPN')
dlg_spec.AvastChromiumWindow.Pane2.Document.GroupBox2.Button4.click()
The resulting output is as follows:
Control Identifiers:
Dialog - 'HMA VPN' (L-149, T68, R551, B638)
['HMA VPNDialog', 'HMA VPN', 'Dialog']
child_window(title="HMA VPN", control_type="Window")
|
| Pane - 'AvastChromiumWindow' (L-149, T68, R551, B638)
| ['Pane', 'AvastChromiumWindowPane', 'AvastChromiumWindow', 'Pane0', 'Pane1']
| child_window(title="AvastChromiumWindow", control_type="Pane")
| |
| | Pane - '' (L-149, T68, R551, B638)
| | ['Pane2', '\ue01bPane', '\ue01bPane0', '\ue01bPane1']
| | |
| | | Document - '   Original IP   91.90.253.244 New IP   80.255.5.198  Original IP   91.90.253.244   Location Germany, Berlin  ' (L-149, T68, R551, B638)
| | | ['\ue01b \ue014 \ue007 Original IP \xa0 91.90.253.244 New IP \xa0 80.255.5.198 \ue019 Original IP \xa0 91.90.253.244 \xa0 Location Germany, Berlin \ue005 Document', 'Document', '\ue01b \ue014 \ue007 Original IP \xa0 91.90.253.244 New IP \xa0 80.255.5.198 \ue019 Original IP \xa0 91.90.253.244 \xa0 Location Germany, Berlin \ue005 ']
| | | child_window(title="   Original IP   91.90.253.244 New IP   80.255.5.198  Original IP   91.90.253.244   Location Germany, Berlin  ", auto_id="7211296", control_type="Document")
| | | |
| | | | GroupBox - '' (L-149, T68, R551, B104)
| | | | ['\ue01bGroupBox', 'GroupBox', '\ue01bGroupBox0', '\ue01bGroupBox1', 'GroupBox0', 'GroupBox1']
| | | | |
| | | | | Button - 'Preferences' (L-134, T76, R-114, B96)
| | | | | ['PreferencesButton', 'Preferences', 'Button', 'Button0', 'Button1']
| | | | | child_window(title="Preferences", auto_id="hma_title__preferences", control_type="Button")
| | | | | |
| | | | | | Static - '' (L-134, T76, R-114, B96)
| | | | | | ['Static', '\ue01b', '\ue01bStatic', 'Static0', 'Static1']
| | | | | | child_window(title="", control_type="Text")
| | | | |
| | | | | Button - 'Minimize' (L486, T76, R506, B96)
| | | | | ['Minimize', 'Button2', 'MinimizeButton']
| | | | | child_window(title="Minimize", control_type="Button")
| | | | | |
| | | | | | Static - '' (L486, T76, R506, B96)
| | | | | | ['\ue014Static', 'Static2', '\ue014']
| | | | | | child_window(title="", control_type="Text")
| | | | |
| | | | | Button - 'Close' (L516, T76, R536, B96)
| | | | | ['Close', 'Button3', 'CloseButton']
| | | | | child_window(title="Close", control_type="Button")
| | | | | |
| | | | | | Static - '' (L516, T76, R536, B96)
| | | | | | ['Static3', '\ue007', '\ue007Static']
| | | | | | child_window(title="", control_type="Text")
| | | |
| | | | GroupBox - '' (L-149, T68, R551, B638)
| | | | ['\ue01bGroupBox2', 'GroupBox2']
| | | | |
| | | | | Image - '' (L111, T120, R291, B300)
| | | | | ['\ue01bImage', 'Image']
| | | | |
| | | | | Button - 'Disconnect' (L126, T323, R276, B383)
| | | | | ['Button4', 'DisconnectButton', 'Disconnect']
| | | | | child_window(title="Disconnect", auto_id="dashboard_switch", control_type="Button")
| | | | |
| | | | | Static - 'Original IP' (L68, T414, R135, B430)
| | | | | ['Original IPStatic', 'Static4', 'Original IP', 'Original IPStatic0', 'Original IPStatic1', 'Original IP0', 'Original IP1']
| | | | | child_window(title="Original IP", control_type="Text")
| | | | |
| | | | | Static - ' ' (L164, T412, R168, B427)
| | | | | ['Static5', '\xa0Static', '\xa0', '\xa0Static0', '\xa0Static1', '\xa00', '\xa01']
| | | | | child_window(title=" ", control_type="Text")
| | | | |
| | | | | Static - '91.90.253.244' (L184, T414, R266, B429)
| | | | | ['Static6', '91.90.253.244Static', '91.90.253.244', '91.90.253.244Static0', '91.90.253.244Static1', '91.90.253.2440', '91.90.253.2441']
| | | | | child_window(title="91.90.253.244", control_type="Text")
| | | | |
| | | | | Static - 'New IP' (L68, T442, R113, B458)
| | | | | ['New IP', 'Static7', 'New IPStatic']
| | | | | child_window(title="New IP", control_type="Text")
| | | | |
| | | | | Static - ' ' (L164, T440, R168, B455)
| | | | | ['Static8', '\xa0Static2', '\xa02']
| | | | | child_window(title=" ", control_type="Text")
| | | | |
| | | | | Static - '80.255.5.198' (L184, T442, R258, B457)
| | | | | ['Static9', '80.255.5.198', '80.255.5.198Static']
| | | | | child_window(title="80.255.5.198", control_type="Text")
| | | | |
| | | | | Button - 'Change IP Address' (L294, T416, R334, B456)
| | | | | ['Change IP AddressButton', 'Button5', 'Change IP Address']
| | | | | child_window(title="Change IP Address", auto_id="ip_change", control_type="Button")
| | | | |
| | | | | Static - 'Original IP' (L95, T428, R162, B444)
| | | | | ['Original IPStatic2', 'Static10', 'Original IP2']
| | | | | child_window(title="Original IP", control_type="Text")
| | | | |
| | | | | Static - ' ' (L191, T426, R196, B441)
| | | | | ['Static11', '\xa0Static3', '\xa03']
| | | | | child_window(title=" ", control_type="Text")
| | | | |
| | | | | Static - '91.90.253.244' (L211, T428, R293, B443)
| | | | | ['Static12', '91.90.253.244Static2', '91.90.253.2442']
| | | | | child_window(title="91.90.253.244", control_type="Text")
| | | | |
| | | | | Button - 'Location Germany, Berlin' (L31, T489, R371, B569)
| | | | | ['Button6', 'Location Germany, BerlinButton', 'Location Germany, Berlin']
| | | | | child_window(title="Location Germany, Berlin", auto_id="dashboard_chooseLocation", control_type="Button")
| | | | | |
| | | | | | Static - ' ' (L51, T507, R55, B522)
| | | | | | ['Static13', '\xa0Static4', '\xa04']
| | | | | | child_window(title=" ", control_type="Text")
| | | | | |
| | | | | | Static - 'Location' (L111, T511, R164, B526)
| | | | | | ['Static14', 'Location', 'LocationStatic']
| | | | | | child_window(title="Location", control_type="Text")
| | | | | |
| | | | | | Static - 'Germany, Berlin' (L111, T529, R219, B546)
| | | | | | ['Static15', 'Germany, Berlin', 'Germany, BerlinStatic']
| | | | | | child_window(title="Germany, Berlin", control_type="Text")
| | | | | |
| | | | | | Static - '' (L331, T519, R351, B539)
| | | | | | ['Static16', '\ue005', '\ue005Static']
| | | | | | child_window(title="", control_type="Text")
| | | | |
| | | | | Button - 'More' (L420, T107, R528, B137)
| | | | | ['More', 'Button7', 'MoreButton']
| | | | | child_window(title="More", control_type="Button")
| | |
| | | Pane - '' (L-149, T68, R551, B638)
| | | ['Pane3', '\ue01bPane2']
Traceback (most recent call last):
File "C:\Users\db533\VirtualBox VMs\pycharm-gunabot\venv\lib\site-packages\pywinauto\application.py", line 250, in __resolve_control
ctrl = wait_until_passes(
File "C:\Users\db533\VirtualBox VMs\pycharm-gunabot\venv\lib\site-packages\pywinauto\timings.py", line 458, in wait_until_passes
raise err
pywinauto.timings.TimeoutError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\db533\PycharmProjects\GunaBot\hma_control.py", line 19, in <module>
dlg_spec.AvastChromiumWindow.Pane2.Document.GroupBox2.Button4.click()
File "C:\Users\db533\VirtualBox VMs\pycharm-gunabot\venv\lib\site-packages\pywinauto\application.py", line 379, in __getattribute__
ctrls = self.__resolve_control(self.criteria)
File "C:\Users\db533\VirtualBox VMs\pycharm-gunabot\venv\lib\site-packages\pywinauto\application.py", line 261, in __resolve_control
raise e.original_exception
File "C:\Users\db533\VirtualBox VMs\pycharm-gunabot\venv\lib\site-packages\pywinauto\timings.py", line 436, in wait_until_passes
func_val = func(*args, **kwargs)
File "C:\Users\db533\VirtualBox VMs\pycharm-gunabot\venv\lib\site-packages\pywinauto\application.py", line 203, in __get_ctrl
dialog = self.backend.generic_wrapper_class(findwindows.find_element(**criteria[0]))
File "C:\Users\db533\VirtualBox VMs\pycharm-gunabot\venv\lib\site-packages\pywinauto\findwindows.py", line 87, in find_element
raise ElementNotFoundError(kwargs)
pywinauto.findwindows.ElementNotFoundError: {'title': 'HMA VPN', 'backend': 'win32', 'process': 40580}
Process finished with exit code 1

Cracked it! The following code let's me toggle the Connect / Disconnect button and change the IP.
from pywinauto import Desktop,Application
vpn_app = Application(backend="uia").start('C:\Program Files\Privax\HMA VPN\Vpn.exe')
dialog=Desktop(backend="uia").HMA
panel0=dialog.Pane
# Command to connect / disconnect the VPN: connect_button.click()
connect_button=panel0.ConnectButton
# Command to change the IP address: changeIP.click()
changeIP=panel0.Button5
# Check VPN state:
# 0 if disconnected
# 1 if connected
print(connect_button.get_toggle_state())
# Command to connect / disconnect the VPN: connect_button.click()
connect_button=panel0.ConnectButton
connect_button.click()
# Command to change the IP address: changeIP.click()
changeIP=panel0.Button5
changeIP.click()
# Check VPN state:
# 0 if disconnected
# 1 if connected
print(connect_button.get_toggle_state())

Web scraping: how to iterate over all the pages from a url and store the result in a table

I am not able to resolve this loop. I want to iterate over all pages from the URL and put all the names in a table. There are 102 pages. Thank you in advance.
[from bs4 import BeautifulSoup
import re
import requests
url = "https://www.prokerala.com/kids/baby-names/hindu/"
r = requests.get(url)
soup = BeautifulSoup(r.content, "html.parser")
next_page = soup.find('a', {'class': 'next'}).get('href')
l=(f'{next_page}')
new_url=url+l
link=soup.find('span', {'class' : \['a name-details girl-names', 'a name-details boy-names'\] })
print("href='>%s'>%s</a>" % (link.get("href"), link.text))
while True:
r = requests.get(new_url)
soup = BeautifulSoup(r.content, "html.parser")
next_page = soup.find('a', {'class': 'next'}).get('href')
link=soup.find('span', {'class' : \['a name-details girl-names', 'a name-details boy-names'\] })
print("href='>%s'>%s</a>" % (link.get("href"), link.text))
l=(f'{next_page}')
new_url=url+l
if BeautifulSoup(r.content, "html.parser")is True:
break][1]
[1]: https://i.stack.imgur.com/SgM4I.png

from bs4 import BeautifulSoup
import requests
import re
from prettytable import PrettyTable
def main(url):
with requests.Session() as req:
p = PrettyTable()
p.field_names = ['Name', 'Url']
for item in range(1, 3):
r = req.get(url.format(item))
soup = BeautifulSoup(r.content, 'html.parser')
for item in soup.findAll("span", class_=re.compile("a name")):
p.add_row([item.a.text, f'{url[:25]}{item.a.get("href")}'])
print(p)
main("https://www.prokerala.com/kids/baby-names/hindu/page-{}.html")
Output:
+---------------+--------------------------------------------------------------------+
| Name | Url |
+---------------+--------------------------------------------------------------------+
| Aabha | https://www.prokerala.com/kids/baby-names/aabha-13597.html |
| Aabheer | https://www.prokerala.com/kids/baby-names/aabheer-2.html |
| Aachal | https://www.prokerala.com/kids/baby-names/aachal-20815.html |
| Aadarsh | https://www.prokerala.com/kids/baby-names/aadarsh-3.html |
| Aadarshini | https://www.prokerala.com/kids/baby-names/aadarshini-20997.html |
| Aadav | https://www.prokerala.com/kids/baby-names/aadav-21329.html |
| Aadavan | https://www.prokerala.com/kids/baby-names/aadavan-19505.html |
| Aadesh | https://www.prokerala.com/kids/baby-names/aadesh-4.html |
| Aadhan | https://www.prokerala.com/kids/baby-names/aadhan-5.html |
| Aadharsha | https://www.prokerala.com/kids/baby-names/aadharsha-20812.html |
| Aadhik | https://www.prokerala.com/kids/baby-names/aadhik-19726.html |
| Aadhikesav | https://www.prokerala.com/kids/baby-names/aadhikesav-21003.html |
| Aadhinath | https://www.prokerala.com/kids/baby-names/aadhinath-21392.html |
| Aadhira | https://www.prokerala.com/kids/baby-names/aadhira-6.html |
| Aadhira | https://www.prokerala.com/kids/baby-names/aadhira-20603.html |
| Aadhith | https://www.prokerala.com/kids/baby-names/aadhith-21626.html |
| Aadhriti | https://www.prokerala.com/kids/baby-names/aadhriti-20601.html |
| Aadhunik | https://www.prokerala.com/kids/baby-names/aadhunik-21198.html |
| Aadhvitha | https://www.prokerala.com/kids/baby-names/aadhvitha-20571.html |
| Aadhya | https://www.prokerala.com/kids/baby-names/aadhya-21327.html |
| Aadi | https://www.prokerala.com/kids/baby-names/aadi-7.html |
| Aadidev | https://www.prokerala.com/kids/baby-names/aadidev-8.html |
| Aadilakshmi | https://www.prokerala.com/kids/baby-names/aadilakshmi-20995.html |
| Aadimoolan | https://www.prokerala.com/kids/baby-names/aadimoolan-20920.html |
| Aadinadh | https://www.prokerala.com/kids/baby-names/aadinadh-20614.html |
| Aadinath | https://www.prokerala.com/kids/baby-names/aadinath-10.html |
| Aadinathan | https://www.prokerala.com/kids/baby-names/aadinathan-24838.html |
| Aadipta | https://www.prokerala.com/kids/baby-names/aadipta-20922.html |
| Aadir | https://www.prokerala.com/kids/baby-names/aadir-11.html |
| Aadish | https://www.prokerala.com/kids/baby-names/aadish-20945.html |
| Aadishree | https://www.prokerala.com/kids/baby-names/aadishree-21389.html |
| Aadit | https://www.prokerala.com/kids/baby-names/aadit-12.html |
| Aaditey | https://www.prokerala.com/kids/baby-names/aaditey-20921.html |
| Aaditeya | https://www.prokerala.com/kids/baby-names/aaditeya-13.html |
| Aaditya | https://www.prokerala.com/kids/baby-names/aaditya-22619.html |
| Aadrika | https://www.prokerala.com/kids/baby-names/aadrika-20925.html |
| Aadvay | https://www.prokerala.com/kids/baby-names/aadvay-22288.html |
| Aadvik | https://www.prokerala.com/kids/baby-names/aadvik-19508.html |
| Aadya | https://www.prokerala.com/kids/baby-names/aadya-19557.html |
| Aadyasha | https://www.prokerala.com/kids/baby-names/aadyasha-20570.html |
| Aafreen | https://www.prokerala.com/kids/baby-names/aafreen-19244.html |
| Aagam | https://www.prokerala.com/kids/baby-names/aagam-16.html |
| Aaghosh | https://www.prokerala.com/kids/baby-names/aaghosh-22289.html |
| Aagney | https://www.prokerala.com/kids/baby-names/aagney-17.html |
| Aagneya | https://www.prokerala.com/kids/baby-names/aagneya-18.html |
| Aahan | https://www.prokerala.com/kids/baby-names/aahan-19.html |
| Aahana | https://www.prokerala.com/kids/baby-names/aahana-21194.html |
| Aahish | https://www.prokerala.com/kids/baby-names/aahish-22290.html |
| Aahlaad | https://www.prokerala.com/kids/baby-names/aahlaad-21.html |
| Aahva | https://www.prokerala.com/kids/baby-names/aahva-22.html |
| Aahvan | https://www.prokerala.com/kids/baby-names/aahvan-23857.html |
| Aakaar | https://www.prokerala.com/kids/baby-names/aakaar-23.html |
| Aakaash | https://www.prokerala.com/kids/baby-names/aakaash-24.html |
| Aakanksha | https://www.prokerala.com/kids/baby-names/aakanksha-19223.html |
| Aakar | https://www.prokerala.com/kids/baby-names/aakar-25.html |
| Aakarshan | https://www.prokerala.com/kids/baby-names/aakarshan-26.html |
| Aakash | https://www.prokerala.com/kids/baby-names/aakash-19650.html |
| Aalap | https://www.prokerala.com/kids/baby-names/aalap-27.html |
| Aalok | https://www.prokerala.com/kids/baby-names/aalok-28.html |
| Aamil | https://www.prokerala.com/kids/baby-names/aamil-17109.html |
| Aamod | https://www.prokerala.com/kids/baby-names/aamod-30.html |
| Aanan | https://www.prokerala.com/kids/baby-names/aanan-23674.html |
| Aanand | https://www.prokerala.com/kids/baby-names/aanand-31.html |
| Aanandhi | https://www.prokerala.com/kids/baby-names/aanandhi-19971.html |
| Aanandswarup | https://www.prokerala.com/kids/baby-names/aanandswarup-32.html |
| Aananthan | https://www.prokerala.com/kids/baby-names/aananthan-26697.html |
| Aanav | https://www.prokerala.com/kids/baby-names/aanav-19506.html |
| Aanchal | https://www.prokerala.com/kids/baby-names/aanchal-26403.html |
| Aandaleeb | https://www.prokerala.com/kids/baby-names/aandaleeb-33.html |
| Aanjaneya | https://www.prokerala.com/kids/baby-names/aanjaneya-34.html |
| Aanjay | https://www.prokerala.com/kids/baby-names/aanjay-23679.html |
| Aapt | https://www.prokerala.com/kids/baby-names/aapt-35.html |
| Aaraadhak | https://www.prokerala.com/kids/baby-names/aaraadhak-37.html |
| Aaradhya | https://www.prokerala.com/kids/baby-names/aaradhya-19272.html |
| Aaranyan | https://www.prokerala.com/kids/baby-names/aaranyan-23680.html |
| Aarathi | https://www.prokerala.com/kids/baby-names/aarathi-24839.html |
| Aarav | https://www.prokerala.com/kids/baby-names/aarav-38.html |
| Aardik | https://www.prokerala.com/kids/baby-names/aardik-26698.html |
| Aarhant | https://www.prokerala.com/kids/baby-names/aarhant-39.html |
| Aarin | https://www.prokerala.com/kids/baby-names/aarin-19496.html |
| Aarit | https://www.prokerala.com/kids/baby-names/aarit-41.html |
| Aariv | https://www.prokerala.com/kids/baby-names/aariv-19507.html |
| Aarna | https://www.prokerala.com/kids/baby-names/aarna-24840.html |
| Aarnav | https://www.prokerala.com/kids/baby-names/aarnav-44.html |
| Aarodeep | https://www.prokerala.com/kids/baby-names/aarodeep-26700.html |
| Aaroha | https://www.prokerala.com/kids/baby-names/aaroha-21497.html |
| Aaromal | https://www.prokerala.com/kids/baby-names/aaromal-26701.html |
| Aarpit | https://www.prokerala.com/kids/baby-names/aarpit-46.html |
| Aarsh | https://www.prokerala.com/kids/baby-names/aarsh-47.html |
| Aarshabh | https://www.prokerala.com/kids/baby-names/aarshabh-23864.html |
| Aarth | https://www.prokerala.com/kids/baby-names/aarth-48.html |
| Aarthi | https://www.prokerala.com/kids/baby-names/aarthi-24320.html |
| Aarti | https://www.prokerala.com/kids/baby-names/aarti-24228.html |
| Aarul | https://www.prokerala.com/kids/baby-names/aarul-19501.html |
| Aarunya | https://www.prokerala.com/kids/baby-names/aarunya-21502.html |
| Aarush | https://www.prokerala.com/kids/baby-names/aarush-49.html |
| Aarushi | https://www.prokerala.com/kids/baby-names/aarushi-50.html |
| Aarya | https://www.prokerala.com/kids/baby-names/aarya-19111.html |
| Aaryah | https://www.prokerala.com/kids/baby-names/aaryah-13617.html |
| Aaryan | https://www.prokerala.com/kids/baby-names/aaryan-51.html |
| Aaryana | https://www.prokerala.com/kids/baby-names/aaryana-13618.html |
| Aaryanna | https://www.prokerala.com/kids/baby-names/aaryanna-13619.html |
| Aashadhar | https://www.prokerala.com/kids/baby-names/aashadhar-21503.html |
| Aashank | https://www.prokerala.com/kids/baby-names/aashank-53.html |
| Aashay | https://www.prokerala.com/kids/baby-names/aashay-54.html |
| Aashi | https://www.prokerala.com/kids/baby-names/aashi-24841.html |
| Aashirya | https://www.prokerala.com/kids/baby-names/aashirya-19779.html |
| Aashish | https://www.prokerala.com/kids/baby-names/aashish-55.html |
| Aashman | https://www.prokerala.com/kids/baby-names/aashman-56.html |
| Aashna | https://www.prokerala.com/kids/baby-names/aashna-19991.html |
| Aashutosh | https://www.prokerala.com/kids/baby-names/aashutosh-57.html |
| Aastha | https://www.prokerala.com/kids/baby-names/aastha-19567.html |
| Aastik | https://www.prokerala.com/kids/baby-names/aastik-60.html |
| Aathira | https://www.prokerala.com/kids/baby-names/aathira-24842.html |
| Aatish | https://www.prokerala.com/kids/baby-names/aatish-62.html |
| Aatmaj | https://www.prokerala.com/kids/baby-names/aatmaj-63.html |
| Aatreya | https://www.prokerala.com/kids/baby-names/aatreya-64.html |
| Aayan | https://www.prokerala.com/kids/baby-names/aayan-65.html |
| Aayu | https://www.prokerala.com/kids/baby-names/aayu-66.html |
| Aayush | https://www.prokerala.com/kids/baby-names/aayush-24843.html |
| Aayushi | https://www.prokerala.com/kids/baby-names/aayushi-26998.html |
| Aayushmaan | https://www.prokerala.com/kids/baby-names/aayushmaan-67.html |
| Abadhya | https://www.prokerala.com/kids/baby-names/abadhya-19727.html |
| Abeesh | https://www.prokerala.com/kids/baby-names/abeesh-19724.html |
| Abeesht | https://www.prokerala.com/kids/baby-names/abeesht-20923.html |
| Aben | https://www.prokerala.com/kids/baby-names/aben-19257.html |
| Abha | https://www.prokerala.com/kids/baby-names/abha-7340.html |
| Abhaiveer | https://www.prokerala.com/kids/baby-names/abhaiveer-23649.html |
| Abhanja | https://www.prokerala.com/kids/baby-names/abhanja-23670.html |
| Abhas | https://www.prokerala.com/kids/baby-names/abhas-19723.html |
| Abhat | https://www.prokerala.com/kids/baby-names/abhat-19722.html |
| Abhati | https://www.prokerala.com/kids/baby-names/abhati-18373.html |
| Abhay | https://www.prokerala.com/kids/baby-names/abhay-154.html |
| Abhaya | https://www.prokerala.com/kids/baby-names/abhaya-7341.html |
| Abhayan | https://www.prokerala.com/kids/baby-names/abhayan-22297.html |
| Abhayananda | https://www.prokerala.com/kids/baby-names/abhayananda-18027.html |
| Abhayaprada | https://www.prokerala.com/kids/baby-names/abhayaprada-18028.html |
| Abhayd | https://www.prokerala.com/kids/baby-names/abhayd-22295.html |
| Abhayd | https://www.prokerala.com/kids/baby-names/abhayd-22296.html |
| Abhaysimha | https://www.prokerala.com/kids/baby-names/abhaysimha-23650.html |
| Abheek | https://www.prokerala.com/kids/baby-names/abheek-155.html |
| Abhey | https://www.prokerala.com/kids/baby-names/abhey-22292.html |
| Abhi | https://www.prokerala.com/kids/baby-names/abhi-156.html |
| Abhibhava | https://www.prokerala.com/kids/baby-names/abhibhava-18029.html |
| Abhicandra | https://www.prokerala.com/kids/baby-names/abhicandra-18030.html |
| Abhidha | https://www.prokerala.com/kids/baby-names/abhidha-18374.html |
| Abhidhya | https://www.prokerala.com/kids/baby-names/abhidhya-18375.html |
| Abhidi | https://www.prokerala.com/kids/baby-names/abhidi-18031.html |
| Abhigita | https://www.prokerala.com/kids/baby-names/abhigita-23855.html |
| Abhigna | https://www.prokerala.com/kids/baby-names/abhigna-24844.html |
| Abhigyaan | https://www.prokerala.com/kids/baby-names/abhigyaan-23671.html |
| Abhihita | https://www.prokerala.com/kids/baby-names/abhihita-18032.html |
| Abhijaat | https://www.prokerala.com/kids/baby-names/abhijaat-157.html |
| Abhijat | https://www.prokerala.com/kids/baby-names/abhijat-18787.html |
| Abhijay | https://www.prokerala.com/kids/baby-names/abhijay-158.html |
| Abhijaya | https://www.prokerala.com/kids/baby-names/abhijaya-18033.html |
| Abhijeet | https://www.prokerala.com/kids/baby-names/abhijeet-17122.html |
| Abhiji | https://www.prokerala.com/kids/baby-names/abhiji-25946.html |
| Abhijit | https://www.prokerala.com/kids/baby-names/abhijit-159.html |
| Abhijith | https://www.prokerala.com/kids/baby-names/abhijith-23839.html |
| Abhijun | https://www.prokerala.com/kids/baby-names/abhijun-160.html |
| Abhijvala | https://www.prokerala.com/kids/baby-names/abhijvala-18034.html |
| Abhik | https://www.prokerala.com/kids/baby-names/abhik-161.html |
| Abhikama | https://www.prokerala.com/kids/baby-names/abhikama-19719.html |
| Abhikansh | https://www.prokerala.com/kids/baby-names/abhikansh-19720.html |
| Abhilasa | https://www.prokerala.com/kids/baby-names/abhilasa-18376.html |
| Abhilash | https://www.prokerala.com/kids/baby-names/abhilash-162.html |
| Abhilasha | https://www.prokerala.com/kids/baby-names/abhilasha-7342.html |
| Abhim | https://www.prokerala.com/kids/baby-names/abhim-19721.html |
| Abhimand | https://www.prokerala.com/kids/baby-names/abhimand-18035.html |
| Abhimani | https://www.prokerala.com/kids/baby-names/abhimani-18036.html |
| Abhimanyu | https://www.prokerala.com/kids/baby-names/abhimanyu-163.html |
| Abhimanyusuta | https://www.prokerala.com/kids/baby-names/abhimanyusuta-18037.html |
| Abhimoda | https://www.prokerala.com/kids/baby-names/abhimoda-18038.html |
| Abhin | https://www.prokerala.com/kids/baby-names/abhin-164.html |
| Abhinabhas | https://www.prokerala.com/kids/baby-names/abhinabhas-18039.html |
| Abhinand | https://www.prokerala.com/kids/baby-names/abhinand-23672.html |
| Abhinanda | https://www.prokerala.com/kids/baby-names/abhinanda-18040.html |
| Abhinandan | https://www.prokerala.com/kids/baby-names/abhinandan-165.html |
| Abhinandana | https://www.prokerala.com/kids/baby-names/abhinandana-18788.html |
| Abhinatha | https://www.prokerala.com/kids/baby-names/abhinatha-18041.html |
| Abhinav | https://www.prokerala.com/kids/baby-names/abhinav-166.html |
| Abhinava | https://www.prokerala.com/kids/baby-names/abhinava-18042.html |
| Abhinavin | https://www.prokerala.com/kids/baby-names/abhinavin-19729.html |
| Abhinay | https://www.prokerala.com/kids/baby-names/abhinay-168.html |
| Abhinaya | https://www.prokerala.com/kids/baby-names/abhinaya-23651.html |
| Abhineet | https://www.prokerala.com/kids/baby-names/abhineet-169.html |
| Abhinit | https://www.prokerala.com/kids/baby-names/abhinit-170.html |
| Abhinithi | https://www.prokerala.com/kids/baby-names/abhinithi-18377.html |
| Abhinivesh | https://www.prokerala.com/kids/baby-names/abhinivesh-171.html |
| Abhiprithi | https://www.prokerala.com/kids/baby-names/abhiprithi-18378.html |
| Abhir | https://www.prokerala.com/kids/baby-names/abhir-172.html |
| Abhiraam | https://www.prokerala.com/kids/baby-names/abhiraam-173.html |
| Abhiraj | https://www.prokerala.com/kids/baby-names/abhiraj-174.html |
| Abhiraksh | https://www.prokerala.com/kids/baby-names/abhiraksh-19733.html |
| Abhirakshit | https://www.prokerala.com/kids/baby-names/abhirakshit-19732.html |
| Abhiram | https://www.prokerala.com/kids/baby-names/abhiram-19728.html |
| Abhirami | https://www.prokerala.com/kids/baby-names/abhirami-24745.html |
| Abhirath | https://www.prokerala.com/kids/baby-names/abhirath-175.html |
| Abhirathi | https://www.prokerala.com/kids/baby-names/abhirathi-18379.html |
+---------------+--------------------------------------------------------------------+

Extracting and plotting data from a MultiIndex DataFrame in pandas

I've managed to get the following table into a pandas DataFrame. It has a multi-dimensional index (file_type, server_count, file_count, thread_count, cacheclear_type) which represents a configuration for some performance measurement. I then have 5 runs for each configuration.
+-----------+--------------+------------+--------------+-----------------+---------+---------+---------+---------+---------+
| | | | | | run_001 | run_002 | run_003 | run_004 | run_005 |
+-----------+--------------+------------+--------------+-----------------+---------+---------+---------+---------+---------+
| file_type | server_count | file_count | thread_count | cacheclear_type | | | | | |
+-----------+--------------+------------+--------------+-----------------+---------+---------+---------+---------+---------+
| gor | 01servers | 05files | 20threads | ccALWAYS | 15.918 | 16.275 | 15.807 | 17.781 | 16.233 |
| | 08servers | 05files | 20threads | ccALWAYS | 17.061 | 15.414 | 16.819 | 15.597 | 16.818 |
| gorz | 01servers | 05files | 20threads | ccALWAYS | 12.285 | 11.218 | 12.009 | 14.122 | 10.991 |
| | 08servers | 05files | 20threads | ccALWAYS | 9.881 | 9.405 | 9.322 | 10.184 | 9.924 |
| gor | 01servers | 10files | 20threads | ccALWAYS | 17.322 | 17.636 | 16.096 | 16.484 | 16.715 |
| | 08servers | 10files | 20threads | ccALWAYS | 17.167 | 17.666 | 15.950 | 18.867 | 16.569 |
| gorz | 01servers | 10files | 20threads | ccALWAYS | 14.718 | 19.553 | 17.930 | 21.415 | 21.495 |
| | 08servers | 10files | 20threads | ccALWAYS | 10.236 | 9.948 | 12.605 | 9.780 | 10.320 |
| gor | 01servers | 15files | 20threads | ccALWAYS | 19.265 | 17.128 | 17.630 | 18.739 | 16.833 |
| | 08servers | 15files | 20threads | ccALWAYS | 23.083 | 22.084 | 25.024 | 24.677 | 20.648 |
| gorz | 01servers | 15files | 20threads | ccALWAYS | 15.401 | 28.282 | 28.727 | 24.645 | 27.509 |
| | 08servers | 15files | 20threads | ccALWAYS | 10.307 | 12.217 | 13.005 | 12.277 | 12.224 |
| gor | 01servers | 20files | 20threads | ccALWAYS | 23.744 | 20.539 | 21.416 | 22.921 | 22.794 |
| | 08servers | 20files | 20threads | ccALWAYS | 35.393 | 36.218 | 35.949 | 35.157 | 37.342 |
| gorz | 01servers | 20files | 20threads | ccALWAYS | 19.505 | 23.756 | 25.767 | 26.575 | 25.239 |
| | 08servers | 20files | 20threads | ccALWAYS | 11.398 | 11.332 | 15.086 | 16.115 | 13.479 |
+-----------+--------------+------------+--------------+-----------------+---------+---------+---------+---------+---------+
I would like to take all the gor,1servers,20threads,ccALWAYS configurations and create one data point for each of the XXfiles configurations. So to begin with I'd like to somehow get a DataFrame that looks like this:
+-----------+--------------+------------+--------------+-----------------+---------+---------+---------+---------+---------+
| | | | | | run_001 | run_002 | run_003 | run_004 | run_005 |
+-----------+--------------+------------+--------------+-----------------+---------+---------+---------+---------+---------+
| file_type | server_count | file_count | thread_count | cacheclear_type | | | | | |
+-----------+--------------+------------+--------------+-----------------+---------+---------+---------+---------+---------+
| gor | 01servers | 05files | 20threads | ccALWAYS | 15.918 | 16.275 | 15.807 | 17.781 | 16.233 |
| gor | 01servers | 10files | 20threads | ccALWAYS | 17.322 | 17.636 | 16.096 | 16.484 | 16.715 |
| gor | 01servers | 15files | 20threads | ccALWAYS | 19.265 | 17.128 | 17.630 | 18.739 | 16.833 |
| gor | 01servers | 20files | 20threads | ccALWAYS | 23.744 | 20.539 | 21.416 | 22.921 | 22.794 |
+-----------+--------------+------------+--------------+-----------------+---------+---------+---------+---------+---------+
How do I do that?

I managed to filter the data using the query() function to get it to look like the second table in the question using the following code:
df.query('file_type == "gor" & server_count == "01servers"').sortlevel(2)