I use Selenium (Python) to run multiple chromedriver instances on a Raspberry Pi. Preferably I would like to run as many instances as the Pi's hardware can manage, hence a lower resource usage will be beneficial. Running a headless chromedriver is unfortunately not possible due to chromedriver not allowing DRM content to load in this mode.
A starting point would most likely be to optimize switches. A very comprehensive list of available switches can be found here. The list contains a lot of switches, and I was wondering if anyone has a list of switches optimizing performance that I can use as a starting point.
In addition I would like to mention that it is not important for me to actually see what happens on the browser window. So, might it be relevant to e.g. prevent images from loading or hide/move the browser window of screen?
Related
I want to know if is possible to capture a lot of actions like a flow and export to selenium, in order to repeat that flow.
For example, I need to uninstall and reinstall and configure a few applications each day several times, the process is always the same, and it's a long process, so in order to avoid navigate between code to capture all IDs and classes, is there any way of doing that?
Kind regards.
I think Selenium IDE is basically what you are looking for. It is a Chrome or Firefox extension that has a record and play back feature, and it is able to export working code in a variety of languages (including python).
Word of caution: tests produced by the tool tend to be pretty unreliable/flaky, you can attain much better stability by coding with WebDriver.
Using pyautogui or something similar you could record the location of each click and either use the color of certain pixels to intiate different stages or waiting x amount of time before click each saved point on screen
I am attempting to quantify how much download quota would be consumed when a certain web page is loaded (in chrome in my case), including all the page's assets (e.g. 'loaded' according to regular human use of the webpage)
Is there a way to achieve this using mainstream techniques (e.g. a python library, selenium, netstat command line utility, curl or something else)?
Note: I guess one very crude way would be to check my ISP stats before/after the page load, but this is fraught with potential inaccuracies, probably most notably the device doing background tasks and the ISP not providing quota estimates fine enough to discern the additional kbs consumed by the page load, so I think this method would not be reliable
There may be better ways, but I found one that seems to work
In chrome, open developer tools (cmd + option + j), click on the 'network' tab, and refresh the page. When it has fully loaded, look for the resources.
Note: to get an accurate reading, it could be important to ensure the 'Disable cache' checkbox is ticked (failing to disallow the cache could underestimate the download quota required)
For the page we're on now, we see it uses 1.5MB without disabling the cache.
Note: the amount seemed to vary for me quite a bit each time I ran it (not always in a downward direction), so depending on the circumstances, it could be worth doing this several times and taking an average.
My project consists of a website where a user inputs a Music XML file and receives a video (similar to synthesia) based on that XML file. I am using python to parse the XML file and get all the useful information. With that information, I am using PyOpenGL with glut to create animations and OpenCV to save each frame to a video.
I am able to run the program locally and it works. Now I am trying to use the program within my Wamp Server. So my question is, how would I go about doing this? My plan was to call the program with PHPs shell_exec() but nothing seems to happen. I've tested shell_exec() on simple test files that returns a string and that works. I have done some research and found I can use xvfb for headless server rendering. Any idea of how I can implement this with PyOpenGL/Glut? Also is it ok to use PHPs shell_exec() or should I be using something else to call my Python program?
First and foremost decide upon you need/want GPU acceleration or not. There's little use in trying to hone out a GPU accelerated OpenGL context creation, if your target system doesn't even have a GPU.
Next you should come to terms with, that you'll no longer be able to use GLUT, because GLUT was implemented with creating on-screen windows in mind.
If you can live without GPUs and rely on software rasterization, you should look into OSMesa https://mesa3d.org/osmesa.html
If you need GPU acceleration, check what GPU you'll be running on. If it's going to be a NVidia one, check out their excellent blog on how to create a headless rendering context with EGL https://devblogs.nvidia.com/egl-eye-opengl-visualization-without-x-server/
If it's an AMD or Intel GPU, then EGL in theory should work as well. However using DRM+GBM will usually yield better results. There's an example project for that to be found at https://github.com/eduble/gl
Got a bit of weird request here, however it's one which I can't really figure out the answer to.
I'm writing a python application that displays web pages and locally stored images.
What I need is a way of displaying a web page using python that is really lightweight and quite fast. The reason for this is that it is running on a Raspberry Pi.
Of course I have many options, I can run it through web browser installed on the Raspbian distribution and run it as a separate process in python, I can download an Arch-Linux compatible browser and run it as a separate process in python and finally I can write my own native python file using Gtk or PyQt.
All of these approaches have their downsides as well as serious overheads. The web browser must also be full screen when I have a web page to display, and minimised when I'm displaying an image.
The main issue I have had with Gtk and PyQt is the way they have to be executed on the main thread - which is impossible as it doesn't align with my multithreaded architecture. The downside to using the web browsers that are pre-installed on raspbian, is that from python you lack control and it's slow. And finally, the issue with using an Arch-Linux browser is that it ends up being messy and hard to control.
What I would Ideally need is a web browser that loads a web page almost instantaneously, or a multithreaded web browser that can handle multiple instances. This way I can buffer one web page in the background whilst another browser is being displayed.
Do you guys have any advice to point me in the right direction? I would've thought that there would be a neat multithreaded python based solution by now, and I would think that's either because no one needs to do what I'm doing (less likely) - or I'm missing something big (more likely)!
Any advice would be appreciated.
James.
I'd use PyQT to display the page but if the way PyQT use threads does not fit within you application, you may just write a minimalist (I'm speaking of ~10 lines of code here) web browser using PyQT, and fork it from your main application ?
The solution that I came to was to use a couple of frame buffer browsers for linux called: netsurf-fb and links2 with -.
However after extensive testing - i decided it was not appropriate to use these due to the fact that they don't have javascript support.
Therefore the end solution was to use a script called cutycapt running in a virtual x window frame buffer called xvfb.
I called it from python using the pxpect library like so:
process=pexpect.spawn(xvfb_run_bin+' --server-args "-screen 0, '+self.width_height+'x24" cutycapt --url='+uri+' --out='+temp_path)
process.wait()
This in my implementation goes off, renders the page and saves the screen shot. An image viewer then fetches the image from my cache and displays it.
If anyone has any further questions - feel free to comment on this question...
IMPORT cutecapt information. If you want to render a whole page using javascript it will take longer... specify the --delay 1000 where it will delay the loading by 1 second in this instance. For me it took around 7000 to get it just right...
i have written winks-up in vala.
It's small and fast and compile well on raspbian.
All the code was optimize to reduce memory occupation.
isn't perfect but was better like nothing
We have begun upgrading hardware and software to a 64-bit architecture using Apache with mod_jk and four Tomcat servers (the new hardware). We need to be able to test this equipment with a large number of simultaneous connections while still actually doing things in the app (logging in, etc.)
I currently am using Python with the Mechanize library to do this, but it's just not cutting it. Threading is not "real" in Python, and multiprocessing makes the local box work harder than the machines we are trying to test since it has to load so much into memory for Mechanize.
The bottom line is that I need something that will really hammer this thing's connections and hold a session to make sure that the sticky sessions are working in mod_jk. I need to be able to code it quickly, it needs to be lightweight, and being able to do true multithreading would be a perk. Other than that, I am open-minded.
Any input will be greatly appreciated. Thanks.
Open Source Testing Tools
Not knowing the full requirements makes it difficult, however something from the list might fit the bill.
In order to accomplish what I wanted to do, I just went back to basics. Mechanize is somewhat bulky, and there was a lot of bloat involved in the main functionality tests I had before. So I started with a clean slate and just used cookielib.CookieJar and urllib2 to build a linear test and then run them in a while 1 loop. This provided enough strain on the Apache system to see how it would react in the new environment, and for the record, it did VERY well.