I'm trying to get some tables with specific filters on this qlikview page, for future analysis: http://transferenciasabertas.planejamento.gov.br/QvAJAXZfc/opendoc.htm?document=painelcidadao.qvw&lang=en-US&host=QVS%40srvbsaiasprd01&anonymous=true
I don't want to do it manually (downloading tables for every filter). Therefore, I searched for API's for Python on qlikview website, but only found qliksense API's for SSE (like this https://github.com/qlik-oss/server-side-extension).
Is there any chance that I could automate the retrieving process that I explained using Python?
Server side extensions are used for something else. They extend Qlik's functionality to process data (for example running some statistical functions on top of the displayed data if such functions do not exists in Qlik natively)
Interestingly is that the portal link (http://transferenciasabertas.planejamento.gov.br) is a QlikView app that later redirects to a Qlik Sense app(s). It seems that anonymous users are allowed on the platform (which makes automating data retrieval easier).
Qlik Sense communicates with the browser via web sockets. So the answer to your question is - yes. You can used Python to connect to the underlying Qlik Sense Engine and make some selections and get the data back.
The not very good news is that I dont think there is dedicated Python library so you'll have to send the raw web socket requests by yourself. The documentation for the Engine API can be found at Qlik's help site
If you are open for JS solution then you can use Qlik's enigma.js library for Engine communication.
The web sockets traffic can be monitored from the browser (to view what data is being send/received and its format)
Related
I have a Python program for customers to query price. Each time a customer can input some necessary information, the program will calculate and return the price to customer. Note: during the calculation process, the program also need to query a third party map service web API to get some information (such as google map API or other similar service).
I have a website developed using web development tools such Wix, Strikingly. It offers a capability to customize a web page by simply input a block of HTML codes. So, I want to study the possibility of using Django to convent my python program into HTML (incl. add some user interface such as text box and button), which can then be pasted into the website to form a unique webpage.
I am not sure if it is doable? Especially, the part of connecting third party map service API. Would Django be able to convert this part automatically to HTML as well? (how does it deal with API key and connection).
Python itself runs only on the console, and is meant to be the backend in site development, whereas HTML is meant only to be the frontend, so no calculation or data fetching. Wix is a frontend tool with some content management that offers customization but still in the frontends (html/css), and there's nothing more you could do with the content management other than using the built in table like feature. Trying to use the html generated by wix will be so much pain due to its css name optimization and making it quite unscalable.
If you don't wish to learn frontend building at all then you could look up other html generator tool for the frontend codes. From there, django itself is capable of building the entire website, using the html you generated as template, and passing the data you've computed into the templates. That's what Django is meant to do. In this case you would need to learn Django itself, which I would recommend if you intend to showcase your project as an interactive program rather just console logs.
Other alternatives include converting your python codes into javascript, which is capable of doing calculations and fetching from APIs, and you can include the javascript code directly in HTML with the tag.
Actual situation:
The client downloads a small pythonscript that is executable.
The client executes it. The script gathers information from the computer and sends the data to the webserver vià POST-Method.
Wanted Situation:
After the webserver recived the data, it should forward the information to the website-session of the client. And the website should display the information.
This is a visual example of the principle:
There is also a example of this principle on Can You Run It.
How can I realize this?
A common way of implementing this is using a RESTful API. Basically the API does not care if the request is from a script or web browser, it just passes data structures to and from clients. The only tricky part to your example is when there are multiple users involved because a secret must be shared between the browser and script. I believe Can You Run It puts this unique secret into the program they ask you to download.
Look into Django Rest Framework for examples of how to implement this.
I want to do automatic searches on a database (in this example www.scopus.com) with a simple python script. I need some place from where to start. For example I would like to do a search and get a list of links and open the links and extract information from the opened pages. Where do I start?
Technically speaking, scopus.com is not "a database", it's a web site that let's you search / consult a database. If you want to programmatically access their service, the obvious way is to use their API, which will mostly requires sending HTTP requests and parsing the HTTP response. You can do this with the standard lib's modules, but you'll certainly save a lot of time using python-requests instead. And you'll certainly want to get some understanding of the HTTP protocol before...
I have a long-running process written in Python 2.7 that I would like to send KML files to my GWT application asynchronously as the KML files are generated.
I have been trying to determine what Python web framework I could use as the back-end with the Python process that could possibly allow the webapp to be hosted on Google AppEngine.
I was able to write a simple python webserver using Cherrypy that sent the kml using JSON from the back-end to GWT using an http request; however, I would like the files to be sent to GWT as they are generated since it may be several minutes between each one. What would be a relatively simple but effective way to achieve this? (Comet? Long-polling? Websockets?)
After researching more python web frameworks, I started experimenting with Tornado because it is non-blocking and seems like it could return data as it is generated possibly using long-polling as mentioned in this answer. However, it looks like GAE requires WSGI which would not allow a Tornado webserver to be non-blocking.
I have read answers to similar questions such as this one. However, I am not sure if updates in web frameworks, GWT, or GAE has changed what is the best option today, or whether some of these answers apply to my case.
What Python web framework would you recommend I use to send data to my asynchronous GWT app using long-polling or another method relatively simply? Could I use this web framework with GAE, or would I need to use something else?
If I understood the problem correctly you might don't need any special framework and you can solve it with what you have: Tasks API and Channel API.
With Tasks API you can perform long task and when the task is complete you can get a notification. You can combine it with the Channel API to push messages directly to the client when a particular task is complete.
You could use also the deferred library to simplify your life with tasks and maybe even using the PubNub for your push notifications, since the setup is easier and you can have many subscribers at the same time.
I'm looking for a bit of web development advice. I'm fairly new to the area but I'm sure there are some gurus out there willing to part with some wisdom.
Objective: I'm interested in controlling a Python application on my computer from my personal web hosted site. I know, this question has been asked several times before but in each case the requirements were a bit different from my own. To reduce the length of this post I'll summarize my objective in a few bullet points:
Personal site is hosted by a web hosting company
Site uses HTML, PHP, MySQL, Python and JavaScript, the majority of everything is coded by me from the ground up
An application that is coded in Python will run on a PC within my home and will communicate with an Arduino board
The app will receive commands from the internet to control actuation via the Arduino, and will transmit sensor data back to the site (such as temperature)
Looking for the communication to be bi-directional, fast and secure
Securing the connection between site and Python app would be most ideal
I'm not looking to connect to the Python application directly, the web server must serve as the 'middle man'
So far I've considered HTTP Post and HTML forms, using sockets (Python app would run as a web server), an IRC bot and reading/writing to a text file stored on the web server.
I was also hoping to have a way to communicate with the Python app without needing to refresh the webpage, perhaps using AJAX or JavaScipt? Maybe with Flash?
Is there something I'm not considering? I feel like I'm missing something. Thanks in advance for the advice!
Just thinking out loud for how I would start out with this. First, regarding the website itself, you can just use what's easiest to you, or to the environment you're in. For example, a basic PHP page will do just fine, but if you can get a site running in Python as well, I'd prefer using the same language all over.
That said, I'm not sure why you would need to use a hosted website? Given that you're already forced to have a externally accessible PC at home for the communication, why not run a webserver on that directly (Apache, Nginx, or even something like CherryPy should do)? That webserver can then communicate with the python process that is running to control your Arduino (by using e.g. Python's xmlrpclib). If you would run things via the hosting company, you would still need some process that can handle external requests securely... something a webserver is quite good at. Just running it yourself gives you all the freedom you want, and simplifies things by lessening the number of components in your solution.
The updates on your site I'd keep quite basic: commands you want to run can be handled in the request handlers of the webserver by just calling the relevant (xmlrpclib) calls. Dynamically updating the page is best done by some AJAX calls I reckon. Based on your story, these updates are easily put in a JSON object, suitable for periodically updating only the relevant segments of your page.