I'm slowly building a web browser in PyQt4 and like the speed i'm getting out of it. However, I want to combine easylist.txt with it. I believe adblock uses this to block http requests by the browser.
How would you go about it using python/PyQt4?
[edit1] Ok. I think i've setup Privoxy. I haven't setup any additional filters and it seems to work. The PyQt4 i've tried to use looks like this
self.proxyIP = "127.0.0.1"
self.proxyPORT= 8118
proxy = QNetworkProxy()
proxy.setType(QNetworkProxy.HttpProxy)
proxy.setHostName(self.proxyIP)
proxy.setPort(self.proxyPORT)
QNetworkProxy.setApplicationProxy(proxy)
However, this does absolutely nothing and I cannot make sense of the docs and can not find any examples.
[edit2] I've just noticed that i'f I change self.proxyIP to my actual local IP rather than 127.0.0.1 the page doesn't load. So something is happening.
I know this is an old question, but I thought I'd try giving an answer for anyone who happens to stumble upon it. You could create a subclass of QNetworkAccessManager and combine it with https://github.com/atereshkin/abpy. Something kind of like this:
from PyQt4.QtNetwork import QNetworkAccessManager
from abpy import Filter
adblockFilter = Filter(file("easylist.txt"))
class MyNetworkAccessManager(QNetworkAccessManager):
def createRequest(self, op, request, device=None):
url = request.url().toString()
doFilter = adblockFilter.match(url)
if doFilter:
return QNetworkAccessManager.createRequest(self, self.GetOperation, QNetworkRequest(QUrl()))
else:
QNetworkAccessManager.createRequest(self, op, request, device)
myNetworkAccessManager = MyNetworkAccessManager()
After that, set the following on all your QWebView instances, or make a subclass of QWebView:
QWebView.page().setNetworkAccessManager(myNetworkAccessManager)
Hope this helps!
Is this question about web filtering?
Then try use some of external web-proxy, for sample Privoxy (http://en.wikipedia.org/wiki/Privoxy).
The easylist.txt file is simply plain text, as demonstrated here: http://adblockplus.mozdev.org/easylist/easylist.txt
lines beginning with [ and also ! appear to be comments, so it is simply a case of sorting through the file, and searching for the correct things in the url/request depending upon the starting character of the line in the easylist.txt file.
Privoxy is solid. If you want it to be completely API based though, check out the BrightCloud web filtering API as well.
Related
I try to download a bingads report using python SDK, but I keep getting an error says: "Type not found: 'Aggregation'" after submitting a report request. I've tried all 4 options mentioned in the following link:
https://github.com/BingAds/BingAds-Python-SDK/blob/master/examples/v13/report_requests.py
Authentication process prior to request works just fine.
I execute the following:
report_request = get_report_request(authorization_data.account_id)
reporting_download_parameters = ReportingDownloadParameters(
report_request=report_request,
result_file_directory=FILE_DIRECTORY,
result_file_name=RESULT_FILE_NAME,
overwrite_result_file=True, # Set this value true if you want to overwrite the same file.
timeout_in_milliseconds=TIMEOUT_IN_MILLISECONDS
)
output_status_message("-----\nAwaiting download_report...")
download_report(reporting_download_parameters)
after a careful debugging, it seems that the program fails when trying to execute a command within "reporting_service_manager.py". Here is workflow:
download_report(self, download_parameters):
report_file_path = self.download_file(download_parameters)
then:
download_file(self, download_parameters):
operation = self.submit_download(download_parameters.report_request)
then:
submit_download(self, report_request):
self.normalize_request(report_request)
response = self.service_client.SubmitGenerateReport(report_request)
SubmitGenerateReport starts a sequence of events ending with a call to "_SeviceCall.init" function within "service_client.py", returning an exception "Type not found: 'Aggregation'"
try:
response = self.service_client.soap_client.service.__getattr__(self.name)(*args, **kwargs)
return response
except Exception as ex:
if need_to_refresh_token is False \
and self.service_client.refresh_oauth_tokens_automatically \
and self.service_client._is_expired_token_exception(ex):
need_to_refresh_token = True
else:
raise ex
Can anyone shed some light? .
Thanks
Please be sure to set Aggregation e.g., as shown here.
aggregation = 'Daily'
If the report type does not use aggregation, you can set Aggregation=None.
Does this help?
This may be a bit late 2 months after the fact but maybe this will help someone else. I had the same error (though I suppose it may not be the same issue). It does look like you did what I did (and I'm sure others will as well): copy-paste the Microsoft example code and tried to run it only to find that it didn't work.
I spent quite some time trying to debug the issue and it looked to me like the XML wasn't being searched correctly. I was using suds-py3 for the script at the time so I tried suds-community and everything just worked after that.
I also re-read the Bing Ads API walkthrough for getting started again and found that they recommend suds-jurko instead.
Long story short: If you want to use the bingads API don't use suds-py3, use either suds-community (which I can confirm works for everything I've used the API for) or suds-jurko (which is the one recommended by Microsoft).
I'm at the moment in the middle of writing my Bachelor thesis and for it creating a database system with Postgres and Flask.
To ensure the safety of my data, I was working on a file to prevent SQL injections, since a user should be able to submit a string via Http request. Since most of my functions which I use to analyze the Http request use Kwargs and a dict based on JSON in the request I was wondering if it is possible to inject python code into those kwargs.
And If so If there are ways to prevent that.
To make it easier to understand what I mean, here are some example requests and code:
def calc_sum(a, b):
c = a + b
return c
#app.route(/<target:string>/<value:string>)
def handle_request(target,value):
if target == 'calc_sum':
cmd = json.loads(value)
calc_sum(**cmd)
example Request:
Normal : localhost:5000/calc_sum/{"a":1, "b":2}
Injected : localhost:5000/calc_sum/{"a":1, "b:2 ): print("ham") def new_sum(a=1, b=2):return a+b":2 }
Since I'm not near my work, where all my code is I'm unable to test it out. And to be honest that my code example would work. But I hope this can convey what I meant.
I hope you can help me, or at least nudge me in the right direction. I've searched for it, but all I can find are tutorials on "who to use kwargs".
Best regards.
Yes you, but not in URL, try to use arguments like these localhost:5000/calc_sum?func=a+b&a=1&b=2
and to get these arguments you need to do this in flask
#app.route(/<target:string>)
def handle_request(target):
if target == 'calc_sum':
func= request.args.get('func')
a = request.args.get('a')
b = request.args.get('b')
result = exec(func)
exec is used to execute python code in strings
Need help to select all elements of a listview using send_message. I want this to work in RDP disconnected mode and hence using pywinauto's send_message api.
My code
from pywinauto import win32defines
app = Application().connect(path = pathToAppEXE)
lvitem = win32structures.LVITEMW()
lvitem.mask = win32defines.LVIF_STATE
lvitem.state = 1
lvitem.stateMask = win32defines.LVIS_SELECTED
app.window_(title_re = "Net Position.*").ListView.send_message(win32defines.LVM_SETITEMSTATE,-1,lvitem)
It does nothing. Maybe I am not getting the lvm flags correctly. Need assistance to fix the code.
Method .get_item(...) (see docs) should return _listview_item object with some available methods: some of them don't involve real click.
Maybe Remote Execution Guide is also useful.
So because I have the unity-gtk-module installed, all gtk-applications export their menu over the dbus SessionBus. My goal is to extract a list of all available menu entries. I've already implemented this with the help of pydbus, but for some reason, this solution is highly unstable and some applications just flat out crash. The unity-gtk-module uses Gio's g_dbus_connection_export_menu_model () to export its GMenuModel modeled menu over dbus, so I thought it would make sense to try to use Gio to process the exported menu. Gio uses the GDBusMenuModel class to retrieve a menu from the bus. Python uses PyGObject for wrapping Gio:
from gi.repository import Gio
connection = Gio.bus_get_sync(Gio.BusType.SESSION, None)
menuModel = Gio.DBusMenuModel.get(connection, [bus-name e.g. ":1.5"], [object-path e.g. "/com/canonical/unity/gtk/window/0"])
Now menuModel should be wrapping the GMenuModel from my application. At this point I'm honestly a bit confused about how exactly the GMenuModel works (the Description is not really helping) but it seems I have to use a GMenuAttributeIter object to iterate through the entries. But when I try this:
iter = Gio.MenuModel.iterate_item_attributes(menuModel, 0) #0 is the index of the root node
this happens:
GLib-GIO-CRITICAL **: g_dbus_menu_model_get_item_attributes: assertion 'proxy->items' failed
GLib-GIO-CRITICAL **: GMenuModel implementation 'GDBusMenuModel' doesn't override iterate_item_attributes() and fails to return sane calues from get_item_attributes()
This probably happens because GDBusMenuModel inherits GMenuModel which provides these methods, but is abstract, so GDBusMenuModel should override them, which it doesn't (see link above, it provides just g_dbus_menu_model_get ()). If this is the case, how am I supposed to actually use this class as a proxy? And if it's not, what am I doing wrong?
I justed logged in to SO the first time after a few years and remembered that I've actually found a solution to this question (I think). Honestly, I can't remember what half of these words even mean, but at the time I wrote a script to accomplish the task posed in the title, and as far as I remember, in the end, it worked out: https://gist.github.com/encomiastical/caa0ee955300bc2a40ef55d123b06212
I'm trying to write a test for an ajax view...The view is configured like so:
#view_config(name='new', context='resource.Events', renderer='json',
request_method='POST', xhr=True)
def event_view(self):
# ...
In my test, I want to create an ajax post so I try:
extra_environ = {'X_REQUESTED_WITH' : 'XmlHttpRequest'}
# also tried setting HTTP_X_REQUESTED_WITH
self.testapp.post('/events/new', params=post_params, extra_environ=extra_environ)
But the post never gets routed to my view. The request.is_xhr param never gets set to True.
Should be a simple answer somewhere, but I could not find it in any of the docs, or elsewhere. Can anyone recommend how to do this?
Thanks!
Looks like I wasn't casing 'xmlhttprequest' correctly. Needs to be:
extra_environ = {'HTTP_X_REQUESTED_WITH' : 'XMLHttpRequest'}
Found this in the code at:
webob/request.py line 472
Unfortunate to need to dig around in the code for this. Also surprised case matters.
In my case it was simply enough to set xhr=True in post request.
self.testapp.post('/events/new', params=post_params, xhr=True)