I'm planning to compile my application to an executable file using Py2Exe. However, I have sensitive URL links in my application that I would like to remain hidden as in encrypted. Regardless if my application is decompiled, the links will still remain encrypted. How would I get say urllib2 to open the encrypted link?
Any help would be appreciated, and or example code that could point me in the right direction.
Thanks!
I don't think urllib2 has an option like it, though what you can do is save the link somewhere else (say a simple database, encrypt them (like a password) and when urllib2 calls the link, you check the hash.
Something like an user authentication.
URL encryption will not save you in this situation. As your software runs in the client machine and somehow decrypts this URL and sends an HTTP request to it, any kid using Wireshark will be able to see your URL.
If the design of your system requires sensitive URLs, the safer way probably involves changes in the design of your HTTP server itself! You have to structure your system in a way that the URLs are not sensitive because you cannot control them. As soon as they are used by your code, they can be captured.
Related
I'm making a simple project where I will have a downloadable scraper on an HTML website. The scraper is made in Python and is converted to a .exe file for downloading purposes. Inside the python code, however, I included a Google app password to an email account, because the scraper sends an email and I need the server to login with an available Google account. Whilst .exe files are hard to get source code for, I've seen that there are ways to do so, and I'm wondering, how could I make it so that anyone who has downloaded the scraper.exe file cannot see the email login details that I will be using to send them an email when the scraper needs to? If possible, maybe even block them from accessing any of the .exe source code or bytecode altogether? I'm using the Python libraries bs4 and requests.
Additionally, this is off-topic, however, as it is my first time developing a downloadable file, even whilst converting the Python file to a .exe file, my antivirus picked it up as a suspicious file. This is like a 50 line web scraper and obviously doesn't have any malicious code within it. How can I make the code be less suspicious to antivirus programs?
Sadly even today,there is no perfect solution to this problem.
The ideal usecase is to provide this secret_password from web application,but in your case seems unlikelly since you are building a rather small desktop app.
The best and easiest way is to create a function providing this secret_password in a separate file,and compile this file with Cython,thing that will obcufate your script(and your secret_password) at a very good extend.Will this protect you from lets say Anonymous or a state security agency?No.Here comes the reasonable thinking about how secret and important really your password is and from who you mainly can be harmed.
Finally before compiling you can 'salt' your script or further obscufate it with bcrypt or other libaries.
As for your second question antiviruses and specifically windows don't like programms running without installers and unsigned.
You can use inno setup to create a real life program installer.
If you want to deal with UAC or other issues related to unsigned programms you can sign your programm(will cost money).
Firstly, why is it even sending them an email? Since they'll be running the .exe, it can pop up a window and offer to save the file. If an email must be sent, it can be from the user's gmail rather than yours.
Secondly, using your gmail account in this way may be against the terms of service. You could get your account suspended, and it may technically be a felony in the US. Consult a lawyer if this is a concern.
To your question, there's basically no way to obfuscate the password that will be more than a mild annoyance to anyone with the least interest. At the end of the day, (a) the script runs under the control of the user, potentially in a VM or a container, potentially with network communications captured; and (b) at some point it has to decrypt and send the password. Decoding and following either the script, or the network communications that it makes will be relatively straightforward for anyone who wants to put in quite modest effort.
There is a site https://www.flashscore.ru/. I need to parse the 'Odds' category from there. But I do not know how to do this, the site has a loading using ajax. But when I look at this ajax I can’t find the very request that I need, all requests are encrypted as well. Tell me how to decrypt requests or where it can be read. I must say right away that I can’t open the browser, because the script will work on the server (without selenium type libraries). If there is a ready-made solution, I will be very happy
I want to do automatic searches on a database (in this example www.scopus.com) with a simple python script. I need some place from where to start. For example I would like to do a search and get a list of links and open the links and extract information from the opened pages. Where do I start?
Technically speaking, scopus.com is not "a database", it's a web site that let's you search / consult a database. If you want to programmatically access their service, the obvious way is to use their API, which will mostly requires sending HTTP requests and parsing the HTTP response. You can do this with the standard lib's modules, but you'll certainly save a lot of time using python-requests instead. And you'll certainly want to get some understanding of the HTTP protocol before...
I'm implementing a link scraping system like Facebook's link share feature, whereby a user enters a url which is passed to our server via ajax, and our server then does a get request (using the requests library) and parses the response html with Beautiful Soup to capture relevant information about the page.
In this type of system, obviously a person can enter any url that they want. I'm trying to imagine what type of security risks our server could be exposed to in this type of scenario? Could such a set up be exploited maliciously?
You probably want to make sure that your server doesn't execute any plugins or copy any videos/images.
Javascript is trickier, if you ignore it you will miss some links, if you execute it then you had better be sure you aren't being used to do something like send spam.
If you are asking on SO you probably aren't sure enough!
You should do a google on RFI/LFI (Remote / Local) File Inclusion Vulnerability and Iframe attacks. If you are safe from these two attacks , then you're good.
I have built quite a few small & large crawling systems. Actually not sure what kind of security risks you are talking about. I am not clear on your requirements.
But if all you are doing is fetch the html using BeautifulSoup & then extracting certain stuff about the page like title tag & meta tag info etc. & then store this data. I dont see any problems.
Unless you are not blindly doing some kind of eval either on the response of the url or on the stuff the user entered you are safe I feel.
I need to be able to block the urls that are stored in a text file on the hard disk using Python. If the url the user tries to visit is in the file, it redirects them to another page instead. How is this done?
There are several proxies written in Python: you can pick one of them and modify it so that it proxies most URLs normally but redirects those in your text file. You'll also need to set IE to use that proxy, of course.
Doing this at the machine level is a weak solution, it would be pretty easy for a technically inclined user to bypass.
Even with a server side proxy it will be very easy to bypass unless you firewall normal http traffic, at a bare minimum block ports 80, 443.
You could program a proxy in python as Alex suggested, but this is a pretty common problem and there are plenty of off the shelf solutions.
That being said, I think that restricting web access will do nothing but aggravate your users.