Develop Reference

Best way to obtain a secure connection using Python urllib

Hope this will be an easy question for someone to answer. I'm in the process of programming an application in Python and part of the application uses an API to obtain a download link and then uses that link to download the corresponding server updates. Currently I'm accomplishing this using urllib.request.urlopen() at the moment and would like to do so securely. Therefore I'm wondering if just specifying https in the URL is enough or if I have to use the context parameter in addition?
The Python documentation is a bit vague in how it handles HTTPS requests but as I understand it right now specifying https in the URL should be sufficient.

In the documentation for urllib.request.urlopen, there is a reference to the http.client.HTTPSConnection class in regards to the context parameter. In the documentation for the HTTPSConnection class, there is a link for security considerations. In here it states:
For client use, if you don’t have any special requirements for your security policy, it is highly recommended that you use the create_default_context() function to create your SSL context. It will load the system’s trusted CA certificates, enable certificate validation and hostname checking, and try to choose reasonably secure protocol and cipher settings.
Given that the documentation on urllib.request.urlopen() shows that context is an optional parameter, you probably don't HAVE to use it to make secure https connections, but given what the security considerations section says, I would use
ssl.create_default_context()
to generate the context just as good practice
urllib.request.urlopen("https://www.stackoverflow.com", context=ssl.create_default_context())
EDIT
Upon reviewing the source code for urllib.request.urlopen, if you do not specify a context but you use an https url, it looks like it will provide a default context for you. If you don't provide a context to urlopen() it will call build_opener() and in THAT function's comments it states
The opener will use several default handlers, including support
for HTTP, FTP and when applicable HTTPS.
So the final answer is you should be fine with providing no context, all it should need is the url

How to make requests from back-end to another server on user’s localhost

I’ve got a standard client-server set-up with ReScript (ReasonML) on the front-end and a Python server on the back-end.
The user is running a separate process on localhost:2000 that I’m connecting to from the browser (UI). I can send requests to their server and receive responses.
Now I need to issue those requests from my back-end server, but cannot do so directly. I’m assuming I need some way of doing it through the browser, which can talk to localhost on the user’s computer.
What are some conceptual ways to implement this (ideally with GraphQL)? Do I need to have a subscription or web sockets or something else?
Are there any specific libraries you can recommend for this (perhaps as examples from other programming languages)?

I think the easiest solution with GraphQL would be to use Subscriptions indeed, the most common Rescript GraphQL clients already have such a feature, at least ReasonRelay, Reason Apollo Hooks and Reason-URQL have it.

AppEnginePlatformWarning - reason to use sockets?

In the Google App Engine standard environment, if you use urllib to make HTTPS requests, you'll get an AppEnginePlatformWarning which says you're using urlfetch instead of sockets.
I found the warning annoying, so I disabled it.
# Use the App Engine Requests adapter. This makes sure that Requests uses
# URLFetch.
requests_toolbelt.adapters.appengine.monkeypatch()
# squelch warning
requests.packages.urllib3.disable_warnings(
requests.packages.urllib3.contrib.appengine.AppEnginePlatformWarning
)
My question is - is there a good reason to switch to sockets? Specifically what is wrong with using urlfetch?

There's nothing wrong with using urlfetch, in fact it is the recommended method for issuing outbound HTTP(S) requests on GAE. From Issuing HTTP(S) Requests (emphasis on requests-related note mine):
App Engine uses the URL Fetch service to issue outbound HTTP(S)
requests.
For details about how the URL Fetch service is implemented and which
headers are sent in a URL Fetch request, see Outbound Requests.
Issuing an HTTP request
To issue an outbound HTTP request, use the urlfetch.fetch
method. For improved code portability, you can also use the Python
standard libraries urllib, urllib2, or httplib to issue HTTP
requests. When you use these libraries in App Engine, they perform
HTTP requests using App Engine's URL Fetch service. You can also use
the third-party requests library as long as you configure it to use
URLFetch.
The sockets support is rather the problematic one in GAE, it comes with a fairly long list of limitations and restrictions, see Sockets Python API Overview, in particular the Limitations and restrictions section.
The warning you see is not from GAE, it's from the 3rd-party requests library you use, which is why I highlighted the note in the above quote. IMHO it's safe to simply ignore/mask the warning in a GAE context.

Is there a toolkit that will provide a web-based API (xmlrpc, REST, whatever) to remote IMAP servers?

I'm working on a Google App Engine project that needs to access IMAP. Context.IO isn't quite powerful enough for my purposes, but I'd like something in the same spirit: I want to log into, access, and manipulate hundreds of IMAP mailboxes from within Google App Engine, using either a third-party service or an application server I put on a dedicated hosting server.
As you might imagine, this is mostly to work around the opening sockets limitation in GAE.
Any recommendations?

I don't know of any pre-made solution, but rolling your own shouldn't be very difficult or take too long. You can build on IMAPClient and SimpleXMLRPCServer on the server and use xmlrpclib on the client.
You would need to think about a way to retain state between calls though, since XmlRPC is a connectionless protocol (as most other RPC mechanisms are as well) and implement some form of service authentication. I have written a class inherited from SimpleXMLRPCServer, which supports SSL connections and HTTP Basic Auth (xmlrpclib already has support for both items). If you're interested in the code, give me a shout.

Have a look at Mailgun; it offers a robust API and supports IMAP V4.*
* IMAP mailboxes are on Mailgun

making urllib request in Python from the client side

I've written a Python application that makes web requests using the urllib2 library after which it scrapes the data. I could deploy this as a web application which means all urllib2 requests go through my web-server. This leads to the danger of the server's IP being banned due to the high number of web requests for many users. The other option is to create an desktop application which I don't want to do. Is there any way I could deploy my application so that I can get my web-requests through the client side. One way was to use Jython to create an applet but I've read that Java applets can only make web-requests to the server it is deployed on and the only way to to circumvent this is to create a server side proxy which leads us back to the problem of the server's ip getting banned.
This might sounds sound like and impossible situation and I'll probably end up creating a desktop application but I thought I'd ask if anyone knew of an alternate solution.
Thanks.

You can use a signed Java applet, they can use the Java security mechanism to enable access to any site.
This tutorial explains exactly what you have to do: http://www-personal.umich.edu/~lsiden/tutorials/signed-applet/signed-applet.html
The same might be possible from a Flash applet. Javascript is also restricted to the published site and doesn't allow being signed or security exceptions like this, AFAIK.

You probably can use AJAX requests made from JavaScript that is a part of client-side.
Use server → client communication to give commands and necessary data to make a request
…and use AJAX communication from client to 3rd party server then.

This depends on the form of "scraping" you intend to do:
You might run into problems running an AJAX call to a third-party site. Please see Screen scraping through AJAX and javascript.
An alternative would be to do it server-side, but to cache the results so that you don't hit the third-party server unnecessarily.
Check out diggstripper on google code.