I've got a website that I wrote in python using the CGI. This was great up until very recently, when the ability to scale became important.
I decided, because it was very simple, to use mod_python. Most of the functionality of my site is stored in a python module which I call to render the various pages. One of the CGI scripts might look like this:
#!/usr/bin/python
import mysite
mysite.init()
mysite.foo_page()
mysite.close()
and in mysite, I might have something like this:
def get_username():
cookie = Cookie.SimpleCookie(os.environ.get("HTTP_COOKIE",""))
sessionid = cookie['sessionid'].value
ip = os.environ['REMOTE_ADDR']
username = select username from sessions where ip = %foo and session = %bar
return(username)
to fetch the current user's username. Problem is that this depends on os.envrion getting populated when os is imported to the script (at the top of the module). Because I'm now using mod_python, the interpreter only loads this module once, and only populates it once. I can't read cookies because it's os has the environment variables of the local machine, not the remote user.
I'm sure there is a way around this, but I'm not sure what it is. I tried re-importing os in the get_username function, but no dice :(.
Any thoughts?
Which version of mod_python are you using? Mod_python 3.x includes a separate Cookie class to make this easier (see here)
Under earlier versions IIRC you can get the incoming cookies inside of the headers_in member of the request object.
Related
I am working on a python package which speaks with an API (using HTTP protocol).
In my package I have a python configuration file that contains the API URL (hardcoded) and other settings.
The API URL is saved in a global variable so I could import it and use it in my package modules. For example:
API_URL = "https://api-url.com"
The configuration file is part of the package, that means once that a user installs the package he also gets the configuration file including the hardcoded URL.
The problem is that in some point in the future the API's URL could change, and all the factuality's of the package will break, and the users will have to update (i.g. pip install etc.) the package.
What is the right way to handle hardcoded URLs in a package?
The common pattern here is to not hardcode the URL base, but instead provide a API class that will get instantiate with the corresponding URL. This class can also get additional config/a config file:
class MyAPI:
def __init__(self, config_file: str = DEFAULT_CONFIG, **overrides):
self.config = build_config(config_file, overrides)
self.url = self.config.url
#property
def value(self):
return requests.get(urljoin(self.url, "./value"))
The user would then use it like this:
api = MyAPI()
print(api.value)
or
api = MyAPI(url="NEW_URL")
print(api.value)
This is as example similar to what praw and wikipedia use.
This is an especially good idea if there are different URL that can be used with the same/a very similar Interface. (Often at least the live product + a test sandbox)
You can design your API so that URIs don't change. This might be achieved using versioning, e.g. serving your API at https://api-url.com/v1 instead of https://api-url.com. In this case users
won't have to update or change anything because any changes to the API won't affect them since the changes will exist only in the newer version of you API served at https://api-url.com/v2.
Another option is to force library clients to pass the URL they want to use. This will solve the issue with hardcoding but won't solve the root cause. API usually change because something has changed in the understanding of the domain, which in turn means there are likely more changes than URLs alone (which will break your library anyway).
In case you think you simply might want to change domain name and don't anticipate any other changes that might affect clients you can use redirect from old URL to the new one.
I am creating a Python script which reads a spreadsheet and issues Rest requests to an external service. A token must be obtained to issue requests to the Rest service, so the script needs a username and password to obtain the oauth2 token.
What's the best or standard way to hide or not have visible information in the Python script?
I recommend using a config file. Let's create a config file and name it config.cfg. The file structure should look more or less like this:
[whatever]
key=qwerertyertywert2345
secret=sadfgwertgrtujdfgh
Then in python you can load it this way:
from configparser import ConfigParser
config = ConfigParser()
config.read('config.cfg')
my_key = config['whatever']['key']
my_secret = config['whatever']['secret']
In general, the most standard way to handle secrets in Python is by putting them in runtime configuration.
You can do that by reading explicitly from external files or using os.getenv to read from environment variables.
Another way is to use a tool like python-decouple, which lets you use the environment (variables), a .env file, and an .ini file in a cascade, so that developers and operations can control the environment in local, dev, and production systems.
I am building simple app which is using Twitter API. What I have to do to hide my Twitter app keys? For example, if I will put my program to the internet and somebody who look up to the code will know my consumer key, access token etc. And if I not include this information into my program, that it won't be work!
I'm assuming by putting on the internet you mean publishing your code on github or such.
In that case you should always separate code and configuration. Put your API keys in an .ini file, i.e. config.ini, then load that file from python program using configparser
Add configuration file to your .gitignore so it would not get added to the source control.
Assuming you're running on a Unix like system, one way to handle this is environment variables.
In your shell you can do this:
export TWITTER_API_KEY=yoursecretapikey
Note that you don't use quotes of any kind for this.
Then in your script:
import os
twitter_key = os.environ.get('TWITTER_API_KEY')
If there is someone out there who has already worked with SOLR and a python library to index/query solr, would you be able to try and answer the following question.
I am using the mySolr python library but there are others out (like pysolr) there and I don't think the problem is related to the library itself.
I have a default multicore SOLR setup, so no authentication required normally. Don't need it to access the admin page at http://localhost:8080/solr/testcore/admin/ either
from mysolr import Solr
solr = Solr('http://localhost:8080/solr/testcore/')
response = solr.search(q='*:*')
print("response")
print(response)
This code used to work but now I get a 401 reply from SOLR ... just like that, no changes have been made to the python virtual env containing mysolr or the SOLR setup. Still...something must have changed somewhere but I'm out of clues.
What could be the causes of a SOLR 401 reponse?
Additional info: This script and mor advanced script do work on another PC, just not on the one I am working on. Also, adding "/select?q=:" behind the url in the browser does return the correct results. So the SOLR is setup correctly, it has probably something to do with my computer itself. Could windows settings (of any kind) have an impact on how SOLR responds to requests from python? The python env itself has been reinstalled several times to no avail.
Thanks in advance!
The problem was: proxy.
If this exact situation was ever to occur to someone and you are behind a proxy, check if your HTTP and HTTPS environmental variables are not set. If they are... this might cause the python session to try using the proxy while it shouldn't (connecting to localhost via proxy).
It didn't cause any trouble for months but out of the blue it did so whether you encounter this or not might be dependent on how your IT setup your proxy or made some other changes...somewhere.
thank you everyone!
I've been asked to set up a FTP server using python that different users can log in to, and depending on their login will display a different file structure.
Part of the structure will be read only, and another part write, read, create and delete.
The file structure and files won't exist on the server, and will have to be built in a lazy way as the user expands folders by querying external servers.
The servers need to, I guess, mimic the FTP interface/protocol from the outside, but work completely differently internally.
I was wondering how big or difficult a job this would be as I need to provide some type of time scale for getting this working.
Is there anything like this out there already? has anyone done something similar before?
Are there any obvious problems of trying to implement this kind of model?
The twisted project would be the obvious place to start; the following example starts a simple FTP server that authenticates users against a password file but also allows anonymous access
from twisted.protocols.ftp import FTPFactory, FTPRealm
from twisted.cred.portal import Portal
from twisted.cred.checkers import AllowAnonymousAccess, FilePasswordDB
from twisted.internet import reactor
p = Portal(FTPRealm('./'),
[AllowAnonymousAccess(), FilePasswordDB("pass.dat")])
f = FTPFactory(p)
reactor.listenTCP(21, f)
reactor.run()
You can easily expand from there. How you implement 'files' and 'directories' is completely up to you.
Why python? I mean what python has to do with it? I'd look for some PAM module, able to create user-specific virtual filesystem structure on login, and if there's no ready one, consider modify some pam_mount, something like that..
http://pam-mount.sourceforge.net