a lot of questions exists that are similar to this, but none of them helped me out. Basically I'm using WSGI start_response() method link. I tried to set a dummy header in the response with the tuple [('Set-Cookie', 'token=THE_TOKEN')] and add it to start response like this:
status = '200 OK'
response = 'Success'
start_response(status,[('Set-Cookie', "DMR_TOKEN=DMR_TOKEN")])
return response
I'm not pretty sure that is working correctly, but it's here setting cookies. Now, let's suppose the header is correct and in following requests I want to authenticate a token. What would be the correct way to catch that cookie/header setted in the past ?
I've been reading and find I need something like this:
(environ.get("HTTP_COOKIE",""))
but that has been yielding empty string all the time, so I'm just assuming the header/cookie is not correctly set.
Thanks guys
I think you need to set the path explicitly to get useful behavior out of cookies, try something like:...
from Cookie import SimpleCookie
def my_app(environ, start_response):
session_cookie = SimpleCookie()
session_cookie['session'] = "somedata"
session_cookie['session']["Path"] = '/'
headers = []
headers.extend(("set-cookie", morsel.OutputString())
for morsel
in session_cookie.values())
start_response("200 OK", headers)
Here's a simple solution I came up with for setting cookies in WSGI that doesn't require an external library:
def set_cookie_header(name, value, days=365):
dt = datetime.datetime.now() + datetime.timedelta(days=days)
fdt = dt.strftime('%a, %d %b %Y %H:%M:%S GMT')
secs = days * 86400
return ('Set-Cookie', '{}={}; Expires={}; Max-Age={}; Path=/'.format(name, value, fdt, secs))
def handler(env, start_response):
content_type = 'text/html'
headers = [('Content-Type', content_type), set_cookie_header('name', 'value')]
start_response('200 OK', headers)
...
Related
I'm working with one pretty old server and for some reason it requires to send cookies in next format (RFC2109 section 4.4). Raw Cookie header:
Cookie: $Version="1"; temp="1234567890";$Path="/";$Domain=".example.com"; session="abcdefgh";$Path="/";$Domain=".example.com"; id="00001111";$Path="/";$Domain=".example.com"
I know, that there's a way to implement that formatting manually using prepared request, but maybe there's some other method?
I'm using python requests.
Code which obviously doesn't work as expected:
from requests import Session
from datetime import datetime, timedelta
from http.cookiejar import DefaultCookiePolicy
sess = Session()
sess.cookies.set_policy(DefaultCookiePolicy(rfc2109_as_netscape=True))
sess.cookies.set(name="temp", value="1234567890", domain=".httpbin.org", path="/", expires=int((datetime.now() + timedelta(days=365)).timestamp()))
sess.cookies.set(name="session", value="abcdefgh", domain=".httpbin.org", path="/", expires=int((datetime.now() + timedelta(days=365)).timestamp()))
sess.cookies.set(name="id", value="00001111", domain=".httpbin.org", path="/", expires=int((datetime.now() + timedelta(days=365)).timestamp()))
resp = sess.get("https://httpbin.org/headers")
print(resp.json())
Upd.
I've tried to set cookie policy to this old standard, but it changed nothing.
Unfortunately, I haven't found other way except using prepared request. Sharing my code for future researchers.
Code:
from requests import Session, Request
def dumb_cookies_request(*args, **kwargs):
session = kwargs.pop('session') if 'session' in kwargs else Session()
req = Request(*args, **kwargs)
prepped = session.prepare_request(req)
if len(session.cookies) > 0:
cookies = ['$Version="1"']
for c in session.cookies:
cookies.append(f'{c.name}="{c.value}";$Path="{c.path}";$Domain="{c.domain}"')
prepped.headers['Cookie'] = '; '.join(cookies)
return session.send(prepped)
Usage:
sess = Session()
# some actions
resp = dumb_cookies_request("GET", "https://httpbin.org/headers", session=sess)
I managed to import queries into another account. I used the endpoint POST function given by Redash, it sort of just applies to just “modifying/replacing”: https://github.com/getredash/redash/blob/5aa620d1ec7af09c8a1b590fc2a2adf4b6b78faa/redash/handlers/queries.py#L178
So actually, if I want to import a new query what should I do? I want to create a new query that doesn’t exist on my account. I’m looking at https://github.com/getredash/redash/blob/5aa620d1ec7af09c8a1b590fc2a2adf4b6b78faa/redash/handlers/queries.py#L84
Following is the function which I made to create new queries if the query_id doesn’t exist.
url = path, api = user api, f = filename, query_id = query_id of file in local desktop
def new_query(url, api, f, query_id):
headers ={'Authorization': 'Key {}'.format(api), 'Content-Type': 'application/json'}
path = "{}/api/queries".format(url)
query_content = get_query_content(f)
query_info = {'query':query_content}
print(json.dumps(query_info))
response = requests.post(path, headers = headers, data = json.dumps(query_info))
print(response.status_code)
I am getting response.status_code 500. Is there anything wrong with my code? How should I fix it?
For future reference :-) here's a python POST that creates a new query:
payload = {
"query":query, ## the select query
"name":"new query name",
"data_source_id":1, ## can be determined from the /api/data_sources end point
"schedule":None,
"options":{"parameters":[]}
}
res = requests.post(redash_url + '/api/queries',
headers = {'Authorization':'Key YOUR KEY'},
json=payload)
(solution found thanks to an offline discussion with #JohnDenver)
TL;DR:
...
query_info = {'query':query_content,'data_source_id':<find this number>}
...
Verbose:
I had a similar problem. Checked redash source code, it looks for data_source_id. I added the data_source_id to my data payload which worked.
You can find the appropriate data_source_id by looking at the response from a 'get query' call:
import json
def find_data_source_id(url,query_number,api)
path = "{}/api/queries/{}".format(url,query_number)
headers ={'Authorization': 'Key {}'.format(api), 'Content-Type': 'application/json'}
response = requests.get(path, headers = headers)
return json.loads(response.text)['data_source_id']
The Redash official API document is so lame, it doesn't give any examples for the documented "Common Endpoints". I was having no idea how I should use the API key.
Instead check this saviour https://github.com/damienzeng73/redash-api-client .
After the first Python3 application is created in a new RedHat OpenShift account, wsgi.py looks like this template:
def application(environ, start_response):
ctype = 'text/plain'
if environ['PATH_INFO'] == '/health':
response_body = "1"
elif environ['PATH_INFO'] == '/env':
response_body = ['%s: %s' % (key, value)
for key, value in sorted(environ.items())]
response_body = '\n'.join(response_body)
else:
ctype = 'text/html'
response_body = '<lots of html>'
response_body = response_body.encode('utf-8')
status = '200 OK'
response_headers = [('Content-Type', ctype),
('Content-Length', str(len(response_body)))]
#
start_response(status, response_headers)
return [response_body ]
#
# Below for testing only
#
if __name__ == '__main__':
from wsgiref.simple_server import make_server
httpd = make_server('localhost', 8051, application)
# Wait for a single request, serve it and quit.
httpd.handle_request()
My question is this: During the execution of this code, the following information appears in the log, "[28/Jun/2016 18:35:08] "GET / HTTP/1.1" 200 14". How do I get at that information from within the python program? Have examined the available variables but can not see it. Is there a function call I can make?
Many thanks for your insights.
.... later ....
From reader fat fantasma I see that I should provide additional information.
The user connects to the website using a URL that looks like this:
http://www.website.com/help?
The "help?" appears after the GET in "GET / HTTP/1.1", when the extension to the base URL appears. The GET phrase appears in the system log but I need access to it within wsgi.py's app() function. I do not see how to do that.
I am trying to unshort a lot of URLs which I have in a urlSet. The following code works most of the time. But some times it takes a very long time to finish. For example I have 2950 in urlSet. stderr tells me that 2900 is done, but getUrlMapping does not finish.
def getUrlMapping(urlSet):
# get the url mapping
urlMapping = {}
#rs = (grequests.get(u) for u in urlSet)
rs = (grequests.head(u) for u in urlSet)
res = grequests.imap(rs, size = 100)
counter = 0
for x in res:
counter += 1
if counter % 50 == 0:
sys.stderr.write('Doing %d url_mapping length %d \n' %(counter, len(urlMapping)))
urlMapping[ getOriginalUrl(x) ] = getGoalUrl(x)
return urlMapping
def getGoalUrl(resp):
url=''
try:
url = resp.url
except:
url = 'NULL'
return url
def getOriginalUrl(resp):
url=''
try:
url = resp.history[0].url
except IndexError:
url = resp.url
except:
url = 'NULL'
return url
Probably it won't help you as it has passed a long time but still..
I was having some issues with Requests, similar to the ones you are having. To me the problem was that Requests took ages to download some pages, but using any other software (browsers, curl, wget, python's urllib) everything worked fine...
Afer a LOT of time wasted, I noticed that the server was sending some invalid headers, for example, in one of the "slow" pages, after Content-type: text/html it began to send header in the form Header-name : header-value (notice the space before the colon). This somehow breaks Python's email.header functionality used to parse HTTP headers by Requests so the Transfer-encoding: chunked header wasn't being parsed.
Long story short: manually setting the chunked property to True of Response objects before asking for the content solved the issue. For example:
response = requests.get('http://my-slow-url')
print(response.text)
took ages but
response = requests.get('http://my-slow-url')
response.raw.chunked = True
print(response.text)
worked great!
I'm creating a custom middleware to django edit response object to act as a censor. I would like to find a way to do a kind of search and replace, replacing all instances of some word with one that I choose.
I've created my middleware object, added it to my MIDDLEWARE_CLASSES in settings and have it set up to process the response. But so far, I've only found methods to add/edit cookies, set/delete dictionary items, or write to the end of the html:
class CensorWare(object):
def process_response(self, request, response):
"""
Directly edit response object here, searching for and replacing terms
in the html.
"""
return response
Thanks in advance.
You can simply modify the response.content string:
response.content = response.content.replace("BAD", "GOOD")
Perhaps my reply little better. When you try to make response.content.replace("BAD", "GOOD"), you will get error, that you cannot do it with strings, because response.content is byte array. I've added syntactic strings 'gen_duration_time_777' and 'server_time_777' to base template. And this works for me.
import time
from datetime import datetime
class StatsMiddleware(object):
duration = 0
def process_request(self, request):
# Store the start time when the request comes in.
request.start_time = time.time()
def process_response(self, request, response):
# Calculate and output the page generation duration
# Get the start time from the request and calculate how long
# the response took.
self.duration = time.time() - request.start_time
response["x-server-time"] = datetime.now().strftime("%d/%m/%Y %H:%M")
response.content = response.content.replace(b"server_time_777", str.encode(response["x-server-time"]))
response["x-page-generation-duration-ms"] = '{:.3f}'.format(self.duration)
response.content = response.content.replace(b"gen_duration_time_777", str.encode(response["x-page-generation-duration-ms"]))
return response