Develop Reference

How to access error reason in Python google.cloud.storage upload methods?

I am using Google's google-cloud-storage Python package for GCS access. When I get a 403 error it could be for many different reasons. Google's SDK by default only provides this message:
('Request failed with status code', 403, 'Expected one of', <HTTPStatus.OK: 200>)")
Using a debugger I can look deeper into the library and find _upload.py has a _process_response method where the true HTTP response can be found, with the following message as part of the result:
"message": "$ACCOUNT does not have storage.objects.delete access to $BLOB."
Q: Is there any way I can access this more useful error code or the raw response?
I am hoping to present to the user the difference between e.g. expired credentials and attempting to do something your credentials do not allow.

What version of google-cloud-storage are you using? With the latest, and this example:
from google.cloud import storage
client = storage.Client.from_service_account_json('service-account.json')
bucket = client.get_bucket('my-bucket-name')
blob = bucket.get_blob('test.txt')
try:
blob.delete()
except Exception as e:
print(e)
It prints the following:
403 DELETE https://storage.googleapis.com/storage/v1/b/my-bucket-name/o/test.txt?generation=1579627133414449: $ACCOUNT does not have storage.objects.delete access to my-bucket-name/test.txt.
The string representation here is roughly the same as e.message:
>>> e.message
'DELETE https://storage.googleapis.com/storage/v1/b/my-bucket-name/o/test.txt?generation=1579627133414449: $ACCOUNT does not have storage.objects.delete access to my-bucket-name/test.txt.'
If you want more structure, you can use e._response.json():
>>> e._response.json()
{
'error': {
'code': 403,
'message': '$ACCOUNT does not have storage.objects.delete access to my-bucket-name/test.txt/test.txt.',
'errors': [{
'message': '$ACCOUNT does not have storage.objects.delete access to my-bucket-name/test.txt/test.txt.',
'domain': 'global',
'reason': 'forbidden'
}]
}
}

Python telegram bot via proxy with authentification

everyone.
I am trying to create a simple telegram bot. In my country telegram is blocked so I need to work via proxy.
from telegram.ext import Updater
from telegram.ext import CommandHandler
import os
def start(bot, update):
print("Hello")
bot.sendMessage(chat_id=512562849, text="Hello.")
REQUEST_KWARGS={
'proxy_url': 'http://93.171.217.48:10996',
'urllib3_proxy_kwargs': {
'username': '***',
'password': '***',
}
}
updater = Updater(token='<BOT_TOKEN>',
request_kwargs=REQUEST_KWARGS)
start_handler = CommandHandler('start', start)
updater.dispatcher.add_handler(start_handler)
updater.start_polling()
But I have the next log
Exception in thread updater:
...
_HTTPConnection.__init__(self, *args, **kw)
TypeError: __init__() got an unexpected keyword argument 'username'
I used the next docs.
The free proxy works ok, but for my goals, it's bad to lose connection time after time. I prefer not to work under VPN either because of some bureaucracy in my company.
Thanks in advance!

I suppose you're using a SOCKS5 proxy. If that's the case, the proxy url protocol should be socks:// instead of https://, in your example:
REQUEST_KWARGS={
'proxy_url': 'socks5://93.171.217.48:10996',
'urllib3_proxy_kwargs': {
'username': '***',
'password': '***',
}

I've just ran the same issue and came up with the following.
1) Make sure we import required function from urllib3:
from urllib3 import make_headers
2) Fill appropriate values:
REQUEST_KWARGS = {
'proxy_url': "http://ip:port",
'urllib3_proxy_kwargs': {
'proxy_headers': make_headers(proxy_basic_auth='username:password')
}
}

I used tor to connect and my problem was fixed.
Download, install and run the tor browser.
Install pysocks dependency. pip install pysocks
And in the code: REQUEST_KWARGS = { 'proxy_url': 'socks5h://127.0.0.1:9150' }

I was behind corporate https proxy with auth and I solved with Working Behind a Proxy which basically proposes this:
TOKEN='YOUR_BOT_TOKEN'
REQUEST_KWARGS={
# "USERNAME:PASSWORD#" is optional, if you need authentication:
'proxy_url': 'http://USERNAME:PASSWORD#PROXY_HOST:PROXY_PORT/',
}
updater = Updater(TOKEN, request_kwargs=REQUEST_KWARGS)

Object 'user' has no atribute 'create' - Django

I have a question about an error that I am getting. I have a query that I am trying to send based on the Synapse Api for my project. I am currently trying to senda request taht creates a new user with the API. Every time I try to to send the request, I get a message that the User object doesnt have a create. Here is the error.
AttributeError at /setup_profile/
type object 'User' has no attribute 'create'
Request Method: POST
Request URL: http://127.0.0.1:8000/setup_profile/
Django Version: 1.8.6
Exception Type: AttributeError
Exception Value:
type object 'User' has no attribute 'create'
Exception Location: C:\Users\OmarJandali\Desktop\opentab\opentab\tab\views.py in createUserSynapse, line 1104
Here is the current code that I have which will create the request to create a new user.
def createUserSynapse(request):
args = {
'email': 'hello#synapsepay.com',
'phone_number': '555-555-5555',
'legal_name': 'Hello McHello',
'note': ':)', # optional
'supp_id': '123abc', # optional
'is_business': True,
'cip_tag': 1
}
user = User.create(client, **args)
print(user)
Now i do know that with a normal query set, I have the object in the following format
User.objects.create(client, **args)
but when i do that I get an error that says
two parameters are being passed and 1 is requires so I think there is to many variables being passed... I am not sure where the error is coming from...
here is the error that I get when I use the User.objects.create(client, ** args)
TypeError at /setup_profile/
create() takes 1 positional argument but 2 were given
Request Method: POST
Request URL: http://127.0.0.1:8000/setup_profile/
Django Version: 1.8.6
Exception Type: TypeError
Exception Value:
create() takes 1 positional argument but 2 were given
Exception Location: C:\Users\OmarJandali\AppData\Local\Programs\Python\Python36\lib\site-packages\django\db\models\manager.py in manager_method, line 127
UPDATED
The client needs to be passed into the api call and it contains the following:
import os
from synapse_pay_rest import Client
args = {
'client_id': 'client_id_...6YiBl',
'client_secret': 'client_secret_...kC3IF',
'fingerprint': '...dd48b',
'ip_address': '127.0.0.1',
'development_mode':True,
'logging':False
}
client = Client(**args)
Also, here is a github link to the api samples that were created by the api developers.
https://github.com/synapsepay/SynapsePayRest-Python
the client has to be passed with the api call

Create method has only key word arguments. Rewrite your code to this:
User.objects.create(client=client, **args)
UPDATE
I just figured out that you are using third party package. So you need to import User class like this from synapse_pay_rest import User as SynapseUser and use SynapseUser in your code: SynapseUser.create(clients, **argss)

Azure module on webservice

I am trying to publish a machine learning model on Azure webservice using python. I am able to deploy the code successfully but when i try to call it through the URL, it's throwing me 'Azure' module doesn't exist. The code basically retrieves a TFIDF model from the container (blob) and use it to predict the new value. The error clearly says, Azure package is missing while trying to run on the webservice and I am not sure how to fix it. Here goes the code:
For deployment:
from azureml import services
from azure.storage.blob import BlobService
#services.publish('7c94eb2d9e4c01cbe7ce1063','f78QWNcOXHt9J+Qt1GMzgdEt+m3NXby9JL`npT7XX8ZAGdRZIX/NZ4lL2CkRkGQ==')
#services.types(res=unicode)
#services.returns(str)
def TechBot(res):
from azure.storage.blob import BlobService
from gensim.similarities import SparseMatrixSimilarity, MatrixSimilarity, Similarity
blob_service = BlobService(account_name='tfidf', account_key='RU4R/NIVPsPOoR0bgiJMtosHJMbK1+AVHG0sJCHT6jIdKPRz3cIMYTsrQ5BBD5SELKHUXgBHNmvsIlhEdqUCzw==')
blob_service.get_blob_to_path('techbot',"2014.csv","df")
df=pd.read_csv("df")
doct = res
To access the url I used the python code from
service.azureml.net
import urllib2
import json
import requests
data = {
"Inputs": {
"input1":
[
{
'res': "wifi wnable",
}
],
},
"GlobalParameters": {
}
}
body = str.encode(json.dumps(data))
#proxies = {"http":"http://%s" % proxy}
url = 'http://ussouthcentral.services.azureml.net/workspaces/7c94eb2de26a45399e4c01cbe7ce1063/services/11943e537e0741beb466cd91f738d073/execute?api-version=2.0&format=swagger'
api_key = '8fH9kp67pEt3C6XK9sXDLbyYl5cBNEwYg9VY92xvkxNd+cd2w46sF1ckC3jqrL/m8joV7o3rsTRUydkzRGDYig==' # Replace this with the API key for the web service
headers = {'Content-Type':'application/json', 'Authorization':('Bearer '+ api_key)}
#proxy_support = urllib2.ProxyHandler(proxies)
#opener = urllib2.build_opener(proxy_support, urllib2.HTTPHandler(debuglevel=1))
#urllib2.install_opener(opener)
req = urllib2.Request(url, body, headers)
try:
response = urllib2.urlopen(req, timeout=60)
result = response.read()
print(result)
except urllib2.HTTPError, error:
print("The request failed with status code: " + str(error.code))
# Print the headers - they include the requert ID and the timestamp, which are useful for debugging the failure
print(error.info())
print(json.loads(error.read()))
The string 'res' will be predicted at the end. As I said it runs perfectly fine if I run as it is in python by calling azure module, problem happens when I access the url.
Any help is appreciated, please let me know if you need more information (I only sohwcased half of my code)

I tried to reproduce the issue via POSTMAN, then I got the error information below as you said.
{
"error": {
"code": "ModuleExecutionError",
"message": "Module execution encountered an error.",
"details": [
{
"code": "85",
"target": "Execute Python Script RRS",
"message": "Error 0085: The following error occurred during script evaluation, please view the output log for more information:\r\n---------- Start of error message from Python interpreter ----------\r\nCaught exception while executing function: Traceback (most recent call last):\n File \"\\server\\InvokePy.py\", line 120, in executeScript\n outframe = mod.azureml_main(*inframes)\n File \"\\temp\\1280677032.py\", line 1094, in azureml_main\n File \"<ipython-input-15-bd03d199b8d9>\", line 6, in TechBot_2\nImportError: No module named azure\n\r\n\r\n---------- End of error message from Python interpreter ----------"
}
]
}
}
According to the error code 00085 & the information ImportError: No module named azure, I think the issue was caused by importing python moduleazure-storage. There was a similar SO thread Access Azure blog storage from within an Azure ML experiment which got the same issue, I think you can refer to its answer try to use HTTP protocol instead HTTPS in your code to resolve the issue as the code client = BlobService(STORAGE_ACCOUNT, STORAGE_KEY, protocol="http").
Hope it helps. Any concern & update, please feel free to let me know.
Update: Using HTTP protocol for BlobService
from azureml import services
from azure.storage.blob import BlobService
#services.publish('7c94eb2d9e4c01cbe7ce1063','f78QWNcOXHt9J+Qt1GMzgdEt+m3NXby9JL`npT7XX8ZAGdRZIX/NZ4lL2CkRkGQ==')
#services.types(res=unicode)
#services.returns(str)
def TechBot(res):
from azure.storage.blob import BlobService
from gensim.similarities import SparseMatrixSimilarity, MatrixSimilarity, Similarity
# Begin: Update code
# Using `HTTP` protocol for BlobService
blob_service = BlobService(account_name='tfidf',
account_key='RU4R/NIVPsPOoR0bgiJMtosHJMbK1+AVHG0sJCHT6jIdKPRz3cIMYTsrQ5BBD5SELKHUXgBHNmvsIlhEdqUCzw==',
protocol='http')
# End
blob_service.get_blob_to_path('techbot',"2014.csv","df")
df=pd.read_csv("df")
doct = res

error while using Django-websocket

I am developing a RESTFUL webservice using Django. On some occassion, we need to push the server object to the connected client without client polling.
We decided to use django-websocket 0.3.0.
I am writing the test cases and tried to connect to the server using nodejs ws client module
My View Function in Django is following:
from django.http import HttpResponse
from django_websocket import accept_websocket, require_websocket
from django.views.decorators.csrf import csrf_exempt
import json, sys, os, time, datetime
#csrf_exempt
#accept_websocket
def home(request) :
if not request.is_websocket():
return HttpResponse('new message')
else:
for message in request.websocket:
message = modify_message(message)
request.websocket.send(message)
request.websocket.close()
My Client Side code in js is like this:-
//Normal Get
var request = require('request');
request('http://127.0.0.1:8000',function(err,resp,flag){
console.log(resp.body);
});
//Opening a websocket
var WebSocket = require('ws');
var ws = new WebSocket('ws://127.0.0.1:8000/', {origin: 'http://127.0.0.1:8000'});
ws.on('open', function() {
console.log('connected');
ws.send(Date.now().toString(), {mask: true});
});
ws.on('close', function() {
console.log('disconnected');
});
ws.on('message', function(data, flags) {
console.log('Roundtrip time: ' + (Date.now() - parseInt(data)) + 'ms', flags);
setTimeout(function() {
ws.send(Date.now().toString(), {mask: true});
}, 500);
});
The first option gets the message as 'new message'
On the other side the second call throws the following error on the client side. On the server side, both commands pass through a 200OK
events.js:72
throw er; // Unhandled 'error' event
^
Error: unexpected server response (200)
at ClientRequest.<anonymous> (../ws/lib/WebSocket.js:603:17)
at ClientRequest.g (events.js:175:14)
at ClientRequest.EventEmitter.emit (events.js:95:17)
at HTTPParser.parserOnIncomingClient [as onIncoming] (http.js:1689:21)
at HTTPParser.parserOnHeadersComplete [as onHeadersComplete] (http.js:120:23)
at Socket.socketOnData [as ondata] (http.js:1584:20)
at TCP.onread (net.js:525:27)
On a side note if I log the request.is_websocket() on both calls it returns false meaning on the server side it never goes into the else part.
Please help me understand what mistake I am doing here
Thank you

Well,
I downloaded their entire code (not pip install) and run the supplied example chat program. Same error. The system sends a 400 response code for any ws:// call.
The git hub project page linked on the pypi site returns a 404 error. No way I can file a bug report. Emailed the developer and didn't get any response.
Probably something should have been broken on the new Django 1.5.2 version.
I consider that this is a dead project and hence moving to a complex but working solution like gevent or twisted.
thanks for your support!!!