Anyone using Django herald for sending notifications?
I've been struggling for days to make it work but the lack of documentation and silent failures made it impossible to debug the issues. It seems that the mails are not being sent if I include an attachment in it.
from herald.base import EmailNotification
def sendMail():
SendThisMail(user, my_modal).send(user=my_user) # creates an error on this line as the file object is closed and inaccessible.
#registry.register_decorator()
class SendThisMail(SomeBaseClass, EmailNotification):
def __init__(self, user, my_modal: models.MyModal):
super().__init__(user, my_modal)
self.subject = "abc"
file = open('.staticfiles/assets/some.pdf', 'rb')
self.attachments = [('attachment_1', File(file))]
self.context = {
**self.context,
'subject': self.subject,
'attachment': self.attachments,
}
self.to_emails = [user.email]
What's wrong with it?
From the project docs:
Each attachment in the list can be one of the following:
A tuple which consists of the filename, the raw attachment data, and the mimetype. It is up to you to get the attachment data
So the relevant parts of your code code should be something like:
data = open('.staticfiles/assets/some.pdf', 'rb').read()
self.attachments = [('attachment_1', data, 'application/pdf')]
I started learning SalesForce and developing apps using django.
I need assistance with uploading a file to salesforce, For that I read simple-salesforce and this that help to upload file using rest and SOAP api.
My question is how do I upload one or more files using simple-salesforce?
Here is the code block I use for uploading files.
def load_attachments(sf, new_attachments):
'''
Method to attach the Template from the Parent Case to each of the children.
#param: new_attachments the dictionary of child cases to the file name of the template
'''
url = "https://" + sf.get_forced_url() + ".my.salesforce.com/services/data/v29.0/sobjects/Attachment/"
bearer = "Bearer " + sf.get_session_id()
header = {'Content-Type': 'application/json', 'Authorization': bearer}
for each in new_attachments:
body = ""
long_name = str(new_attachments[each]).split(sep="\\")
short_name = long_name[len(long_name) - 1]
with open(new_attachments[each], "rb") as upload:
body = base64.b64encode(upload.read())
data = json.dumps({
'ParentId': each,
'Name': short_name,
'body': body
})
response = requests.post(url, headers=header, data=data)
print(response.text)
Basically, to send the file, you need to use the requests module and submit the file via a post transaction. The post transaction requires the URL to which the request is sent, the header information, and the data.
Here, sf is the instance of returned by the simple-salesforce initialization. Since my instance uses custom domains, I had to create my own function in simple-salesforce to handle that; I call it get_forced_url(). Note: The URL is may be different for you depending on which version you are using [the v29.0 portion may change].
Then I set up my bearer and header.
The next thing is a loop that submits a new attachment for each attachment in a map from Parent ID to the File I wish to upload. This is important to note, attachments must have a Parent Object so you need to know the ParentId. For each attachment, I blank out the body, create a long and short name for the attachment. Then the important part. On attachments, the actual data of the file is stored as a base-64 binary array. So the file must be opened as binary, hence the "rb" and then encoded to base-64.
Once the file has been parsed to base-64 binary, I build my json string where ParentId is the object ID of the parent object, the Name is the short name, and the body is the base-64 encoded string of data.
Then the file is submitted to the URL with the headers and data. Then I print the response so I could watch it happening.
To upload files, you only need simple-salesforce
Complete example, including creating Account, Contact and Case. Then attaching the file to Case.
#Create Account, Contact and Case
AccountID = sf.Account.create({'Name':'Test12','Phone':'987654321'})["id"]
ContactID = sf.Contact.create({'LastName':'Smith2','Email':'example3#example.com'})["id"]
CaseID = sf.Case.create({'AccountId':AccountID,'ContactId':ContactID,'Description':'Test4321','Subject':'Test4321'})
#Convert image to Base64
import json, base64
with open('test1.png', mode='rb') as file:
img = file.read()
image = base64.encodebytes(img).decode('utf-8')
#The simple example
sf.Attachment.create({'ParentId': CaseID["id"],'Name':'TestFile1','body': image,'ContentType':'image/png'})
And how to change the 'one-file' example to multiple files
sf.bulk.Attachment.insert([
{'ParentId': CaseID["id"],'Name':'TestFile2','body': image,'ContentType':'image/png'},
{'ParentId': CaseID["id"],'Name':'TestFile3','body': image,'ContentType':'image/png'},
{'ParentId': CaseID["id"],'Name':'TestFile4','body': image,'ContentType':'image/png'},
{'ParentId': CaseID["id"],'Name':'TestFile5','body': image,'ContentType':'image/png'},
{'ParentId': CaseID["id"],'Name':'TestFile6','body': image,'ContentType':'image/png'},
{'ParentId': CaseID["id"],'Name':'TestFile7','body': image,'ContentType':'image/png'},
{'ParentId': CaseID["id"],'Name':'TestFile8','body': image,'ContentType':'image/png'},
{'ParentId': CaseID["id"],'Name':'TestFile9','body': image,'ContentType':'image/png'}],batch_size=1000,use_serial=True)
(you know how to fix the rest)
I am trying to upload a file to JIRA via its REST API using the python lib found here: jira python documentation
It seems pretty straight forward I wrote a method that allows me to pass an issue and then it attaches a filename. and one that lets me retrieve an issue from JIRA.
from jira.client import JIRA
class JIRAReport (object):
def attach(self,issue):
print 'Attaching... '
attachment = self.jira.add_attachment(issue, attachment=self.reportpath, filename='Report.xlsx')
print 'Success!'
def getissue(self):
if not self.issue == None:
return self.jira.issue(self.issue)
return None
then in my main script I am getting the issue and attaching the file to an issue I retrieved from JIRA
report = JiraReport()
report.issue = 'ProjectKey-1'
report.reportpath = '../report_upload/tmp/' + filename
issue = report.getissue()
if not issue == None:
report.attach(issue)
else:
print "No Issue with Key Found"
I am able to get the issue/create issues if needed but when using the self.jira.add_attachment() method I am getting 405 Method Not Allowed.
The file exists and is able to be opened.
Here is the add_attachment() method from the source code:
def add_attachment(self, issue, attachment, filename=None):
"""
Attach an attachment to an issue and returns a Resource for it.
The client will *not* attempt to open or validate the attachment; it expects a file-like object to be ready
for its use. The user is still responsible for tidying up (e.g., closing the file, killing the socket, etc.)
:param issue: the issue to attach the attachment to
:param attachment: file-like object to attach to the issue, also works if it is a string with the filename.
:param filename: optional name for the attached file. If omitted, the file object's ``name`` attribute
is used. If you aquired the file-like object by any other method than ``open()``, make sure
that a name is specified in one way or the other.
:rtype: an Attachment Resource
"""
if isinstance(attachment, string_types):
attachment = open(attachment, "rb")
# TODO: Support attaching multiple files at once?
url = self._get_url('issue/' + str(issue) + '/attachments')
fname = filename
if not fname:
fname = os.path.basename(attachment.name)
content_type = mimetypes.guess_type(fname)[0]
if not content_type:
content_type = 'application/octet-stream'
files = {
'file': (fname, attachment, content_type)
}
r = self._session.post(url, files=files, headers=self._options['headers'])
raise_on_error(r)
attachment = Attachment(self._options, self._session, json.loads(r.text)[0])
return attachment
It is mentioned in documentation that as a argument they expect file-like object.
Try to do something like :
file_obj = open('test.txt','rb')
jira.add_attachment(issue,file_obj,'test.txt')
file_obj.close()
Check that the URL that you are specifying for JIRA (if using the on-demand service) is https://instance.atlassian.net.
I just hit this as well, and it sends a POST request to http://instance.atlassian.net and gets redirected to https://instance.atlassian.net, but the client sends a GET request to the redirected address (see: https://softwareengineering.stackexchange.com/questions/99894/why-doesnt-http-have-post-redirect for more information)
I'd like to download several files with GAE Python code.
My current code is like below
import webapp2, urllib
url1 = 'http://dummy/sample1.jpg'
url2 = 'http://dummy/sample2.jpg'
class DownloadHandler(webapp2.RequestHandler):
def get(self):
#image1
self.response.headers['Content-Type'] = 'application/octet-stream'
self.response.headers['Content-Disposition'] = 'attachment; filename="' + 'sample1.jpg' + '"'
f = urllib.urlopen(url1)
data = f.read()
self.response.out.write(data)
#image2
self.response.headers['Content-Type'] = 'application/octet-stream'
self.response.headers['Content-Disposition'] = 'attachment; filename="' + 'sample2.jpg' + '"'
f = urllib.urlopen(url2)
data = f.read()
self.response.out.write(data)
app = webapp2.WSGIApplication([('/.*', DownloadHandler)],
debug=True)
I expected to occur download dialogue twice with this code, but actually occurred once, and only sample2.jpg was downloaded.
How can you handle download dialogue several times?
I'd actually like to realize some other functions adding above as well.
To display progressing message on the browser such as
sample1.jpg was downloaded
sample2.jpg was downloaded
sample3.jpg was downloaded ...
And redirect to the other page after downloading files.
When I wrote a code such as
self.redirect('/otherpage')
after
self.response.out.write(data)
Only redirect had happened and didn't occur download procedure.
Would you give me any ideas to solve it please.
I'm using python2.7
Two things.
You cannot write two files in one response that has a Content-Type of application/octet-stream. To stuff multiple files in in the response, you would have to encode your response with multipart/form-data or multipart/mixed and hope that the client would understand that and parse it and show two download dialogues
Once you've already called self.response.out.write(…), you shouldn't be setting any more headers.
To me it seems that the most foolproof option would be to serve an HTML file that contains something like:
<script>
window.open('/path/to/file/1.jpg');
window.open('/path/to/file/1.jpg');
</script>
… and then handle those paths using different handlers.
Another option would be to zip the two files and serve the zipfile to the client, though it may or may not be preferable in your case.
I reached the goal what I wanted to do.
As user interaction, generating html sources include below
<script type="text/javascript">
window.open("/download?url=http://dummy/sample1.jpg")
window.open("/download?url=http://dummy/sample2.jpg")
</script>
then created new windows are handled with this code.
class DownloadHandler(webapp2.RequestHandler):
def get(self):
url = self.request.get('url')
filename = str(os.path.basename(url))
self.response.headers['Content-Type'] ='application/octet-stream'
self.response.headers['Content-Disposition'] = 'attachment; filename="%s"' % (filename)
data = urllib.urlopen(url).read()
self.response.out.write(data)
app = webapp2.WSGIApplication([('/download', DownloadHandler)], debug=True)
Thank you, Attila.
Can you produce a Python example of how to download a Google Sheets spreadsheet given its key and worksheet ID (gid)? I can't.
I've scoured versions 1, 2 and 3 of the API. I'm having no luck, I can't figure out their compilcated ATOM-like feeds API, the gdata.docs.service.DocsService._DownloadFile private method says that I'm unauthorized, and I don't want to write an entire Google Login authentication system myself. I'm about to stab myself in the face due to frustration.
I have a few spreadsheets and I want to access them like so:
username = 'mygooglelogin#gmail.com'
password = getpass.getpass()
def get_spreadsheet(key, gid=0):
... (help!) ...
for row in get_spreadsheet('5a3c7f7dcee4b4f'):
cell1, cell2, cell3 = row
...
Please save my face.
Update 1: I've tried the following, but no combination of Download() or Export() seems to work. (Docs for DocsService here)
import gdata.docs.service
import getpass
import os
import tempfile
import csv
def get_csv(file_path):
return csv.reader(file(file_path).readlines())
def get_spreadsheet(key, gid=0):
gd_client = gdata.docs.service.DocsService()
gd_client.email = 'xxxxxxxxx#gmail.com'
gd_client.password = getpass.getpass()
gd_client.ssl = False
gd_client.source = "My Fancy Spreadsheet Downloader"
gd_client.ProgrammaticLogin()
file_path = tempfile.mktemp(suffix='.csv')
uri = 'http://docs.google.com/feeds/documents/private/full/%s' % key
try:
entry = gd_client.GetDocumentListEntry(uri)
# XXXX - The following dies with RequestError "Unauthorized"
gd_client.Download(entry, file_path)
return get_csv(file_path)
finally:
try:
os.remove(file_path)
except OSError:
pass
The https://github.com/burnash/gspread library is a newer, simpler way to interact with Google Spreadsheets, rather than the old answers to this that suggest the gdata library which is not only too low-level, but is also overly-complicated.
You will also need to create and download (in JSON format) a Service Account key: https://console.developers.google.com/apis/credentials/serviceaccountkey
Here's an example of how to use it:
import csv
import gspread
from oauth2client.service_account import ServiceAccountCredentials
scope = ['https://spreadsheets.google.com/feeds']
credentials = ServiceAccountCredentials.from_json_keyfile_name('credentials.json', scope)
docid = "0zjVQXjJixf-SdGpLKnJtcmQhNjVUTk1hNTRpc0x5b9c"
client = gspread.authorize(credentials)
spreadsheet = client.open_by_key(docid)
for i, worksheet in enumerate(spreadsheet.worksheets()):
filename = docid + '-worksheet' + str(i) + '.csv'
with open(filename, 'wb') as f:
writer = csv.writer(f)
writer.writerows(worksheet.get_all_values())
In case anyone comes across this looking for a quick fix, here's another (currently) working solution that doesn't rely on the gdata client library:
#!/usr/bin/python
import re, urllib, urllib2
class Spreadsheet(object):
def __init__(self, key):
super(Spreadsheet, self).__init__()
self.key = key
class Client(object):
def __init__(self, email, password):
super(Client, self).__init__()
self.email = email
self.password = password
def _get_auth_token(self, email, password, source, service):
url = "https://www.google.com/accounts/ClientLogin"
params = {
"Email": email, "Passwd": password,
"service": service,
"accountType": "HOSTED_OR_GOOGLE",
"source": source
}
req = urllib2.Request(url, urllib.urlencode(params))
return re.findall(r"Auth=(.*)", urllib2.urlopen(req).read())[0]
def get_auth_token(self):
source = type(self).__name__
return self._get_auth_token(self.email, self.password, source, service="wise")
def download(self, spreadsheet, gid=0, format="csv"):
url_format = "https://spreadsheets.google.com/feeds/download/spreadsheets/Export?key=%s&exportFormat=%s&gid=%i"
headers = {
"Authorization": "GoogleLogin auth=" + self.get_auth_token(),
"GData-Version": "3.0"
}
req = urllib2.Request(url_format % (spreadsheet.key, format, gid), headers=headers)
return urllib2.urlopen(req)
if __name__ == "__main__":
import getpass
import csv
email = "" # (your email here)
password = getpass.getpass()
spreadsheet_id = "" # (spreadsheet id here)
# Create client and spreadsheet objects
gs = Client(email, password)
ss = Spreadsheet(spreadsheet_id)
# Request a file-like object containing the spreadsheet's contents
csv_file = gs.download(ss)
# Parse as CSV and print the rows
for row in csv.reader(csv_file):
print ", ".join(row)
You might try using the AuthSub method described in the Exporting Spreadsheets section of the documentation.
Get a separate login token for the spreadsheets service and substitue that for the export. Adding this to the get_spreadsheet code worked for me:
import gdata.spreadsheet.service
def get_spreadsheet(key, gid=0):
# ...
spreadsheets_client = gdata.spreadsheet.service.SpreadsheetsService()
spreadsheets_client.email = gd_client.email
spreadsheets_client.password = gd_client.password
spreadsheets_client.source = "My Fancy Spreadsheet Downloader"
spreadsheets_client.ProgrammaticLogin()
# ...
entry = gd_client.GetDocumentListEntry(uri)
docs_auth_token = gd_client.GetClientLoginToken()
gd_client.SetClientLoginToken(spreadsheets_client.GetClientLoginToken())
gd_client.Export(entry, file_path)
gd_client.SetClientLoginToken(docs_auth_token) # reset the DocList auth token
Notice I also used Export, as Download seems to give only PDF files.
(Jul 2016) All other answers are pretty much outdated or will be, either because they use GData ("Google Data") Protocol, ClientLogin, or AuthSub, all of which have been deprecated. The same is true for all code or libraries that use the Google Sheets API v3 or older.
Modern Google API access occurs using API keys (for accessing public data), OAuth2 client IDs (for accessing data owned by users), or service accounts (for accessing data owned by applications/in the cloud) primarily with the Google Cloud client libraries for GCP APIs and Google APIs Client Libraries for non-GCP APIs. For this task, it would be the latter for Python.
To make it happen your code needs authorized access to the Google Drive API, perhaps to query for specific Sheets to download, and then to perform the actual export(s). Since this is likely a common operation, I wrote a blogpost sharing a code snippet that does this for you. If you wish to pursue this even more, I've got another pair of posts along with a video that outlines how to upload files to and download files from Google Drive.
Note that there is also a Google Sheets API v4, but it's primarily for spreadsheet-oriented operations, i.e., inserting data, reading spreadsheet rows, cell formatting, creating charts, adding pivot tables, etc., not file-based request like exporting where the Drive API is the correct one to use.
I wrote a blog post that demos exporting a Google Sheet as CSV from Drive. The core part of the script:
# setup
FILENAME = 'inventory'
SRC_MIMETYPE = 'application/vnd.google-apps.spreadsheet'
DST_MIMETYPE = 'text/csv'
DRIVE = discovery.build('drive', 'v3', http=creds.authorize(Http()))
# query for file to export
files = DRIVE.files().list(
q='name="%s" and mimeType="%s"' % (FILENAME, SRC_MIMETYPE), orderBy='modifiedTime desc,name').execute().get('files', [])
# export 1st match (if found)
if files:
fn = '%s.csv' % os.path.splitext(files[0]['name'].replace(' ', '_'))[0]
print('Exporting "%s" as "%s"... ' % (files[0]['name'], fn), end='')
data = DRIVE.files().export(fileId=files[0]['id'], mimeType=DST_MIMETYPE).execute()
if data:
with open(fn, 'wb') as f:
f.write(data)
print('DONE')
To learn more about using Google Sheets with Python, see my answer for a similar question. You can also download a Sheet in XLSX and other formats supported by Drive.
If you're completely new to Google APIs, then you need to take a further step back and review these videos first:
How to use Google APIs & create API projects -- the UI has changed but the concepts are still the same
Walkthrough of authorization boilerplate code (Python) -- you can use any supported language to access Google APIs; if you don't do Python, use it as pseudocode to help get you started
Listing your files in Google Drive and code deep dive post
If you already have experience with Google Workspace (formerly G Suite, Google Apps, Google "Docs") APIs and want to see more videos on using both APIs:
Sheets API video library
Drive API video library
Google Workspace (G Suite) Dev Show video series I produced
This no longer works as of gdata 2.0.1.4:
gd_client.SetClientLoginToken(spreadsheets_client.GetClientLoginToken())
Instead, you have to do:
gd_client.SetClientLoginToken(gdata.gauth.ClientLoginToken(spreadsheets_client.GetClientLoginToken()))
I wrote pygsheets as an alternative to gspread, but using google api v4. It has an export method to export spreadsheet.
import pygsheets
gc = pygsheets.authorize()
# Open spreadsheet and then workseet
sh = gc.open('my new ssheet')
wks = sh.sheet1
#export as csv
wks.export(pygsheets.ExportType.CSV)
The following code works in my case (Ubuntu 10.4, python 2.6.5 gdata 2.0.14)
import gdata.docs.service
import gdata.spreadsheet.service
gd_client = gdata.docs.service.DocsService()
gd_client.ClientLogin(email,password)
spreadsheets_client = gdata.spreadsheet.service.SpreadsheetsService()
spreadsheets_client.ClientLogin(email,password)
#...
file_path = file_path.strip()+".xls"
docs_token = gd_client.auth_token
gd_client.SetClientLoginToken(spreadsheets_client.GetClientLoginToken())
gd_client.Export(entry, file_path)
gd_client.auth_token = docs_token
I've simplified #Cameron's answer even further, by removing the unnecessary object orientation. This makes the code smaller and easier to understand. I also edited the url, which might work better.
#!/usr/bin/python
import re, urllib, urllib2
def get_auth_token(email, password):
url = "https://www.google.com/accounts/ClientLogin"
params = {
"Email": email, "Passwd": password,
"service": 'wise',
"accountType": "HOSTED_OR_GOOGLE",
"source": 'Client'
}
req = urllib2.Request(url, urllib.urlencode(params))
return re.findall(r"Auth=(.*)", urllib2.urlopen(req).read())[0]
def download(spreadsheet, worksheet, email, password, format="csv"):
url_format = 'https://docs.google.com/spreadsheets/d/%s/export?exportFormat=%s#gid=%s'
headers = {
"Authorization": "GoogleLogin auth=" + get_auth_token(email, password),
"GData-Version": "3.0"
}
req = urllib2.Request(url_format % (spreadsheet, format, worksheet), headers=headers)
return urllib2.urlopen(req)
if __name__ == "__main__":
import getpass
import csv
spreadsheet_id = "" # (spreadsheet id here)
worksheet_id = '' # (gid here)
email = "" # (your email here)
password = getpass.getpass()
# Request a file-like object containing the spreadsheet's contents
csv_file = download(spreadsheet_id, worksheet_id, email, password)
# Parse as CSV and print the rows
for row in csv.reader(csv_file):
print ", ".join(row)
I'm using this:
curl 'https://docs.google.com/spreadsheets/d/1-lqLuYJyHAKix-T8NR8wV8ZUUbVOJrZTysccid2-ycs/gviz/tq?tqx=out:csv' on a sheet that is set to publicly readable.
So you would need a python version of curl, if you can work with public sheets.
If you have a sheet with some tabs you don't want to reveal, create a new sheet, and import the ranges you want to publish into tabs on it.
Downloading a spreadsheet from google doc is pretty simple using sheets.
You can follow the detailed documentation on
https://pypi.org/project/gsheets/
or follow the below-given steps. I recommend reading through the documentation for better coverage.
pip install gsheets
Log in to the Google Developers Console with the Google account whose spreadsheets you want to access. Create (or select) a project and enable the Drive API and Sheets API (under Google Apps APIs).
Go to the Credentials for your project and create New credentials > OAuth client ID > of type Other. In the list of your OAuth 2.0 client IDs click Download JSON for the Client ID you just created. Save the file as client_secrets.json in your home directory (user directory).
Use the following code snippet.
from gsheets import Sheets
sheets = Sheets.from_files('client_secret.json')
print(sheets) # will ensure authenticate connection
s = sheets.get("{SPREADSHEET_URL}")
print(s) # will ensure your file is accessible
s.sheets[1].to_csv('Spam.csv', encoding='utf-8', dialect='excel') # will download the file as csv
This isn't a complete answer, but Andreas Kahler wrote up an interesting CMS solution using Google Docs + Google App Engline + Python. Not having any experience in the area, I cannot see exactly what portion of the code may be of use to you, but check it out. I know it interfaces with a Google Docs account and plays with files, so I have a feeling you'll recognize what's going on. It should at least point you in the right direction.
Google AppEngine + Google Docs + Some Python = Simple CMS
Gspread is indeed a big improvement over GoogleCL and Gdata (both of which I've used and thankfully phased out in favor of Gspread). I think that this code is even quicker than the earlier answer to get the contents of the sheet:
username = 'sdfsdfsds#gmail.com'
password = 'sdfsdfsadfsdw'
sheetname = "Sheety Sheet"
client = gspread.login(username, password)
spreadsheet = client.open(sheetname)
worksheet = spreadsheet.sheet1
contents = []
for rows in worksheet.get_all_values():
contents.append(rows)
(Mar 2019, Python 3) My data is usually not sensitive and I use usually table format similar to CSV.
In such case, one can simply publish to the web the sheet and than use it as a CSV file on a server.
(One publishes it using File -> Publish to the web ... -> Sheet 1 -> Comma separated values (.csv) -> Publish).
import csv
import io
import requests
url = "https://docs.google.com/spreadsheets/d/e/<GOOGLE_ID>/pub?gid=0&single=true&output=csv" # you can get the whole link in the 'Publish to the web' dialog
r = requests.get(url)
r.encoding = 'utf-8'
csvio = io.StringIO(r.text, newline="")
data = []
for row in csv.DictReader(csvio):
data.append(row)