Is it possible to upload an image from an external site (i.e. www.example.com/image.png) to the Google App Engine? This is not the same as uploading a file from a user submitted form.
If it's possible, any solutions?
I'm also using Google Cloud Storage, so if anybody has found a way to accomplish the same thing by uploading straight to Google Cloud Storage, please let me know.
-- UPDATE ---
I've followed the example from here - https://developers.google.com/appengine/docs/python/googlecloudstorageclient/getstarted and
replaced the text write "abcd" with this:
url="example.com/image.jpeg"
opener1 = urllib2.build_opener()
page1 = opener1.open(url)
write_retry_params = gcs.RetryParams(backoff_factor=1.1)
gcs_file = gcs.open(filename,
'w',
content_type='image/jpeg',
options={'x-goog-meta-foo': 'foo',
'x-goog-meta-bar': 'bar'},
retry_params=write_retry_params)
gcs_file.write(page1.read())
gcs_file.close()
The problem is - when I run this code, it tries to download the image to my computer (the client) instead of downloading to the gcs_file. I get the download file popup.... That's not what I'm trying to do. What am I doing wrong?
Yes, you are looking at using the google cloud storage client library. Specifically this:
https://developers.google.com/appengine/docs/python/googlecloudstorageclient/functions#open
Open it for writing then write the url fetched image.
Related
Im using Google Drive API for creating and opening html file. But the problem is that the document opens with the technical content (links to css, js files, html tags ...) like this
How to make it so that it would open correctly, in a user-friendly form?
part of my google-api code
def file_to_drive(import_file=None):
service = build('drive', 'v3', credentials=creds)
file_name = import_file
media_body = MediaFileUpload(file_name, resumable=True, mimetype='text/html')
body = {
'title': file_name,
'description': 'Uploaded By You'}
file = service.files().create(body=body, media_body=media_body, fields='id')
The google drive API is a file store api. It allows you to upload and download files. It does not have the ability to open files. You could share a link to the file with someone that has access then when they click on the link it will open for them in the google drive web application.
The only api able to open files for editing would be the Google docs api which gives you limited ability to open google doc files. that however would require that you covert your html file to a google docs format. Even if this was an option you would need to create your own "user friendly form" Google apis return data as json and not user friendly options thats not what APIs are for.
I wrote a python program which connected various platforms' API for file downloading purposes here. The program is currently running on my local machine (laptop) without a problem (all downloaded files saved to my local drive of course).
Here is my real question, without Google Compute Engine, is it possible to deploy the very same python program using Google App Engine? If yes, how could I save my files (via API calls) to Google Cloud Storage here?
Thanks.
Is this a Web App? If so you deploy it using GOOGLE APP ENGINE standard or flexible.
In order to send files to Cloud Storage, try the example in the python-docs-samples repo (folder appengine/flexible/storage/):
# [START upload]
#app.route('/upload', methods=['POST'])
def upload():
"""Process the uploaded file and upload it to Google Cloud Storage."""
uploaded_file = request.files.get('file')
if not uploaded_file:
return 'No file uploaded.', 400
# Create a Cloud Storage client.
gcs = storage.Client()
# Get the bucket that the file will be uploaded to.
bucket = gcs.get_bucket(CLOUD_STORAGE_BUCKET)
# Create a new blob and upload the file's content.
blob = bucket.blob(uploaded_file.filename)
blob.upload_from_string(
uploaded_file.read(),
content_type=uploaded_file.content_type
)
# The public URL can be used to directly access the uploaded file via HTTP.
return blob.public_url
# [END upload]
I've recently deployed my python GAE app from the development server and my image upload function stopped working properly...
After a bit of testing, it seems that the get_uploads function from blobstore is returning an empty list and hence I get an out of index error from the upload handler (also tried the get_file_infos function and had the same result)
However, when I check the GCS browser, the file is properly uploaded so my problem seems to be that I can't find a way to extract the image link from the post to Upload Handler
Anybody have clues as to why this is happening? and if there's a way around this?
(The form uses a post method with multipart/form-data so hopefully that isn't an issue)
Here's the function I'm calling to post to the upload handler:
upload_url = blobstore.create_upload_url('/upload', gs_bucket_name='BUCKET')
result = urlfetch.fetch(url= upload_url,
payload=self.request.body,
method=urlfetch.POST,
headers=self.request.headers)
And here's the code for the upload handler:
class UploadHandler(blobstore_handlers.BlobstoreUploadHandler):
def post(self):
upload_files = self.get_uploads('file')
blob_info = upload_files[0]
self.response.write(str(blob_info.key()))
What do you try to do?
It looks like you try to post a received body to GCS. Why not write it using the Google Cloud Storage Client Library.
with gcs.open(gcs_filename, 'w', content_type, options={b'x-goog-acl': b'public-read'}) as f:
f.write(blob)
I currently use the following code for allowing my users to upload files;
uploadurl = blobstore.create_upload_url('/process?session=' + session, gs_bucket_name='mybucketname')
and I can serve images like this;
imgurl = get_serving_url(blob_key, size=1600, crop=False, secure_url=True)
After content is uploaded using the method in the first code snipped, the blob key contains encoded_gs_file: and that's how it knows to serve it from Google Cloud Service and not the blobstore as standard.
However, I'm unsure how I'd serve any other kind of file (for example .pdf, or .rtf). I do not want the content to be displayed in the browser, but rather sent to the client as a download (so they get the save file dialog and choose a location on their computer to save it).
How would I go about doing this? Thanks.
Using a google serving_url works only for images.
To serve a pdf from the blobstore you can use:
class DynServe(blobstore_handlers.BlobstoreDownloadHandler):
def get(self, resource):
(blob_key, extension) = resource.rpartition('.')[::2]
blob_info = blobstore.BlobInfo.get(blob_key)
if not blob_info:
logging.error('Blob NOT FOUND %s' % resource)
self.abort(404)
self.response.headers[b'Content-Type'] = mimetypes.guess_type(blob_info.filename)
self.send_blob(blob_key, save_as=blob_info.filename)
The webapp2 route for this handler looks like:
webapp2.Route(r'/dynserve/<resource:(.*)>', handler=DynServe)
To serve:
PDF download
I'm going to answer my own question based on the answer from #voscausa
This is what my handler looks like (inside a file named view.py);
class DynServe(blobstore_handlers.BlobstoreDownloadHandler):
def get(self, resource):
blob_key = resource
if not blobstore.get(blob_key):
logging.warning('Blob NOT FOUND %s' % resource)
self.abort(404)
return
else:
blob_info = blobstore.BlobInfo.get(blob_key)
self.send_blob(blob_key, save_as=blob_info.filename)
We need this in app.yaml;
- url: /download/.*
script: view.app
secure: always
secure: always is optional, but I always use it while handling user data.
Put this at the bottom of view.py;
app = webapp.WSGIApplication([('/download/([^/]+)?', DynServe),
], debug=False)
Now visit /download/BLOB_KEY_HERE. (you can check the datastore for your blob key)
That's a fully working example which works with both the standard blob store AND Google Cloud Service.
NOTE: All blob keys which are part of the GCS will start with encoded_gs_file: and the ones which don't are in the standard blobstore; app engine automatically uses this to determine where to locate the file
I use python Appengine. I'm trying to create a link on a webpage, which a user can click to download a csv file. How can I do this?
I've looked at csv module, but it seems to want to open a file on the server, but appengine doesn't allow that.
I've looked at remote_api, but it seems that its only for uploading or downloading using app config, and from account owner's terminal.
Any help thanks.
Pass a StringIO object as the first parameter to csv.writer; then set the content-type and content-disposition on the response appropriately (probably "text/csv" and "attachment", respectively) and send the StringIO as the content.
I used this code:
self.response.headers['Content-Type'] = 'application/csv'
writer = csv.writer(self.response.out)
writer.writerow(['foo','foo,bar', 'bar'])
Put it in your handler's get method. When user requests it, user's browser will download the list content automatically.
Got from: generating a CSV file online on Google App Engine