I'm new to AWS Chalice and I'm running into obstacles during deployment--essentially, everything works fine when I run chalice local, I can go to the route I've defined and it will return the related JSON data. However, once deployed and I try accessing the same route, I get the HTTP 502 Bad Gateway error:
{
"message": "Internal server error"
}
Is there something I'm missing when I set up my AWS IAM roles? Or, more generally, is there some level of configuration with AWS beyond just setting a config file with the access key and secret in the .aws directory on my system? Below is the basic boilerplate code I'm running when I get the error:
from chalice import Chalice
import json, datetime, os
from chalicelib import config
app = Chalice(app_name='trading-bot')
app.debug = True
#app.route('/quote')
def quote():
return {"hello": "world"}
Please let me know if there's any more details I can provide; again I'm new to Chalice and AWS in general so it may be some simple settings I need to update on my profile.
Thanks!
Related
Hi I have an application that I would like to deploy on heroku. The question is how would I deploy a streamlit app with secrets.toml?
Currently the connection can be done locally via this
credentials = service_account.Credentials.from_service_account_info(
st.secrets["gcp_service_account"])
However when I deploy it to heroku, this doesn't seem to connect.
Please help.
On heroku I entered the gcp_service_account credentials as a config var (from the heroku dashboard go to 'Settings' --> 'Reveal Config Vars' as below:
Instead of st.secrets["<key>"], use os.environ["<key>"] in your python code as below:
gsheet_url = os.environ['private_gsheets_url']
For nested secrets like the gcp service account credentials, I first parse the json string as below:
parsed_credentials = json.loads(os.environ["gcp_service_account"])
credentials = service_account.Credentials.from_service_account_info(parsed_credentials,scopes=scopes)
Hope this helps.
I'm using the Firebase Admin Python SDK to read/write data to Firestore. I've created a service account with the necessary permissions and saved the credentials .json file in the source code (I know this isn't the most secure, but I want to get the thing running before fixing security issues). When testing the integration locally, it works flawlessly. But after deploying to GCP, where our service is hosted, calls to Firestore don't work properly and retry for a while before throwing 503 Deadline Exceeded errors. However, SSHing into a GKE pod and calling the SDK manually works without issues. It's just when the SDK is used in code flow that causes problems.
Our service runs in Google Kubernetes Engine in one project (call it Project A), but the Firestore database is in another project (call it project B). The service account that I'm trying to use is owned by Project B, so it should still be able to access the database even when it is being initialized from inside Project A.
Here's how I'm initiating the SDK:
from firebase_admin import get_app
from firebase_admin import initialize_app
from firebase_admin.credentials import Certificate
from firebase_admin.firestore import client
from google.api_core.exceptions import AlreadyExists
credentials = Certificate("/path/to/credentials.json")
try:
app = initialize_app(credential=credentials, name="app_name")
except ValueError:
app = get_app(name="app_name")
client = client(app=app)
Another wrinkle is that another part of our code is able to successfully use the same service account to produce Firebase Access Tokens. The successful code is:
import firebase_admin
from firebase_admin import auth as firebase_admin_auth
if "app_name" in firebase_admin._apps:
# Already initialized
app = firebase_admin.get_app(name="app_name")
else:
# Initialize
credentials = firebase_admin.credentials.Certificate("/path/to/credentials.json")
app = firebase_admin.initialize_app(credential=credentials, name="app_name")
firebase_token = firebase_admin_auth.create_custom_token(
uid="id-of-user",
developer_claims={"admin": is_admin, "site_slugs": read_write_site_slugs},
app=app,
)
Any help appreciated.
Turns out that the problem here was a conflict between gunicorn's gevents and the SDK's use of gRCP. Something related to websockets. I found the solution here. I added the following code to our Django app's settings:
import grpc.experimental.gevent as grpc_gevent
grpc_gevent.init_gevent()
I'm following a mooc for building quickly a website in flask.
I'm using Cloud9 but i'm unable to watch my preview on it, i get an :
"Unable to load http preview" :
the code is really simple, here the views.py code
from flask import Flask, render_template
app = Flask(__name__)
# Config options - Make sure you created a 'config.py' file.
app.config.from_object('config')
# To get one variable, tape app.config['MY_VARIABLE']
#app.route('/')
def index():
return "Hello world !"
if __name__ == "__main__":
app.run()
And the preview screen, is what I get when I execute
python views.py
Thank you in advance
you need to make FLASK_APP environment variable, and flask application is not running like python views.py but flask run. Quick start
# give an environment variable, give the absolute path or relative
# path to you flask app, in your case it is `views.py`
export FLASK_APP=views.py
#after this run flask application
flask run
I faced the same problem. There is no way we can preview http endpoints directly. Although in AWS documentation they have asked to follow certain steps, but those too wont work. Only way is to access it using instance public address and exposing required ports. Read here for this.
I built a simple python application to be run on the Google App Engine. Code:
import webapp2
from oauth2client.contrib.appengine import AppAssertionCredentials
from apiclient.discovery import build
from googleapiclient import discovery
from oauth2client.client import GoogleCredentials
class MainPage(webapp2.RequestHandler):
def get(self):
self.response.headers['Content-Type'] = 'text/plain'
self.response.write('BigQuery App')
credentials = AppAssertionCredentials(
'https://www.googleapis.com/auth/sqlservice.admin')
service = discovery.build('bigquery', 'v2', credentials=credentials)
projectId = '<Project-ID>'
query_request_body = {
"query": "SELECT a from Data.test LIMIT 10"
}
request = service.jobs().query(projectId=projectId, body=query_request_body)
response = request.execute()
self.response.write(response)
app = webapp2.WSGIApplication([
('/', MainPage),
], debug=True)
I am able to deploy this code locally (http://localhost:8080) and everything works correctly, however I get the following error 500 Server Error when I try to deploy it to GAE using:
appcfg.py -A <Project-Id> -V v1 update .
This is the error I get from the Error Report Console:
error: An error occured while connecting to the server: DNS lookup failed for URL:http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/https://www.googleapis.com/auth/sqlservice.admin/?recursive=True
I believe it is an auth issue and to make sure my service account was authorized I went through the gcloud authentification for service accounts and I also set the set environment variables from the SDK.
I have been trying to get around this for a while, any pointers are very appreciated. Thank you.
Also, I have been using Service Account Auth by following these docs: https://developers.google.com/identity/protocols/OAuth2ServiceAccount where it says that I shouldn't be able to run AppAsseritionCredenitals locally, which adds to my confusion because I actually can with no errors.
EDIT:
After reuploading and reauthorizing my service account I was able to connect to the server. However, the authorization error continues with this:
HttpError: <HttpError 403 when requesting https://www.googleapis.com/bigquery/v2/projects/sqlserver-1384/queries?alt=json returned "Insufficient Permission">
To fix the "error while connecting to the server", follow the instructions listed in this answer: https://stackoverflow.com/questions/31651973/default-credentials-in-google-app-engine-invalid-credentials-error#=
and then re-upload the app
Then, to fix the HttpError 403 when requesting ... returned "Insufficient Permission", you have to change the scope you were requesting. In my case I was requesting:
credentials = AppAssertionCredentials(
'https://www.googleapis.com/auth/sqlservice.admin')
however, the correct scope for Google BigQuery is: https://www.googleapis.com/auth/bigquery. Which looks like this:
credentials = AppAssertionCredentials(
'https://www.googleapis.com/auth/bigquery')
If you are using a different API, use whichever scope is outlined in the documentations.
I have a Flask app which looks like this:
from flask import Flask
import boto3
application = Flask(__name__)
#application.route("/")
def home():
return "Server successfully loaded"
#application.route("/app")
def frontend_from_aws():
s3 = boto3.resource("s3")
frontend = s3.Object(bucket_name = "my_bucket", key = "frontend.html")
return frontend.get()["Body"].read()
if __name__ == "__main__":
application.debug = True
application.run()
Everything works perfectly when I test locally, but when I deploy the app to Elastic Beanstalk the second endpoint gives an internal server error:
The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.
I didn't see anything alarming in the logs, though I'm not completely sure I'd know where to look. Any ideas?
Update: As a test, I moved frontend.html to a different bucket and modified the "/app" endpoint accordingly, and mysteriously it worked fine. So apparently this has something to do with the settings for the original bucket. Does anybody know what the right settings might be?
I found a quick and dirty solution: IAM policies (AWS console -> Identity & Access Management -> Policies). There was an existing policy called AmazonS3FullAccess, and after I attached aws-elasticbeanstalk-ec2-role to it my app was able to read and write to S3 at will. I'm guessing that more subtle access management can be achieved by creating custom roles and policies, but this was good enough for my purposes.
Did you set up your AWS credentials on your Elastc Beanstalk instance as they are on your local machine (i.e. in ~/.aws/credentials)?