I using pymysql for connection to AWS Mysql Aurora DB. I created lambda function, which should return the data as json for use in the js framework.
def lambda_handler(event, context):
responses = []
try:
conn = pymysql.connect(rds_host, user=name, passwd=password, db=db_name, connect_timeout=5)
except pymysql.MySQLError as e:
...
with conn.cursor() as cur:
cur.execute('SELECT * FROM Plans')
conn.commit()
for row in cur:
resp = {
"id": row[0],
"name": row[1],
...
...
"type": row[14],
}
responses.append(resp)
return responses
This code return list of dict.
I try to use json module result = json.dumps(responses), but this code return str
How can I get json?
The json module won't return you a JSON object.
json.loads()
will return a Python object using this formatting table.
json.dumps()
will return you a JSON formatted str using this formatting table.
You will be able to create a .json file using following code :
with open ('path/to/myfile.json', 'w') as myfile:
json.dump(responses, my_file)
Related
I have the following code to update a url in a table
from mysql.connector import Error
import mysql.connector
def update_url(self, data):
target_url = data['target_url']
try:
cnx = mysql.connector.connect(
host=os.getenv('mysql_host'),
port=os.getenv('mysql_port'),
user=os.getenv('mysql_user'),
password=os.getenv('mysql_password'),
database=os.getenv('mysql_database')
)
# Create Cursor
cur = cnx.cursor(dictionary=True)
# update target_url
cur.execute('''UPDATE client_url_info
SET target_url=%s
WHERE id=%s''', (data['client_id'], target_url))
cnx.commit()
change_url = {
'status': True,
'msg': 'Target URL has been successfully updated'
}
return change_url
except mysql.connector.Error as err:
change_error_url = {
'status': 'MYSQL_ERROR',
'msg': "Failed in MySQL: {}".format(err)
}
return change_error_url
finally:
if (cnx.is_connected()):
cnx.close()
for some reason I can add data and delete data, but when I try to use update it does not work.
based on other questions, the issue could be cause by the missing of commit() which in my case is included in the code.
I don´t know if I´m not looking right, the code seems to be fine and actually returns the successful msg after execution.
Table info:
Is there any other configuration I´m missing?
thanks in advance
I have a simple AWS Lambda function connected to a mysql rds. When I update a field in my apps UI it updates it in the database when viewed from the MySQL workbench but when using the Lambda function it returns the same value until I redeploy the function and then it gives me the new correct value
"""Search Function for Lambda"""
from urllib.parse import unquote
import json
import pymysql
# Configuration Values
##CONFIG VALUES REMOVED
# Connection
connection = pymysql.connect(ENDPOINT, user=USERNAME,
passwd=PASSWORD, db=DATABASE_NAME)
def get(event, context):
"""Takes searches key from event and searches the database on that key"""
print(context)
cursor = connection.cursor()
search_key = unquote(event['pathParameters']['search_key'])
cmd = ('SELECT * from LCI_Data WHERE Description Like "{}"'.format("%"+search_key+"%"))
cursor.execute(cmd)
result = [dict((cursor.description[i][0], value)
for i, value in enumerate(row)) for row in cursor.fetchall()]
response = {
"statusCode": 200,
"body": json.dumps(result),
"headers": {
"Access-Control-Allow-Origin": "*",
"Access-Control-Allow-Credentials": "true"
}
}
return response
Moving
connection = pymysql.connect(ENDPOINT, user=USERNAME,
passwd=PASSWORD, db=DATABASE_NAME)
into the get() function solved my issues.
Am a beginner to AWS services and python, i used below code in lambda to connect
RDS and invoke this in API gateway
After Successful execution of below code it is returning null.
#!/usr/bin/python
import sys
import logging
import pymysql
import json
rds_host="host"
name="name"
password="password"
db_name="DB"
port = 3306
def save_events(event):
"""
This function fetches content from mysql RDS instance
"""
result = []
conn = pymysql.connect(rds_host, user=name, passwd=password,
db=db_name,connect_timeout=30)
with conn.cursor() as cur:
cur.execute("SELECT * FROM exercise WHERE bid = '1'")
for row in cur:
result.append(list(row))
print ("Data from RDS...")
print (result)
cur.close()
print(json.dumps({'bodyParts':result}))
def lambda_handler(event, context):
save_events(event)
As pointed out in a comment by #John Gordon, you need to return something from your lambda_handler function.
It should be something like:
def lambda_handler(event, context):
save_events(event)
return {
"statusCode": 200,
"result": "Here is my result"
}
Additionally, I don't see any return statement from save_events either.
As the title indicates, how does one use python to elegantly access an API and parse and save the JSON contents onto a relational database (MYSQL) for later access?
Here, I saved the data onto a pandas object. But how do I create a mysql database, save the json contents onto it, and access the contents for later use?
# Libraries
import json, requests
import pandas as pd
from pandas.io.json import json_normalize
# Set URL
url = 'https://api-v2.themuse.com/jobs'
# For loop to
for i in range(100):
data = json.loads(requests.get(
url=url,
params={'page': i}
).text)['results']
data_norm = pd.read_json(json.dumps(data))
You create your Mysql table on your server using something like Mysql Workbench CE. then in python you do this. I wasnt sure if you want to use data in for loop or data_norm so for ease of use, here some functions. insertDb() can be put in your for loop, since data will be overwriten by itself in every iteration.
import MySQLdb
def dbconnect():
try:
db = MySQLdb.connect(
host='localhost',
user='root',
passwd='password',
db='nameofdb'
)
except Exception as e:
sys.exit("Can't connect to database")
return db
def insertDb():
try:
db = dbconnect()
cursor = db.cursor()
cursor.execute("""
INSERT INTO nameoftable(nameofcolumn) \
VALUES (%s) """, (data))
cursor.close()
except Exception as e:
print e
If this is merely for storage for processing later, kind of like a cache, a varchar field is enough. If however you need to retrieve some structured jdata, JSON field is what you need.
I'm in a small dilemma. I'm using Python's version 2.7 module MySQLdb to grab a list from a table. The code is simple:
#!/usr/bin/python
import json
import MySQLdb
db_host = "localhost"
db_user = "xxx"
db_passwd = "yyy"
db_table = "table"
try:
db = MySQLdb.connect(host=db_host, user=db_user, passwd=db_passwd, db=db_table)
cursor = db.cursor()
cursor.execute("""SELECT serial FROM devices WHERE registered_id IS NOT NULL AND registered_id <>''""")
devices = cursor.fetchall()
print devices
except:
print "Something went wrong with the MySQL"
Printing this comes out as:
(('00000000762c1d3c',), ('000000003ad192f2',), ('00000000ca91760d',),
('000000004c9898aa',))
(I shortened it down because it was quite lengthy.)
How do I get this to list to be parsed correctly into JSON so that it looks like:
{"devices": ['00000000762c1d3c', '000000003ad192f2', '00000000ca91760d', '000000004c9898aa']}
Thank you for your suggestions!
data = {"devices": [item[0] for item in devices]}
json_data = json.dumps(data)