I want to save an API response, on some table of my database, I'm using Postgres along with psycopg2.
This is my code:
import json
import requests
import psycopg2
def my_func():
response = requests.get("https://path/to/api/")
data = response.json()
while data['next'] is not None:
response = requests.get(data['next'])
data = response.json()
for item in data['results']:
try:
connection = psycopg2.connect(user="user",
password="user",
host="127.0.0.1",
port="5432",
database="mydb")
cursor = connection.cursor()
postgres_insert_query = """ INSERT INTO table_items (NAME VALUES (%s)"""
record_to_insert = print(item['name'])
cursor.execute(postgres_insert_query, record_to_insert)
connection.commit()
count = cursor.rowcount
print (count, "success")
except (Exception, psycopg2.Error) as error :
if(connection):
print("error", error)
finally:
if(connection):
cursor.close()
connection.close()
my_func()
I mean, I just wanted to sort of "print" all the resulting data from my request into the db, is there a way to accomplish this?
I'm a bit confused as You can see, I mean, what could be some "print" equivalent to achieve this?
I mean, I just want to save from the API response, the name field, into the database table. Or actually INSERT that, I guess psycopg2 has some sort of function for this circumstance?
Any example You could provide?
EDIT
Sorry, I forgot, if I run this code it will throw this:
PostgreSQL connection is closed
A particular name
Failed to insert record into table_items table syntax error at or near "VALUES"
LINE 1: INSERT INTO table_items (NAME VALUES (%s)
There are a few issues here. I'm not sure what the API is or what it is returning, but I will make some assumptions and suggestions based on those.
There is a syntax error in your query, it is missing a ) it should be:
postgres_insert_query = 'INSERT INTO table_items (NAME) VALUES (%s)'
(I'm also assuming thatNAME` is a real column in your database).
Even with this correction, you will have a problem since:
record_to_insert = print(item['name']) will set record_to_insert to None. The return value of the print function is always None. The line should instead be:
record_to_insert = item['name']
(assuming the key name in the dict item is actually the field you're looking for)
I believe calls to execute must pass replacements as a tuple so the line: cursor.execute(postgres_insert_query, record_to_insert) should be:
cursor.execute(postgres_insert_query, (record_to_insert,))
Related
Requirement: 1. I want to create python API which will help to insert data in big query table and this API will host in swagger/postman, from there user can provide input data so that it will get reflected in big query table.
Can anyone help me to find out suitable solution with code
import sqlite3 as sql
from google.cloud import bigquery
from google.oauth2 import service_account
credentials = service_account.Credentials.from_service_account_file('path/to/file.json')
project_id = 'project_id'
client = bigquery.Client(credentials= credentials,project=project_id)
def add_data(group_name, user_name):
try:
# Connecting to database
con = sql.connect('shot_database.db')
# Getting cursor
c = con.cursor()
# Adding data
job_config.use_legacy_sql = True
query_job = client.query("""
INSERT INTO `table_name` (group, user)
VALUES (%s, %s)""",job_config = job_config)
results = query_job.result() # Wait for the job to complete.
# Applying changes
con.commit()
except:
print("An error has occured")
The code you provided is a mix of SQLite and BigQuery, but it likes that you're trying to use BigQuery to insert data into a table. To insert data into a BigQuery table using Python, you can use the insert_data() method of the Client class. Here's I am adding an example of how you can use this method to insert data into a table called "mytable" in a dataset called "mydataset":
# Define the data you want to insert
data = [
{
"group": group_name,
"user": user_name
}
]
# Insert the data
table_id = "mydataset.mytable"
errors = client.insert_data(table_id, data)
if errors == []:
print("Data inserted successfully")
else:
print("Errors occurred while inserting data:")
print(json.dumps(errors, indent=2))
Then, You can create an API using Flask or Django and call the add_data method which you have defined to insert data into big query table.
I have one page HTML with few submit buttons with different names and values. I have all the names of the buttons in one db in SQL. When the user enter in the page usually he click just in one button each time and the form is submitted. How can I loop through this buttons in my flask/python program?
connection = sqlite3.connect('world.db')
cursor = connection.cursor()
sqlite_select_query = """SELECT name FROM countries"""
cursor.execute(sqlite_select_query)
records = cursor.fetchall()
for row in records:
wish = request.form[row[0]]
try:
db.execute("INSERT INTO userinput (user_id, wish, country_name) VALUES (?, ?, ?)", user_id, wish, row[0])
except ValueError:
db.execute("UPDATE userinput SET wish = ? WHERE user_id =? AND country_name=?", wish, user_id, row[0] )
except:
pass
I have one error 400 Bad request.
Ps.: I already tried to put the value of row between ' and ". Doesn't changed anything.
If I put my request.form inside the try session, it just pass.
I put before few "prints" to see where is the error and see that is in the request form.
If I put the name direct in request form and outside the loop for example ‘’’ request.form[“Brazil”]’’’ I can insert in the database with no problems.
Thanks.
I am pretty sure request.form does not contain all countries, just the one associatted button is clicked. You could look at server logs, probably you see info about missing key in request.form. So what you want to is find that specific one:
wish = request.form.get(row[0])
if wish is None:
continue
I've tried to use couple of methods to insert data into mysql database but getting error in all:
In the first method:
sql = ('''Insert into lgemployees (EmpID,Name,Gender,DOB,Address,PhoneNumber,Email)
VALUES (%d,$s,$s,$s,$s,$d,$s)''', (eid, name, gen, dob, add, mob, email))
mycursor.execute(sql)
mycursor.commit()
Error in this approach:
'tuple' object has no attribute 'encode'
2nd method:
sql = "Insert into lgemployees (EmpID,Name,Gender,DOB,Address,PhoneNumber,Email) VALUES(?,?,?,?,?,?,?,)"
val = (eid, name, gen, dob, add, mob, email)
mycursor.execute(sql, val)
mycursor.commit()
Error in this approach :
"Not all parameters were used in the SQL statement")
mysql.connector.errors.ProgrammingError: Not all parameters were used in the SQL statement
I've troubleshooted a lot from my end but no luck. Can any one please help as where am I wrong or what else can be a good option to insert data into mysql from python.
I dont know where you error is at, but ive tested with this code and it works.
insert_tuple = (eid, name, gen, dob, add, mob, email)
sql = """INSERT INTO lgemployees (`EmpID `,
`Name`,`Gender`, `DOB`, `Address`, `PhoneNumber`, `Email`)
VALUES (%s,%s,%s,%s,%s,%s,%s)"""
mycursor = mySQLconnection.cursor()
mycursor.execute(sql, insert_tuple)
mySQLconnection.commit()
mycursor.close()
your code throws this because one of the parameters are empty or are in a format it cant read.
"Not all parameters were used in the SQL statement")
mysql.connector.errors.ProgrammingError: Not all parameters were used in the SQL statement
I am having a hard time understanding why psycopg2 has a problem with the word 'user'. I am trying to insert values into a table called user with the columns user_id, name, password. I am getting a programmingError: syntax error at or near "user". open_cursor() is a function used to open a cursor for database operations.
Here is my code:
query = """INSERT INTO user (name, password) VALUES (%s, %s);"""
data = ('psycouser', 'sha1$ba316b$52dd71da1e331247f0a7ab869e1b072210add9c1')
with open_cursor() as cursor:
cursor.execute(query, data)
print "Done."
because user is a part of sql language.
try taking it in dbl quotes:
query = 'INSERT INTO "user" (name, password) VALUES (%s, %s);'
I'm trying
title = "Title here"
url = "http://www.mysite.com/url-goes-here"
cursor.execute("""INSERT INTO `videos_justicevids` (`title`, `pageurl`) VALUES (%s, %s)""",(title, url))
I'm not getting an error, but it's not inserting into the database.
You need to commit it.
connection.commit()