Develop Reference

POSTGRESQL Queries using Python

I am trying to access tables from a database using python. There was some code on the website: https://rnacentral.org/help/public-database
import psycopg2.extras
def main():
conn_string = "host='hh-pgsql-public.ebi.ac.uk' dbname='pfmegrnargs' user='reader' password='NWDMCE5xdipIjRrp'"
conn = psycopg2.connect(conn_string)
cursor = conn.cursor(cursor_factory=psycopg2.extras.DictCursor)`
# retrieve a list of RNAcentral databases
query = "SELECT * FROM rnc_database"
cursor.execute(query)
for row in cursor:
print(row)`
When i run this code, i get back a list of databases:
I want to access tables from one of these databases but I don't know what the schema for those tables are or what the values in each list returned represents. I have been looking at 'postgresql to python' resources but all of them are about accessing tables when you know the name of the tables and the columns within.... Is there code for how I can access the table names from the database?
Thank You
Edit: sorry, i thought i linked the website before

The dataset you want to use has schema diagram here https://rnacentral.org/help/public-database
For general purpose I would use something like https://dbeaver.io/ tool it will show you all the schemas in the db and tables inside the schema and so forth. The DBeaver settings to connect to your db would look like this
If you want to keep using python script to explore the db this sql query
SELECT *
FROM pg_catalog.pg_tables
WHERE schemaname != 'pg_catalog' AND
schemaname != 'information_schema';
Should help you.

how to insert python logs in postgresql table?

I need to insert the logs from my test case into a table in postgresql data base.
I was able to connect to the db but I can't figure out how to insert this line result in the tabble, I have tried the below but it doesnt work
import logging
import psycopg2
from io import StringIO
from config import config
params = config()
conn = psycopg2.connect(**params)
print(conn)
curr = conn.cursor()
try:
if not hw.has_connection():
logging.error('Failure: Unable to reach websb! ==> '+ str(driver.find_element_by_xpath('//span[#jsselect="heading" and #jsvalues=".innerHTML:msg"]').text))
return
elif hw.is_websiteReachable():
logging.info("Success: website is reachable!")
curr.execute("""INSERT INTO db_name(logs) VALUES (%s)""", ("Success: website is reachable!"))
conn.commit()
except:
logging.error("Failure: Unable to reach website!")
return
Iam a total beginner in this. I have searched but I couldnt find a clear example or guide about it. the above code throws the exception eventhough the website is reachable. sorry if I sound dumb.

It looks like you're incorrectly constructing your SQL statement. Instead of INSERT INTO db_name(table_name) ... it should be INSERT INTO table_name(column_name) .... If you've correctly connected to the appropriate database in your connection settings, you usually don't have to specify the database name each time you write your SQL.
Therefore I would recommend, the following modification (assuming your table is called logs and it has a column named message):
# ...
sql = 'INSERT INTO logs(message) VALUES (%s);'
msg = 'Success: website is reachable!'
curr.execute(sql, (msg))
conn.commit()
You can read the pyscopg2 docs here for more information as well if that would help with passing named parameters to your SQL queries in Python.

You can check a good solution that I personally use in my in-server projects. You just need to give a connection-string to the CRUD object and all the things will be done. For Postgres you can use:
'postgresql+psycopg2://username:password#host:port/database'
or
'postgresql+pg8000://username:password#host:port/database'
for more details check SQLAlchemy Engine Configuration.

MySQL data is caching with google cloud SQL and SQLAlchemy [duplicate]

This question already has answers here:
Why are some mysql connections selecting old data the mysql database after a delete + insert?
(2 answers)
Closed 8 months ago.
This is my first question on stackoverflow, so please correct me if I do something wrong :).
My data from a database hosted at Google Cloud SQL is caching with Flask-SQLAlchemy. When I add a new record and try to get that record it doesn't exist.
I am using a script that adds records and I use one to get records.
I first tried it with SQLite, that worked perfectly. But with MySQL at Google Cloud SQL it doesn't.
Every time I add/change something to the database I use db.session.commit()
I use the pymysql module with this: pymysql.install_as_MySQLdb()
And my connection URI looks like this: mysql://...
Edit:
This is what I use to add a new record (my script is for adding jokes):
new_joke = Jokes(joke, user["username"], user["id"], avatar_url, "0")
db.session.add(new_joke)
db.session.commit()
And this is what I use to get a record (random):
jokes = Jokes.query.all()
randint = random.randint(0, len(jokes) - 1)
joke = jokes[randint]

I found an answer!
By doing db.session.commit() before making a query, it refreshes it's cache.
My code now looks like this:
db.session.commit()
jokes = Jokes.query.all()
randint = random.randint(0, len(jokes) - 1)
joke = jokes[randint]

How to print MongoDB databases in Python

I have a MongoDB database that is storing data from ROS topics that my robot is logging. I am trying to print the data in MongoDB by using the following python script:
from pymongo import MongoClient
client = MongoClient('cpr-j100-0101', 62345)
db1 = client.front_scan
db2 = client.cmd_vel
db3 = client.odometry_filtered
print db1
print db2
print db3
but I dont get the result I want when I run this script. I have attached the result of running this script as an image. Instead of this, I would like to actually be able to access the data within mongoDB.enter image description here

You can't print a database before accessing it. First, you need to select what database you need to print. For an example let's think you have 2 collections in db1 as coll1 and coll2. By printing the database means you are going to print the documents of the collections that are in the database.
from pymongo import MongoClient
client = MongoClient('mongodb://localhost:27017/')
db = client.myDatabase
#my dummy database is myDatabase.
coll1 = db.coll1 #selecting the coll1 in myDatabase
for document in coll1.find():
print (document)
so from the above code, you can print all the documents in the coll1 collection of myDatabase. The same manner you can print the databases one by one.

With this script you actually don't do much. You just create three databases and that's basically it. You never insert data, or read data from the database. You're just printing the database object.
I believe, that MongoDB Manual should be useful...

Save List To PyMongo

Never used PyMongo so I'm new to this stuff. I want to be able to save one of my lists to MongoDB. For example, I have a list imageIds = ["zw8SeIUW", "f28BYZ"], which is appended to frequently. After each append, the list imageIds should be saved to the database.
import pymongo
from pymongo import MongoClient
db = client.databaseForImages
and then later
imageIds.append(data)
db.databaseForImages.save(imageIds)
Why doesn't this work? What is the solution?

First, if you don't know what a python dict is, I recommend brushing up on Python fundamentals . Check out Google's Python Class or Learn Python the Hard Way. Otherwise, you will be back here every 10 minutes with a new question...
Now, you have to connect to the mongoDB server/instance:
client = MongoClient('hostname', port_number)
Connect to a database:
db = client.imagedb
Then save the record to the collection "image_data".
record = {'image_ids': imageIds}
db.image_data.save(record)
Using save(), the record dict is updated with an '_id' field which now points to the record in this collection. To update it with a new appended imageIds:
record['image_ids'] = imageIds # Already contains the original _id
db.image_data.save(record)