I have a MongoDB database that is storing data from ROS topics that my robot is logging. I am trying to print the data in MongoDB by using the following python script:
from pymongo import MongoClient
client = MongoClient('cpr-j100-0101', 62345)
db1 = client.front_scan
db2 = client.cmd_vel
db3 = client.odometry_filtered
print db1
print db2
print db3
but I dont get the result I want when I run this script. I have attached the result of running this script as an image. Instead of this, I would like to actually be able to access the data within mongoDB.enter image description here
You can't print a database before accessing it. First, you need to select what database you need to print. For an example let's think you have 2 collections in db1 as coll1 and coll2. By printing the database means you are going to print the documents of the collections that are in the database.
from pymongo import MongoClient
client = MongoClient('mongodb://localhost:27017/')
db = client.myDatabase
#my dummy database is myDatabase.
coll1 = db.coll1 #selecting the coll1 in myDatabase
for document in coll1.find():
print (document)
so from the above code, you can print all the documents in the coll1 collection of myDatabase. The same manner you can print the databases one by one.
With this script you actually don't do much. You just create three databases and that's basically it. You never insert data, or read data from the database. You're just printing the database object.
I believe, that MongoDB Manual should be useful...
Related
I am trying to access tables from a database using python. There was some code on the website: https://rnacentral.org/help/public-database
import psycopg2.extras
def main():
conn_string = "host='hh-pgsql-public.ebi.ac.uk' dbname='pfmegrnargs' user='reader' password='NWDMCE5xdipIjRrp'"
conn = psycopg2.connect(conn_string)
cursor = conn.cursor(cursor_factory=psycopg2.extras.DictCursor)`
# retrieve a list of RNAcentral databases
query = "SELECT * FROM rnc_database"
cursor.execute(query)
for row in cursor:
print(row)`
When i run this code, i get back a list of databases:
I want to access tables from one of these databases but I don't know what the schema for those tables are or what the values in each list returned represents. I have been looking at 'postgresql to python' resources but all of them are about accessing tables when you know the name of the tables and the columns within.... Is there code for how I can access the table names from the database?
Thank You
Edit: sorry, i thought i linked the website before
The dataset you want to use has schema diagram here https://rnacentral.org/help/public-database
For general purpose I would use something like https://dbeaver.io/ tool it will show you all the schemas in the db and tables inside the schema and so forth. The DBeaver settings to connect to your db would look like this
If you want to keep using python script to explore the db this sql query
SELECT *
FROM pg_catalog.pg_tables
WHERE schemaname != 'pg_catalog' AND
schemaname != 'information_schema';
Should help you.
I need to insert the logs from my test case into a table in postgresql data base.
I was able to connect to the db but I can't figure out how to insert this line result in the tabble, I have tried the below but it doesnt work
import logging
import psycopg2
from io import StringIO
from config import config
params = config()
conn = psycopg2.connect(**params)
print(conn)
curr = conn.cursor()
try:
if not hw.has_connection():
logging.error('Failure: Unable to reach websb! ==> '+ str(driver.find_element_by_xpath('//span[#jsselect="heading" and #jsvalues=".innerHTML:msg"]').text))
return
elif hw.is_websiteReachable():
logging.info("Success: website is reachable!")
curr.execute("""INSERT INTO db_name(logs) VALUES (%s)""", ("Success: website is reachable!"))
conn.commit()
except:
logging.error("Failure: Unable to reach website!")
return
Iam a total beginner in this. I have searched but I couldnt find a clear example or guide about it. the above code throws the exception eventhough the website is reachable. sorry if I sound dumb.
It looks like you're incorrectly constructing your SQL statement. Instead of INSERT INTO db_name(table_name) ... it should be INSERT INTO table_name(column_name) .... If you've correctly connected to the appropriate database in your connection settings, you usually don't have to specify the database name each time you write your SQL.
Therefore I would recommend, the following modification (assuming your table is called logs and it has a column named message):
# ...
sql = 'INSERT INTO logs(message) VALUES (%s);'
msg = 'Success: website is reachable!'
curr.execute(sql, (msg))
conn.commit()
You can read the pyscopg2 docs here for more information as well if that would help with passing named parameters to your SQL queries in Python.
You can check a good solution that I personally use in my in-server projects. You just need to give a connection-string to the CRUD object and all the things will be done. For Postgres you can use:
'postgresql+psycopg2://username:password#host:port/database'
or
'postgresql+pg8000://username:password#host:port/database'
for more details check SQLAlchemy Engine Configuration.
I have written the below Python program to create a MongoDB.
import pymongo
myclient = pymongo.MongoClient('mongodb://localhost:27017/')
mydb = myclient['mydatabase']
dblist = myclient.list_database_names()
if "mydatabase" in dblist:
print("The database exists.")
else :
print("The database does not exists.")
But while I execute the program , I get the result as
The database does not exists.
Why so? Is there something that I am missing ?
I am following the code mentioned in W3Schools Tutorial
In MongoDB the database is not actually created until you put some data into it, try put a document inside and check again.
Important: In MongoDB, a database is not created until it gets content!
Databases are lazily created. Insert something into a collection in that database and they (database/collection) will spring into existence.
Recently I used the python module dataset to manipulate and store information. As result, I have a collection of sqlite databases, let say file1.db, file2.db and so on. Moreover, each of the databases contains the same table.
With dataset I can easily connect and inspect the databases with the code:
>>> db1 = dataset.connect('sqlite:////path/file1.db')
>>> table1 = db1[u'tweet']
Assuming I want to keep databases separated in many files, what is the preferable way to inspect all the databases with dataset?
I am looking at something better to this:
>>> db1 = dataset.connect('sqlite:////path/file1.db')
>>> db2 = dataset.connect('sqlite:////path/file2.db')
>>> tables = [db1[u'tweet'],db2[u'tweet']]
>>> for table in tables:
for tweet in table:
print(tweet['text'])
I don't know a clean solution to this, but it might be interesting to use an in-memory SQLite database for this scenario:
mem_db = dataset.connect('sqlite:///')
databases = ['sqlite:////path/file1.db']
for uri in databases:
db1 = dataset.connect(uri)
for row in db1['table']:
mem_db.insert(row)
There's also an insert_many call, I believe, which may be faster for bulk transfer.
I want to connect to two databases using Python and, later on, use tables from both of the databases. How can I do this? Is the following code correct?
con = mdb.connect(host=MY_HOST, user=MY_USER, passwd=MY_PASS, db1=MY_DB1, db2=MY_DB2)
If you don't specify the database in your connect call, you can write queries against multiple databases at once. The documentation says that db is not required.
db = _mysql.connect('localhost', 'user', 'passwd')
then
SELECT u.*, i.* FROM db1.users u LEFT JOIN db2.items i ON u.id = i.user_id
But it'll only work if the two databases are on the same server.
Just make two separate connections
con1 = mdb.connect (host=MY_HOST, user=MY_USER, passwd=MY_PASS, db1=MY_DB1)
con2 = mdb.connect (host=MY_HOST2, user=MY_USER2, passwd=MY_PASS2, db2=MY_DB2)
and use them independently just as you would when using one database.
You have two possibilities:
1) Take the data into a pandas DataFrame and use it to create table in second server.
2) BCP out data from server1 and load it into second server
The two (or more) connections can live in a single variable e.g. a dictionary. This becomes very handy if you want to perform one operation on all the connections at once. For example closing all of them. Here's an example:
connections = {
'conn1': mdb.connect(host, user, passwd, db),
'conn2': mdb.connect(host, user, passwd, db)
}
You can retrieve each connection with its key:
connections['conn1'].execute('SELECT ...')
And you can close all the connections in one line:
[conn.close() for key, conn in connections.items()]