I'm working with mongodb and python. So I have a huge JSON that is within the database in mongodb, and I wanted to make a list where he gave a "print" on the screen of the things I looked for.
db.getCollection('request').aggregate(
{$match : {"kind" :"qpxExpress#tripsSearch"}},
{$unwind:"$trips.tripOption"},
{$project: {"trips.tripOption.saleTotal":1}}
)
I use it to look inside the JSON, but I wanted one of the print on the screen with a code in python.
Here is a JSON exempo that I get the information:
http://codebeautify.org/jsonviewer/cb959272
I've tried to do something but doesn't work:
from pymongo import MongoClient
client = MongoClient()
cursor = db.getCollection('request').aggregate(
{$match : {"kind" :"qpxExpress#tripsSearch"}},
{$unwind:"$trips.tripOption"},
{$project:{"trips.tripOption.saleTotal":1}}
)
for saleTotal in cursor:
print(saleTotal)
Something like this should work (assuming your DB name is test):
from pymongo import MongoClient
client = MongoClient()
db = client.test
cursor = db.request.aggregate(
[
{$match : {"kind" :"qpxExpress#tripsSearch"}},
{$unwind:"$trips.tripOption"},
{$project:{"trips.tripOption.saleTotal":1}}
]
)
for saleTotal in cursor:
print(saleTotal)
Related
import pymongo
client=pymongo.MongoClient("mongodb+srv://Debaditya:<password>#pycluster.d6kvk.gcp.mongodb.net/<db>?retryWrites=true&w=majority")
db = client.test
db.command("createUser", "sample", roles=["read"])
Help me pls the code is given above
Here is the sample code for inserting a document in a collection.
from pymongo import MongoClient
client = MongoClient("mongodb+srv://Debaditya:<password>#pycluster.d6kvk.gcp.mongodb.net/<db>?retryWrites=true&w=majority")
db = client.test
db.students.insert_one({"name": "Test User", "rollNo": 37})
In above example students is the collection name where you have inserted the above row.
If you want to list the rows in the collection.
list(db.students.find())
Just got through the documention of pymongo here
I´m pretty new learning Python and all the containing stuff.
I tried to make my first little steps, installing a MongoDB (working) and connecting to it.
from pymongo import MongoClient
from pprint import pprint
from random import randint
client = MongoClient('localhost', 27017)
db = client.test
collection = db.users
user = {"id": 1, "username": "Test"}
user_id = collection.insert_one(user).inserted_id
print(user_id)
This is the complete code.
pymongo Version: 3.7.2 checked with:
pip freeze | grep pymongo
Output: pymongo==3.7.2
Python Version: 3.7.1
If I try to execute my tiny script, the following error occurs:
'Collection' object is not callable.
If you meant to call the 'insert_one' method on a 'Collection'
object it is
failing because no such method exists.
Where is my Fault?
A little research showed that in pymongo v2 the ".insert_one" is ".insert", but the 3.7.2 version is installed so I should (and must) use the ".insert.one", not the ".insert"
insert_one according with pymongo documentation exists, for server version >= 3.2...
the use is:
user = {'x': 1}
result = db.test.insert_one(user)
result.inserted_id
more complete explanation about insert_one:
>>> db.test.count_documents({'x': 1})
0
>>> result = db.test.insert_one({'x': 1})
>>> result.inserted_id
ObjectId('54f112defba522406c9cc208')
>>> db.test.find_one({'x': 1})
{u'x': 1, u'_id': ObjectId('54f112defba522406c9cc208')}
the following below, I executed and works fine:
# importing client mongo to make the connection
from pymongo import MongoClient
print("--- Exemplo pymongo Connection ---")
# Connection to MongoDB
client = MongoClient('localhost', 27017)
# Selection the Database
db = client.python
# Select the collection
collection = db.users
# Set up a document
user = {"id": 1, "username": "Test"}
# insert one document into selected document
result = collection.insert_one(user)
# Selection just one document from collection
#result = collection.find_one()
# removing the document inserted
collection.delete_one(user)
# print the inserted_id
print("inserted_id: ", result.inserted_id)
Pymongo Documentation
I'm new to Python and I'm building a simple CRUD app using Flask. Here's how I'm doing it:
from flask import Flask,request
import pymysql.cursors
connection = pymysql.connect(
host='localhost',
db='mydb',
user='root',
password='password',
cursorclass='pymysql.cursors.DictCursor
)
#app.route('/login',methods=['POST'])
def login():
cursor = connection.cursor(pymysql.cursors.DictCursor)
cursor.execute("SELECT id,hash,displayName,attempt,status FROM users WHERE id=%s", (request.form['username']))
user = cursor.fetchone()
pprint(user)
But this code outputs something like this thing:
{u'displayName': 'John Smith',
u'hash': 'somehash.asdf!###$',
u'id': 'developer',
u'attempt': 0,
u'status': 1
}
The thing is, I can't seem to get these attributes using the standard user.hash syntax. Am I doing something wrong? I need to either:
convert it to JSON-like structure
get the properties of user when it's presented in this form
When using a pymysql.cursors.DictCursor cursor the fetched data is returned as a dictionary, therefore you can use all the common dict traversal/accessing/modifying methods on it.
This means that you can access your returned hash as: user["hash"].
When it comes to JSON, Python comes with a built-in json module that can readily convert your retrieved dict into a JSON string, so in your case, to get a JSON representation of the returned user dictionary use:
import json
json_string = json.dumps(user)
print(json_string) # or do whatever you need with it
As the title indicates, how does one use python to elegantly access an API and parse and save the JSON contents onto a relational database (MYSQL) for later access?
Here, I saved the data onto a pandas object. But how do I create a mysql database, save the json contents onto it, and access the contents for later use?
# Libraries
import json, requests
import pandas as pd
from pandas.io.json import json_normalize
# Set URL
url = 'https://api-v2.themuse.com/jobs'
# For loop to
for i in range(100):
data = json.loads(requests.get(
url=url,
params={'page': i}
).text)['results']
data_norm = pd.read_json(json.dumps(data))
You create your Mysql table on your server using something like Mysql Workbench CE. then in python you do this. I wasnt sure if you want to use data in for loop or data_norm so for ease of use, here some functions. insertDb() can be put in your for loop, since data will be overwriten by itself in every iteration.
import MySQLdb
def dbconnect():
try:
db = MySQLdb.connect(
host='localhost',
user='root',
passwd='password',
db='nameofdb'
)
except Exception as e:
sys.exit("Can't connect to database")
return db
def insertDb():
try:
db = dbconnect()
cursor = db.cursor()
cursor.execute("""
INSERT INTO nameoftable(nameofcolumn) \
VALUES (%s) """, (data))
cursor.close()
except Exception as e:
print e
If this is merely for storage for processing later, kind of like a cache, a varchar field is enough. If however you need to retrieve some structured jdata, JSON field is what you need.
I ve got a mongo database and I want to retrieve all documents with a cursor operation like in mongodb java api. I want to retrieve all username of database based on this cursor iteration. My code is like the above:
import pymongo
from pymongo import MongoClient
client = MongoClient('...', 27017)
db = client.test_database
db = client['...']
collection = db.test_collection
collection = db["..."]
result = collection.find()
obj = next(result, None)
if obj:
username= obj['username']
print username
I want for the collection to print all the usernames.
Just loop over the results and print the username. There's no reason to play with next()
.
for obj in collection.find():
print obj['username']