I am trying to connect to a Meteor Mongo database through pymongo. Here's the code:
def get_mongo_url(site):
# return "mongodb://client-xxxxx:yyyyy#production-db-c1.meteor.io:27017/site"
import subprocess
p = subprocess.Popen(['meteor', 'mongo', '--url', site], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate()
print out
return out
from pymongo import MongoClient
client = MongoClient(get_mongo_url("mysite.com"))
And the error (the print statement yields a correct url)
>> mongodb://client-xxxxx:yyyyy#production-db-c1.meteor.io:27017/site
Traceback (most recent call last):
File "private/test.py", line 46, in <module>
client = pymongo.MongoClient(get_mongo_url(METEOR_SITE))
File "/Library/Python/2.7/site-packages/pymongo/mongo_client.py", line 369, in __init__
raise ConfigurationError(str(exc))
pymongo.errors.ConfigurationError: command SON([('authenticate', 1), ('user', u'client-xxxxx'), ('nonce', u'zzzzz'), ('key', u'ttttt')]) failed: auth fails
If I run meteor mongo --url mysite.com, copy the result into the return ... at the top of the function and uncomment it, the connection works. Why can't I connect programmatically?
The subprocess code appends a line feed character \n to the end of the url.
You need to strip that with .rstrip()
The right way to do that is to replace the return in your function with
return out.rstrip()
For confirmation purposes I will show what happens with the function as-is and
rstrip() applied/unapplied to the return.
murl = get_mongo_url('').rstrip()
mongodb://client-faf1d0db:746d8f43-367b-dde2-b69a-039ff8b9f7fa#production-db-a1.meteor.io:27017/_meteor_com
client = pymongo.MongoClient(murl)
Worked OK
murl = get_mongo_url('')
mongodb://client-3578a20b:d4ddeec9-6d24-713e-8ddb-c357b664948a#production-db-a1.meteor.io:27017/_meteor_com
client = pymongo.MongoClient(murl)
Traceback (most recent call last):
File "", line 1, in
File "/home/action/.local/lib/python2.7/site-packages/pymongo/mongo_client.py", line 383, in init
raise ConfigurationError(str(exc))
pymongo.errors.ConfigurationError: command SON([('authenticate', 1), ('user', u'client-3578a20b'), ('nonce', u'e14e2bdb3d8484b9'), ('key', u'9
c101b78ff1a617a9c5f0def36c7e3d9')]) failed: auth fails
Failed without the rstrip.
murl = get_mongo_url('')
mongodb://client-1a193a61:4c9c572e-22e3-4b7e-44a1-dc76bfb65e86#production-db-a1.meteor.io:27017/_meteor_com
client = pymongo.MongoClient(murl)
Traceback (most recent call last):
File "", line 1, in
File "/home/action/.local/lib/python2.7/site-packages/pymongo/mongo_client.py", line 383, in init
raise ConfigurationError(str(exc))
pymongo.errors.ConfigurationError: command SON([('authenticate', 1), ('user', u'client-1a193a61'), ('nonce', u'a2576142b1a33d8b'), ('key', u'4
419c490bcdcc65b20f2950c3b106d59')]) failed: auth fails
Failed again (no rsrtip)
murl = get_mongo_url('').rstrip()
mongodb://client-ce463608:d7dc6be0-499f-1808-43e1-fdfb8b6e8ebc#production-db-a1.meteor.io:27017/_meteor_com
client = pymongo.MongoClient(murl)
Worked (rstrip used).
The following is general info on mongodb URLs. You may know this already.
The URL that pymongo wants is not a web URL but a URL-like specifier for a mongo database connection.
For a development environment, the mongodb is usually set up on port 3001, which is not the default mongodb port for a production server.
Meteor applications can be configured to use a mongodb hosted anywhere. It does not have to be on the same server that serves the meteor content. The specification of this is done through the mongodb:// URL which is what pymongo wants. pymongo doesn't depend on the meteor website url, which can be very different from the mongodb url.
Here is some code I am using
import pymongo
MONGO_URL = r'mongodb://localhost:3001/meteor'
###
def connect():
client = pymongo.MongoClient(MONGO_URL)
return client
def findUser(c, email):
users = c.meteor.users
return users.find_one({"emails.address": email})
According to the mongodb site on Github, The MONGO_URL format is
mongodb://[username:password#]host1[:port1][,host2[:port2],...[,hostN[:portN]]][/[database][?options]]
so the mongodb url mongodb://localhost:3001/meteor can be interpreted like this:
* mongodb:// means this describes a mongodb connection
* localhost means connect locally
* :3001 means use non-standard port number 3001. this is how "meteor run" sets up mongo
* /meteor means connect to the database called "meteor"
Related
I need to connect a python script to the Google Cloud SQL database using SSL. I am aware how to connect to the database only by ip address but I do not understand how to use the SSL certificate (3 .pem certificates), token ID etc that I have received in the .json file using python. I have tried some codes and they do not work nor do I understand what is going on.
import psycopg2
import psycopg2.extensions
import os
import stat
from google.cloud import storage
def main():
con = connect()
connection = con.connect_to_db()
result = connection.execute('SELECT * FROM
personaformation').fetchall()
for row in result:
votes.append({
'fname': row[0],
'lname': row[1],
'email': row[2]
})
print(votes[0])
def connect_to_db(self):
# Get keys from GCS
client = storage.Client()
bucket = client.get_bucket(weightssl)
bucket.get_blob('C:/Users/tasne/Downloads/serverca.pem').
download_to_filename('server-ca.pem')
bucket.get_blob('C:/Users/tasne/Downloads/clientkey.pem').
download_to_filename('client-key.pem')
os.chmod("client-key.pem", stat.S_IRWXU)
bucket.get_blob('C:/Users/tasne/Downloads/clientcert.pem').
download_to_filename('client-cert.pem')
sslrootcert = 'server-ca.pem'
sslkey = 'client-key.pem'
sslcert = 'client-cert.pem'
print("reached here")
con = psycopg2.connect(
host='37.246.65.223',
dbname='personalformation',
user='hello',
password='world',
sslmode = 'verify-full',
sslrootcert=sslrootcert,
sslcert=sslcert,
sslkey=sslkey)
return con
When I try the codes this is the error that I get, however I do not know where to specify the credentials nor am I using compute engine because this is not an application that I am creating but only a python script
C:\Users\tasne\PycharmProjects\project1\venv\Scripts\python.exe C:/Users/tasne/Downloads/database2.py
Traceback (most recent call last):
File "C:/Users/tasne/Downloads/database2.py", line 21, in <module>
credentials = GoogleCredentials.get_application_default()
File "C:\Users\tasne\PycharmProjects\project1\venv\lib\site-packages\oauth2client\client.py", line 1288, in get_application_default
return GoogleCredentials._get_implicit_credentials()
File "C:\Users\tasne\PycharmProjects\project1\venv\lib\site-packages\oauth2client\client.py", line 1278, in _get_implicit_credentials
raise ApplicationDefaultCredentialsError(ADC_HELP_MSG)
oauth2client.client.ApplicationDefaultCredentialsError: The Application Default Credentials are not available. They are available if running in Google Compute Engine. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
I am trying to create a P2P node using python (pyp2p) but I am getting this error:
Eamons-MacBook-Pro:blockchain eamonwhite$ python3 serveralice.py
HTTP Error 404: Not Found
HTTP Error 404: Not Found
HTTP Error 404: Not Found
HTTP Error 404: Not Found
Traceback (most recent call last):
File "/Users/eamonwhite/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pyp2p/net.py", line 732, in start
rendezvous_con = self.rendezvous.server_connect()
File "/Users/eamonwhite/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pyp2p/rendezvous_client.py", line 92, in server_connect
con.connect(server["addr"], server["port"])
File "/Users/eamonwhite/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pyp2p/sock.py", line 189, in connect
self.s.bind((src_ip, 0))
TypeError: str, bytes or bytearray expected, not NoneType
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "serveralice.py", line 10, in <module>
alice.start()
File "/Users/eamonwhite/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pyp2p/net.py", line 735, in start
raise Exception("Unable to connect to rendezvous server.")
Exception: Unable to connect to rendezvous server.
My relevant code looks like this:
from uuid import uuid4
from blockchain import Blockchain
from flask import Flask, jsonify, request
from pyp2p.net import *
import time
#Setup Alice's p2p node.
alice = Net(passive_bind="192.168.1.131", passive_port=44444, interface="en0", node_type="passive", debug=1)
alice.start()
alice.bootstrap()
alice.advertise()
while 1:
for con in alice:
for reply in con:
print(reply)
time.sleep(1)
...
It is getting stuck on the Net function right at the beginning - something to do with the rendezvous package. The IP is my IP on the my network, and I port forwarded 44444 although I'm not sure if I need to do that or not. Thanks.
I am new to this, apparently with the way the server code was configured, it needed a rendezvous server to work (a node that handles all the other nodes). It is in net.py of the pyp2p package:
# Bootstrapping + TCP hole punching server.
rendezvous_servers = [
{
"addr": "162.243.213.95",
"port": 8000
}
]
The address was the problem, obviously it is just a placeholder IP. So then I realized I needed my own rendezvous server, and I used this code - https://raw.githubusercontent.com/StorjOld/pyp2p/master/pyp2p/rendezvous_server.py.
However I had to debug this file a little, it ended up needing to have import sys, import time and import re statements at the top before it would work. Now I am going to host it on my raspberry pi so that it is always up to handle nodes :)
I'm trying to connect from the one machine to another remote server with installed RabbitMQ.
The RabbitMQ is working perfectly locally, but when I connect to it from the another machine then an error is occurs:
root#xxx:~# python3 rabbitmq.py
Traceback (most recent call last):
File "rabbitmq.py", line 8, in <module>
connection = pika.BlockingConnection(pika.ConnectionParameters(parameters))
File "/usr/local/lib/python3.4/dist-packages/pika/connection.py", line 652, in __init__
self.host = host
File "/usr/local/lib/python3.4/dist-packages/pika/connection.py", line 392, in host
(value,))
TypeError: host must be a str or unicode str, but got <ConnectionParameters host=111.111.111.111 port=5672 virtual_host=product ssl=False>
root#xxx:~#
TypeError: host must be a str or unicode str, but got ConnectionParameters host=111.111.111.111 port=5672
virtual_host=product ssl=False
The Python code on other remote machine:
import pika
credentials = pika.PlainCredentials(username='remoteuser', password='mypassword')
parameters = pika.ConnectionParameters(host='111.111.111.111', port=5672, virtual_host='/', credentials=credentials)
#connection = pika.BlockingConnection(pika.ConnectionParameters('111.111.111.111:15672')) # --- it doesn't work too
connection = pika.BlockingConnection(pika.ConnectionParameters(parameters))
channel = connection.channel()
channel.queue_declare(queue='hello')
channel.basic_publish(exchange='', routing_key='hello', body='Hello World!')
print(" [x] Sent 'Hello World!'")
connection.close()
User "remoteuser" has admin rights and access to virtual host "/"
http://111.111.111.111:15672/#/users
Name Tags Can access virtual hosts Has password
remoteuser administrator / ●
What is the problem?
you have double wrapped parameters, change:
connection = pika.BlockingConnection(pika.ConnectionParameters(parameters))
to:
connection = pika.BlockingConnection(parameters)
I am refering to the http://api.mongodb.org/python/current/examples/authentication.html site for authentication mechanism examples. I have created a User administrator and using its credentials I created a user for my 'reporting' database. Now i need to access the same through pymongo using the username and password. I tried the following commands in python shell. Is this the right way as my authentication is failing.
from pymongo import MongoClient
client = MongoClient('localhost')
client.reporting.authenticate('reportsUser', '123456', mechanism='MONGODB-CR')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/dist-packages/pymongo/database.py", line 746, in authenticate
self.connection._cache_credentials(self.name, credentials)
File "/usr/lib/python2.7/dist-packages/pymongo/mongo_client.py", line 441, in _cache_credentials
auth.authenticate(credentials, sock_info, self.__simple_command)
File "/usr/lib/python2.7/dist-packages/pymongo/auth.py", line 214, in authenticate
auth_func(credentials[1:], sock_info, cmd_func)
File "/usr/lib/python2.7/dist-packages/pymongo/auth.py", line 194, in _authenticate_mongo_cr
cmd_func(sock_info, source, query)
File "/usr/lib/python2.7/dist-packages/pymongo/mongo_client.py", line 607, in __simple_command
helpers._check_command_response(response, None, msg)
File "/usr/lib/python2.7/dist-packages/pymongo/helpers.py", line 147, in _check_command_response
raise OperationFailure(msg % errmsg, code)
pymongo.errors.OperationFailure: command SON([('authenticate', 1), ('user', u'reportsUser'), ('nonce', u'f8158a24f1c61650'), ('key', u'14cea216c54b93bae20acd2e076bb785')]) failed: auth failed
As an FYI, you can use the URI string format as well. The pseudocode looks like this:
pymongo.MongoClient('mongodb://user:password#server:port/')
Here's a simple connection code block with auth:
import pymongo
conn = pymongo.MongoClient('mongodb://root:pass#localhost:27017/')
db = conn['database']
coll = db['collection']
There are more options for the query string here: http://docs.mongodb.org/manual/reference/connection-string/
Hope that helps = looks like you already have it though. Happy coding!!
Just adding more to provided solutions.
I have been using as URI connection string and credentials being provided as f string it helps to reduce number of lines. One thing to note is about special characters in password where we convert using urllib package as shown below.
import urllib.parse
from pymongo import MongoClient
host = "localhost"
port = 27017
user_name = "myuser"
pass_word = "Pass#123"
db_name = "mydb" # database name to authenticate
# if your password has '#' then you might need to escape hence we are using "urllib.parse.quote_plus()"
client = MongoClient(f'mongodb://{user_name}:{urllib.parse.quote_plus(pass_word)}#{host}:{port}/{db_name}')
It's worked for me.
Here you can connect mongodb to python by using authentication username and password.
import pymongo
DATABASE_NAME = "your_database_name"
DATABASE_HOST = "localhost"
DATABASE_USERNAME = "database_username"
DATABASE_PASSWORD = "database_password"
try:
myclient = pymongo.MongoClient( DATABASE_HOST )
myclient.test.authenticate( DATABASE_USERNAME , DATABASE_PASSWORD )
mydb = myclient[DATABASE_NAME]
print("[+] Database connected!")
except Exception as e:
print("[+] Database connection error!")
raise e
By default Mongodb uses 27017 port
Hi I am trying to write a simple thrift server in python (named PythonServer.py) with a single method that returns a string for learning purposes. The server code is below. I am having the following errors in the Thrift's python libraries when I run the server. Has anyone experienced this problem and suggest a workaround?
The execution output:
Starting server
Traceback (most recent call last):
File "/home/dae/workspace/BasicTestEnvironmentV1.0/src/PythonServer.py", line 38, in <module>
server.serve()
File "usr/lib/python2.6/site-packages/thrift/server/TServer.py", line 101, in serve
File "usr/lib/python2.6/site-packages/thrift/transport/TSocket.py", line 136, in listen
File "usr/lib/python2.6/site-packages/thrift/transport/TSocket.py", line 31, in _resolveAddr
TypeError: getaddrinfo() argument 1 must be string or None
PythonServer.java
port = 9090
import MyService as myserv
#from ttypes import *
# Thrift files
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol
from thrift.server import TServer
# Server implementation
class MyHandler:
# return server message
def sendMessage(self, text):
print text
return 'In the garage!!!'
# set handler to our implementation
handler = MyHandler()
processor = myserv.Processor(handler)
transport = TSocket.TServerSocket(port)
tfactory = TTransport.TBufferedTransportFactory()
pfactory = TBinaryProtocol.TBinaryProtocolFactory()
# set server
server = TServer.TThreadedServer(processor, transport, tfactory, pfactory)
print 'Starting server'
server.serve() ##### LINE 38 GOES HERE ##########
Your problem is the line:
transport = TSocket.TServerSocket(port)
When calling TSocket.TServerSocket which a single argument, the value is treated as a host identifier, hence the error with getaddrinfo().
To fix that, change the line to:
transport = TSocket.TServerSocket(port=port)
I had this problem while running PythonServer.py ...
I changed this line
transport = TSocket.TServerSocket(9090)
to
transport = TSocket.TServerSocket('9090')
and my server started running.
I had a similar problem. My fix is
TSocket.TServerSocket('your server ip',port)