Develop Reference

python-ldap3 is unable to add user to and existing LDAP group

I am able successfully connect using LDAP3 and retrieve my LDAP group members as below.
from ldap3 import Server, Connection, ALL, SUBTREE
from ldap3.extend.microsoft.addMembersToGroups import ad_add_members_to_groups as addMembersToGroups
>>> conn = Connection(Server('ldaps://ldap.****.com:***', get_info=ALL),check_names=False, auto_bind=False,user="ANT\*****",password="******", authentication="NTLM")
>>>
>>> conn.open()
>>> conn.search('ou=Groups,o=****.com', '(&(cn=MY-LDAP-GROUP))', attributes=['cn', 'objectclass', 'memberuid'])
it returns True and I can see members by printing
conn.entries
>>>
The above line says MY-LDAP-GROUP exists and returns TRUE while searching but throws LDAP group not found when I try to an user to the group as below
>>> addMembersToGroups(conn, ['myuser'], 'MY-LDAP-GROUP')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/****/anaconda3/lib/python3.7/site-packages/ldap3/extend/microsoft/addMembersToGroups.py", line 69, in ad_add_members_to_groups
raise LDAPInvalidDnError(group + ' not found')
ldap3.core.exceptions.LDAPInvalidDnError: MY-LDAP-GROUP not found
>>>

The above line says MY-LDAP-GROUP exists and returns TRUE
Returning True just means that the search succeeded. It doesn't mean that anything was found. Is there anything in conn.entries?
But I suspect your real problem is something different. If this is the source code for ad_add_members_to_groups, then it is expecting the distinguishedName of the group (notice the parameter name group_dn), but you're passing the cn (common name). For example, your code should be something like:
addMembersToGroups(conn, ['myuser'], 'CN=MY-LDAP-GROUP,OU=Groups,DC=example,DC=com')
If you don't know the DN, then ask for the distinguishedName attribute from the search.
A word of warning: that code for ad_add_members_to_groups retrieves all the current members before adding the new member. You might run into performance problems if you're working with groups that have large membership because of that (e.g. if the group has 1000 members, it will load all 1000 before adding anyone). You don't actually need to do that (you can add a new member without looking at the current membership). I think what they're trying to avoid is the error you get when you try to add someone who is already in the group. But I think there are better ways to handle that. It might not matter to you if you're only working with small groups.

After so many trial and errors, I got frustrated and used the older python-ldap library to add existing users. Now my code is a mixture of ldap3 and ldap.
I know this is not what the OP has desired. But this may help someone.
Here the user Dinesh Kumar is already part of a group group1. I am trying to add him
to another group group2 which is successful and does not disturb the existing group
import ldap
import ldap.modlist as modlist
def add_existing_user_to_group(user_name, user_id, group_id):
"""
:return:
"""
# ldap expects a byte string.
converted_user_name = bytes(user_name, 'utf-8')
converted_user_id = bytes(user_id, 'utf-8')
converted_group_id = bytes(group_id, 'utf-8')
# Add all the attributes for the new dn
ldap_attr = {}
ldap_attr['uid'] = converted_user_name
ldap_attr['cn'] = converted_user_name
ldap_attr['uidNumber'] = converted_user_id
ldap_attr['gidNumber'] = converted_group_id
ldap_attr['objectClass'] = [b'top', b'posixAccount', b'inetOrgPerson']
ldap_attr['sn'] = b'Kumar'
ldap_attr['homeDirectory'] = b'/home/users/dkumar'
# Establish connection to server using ldap
conn = ldap.initialize(server_uri, bytes_mode=False)
bind_resp = conn.simple_bind_s("cn=admin,dc=testldap,dc=com", "password")
dn_new = "cn={},cn={},ou=MyOU,dc=testldap,dc=com".format('Dinesh Kumar','group2')
ldif = modlist.addModlist(ldap_attr)
try:
response = conn.add_s(dn_new, ldif)
except ldap.error as e:
response = e
print(" The response is ", response)
conn.unbind()
return response

how to use the curl POST function in Linux terminal to create a document in python

Am trying to add a new record into a python document and I think am stuck with an issue caused by the curl post function. I have attached the python file and the error received when posting to my url. Could anyone kindly point me towards the right direction.
I dont understand the error code to identify whether the problem comes from the python code bu I do suspect an issue with the curl url.
#!/usr/bin/python
import json
from bson import json_util
from bson.json_util import dumps
import bottle
from bottle import route, run, request, abort
#imports for database
from pymongo import MongoClient
connection = MongoClient('localhost', 27017)
db = connection['city']
collection = db['inspections']
# set up URI paths for REST service
#route('/hello', method='GET')
def get_hello():
word = '"'+request.GET.get('name', None)+'"'
string="{hello:"+word+"}"
return json.loads(json.dumps(string, indent=4, default=json_util.default))
#route('/strings', method='POST')
def run_post():
first = '"'+request.json.get('string1')+'"'
second = '"'+request.json.get('string2')+'"'
data="{first:"+first+",second:"+ second+"}"
return json.loads(json.dumps(data, indent=4, default=json_util.default))
#route('/create', method='POST')
def run_create():
myid = request.json.get('id')
print(myid)
cert_number = request.json.get('certificate_number')
bus_name = request.json.get('business_name')
date = request.json.get('date')
result = request.json.get('result')
sector = request.json.get('sector')
added_id = collection.insert({"id":myid,"certificate_number":cert_number,"business_name":bus_name,"date":date,"result":result,"sector":sector})
added_doc = collection.find_one({"_id":added_id})
return json.loads(json.dumps(added_doc, indent=4, default=json_util.default))
#url does not allow spacing when passing an argument,
#therefore i use underscores when passing the business_name and the remove them
#when creating the query
#route('/read', method='GET')
def get_read():
word = request.params.get('business_name')
word = word.replace("_"," ")
found_doc = collection.find({"business_name":{'$regex':word}}) #will still get results when user pass parameter with white space
return dumps(found_doc)
#route('/update', method='GET')
def get_update(rslt = "Violation Issued"):
myid = request.query.id
query = { "id" :myid}
new_update = { "$set":{"result":rslt}}
collection.update_one(query,new_update)
updated_doc = collection.find_one({"id":myid})
return json.loads(json.dumps(updated_doc, indent=4, default=json_util.default))
#route('/delete', method='GET')
def get_update():
myid = request.query.id
query = {"id" :myid};
print(query)
result = collection.delete_one(query)
return "document with id "+myid+" Has beed deleted from the City Collection"
if __name__ == '__main__':
run(debug=True,reloader = True)
#run(host='localhost', port=8080)
Error:
Returned HTML:
python error:

The Problem is that at one point in the json in your curl request you used “ instead of ". Therefore the json parser throws an error.
So instead of
"business_name" : “ACME Test INC."
write:
"business_name" : "ACME Test INC."

Not sure if you solved this but here we go. Jakob was correct that you used “ instead of "
Next, get the values from the document you are inserting
data = request.json (Contains parsed content)
Assign a value to the variables you need such as id
id = data['id']
Store all the values in a dictionary variable (I think it is much cleaner this way)
document = {"id":myid,"certificate_number":cert_number,"business_name":bus_name,"date":date,"result":result,"sector":sector}
Lastly, use insert_one and catch errors
try:
collection.insert_one(document)
print("CREATED NEW DOCUMENT")
except Exception as e:
print("ERROR: ", e)
Also, there are several ways to fix the "space" problem you mention for cURL requests.
One way is to simply add a %20 between spaces like so:
ACME%20TEST%20INC.
I hope that helps.

Python 3.5 / Pastebin "Bad API request, invalid api_option"

I'm working on a twitch irc bot and one of the components I wanted to have available was the ability for the bot to save quotes to a pastebin paste on close, and then retrieve the same quotes on start up.
I've started with the saving part, and have hit a road block where I can't seem to get a valid post, and I can't figure out a method.
#!/usr/bin/env python3
import urllib.parse
import urllib.request
# --------------------------------------------- Pastebin Requisites --------------------------------------------------
pastebin_key = 'my pastebin key' # developer api key, required. GET: http://pastebin.com/api
pastebin_password = 'password' # password for pastebin_username
pastebin_postexp = 'N' # N = never expire
pastebin_private = 0 # 0 = Public 1 = unlisted 2 = Private
pastebin_url = 'http://pastebin.com/api/api_post.php'
pastebin_username = 'username' # user corresponding with key
# --------------------------------------------- Value clean up --------------------------------------------------
pastebin_password = urllib.parse.quote(pastebin_password, safe='/')
pastebin_username = urllib.parse.quote(pastebin_username, safe='/')
# --------------------------------------------- Pastebin Functions --------------------------------------------------
def post(title, content): # used for posting a new paste
pastebin_vars = {'api_option': 'paste', 'api_user_key': pastebin_username, 'api_paste_private': pastebin_private,
'api_paste_name': title, 'api_paste_expire_date': pastebin_postexp, 'api_dev_key': pastebin_key,
'api_user_password': pastebin_password, 'api_paste_code': content}
try:
str_to_paste = ', '.join("{!s}={!r}".format(key, val) for (key, val) in pastebin_vars.items()) # dict to str :D
str_to_paste = str_to_paste.replace(":", "") # remove :
str_to_paste = str_to_paste.replace("'", "") # remove '
str_to_paste = str_to_paste.replace(")", "") # remove )
str_to_paste = str_to_paste.replace(", ", "&") # replace dividers with &
urllib.request.urlopen(pastebin_url, urllib.parse.urlencode(pastebin_vars)).read()
print('did that work?')
except:
print("post submit failed :(")
print(pastebin_url + "?" + str_to_paste) # print the output for test
post("test", "stuff")
I'm open to importing more libraries and stuff, not really sure what I'm doing wrong after working on this for two days straight :S

import urllib.parse
import urllib.request
PASTEBIN_KEY = 'xxx'
PASTEBIN_URL = 'https://pastebin.com/api/api_post.php'
PASTEBIN_LOGIN_URL = 'https://pastebin.com/api/api_login.php'
PASTEBIN_LOGIN = 'my_login_name'
PASTEBIN_PWD = 'yyy'
def pastebin_post(title, content):
login_params = dict(
api_dev_key=PASTEBIN_KEY,
api_user_name=PASTEBIN_LOGIN,
api_user_password=PASTEBIN_PWD
)
data = urllib.parse.urlencode(login_params).encode("utf-8")
req = urllib.request.Request(PASTEBIN_LOGIN_URL, data)
with urllib.request.urlopen(req) as response:
pastebin_vars = dict(
api_option='paste',
api_dev_key=PASTEBIN_KEY,
api_user_key=response.read(),
api_paste_name=title,
api_paste_code=content,
api_paste_private=2,
)
return urllib.request.urlopen(PASTEBIN_URL, urllib.parse.urlencode(pastebin_vars).encode('utf8')).read()
rv = pastebin_post("This is my title", "These are the contents I'm posting")
print(rv)
Combining two different answers above gave me this working solution.

First, your try/except block is throwing away the actual error. You should almost never use a "bare" except clause without capturing or re-raising the original exception. See this article for a full explanation.
Once you remove the try/except, and you will see the underlying error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "paste.py", line 42, in post
urllib.request.urlopen(pastebin_url, urllib.parse.urlencode(pastebin_vars)).read()
File "/usr/lib/python3.4/urllib/request.py", line 161, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.4/urllib/request.py", line 461, in open
req = meth(req)
File "/usr/lib/python3.4/urllib/request.py", line 1112, in do_request_
raise TypeError(msg)
TypeError: POST data should be bytes or an iterable of bytes. It cannot be of type str.
This means you're trying to pass a unicode string into a function that's expecting bytes. When you do I/O (like reading/writing files on disk, or sending/receiving data over HTTP) you typically need to encode any unicode strings as bytes. See this presentation for a good explanation of unicode vs. bytes and when you need to encode and decode.
Next, this line:
urllib.request.urlopen(pastebin_url, urllib.parse.urlencode(pastebin_vars)).read()
Is throwing away the response, so you have no way of knowing the result of your API call. Assign this to a variable or return it from your function so you can then inspect the value. It will either be a URL to the paste, or an error message from the API.
Next, I think your code is sending a lot of unnecessary parameters to the API and your str_to_paste statements aren't necessary.
I was able to make a paste using the following, much simpler, code:
import urllib.parse
import urllib.request
PASTEBIN_KEY = 'my-api-key' # developer api key, required. GET: http://pastebin.com/api
PASTEBIN_URL = 'http://pastebin.com/api/api_post.php'
def post(title, content): # used for posting a new paste
pastebin_vars = dict(
api_option='paste',
api_dev_key=PASTEBIN_KEY,
api_paste_name=title,
api_paste_code=content,
)
return urllib.request.urlopen(PASTEBIN_URL, urllib.parse.urlencode(pastebin_vars).encode('utf8')).read()
Here it is in use:
>>> post("test", "hello\nworld.")
b'http://pastebin.com/v8jCkHDB'

I didn't know about pastebin until now. I read their api and tried it for the first time, and it worked perfectly fine.
Here's what I did:
I logged in to fetch the api_user_key.
Included that in the posting along with api_dev_key.
Checked the website, and the post was there.
Here's the code:
import urllib.parse
import urllib.request
def post(url, params):
data = urllib.parse.urlencode(login_params).encode("utf-8")
req = urllib.request.Request(login_url, data)
with urllib.request.urlopen(req) as response:
return response.read()
# Logging in to fetch api_user_key
login_url = "http://pastebin.com/api/api_login.php"
login_params = {"api_dev_key": "<the dev key they gave you",
"api_user_name": "<username goes here>",
"api_user_password": "<password goes here>"}
api_user_key = post(login_url, login_params)
# Posting some random text
post_url = "http://pastebin.com/api/api_post.php"
post_params = {"api_dev_key": "<the dev key they gave you",
"api_option": "paste",
"api_paste_code": "<head>Testing</head>",
"api_paste_private": "0",
"api_paste_name": "testing.html",
"api_paste_expire_date": "10M",
"api_paste_format": "html5",
"api_user_key": api_user_key}
response = post(post_url, post_params)
Only the first three parameters are needed for posting something, the rest are optional.

fwy the API doesn't seem to accept http requests as of writing this, so make sure to have the urls in the format of https://pas...

Tweepy follow Twitter users from text file using user_id (Python script)

I am trying to follow user_ids from a text file with 27 user_ids - one per line, e.g. :
217275660
234874181
27213931
230766319
83695362
234154065
68385750
I have some code that seems to work except it says user_id isn't defined (see pasted error at the bottom)... but user_id should be the correct variable in Tweepy... with or without [,follow] afterwards. My code is as follows:
# Script to follow Twitter users from text file containing user IDs (one per line)
# Header stuff I've just thrown in from another script to authenticate
import json
import time
import tweepy
import pprint
from tweepy.parsers import RawParser
from auth import TwitterAuth
from datetime import datetime
auth = tweepy.OAuthHandler(TwitterAuth.consumer_key, TwitterAuth.consumer_secret)
auth.set_access_token(TwitterAuth.access_token, TwitterAuth.access_token_secret)
rawParser = RawParser()
api = tweepy.API(auth_handler = auth, parser = rawParser)
# Open to_follow.txt
to_follow = [line.strip() for line in open('to_follow.txt')]
print to_follow
# Follow everyone from list?!
for user_id in to_follow:
try:
api.create_friendship(user_id)
except tweepy.TweepError as e:
print e
continue
print "Done."
The error is:
$ python follow.py
['217275660', '234874181', '27213931', '230766319', '83695362', '234154065', '68385750', '94981006', '215003131', '30921943', '234526708', '229259895', '88973663', '108144701', '233419650', '70622223', '95445695', '21756719', '229243314', '18162009', '224705840', '49731754', '19352387', '80815034', '17493612', '23825654', '102493081']
Traceback (most recent call last):
File "follow.py", line 30, in <module>
api.create_friendship(user_id)
NameError: name 'user_id' is not defined

Indeed user_id isn't defined anywhere in your code.
I think you meant to do:
for user_id in to_follow:
The user_id's are in the list
And on a side note - the following code is better and more orthodox:
with open('to_follow.txt') as f:
for line in f:
user_id = line.strip()
api.create_friendship(user_id)

How to pass SOAP headers into python SUDS that are not defined in WSDL file

I have a camera on my network which I am trying to connect to with suds but suds doesn't send all the information needed. I need to put extra soap headers not defined in the WSDL file so the camera can understand the message. All the headers are contained in a SOAP envelope and then the suds command should be in the body of the message.
I have checked the suds website
and it says to pass in the headers like so: (This passes in the element as a header but I have an envelope so I'm not sure how to input this)
from suds.sax.element import Element
client = client(url)
ssnns = ('ssn', 'http://namespaces/sessionid')
ssn = Element('SessionID', ns=ssnns).setText('123')
client.set_options(soapheaders=ssn)
result = client.service.addPerson(person)
Now, I am not sure how I would implement this. Say for example, I have the below header:
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://www.w3.org/2003/05/soap-envelope" xmlns:SOAP
ENC="http://www.w3.org/2003/05/soap-encoding"
<wsa:MessageID SOAP-ENV:mustUnderstand="true">urn:uuid:43268c01-f09c6</wsa:MessageID>
<SOAP-ENV:Header>
Using this or a similar example does anyone know how I would pass a valid SOAP message to the targeted service?
Thanks

I have worked out how to enter in new headers and namespaces in suds.
As stated above you create an Element and pass it in as a soapheader as so:
from suds.sax.element import Element
client = client(url)
ssnns = ('ssn', 'http://namespaces/sessionid')
ssn = Element('SessionID', ns=ssnns).setText('123')
client.set_options(soapheaders=ssn)
result = client.service.addPerson(person)
But if you would like to add a namespace I have found adding a prefix seem's to do the trick. So when you create one of the elements you add addPrefix. I'm not sure if this was the way it was intended to be done but it work's.
ssn = Element('SessionID', ns=ssnns).setText('123').addPrefix(p='SOAP-ENC', u='http://www.w3.org/2003/05/soap-encoding')
The p = 'SOAP-ENC' can be any prefix eg. wsa and the u = http://address is the address of the namespace.
A complete script that would run could be:
#!/usr/local/bin/python2.6
import suds
#import logging
from suds.client import Client
from suds.sax.element import Element
from suds.sax.attribute import Attribute
from suds.xsd.sxbasic import Import
def absoluteMove():
# connects to WSDL file and stores location in variable 'client'
client = Client('http://10.10.10.10/p.wsdl')
client.options.location = 'http://10.10.10.10:32963'
# Create the header
wsans = ('wsa', 'http://schemas.xmlsoap.org/ws/2004/08/addressing')
mustAttribute = Attribute('SOAP-ENV:mustUnderstand', 'true')
n1s = ('SOAP-ENC', 'http://www.w3.org/2003/05/soap-encoding')
msgId = Element('Element').addPrefix(p='SOAP-ENC', u='http://www.w3.org/2003/05/soap-encoding')
msgId2 = Element('Address', ns=wsans).setText('http://schemas.xmlsoap.org/ws/2004/08/addressing/role/anonymous')
msgId1 = Element('ReplyTo', ns=wsans).insert(msgId2)
msgId1.append(mustAttribute)
msgId3 = Element('To', ns=wsans).setText('http://10.10.10.10:32954')
msgId3.append(mustAttribute)
client.set_options(soapheaders=[msgId, msgId1, msgId3, msgId2])
# Create 'token' object to pass as an argument using the 'factory' namespace
token = client.factory.create('ns4:ReferenceToken')
# Create 'dest' object to pass as an argument and values passed to this object
dest = client.factory.create('ns4:PTZVector')
dest.PanTilt._x=1
dest.PanTilt._y=4.9
dest.Zoom._x=1
# Create 'speed' object to pass as an argument and values passed to this object
speed = client.factory.create('ns4:PTZSpeed')
speed.PanTilt._x=0
speed.PanTilt._y=0
speed.Zoom._x=1
# 'AbsoluteMove' method invoked passing in the new values entered in the above objects
try:
result = client.service.AbsoluteMove(token, dest, speed)
print "absoluteMove result ", result
return result
except suds.WebFault, e:
print "suds.WebFaults caught: "
print e
if __name__ == '__main__': result = absoluteMove()
This moves the camera. To change the type of soap-envelope check my next question.
You can add logging into this script whci allow's you to check what xml command you have sent which is handy:
import logging
logging.basicConfig(level=logging.INFO)
logging.getLogger('suds.client').setLevel(logging.DEBUG)
The location can be put into the script as an option if the location is not in the wsdl file.