I'm using chef server, configuring few different nodes / environments.
when asking for env attributes using the pychef api, few times in a row (when refreshing a web page using python server calling chef server) im getting ChefServerNotFoundError (the first few times are fine, and third exception is raised)
I guess that there is kind of firewall / anti ddos attacks on this server, but i can not figure out how to edit these settings.
anyone have any idea?
this is a part of the method (that is called 3 times and throws an exception):
env_nodes = Search('node').query('chef_environment: {0}'.format(env_name))
nodes_dict = {}
for n in env_nodes:
node = Node(n['name'])
nodes_dict[node.name] = node['ipaddress']`
and this is the traceback:
File "C:\env\lib\site-packages\chef\search.py", line 91, in __getitem__
row_value = self.data['rows'][value]
File "C:\env\lib\site-packages\chef\search.py", line 59, in data
self._data = self.api[self.url]
TypeError: 'NoneType' object is not subscriptable`
When using PyChef in a webapp or other multi-threaded system you should really pass in the API object explicitly. There is a system to track a default API target in a threadlocal for the purposes of making simple scripts easier, but in retrospect this was probably a mistake as it leads to these confusing issues. This would be a better version of that code, also faster:
nodes_dict = {row.object.name: row.object['ipaddress'] for row in Search('node', 'chef_environment:{}'.format(env_name), api=api)}
Where api is the return value of chef.autoconfigure() or some other ChefAPI object.
Related
I'd like to setup a "timeout" for the ldap library (python-ldap-2.4.15-2.el7.x86_64) and python 2.7
I'm forcing my /etc/hosts to resolve an non existing IP address in
order to raise the timeout.
I've followed several examples and took a look at the documentation or questions like this one without luck.
By now I've tried forcing global timeouts before the initialize:
ldap.set_option(ldap.OPT_NETWORK_TIMEOUT, 1)
ldap.set_option(ldap.OPT_TIMEOUT, 1)
ldap.protocol_version=ldap.VERSION3
Forced the same values at object level:
ldap_obj = ldap.initialize("ldap://%s:389" % LDAPSERVER ,trace_level=9)
ldap_obj.set_option(ldap.OPT_NETWORK_TIMEOUT, 1)
ldap_obj.set_option(ldap.OPT_TIMEOUT, 1)
I've also tried with:
ldap.network_timeout = 1
ldap.timelimit = 1
And used both methods, search and search_st (The synchronous form with timeout)
Finally this is the code:
def testLDAPConex( LDAPSERVER ) :
"""
Check ldap
"""
ldap.set_option(ldap.OPT_NETWORK_TIMEOUT, 1)
ldap.set_option(ldap.OPT_TIMEOUT, 1)
ldap.protocol_version=ldap.VERSION3
ldap.network_timeout = 1
ldap.timelimit = 1
try:
ldap_obj = ldap.initialize("ldap://%s:389" % LDAPSERVER ,trace_level=9)
ldap_result_id = ldap_obj.search("ou=people,c=this,o=place", ldap.SCOPE_SUBTREE, "cn=user")
I've printed the constants of the object OPT_NETWORK_TIMEOUT and OPT_TIMEOUT, the values were assigned correctly.
The execution time is 56s everytime and I'm not able to control the amount of seconds for the timeout.
Btw, the same code in python3 does work as intended:
real 0m10,094s
user 0m0,072s
sys 0m0,013s
After some testing I've decided to rollback the VM to a previous state.
The Networking Team were doing changes across the network configuration, that might have changed the behaviour over the interface and some kind of bug when the packages were being sent over the wire in:
ldap_result_id = ldap_obj.search("ou=people,c=this,o=place", ldap.SCOPE_SUBTREE, "cn=user")
After restoring the VM, which included a restart, the error of not being able to reach LDAP raises as expected.
I'm trying to use BLE library for python to communicate with one Nordic nrf51844 chipset. Because one characteristic is notification enabled, I need to enable notification from client side by setting the descriptor Client Characteristic Configuration to 0x0001. But I failed to get the descriptor with the call "characteristic.find_descriptor()" to get it. I also tried to print out all of descriptors discovered, but looks like there is no luck to get it work.
Below is the code I'm using to discover characteristics and its descriptor referred to the example of Adafruit BLE library:
def enable_notification(characteristic):
_enableDesc = characteristic.find_descriptor(CLIENT_CHARACTERISTIC_CONFIG_DESCRIPTOR_UUID)
_cmd = bytearray([0x01, 0x00])
_enableDesc.write_value(_cmd)
def handle_device_message(device):
global _status
# Once connected do everything else in a try/finally to make sure the device
# is disconnected when done.
# Wait for service discovery to complete for at least the specified
# service and characteristic UUID lists. Will time out after 60 seconds
# (specify timeout_sec parameter to override).
_logger.info('Discovering services...')
device.discover([HES_SERVICE_UUID], [DATA_CHAR_UUID, STATUS_CHAR_UUID, FACTOR_CHAR_UUID])
# Find the HES service and its characteristics.
hes = device.find_service(HES_SERVICE_UUID)
dataC = hes.find_characteristic(DATA_CHAR_UUID)
statusC = hes.find_characteristic(STATUS_CHAR_UUID)
#factorC = hes.find_characteristic(FACTOR_CHAR_UUID)
dataC.list_descriptors()
statusC.list_descriptors()
enable_notification(dataC)
enable_notification(statusC)
But it always failed at "characteristic.find_descriptor()" with below error:
_enableDesc =
characteristic.find_descriptor(CLIENT_CHARACTERISTIC_CONFIG_DESCRIPTOR_UUID)
File "build/bdist.macosx-10.11-x86_64/egg/Adafruit_BluefruitLE/interfaces/gatt.py", line 98, in find_descriptor
File "build/bdist.macosx-10.11-x86_64/egg/Adafruit_BluefruitLE/corebluetooth/gatt.py", line 124, in list_descriptors
File "build/bdist.macosx-10.11-x86_64/egg/Adafruit_BluefruitLE/corebluetooth/metadata.py", line 63, in get_all
TypeError: 'NoneType' object is not iterable
I looked into the source code of library, but can't find the interface to get descriptors explicitly. Can anyone help me on this?
Thanks!
Finally I figured out by checking the API of IOS to set notification. It should be set by calling setNotify for characteristic instead of writeValue for descriptor. And for the descriptor stuff, it shows we need to wait for some time before all descriptors are discovered and returned. Might be the issue implemented with Python. Not really verified with IOS native program.
BTW, after setting the notification, we need to wait for some time as well before the device sends notification to client.
Will get a Linux box to verify the implementation with blueZ is working well.
I have been learning how to use Amazon S3 API by using the open source Eucalyptus. So far I have been able to successfully use REST, but now I would also like to use SOAP. I seem to be having trouble generating the correct signature for my request. The service is giving me a 403 Forbidden error:
Traceback (most recent call last):
File "soap.py", line 31, in <module>
r = w.download_file('mybucket', 'test.txt')
File "soap.py", line 27, in download_file
r = self.client.service.ListAllMyBuckets(self.access_key, timestamp, signature)
File "/usr/lib/python2.6/site-packages/suds/client.py", line 521, in __call__
return client.invoke(args, kwargs)
File "/usr/lib/python2.6/site-packages/suds/client.py", line 581, in invoke
result = self.send(soapenv)
File "/usr/lib/python2.6/site-packages/suds/client.py", line 619, in send
description=tostr(e), original_soapenv=original_soapenv)
File "/usr/lib/python2.6/site-packages/suds/client.py", line 677, in process_reply
raise Exception((status, description))
Exception: (403, u'Forbidden')
My code is in Python 2 and uses the SUDS-Jurko library for sending SOAP requests:
from suds.client import Client
class WalrusSoap:
wsdl_url = 'https://s3.amazonaws.com/doc/2006-03-01/AmazonS3.wsdl'
server_url = 'https://localhost:8773/services/Walrus'
def __init__(self, access_key, secret_key):
self.access_key = access_key
self.secret_key = secret_key
self.client = Client(self.wsdl_url)
self.client.wsdl.services[0].setlocation(self.server_url)
#print self.client
def create_signature(self, operation, timestamp):
import base64, hmac, hashlib
h = hashlib.sha1(self.secret_key)
h.update("AmazonS3" + operation + timestamp)
#h = hmac.new(key=self.secret_key, msg="AmazonS3" + operation + timestamp, digestmod=hashlib.sha1)
return base64.encodestring(h.digest()).strip()
def download_file(self, bucket, filename):
from time import gmtime, strftime
timestamp = strftime('%Y-%m-%dT%H:%M:%S.001Z', gmtime())
print(timestamp)
signature = self.create_signature('ListAllMyBuckets', timestamp)
print(signature)
r = self.client.service.ListAllMyBuckets(self.access_key, timestamp, signature)
return r
w = WalrusSoap(access_key='MOBSE7FNS6OC5NYC75PG8', secret_key='yxYZmSLCg5Xw6rQVgoIuVLMAx3hZRlxDc0VOJqox')
r = w.download_file('mybucket', 'test.txt')
print(r)
I changed the server endpoint, because otherwise the WSDL points to the regular S3 servers at Amazon. I also have two different ways of creating the signature in my create_signature function. I was swapping between one and the other by simply commenting out the second one. Neither of the two seem to work. My question is what am I doing wrong?
SOAP Authentication: http://docs.aws.amazon.com/AmazonS3/latest/dev/SOAPAuthentication.html
SUDS-Jurko Documentation: https://bitbucket.org/jurko/suds/overview
Edit: I realized I forgot to include an example of what timestamp and signature is printed for debugging purposes.
2014-12-05T00:27:41.001Z
0h8vxE2+k10tetXZQJxXNnNUjjw=
Edit 2: Also, I know that the download_file function does not download a file :) I am still in testing/debug phase
Edit 3: I am aware that REST is better to use, at least according to Amazon. (Personally I think REST is better also.) I am also already aware that SOAP is deprecated by Amazon. However I would like to go down this path anyways, so please do me a favor and do not waste my time with links to the deprecation. I assure you that while writing this SOAP code, I was already well aware of the deprecation. In fact one of the links I posted has the deprecation notice printed at the top of its page. However, if you have evidence showing that Walrus completely ditches SOAP or that they stopped working on the SOAP portion, I would like to see something like that. But please do not tell me Amazon has deprecated SOAP.
The S3 SOAP API does not support "new" features so the REST API should be used where possible:
http://docs.aws.amazon.com/AmazonS3/latest/dev/SOAPAPI3.html
https://forums.aws.amazon.com/message.jspa?messageID=77821
IIRC recent versions of Eucalyptus do not support SOAP with S3.
That said, the signature looks good to me so I would check if the client/service hosts have the correct time, if there is a difference of more than 15 minutes authentication would fail.
You could also check the cloud-error.log on the Walrus service host as there may be more details on the failure there.
Eucalyptus does not support SOAP for S3 as of Eucalyptus version 4.0.0.
If you are using an older version of Eucalyptus (pre 4.0), then SOAP should work. Note, however, that the wsdl provided by S3 is not necessarily current or accurate for even their own service. S3 is notorious for changing the API without version or wsdl bumps, particularly since they stopped updating the SOAP API. So, there are likely responses from Walrus that do not conform to the published WSDL because our XML was updated based on the responses we see from S3 (via REST) and the SOAP and REST responses diverged. The signatures should be compatible, however, so worst case you would see 500 or 400 errors, but not 403 responses.
My recommendation is if you really want to learn the S3 SOAP API, you'll have to use S3 proper. The S3 SOAP support in Eucalyptus is there pre-4.0 but may not be compliant with the current state of S3 SOAP--we stopped testing against the S3 SOAP API directly when the AWS SDKs stopped using SOAP in favor of better REST support since that was the API moving forward.
Eucalyptus does support SOAP (see ZachH's comment about deprecation):
https://www.eucalyptus.com/docs/eucalyptus/4.0.2/schemas/aws-apis/s3/index.html
I decided to scrap using python, and I produced some working code with C# instead. Turns out C# has better SOAP libraries.
I've been trying to create content in my plone site using the Dexterity tool createContentInContainer.
I wrote a script that runs under my zopepy instance, and it accomplishes the following:
Selects data from a SQL table.
Creates a list of tuples that mirrors the custom content type defined in my product.
I know I'm extremely naive in my approach, but I've created a connection to the applications database by:
storage = FileStorage.FileStorage('.../var/filestorage/Data.fs')
db = DB(storage)
conn = db.open()
dbroot = conn.root()
I'm trying to create content by:
createContentInContainer(dbroot['Application']['myapp']['existingfolder'], portal_type, checkConstraints=False, content=item)
portal_type is previously set to my custom content type. item has been both the list of tuples passed to the content's interface (which throws a Could not adapt TypeError) as well as an unregistered adapter that inherits from the interface.
The interface for the type is registered in mysite.Widget.xml in profiles/defualt/types, but the script keeps throwing:
Traceback (most recent call last):
File "./bin/zopepy", line 345, in <module>
execfile(__file__)
File "importdex.py", line 105, in <module>
createContentInContainer(dbroot['Application']['myapp']['existingfolder'], portal_type, checkConstraints=False, content=item)
File "env/mysite/eggs/plone.dexterity-1.0-py2.7.egg/plone/dexterity/utils.py", line 149, in createContentInContainer
content = createContent(portal_type, **kw)
File "env/mysite/eggs/plone.dexterity-1.0-py2.7.egg/plone/dexterity/utils.py", line 105, in createContent
fti = getUtility(IDexterityFTI, name=portal_type)
File "env/mysite/eggs/zope.component-3.9.5-py2.7.egg/zope/component/_api.py", line 169, in getUtility
raise ComponentLookupError(interface, name)
zope.component.interfaces.ComponentLookupError: (<InterfaceClass plone.dexterity.interfaces.IDexterityFTI>, 'mysite.Widget')
As I've mentioned, I know I'm extremely naive in my approach, and I probably deserve a slap on the hand. I apologize if I've presented my question in a confusing manner.
My questions are these:
Can I instantiate createContentInContainer from zopepy? Is my rigged connection enough or does the script need to be run within the application to inherit stuff that Dexterity/FTI needs to accomplish what I'm asking?
Do I need an adapter? The one i have inherits from grok.Adapter and passes the interface to grok.provides and grok.context, but should it declare properties based on the entirety of the content schema?
The list of tuples is arbitrary. It just seemed like the thing to do given the structure of the ZODB. If I declare the schema of the content type as properties in a registered adapter, the data should be crafted to conform to attributes of an object (the adapter), right?
You need to set up a little more context for your code to work. The Plone site acts as a local component registry, for example.
You are also better off using the bin/instance run [scriptname] command, it'll set up the database connection for you and pass the root object as app to your script. In that script, use the following boilerplate to get the rest of the scaffolding up:
import transaction
from zope.app.component.hooks import setSite
from Testing.makerequest import makerequest
from AccessControl.SecurityManagement import newSecurityManager
plone_site_id = 'Plone' # Adjust as needed.
app = makerequest(app)
site = app[plone_site_id]
setSite(site)
user = app.acl_users.getUser('admin').__of__(site.acl_users)
newSecurityManager(None, user)
With these in place you'll have everything you need to run your code. Don't forget to call transaction.commit() at the end. Your Plone site is reachable in the local variable site.
I'm trying OpenID support for Google App Engine on a small project i have on my machine but when i call:
users.create_login_url(federated_identity = provider_url)
i get this error:
google_appengine/google/appengine/api/user_service_pb.py", line 178, in ByteSize
n += self.lengthString(len(self.destination_url_))
TypeError: object of type 'NoneType' has no len()
provider_url is https://www.google.com/accounts/o8/id
any clue?
You should normally pass a dest_url parameter to create_login_url, unless you're certain that there is a "current request" whose url you want to use instead. Apparently, the latter condition does not obtain, so the destination url stays None, which causes the problem you're observing. Passing an explicit dest_url should fix it.