Im having troubles running the following code:
from google.cloud import bigquery
client = bigquery.Client.from_service_account_json(BQJSONKEY,project = BQPROJECT)
dataset = client.dataset(BQDATASET)
assert not dataset.exists()
The following error pop up:
'DatasetReference' object has no attribute 'exists'
Similarly when i do:
table = dataset.table(BQTABLE)
i get: 'TableReference' object has no attribute 'exists'
However, according to the docs it should work:
https://googlecloudplatform.github.io/google-cloud-python/stable/bigquery/usage.html#datasets
here is my pip freeze (the part with google-cloud):
gapic-google-cloud-datastore-v1==0.15.3
gapic-google-cloud-error-reporting-v1beta1==0.15.3
gapic-google-cloud-logging-v2==0.91.3
gevent==1.2.2
glob2==0.5
gmpy2==2.0.8
google-api-core==0.1.1
google-auth==1.2.1
google-cloud==0.30.0
google-cloud-bigquery==0.28.0
google-cloud-bigtable==0.28.1
google-cloud-core==0.28.0
google-cloud-datastore==1.4.0
google-cloud-dns==0.28.0
google-cloud-error-reporting==0.28.0
google-cloud-firestore==0.28.0
google-cloud-language==1.0.0
google-cloud-logging==1.4.0
google-cloud-monitoring==0.28.0
google-cloud-pubsub==0.29.1
google-cloud-resource-manager==0.28.0
google-cloud-runtimeconfig==0.28.0
google-cloud-spanner==0.29.0
google-cloud-speech==0.30.0
google-cloud-storage==1.6.0
google-cloud-trace==0.16.0
google-cloud-translate==1.3.0
google-cloud-videointelligence==0.28.0
google-cloud-vision==0.28.0
google-gax==0.15.16
google-resumable-media==0.3.1
googleapis-common-protos==1.5.3
I wonder how can i fix it and make it work?
Not sure how you got to this docs but you should be using these as reference:
https://googlecloudplatform.github.io/google-cloud-python/latest/bigquery/usage.html#datasets
Code for 0.28 would be something like:
dataset_refence = client.dataset(BQDATASET)
dataset = client.get_dataset(dataset_reference)
assert dataset.created is not None
I think you forgot to create the dataset before calling exists()
dataset = client.dataset(BQDATASET)
dataset.create()
assert not dataset.exists()
Related
I'm currently developing locally an Azure function that communicates with Microsoft Sentinel, in order to fetch the alert rules from it, and more specifically their respective querys :
credentials = AzureCliCredential()
alert_rules_operations = SecurityInsights(credentials, SUBSCRIPTION_ID).alert_rules
list_alert_rules = alert_rules_operations.list(resource_group_name=os.getenv('RESOURCE_GROUP_NAME'), workspace_name=os.getenv('WORKSPACE_NAME'))
The issue is that when I'm looping over list_alert_rules, and try to see each rule's query, I get an error:
Exception: AttributeError: 'FusionAlertRule' object has no attribute 'query'.
Yet, when I check their type via the type() function:
list_alert_rules = alert_rules_operations.list(resource_group_name=os.getenv(
'RESOURCE_GROUP_NAME'), workspace_name=os.getenv('WORKSPACE_NAME'))
for rule in list_alert_rules:
print(type(rule))
##console: <class 'azure.mgmt.securityinsight.models._models_py3.ScheduledAlertRule'>
The weirder issue is that this error appears only when you don't print the attribute. Let me show you:
Print:
for rule in list_alert_rules:
query = rule.query
print('query', query)
##console: query YAY I GET WHAT I WANT
No print:
for rule in list_alert_rules:
query = rule.query
...
##console: Exception: AttributeError: 'FusionAlertRule' object has no attribute 'query'.
I posted the issue on the GitHub repo, but I'm not sure whether it's a package bug or a runtime issue. Has anyone ran into this kind of problems?
BTW I'm running Python 3.10.8
TIA!
EDIT:
I've tried using a map function, same issue:
def format_list(rule):
query = rule.query
# print('query', query)
# query = query.split('\n')
# query = list(filter(lambda line: "//" not in line, query))
# query = '\n'.join(query)
return rule
def main(mytimer: func.TimerRequest) -> None:
# results = fetch_missing_data()
credentials = AzureCliCredential()
alert_rules_operations = SecurityInsights(
credentials, SUBSCRIPTION_ID).alert_rules
list_alert_rules = alert_rules_operations.list(resource_group_name=os.getenv(
'RESOURCE_GROUP_NAME'), workspace_name=os.getenv('WORKSPACE_NAME'))
list_alert_rules = list(map(format_list, list_alert_rules))
I have tried with same as you used After I changed like below; I get the valid response.
# Management Plane - Alert Rules
alertRules = mgmt_client.alert_rules.list_by_resource_group('<ResourceGroup>')
for rule in alertRules:
# Try this
test.query = rule.query //Get the result
#print(rule)
if mytimer.past_due:
logging.info('The timer is past due!')
Instead of this
for rule in list_alert_rules:
query = rule.query
Try below
for rule in list_alert_rules:
# Try this
test.query = rule.query
Sorry for the late answer as I've been under tons of work these last few days.
Python has an excellent method called hasattr that checks if the object contains a specific key.
I've used it in the following way:
for rule in rules:
if hasattr(rule, 'query'):
...
The reason behind using this is because the method returns object of different classes, however inherited from the one same mother class.
Hope this helps.
I'm trying to setup stream-framework the one here not the newer getstream. I've setup the Redis server and the environment properly, the issue I'm facing is in creating the activities for a user.
I've been trying to create activities, following the documentation to add an activity but it gives me an error message as follows:
...
File "/Users/.../stream_framework/activity.py", line 110, in serialization_id
if self.object_id >= 10 ** 10 or self.verb.id >= 10 ** 3:
AttributeError: 'int' object has no attribute 'id'
Here is the code
from stream_framework.activity import Activity
from stream_framework.feeds.redis import RedisFeed
class PinFeed(RedisFeed):
key_format = 'feed:normal:%(user_id)s'
class UserPinFeed(PinFeed):
key_format = 'feed:user:%(user_id)s'
feed = UserPinFeed(13)
print(feed)
activity = Activity(
actor=13, # Thierry's user id
verb=1, # The id associated with the Pin verb
object=1, # The id of the newly created Pin object
)
feed.add(activity) # Error at this line
I think there is something missing in the documentation or maybe I'm doing something wrong. I'll be very grateful if anyone helps me get the stream framework working properly.
The documentation is inconsistent. The verb you pass to the activity should be (an instance of?*) a subclass of stream_framework.verbs.base.Verb. Check out this documentation page on custom verbs and the tests for this class.
The following should fix the error you posted:
from stream_framework.activity import Activity
from stream_framework.feeds.redis import RedisFeed
from stream_framework.verbs import register
from stream_framework.verbs.base import Verb
class PinFeed(RedisFeed):
key_format = 'feed:normal:%(user_id)s'
class UserPinFeed(PinFeed):
key_format = 'feed:user:%(user_id)s'
class Pin(Verb):
id = 5
infinitive = 'pin'
past_tense = 'pinned'
register(Pin)
feed = UserPinFeed(13)
activity = Activity(
actor=13,
verb=Pin,
object=1,
)
feed.add(activity)
I quickly looked over the code for Activity and it looks like passing ints for actor and object should work. However, it is possible that these parameters are also outdated in the documentation.
* The tests pass in classes as verb. However, the Verb base class has the methods serialize and __str__ that can only be meaningfully invoked if you have an object of this class. So I'm still unsure which is required here. It seems like in the current state, the framework never calls these methods, so classes still work, but I feel like the author originally intended to pass instances.
With the help of great answer by #He3lixxx, I was able to solve it partially. As the package is no more maintained, the package installs the latest Redis client for python which was creating too many issues so by installation redis-2.10.5 if using stream-framework-1.3.7, should fix the issue.
I would also like to add a complete guide to properly add activity to a user feed.
Key points:
If you are not using feed manager, then make sure to first insert the activity before you add it to the user with feed.insert_activity(activity) method.
In case of getting feeds with feed[:] throws an error something like below:
File "/Users/.../stream_framework/activity.py", line 44, in get_hydrated
activity = activities[int(self.serialization_id)]
KeyError: 16223026351730000000001005L
then you need to clear data for that user using the key format for it in my case the key is feed:user:13 for user 13, delete it with DEL feed:user:13, In case if that doesn't fix the issue then you can FLUSHALL which will delete everything from Redis.
Sample code:
from stream_framework.activity import Activity
from stream_framework.feeds.redis import RedisFeed
from stream_framework.verbs import register
from stream_framework.verbs.base import Verb
class PinFeed(RedisFeed):
key_format = 'feed:normal:%(user_id)s'
class UserPinFeed(PinFeed):
key_format = 'feed:user:%(user_id)s'
class Pin(Verb):
id = 5
infinitive = 'pin'
past_tense = 'pinned'
register(Pin)
feed = UserPinFeed(13)
print(feed[:])
activity = Activity(
actor=13,
verb=Pin,
object=1)
feed.insert_activity(activity)
activity_id = feed.add(activity)
print(activity_id)
print(feed[:])
I am using python to call functions in the software OpenLCA and processing the results outside of openLCA. openLCA provides an implementation of an JSON-RPC based protocol for inter-process communication (IPC). However, I get the following error when I am trying to change the value of the Test_Parameter to 2000: 'NoneType' object has no attribute 'append'.
I am following the exact instructions as mentioned here: https://github.com/GreenDelta/olca-ipc.py
this is my code
import olca
import uuid
client = olca.Client(8080)
setup = olca.CalculationSetup()
redef = olca.ParameterRedef()
redef.name = 'Test_Parameter'
redef.value = 2000
setup.parameter_redefs.append(redef)
setup.calculation_type = olca.CalculationType.SIMPLE_CALCULATION
setup.impact_method = client.find(olca.ImpactMethod, 'ReCiPe Midpoint (H)')
setup.product_system = client.find(olca.ProductSystem, 'Test_Process')
setup.amount = 1.0
result = client.calculate(setup)
client.excel_export(result, 'hellotest.xlsx')
client.dispose(result)
any help would be very appreciated.
many many thanks
https://github.com/GreenDelta/olca-ipc.py/blob/21effee30c41a99eb18f6b348f87cff53c23ea8c/olca/schema.py#L206
It seems the client.parameter_redefs is initialized to None. I didn't see a defined way of initializing it through the class so maybe you're expected to do it directly?
setup = olca.CalculationSetup()
setup.parameter_redefs = []
I've been asked to translate some words, and I'm using Python to do it. Yandex has an API that is supposed to be used withing Python, documentation here :
https://pypi.org/project/yandex-translater/1.0/
I followed the steps, but Always get the same error that seems to be withing the API, or maybe I'm not setting Something right in my code.
The code goes as follow :
from yandex import Translater
tr = Translater()
tr.set_key('my API key not given here')
tr.set_text("Hello World")
tr.set_from_lang('en')
tr.set_to_lang('fr')
result = tr.translate()
print(result)
I then get this error :
File "C:\Users\BMQT\Desktop\Scraping\test.py", line 2, in <module>
tr = Translater()
File "C:\Program Files\Python37\lib\site-packages\yandex\Translater.py", line 23, in __init__
self.default_ui = locale.getlocale()[0].split('_')[0]
AttributeError: 'NoneType' object has no attribute 'split'
A quick look if you need in the translater.py goes as follow for line 23 :
self.default_ui = locale.getlocale()[0].split('_')[0]
Is the API broken or am I wrong in my code? Thanks for the answers!
I've used another api module called yandex_translate, and it works fine.
from yandex_translate import YandexTranslate
translate = YandexTranslate('mykey')
traduction =('Translate:', translate.translate('bonjour', 'fr-ar'))
print(traduction)
Don't know what was wrong with the previous one, maybe outdated.
translater object need to be created like this: tr = Translater.Translater()
from yandex import Translater
tr = Translater.Translater()
tr.set_key('my API key not given here')
tr.set_text("Hello World")
tr.set_from_lang('en')
tr.set_to_lang('fr')
result = tr.translate()
print(result)
I am baffled with the following problem.
I am using Flask, flask-pymongo extension, mongodb version v2.2.0-rc0, pdfile version 4.5
This is my route:
#app.route("/check/<id>")
def check(id):
doc=conn.db.msg.find_one({'_id':id})
return render_template('base.html',name=doc)
the id is a valid _id from a document in the msg collection, but ALWAYS return None.
I have tried:
pasing the ObjectId(id) but returns errortype: ObjectId not callable
pasing the id as str(id) returns None
Any thoughts?
UPDATE:
this how the full url looks like:
http://127.0.0.1:8000/check/5030b628895b800de1a9a792
UPDATE2:
I found a similar question with (answer) for ruby. Not sure how I can translate it to python, what sort imports/modules do I need?
How can I retrieve a document by _id?
UPDATE3:
I tried:
import bson
#app.route("/check/<id>")
def check(id):
id2="'"+id+"'"
doc=conn.db.msg.find_one({'_id':bson.objectid(id2) })
return render_template('base.html',name=doc)
but I get TypeError: 'module' object is not callable (it doesnt work with id either)
when I reached 1500 I will suggest a frustation tag :-S
UPDATE4:
Finally I got it up & running!
here it is my solution:
import bson
#app.route("/check/<id>")
def check(id):
doc=conn.db.msg.find_one({'_id':bson.ObjectId(oid=str(id))})
return render_template('base.html',name=doc)
You might also want to try using ObjectId from the bson.objectid module, like so:
from bson.objectid import ObjectId
In that case, you won't need to provide the oid kwarg. You'll just do something like this:
db_conn.msg.find_one({'_id': ObjectId(my_oid)})