I am trying to apply send_active_mq on each row of an rdd.
def send_to_active_mq(json_string) :
k = str(json_string)
conn.send(body=k,destination='dwEmailsQueue2')
json_rdd_to_send.foreach(send_to_active_mq)
I applied the same way as suggested in the spark documentation
http://spark.apache.org/docs/1.1.1/api/python/pyspark.rdd.RDD-class.html#foreach
But I am getting the following error.
AttributeError: 'builtin_function_or_method' object has no attribute '__code__'
Is there something fundamentally wrong I am doing here.
This is probably connected to conn object. Try:
def send_to_active_mq(json_strings):
conn = ... # Initalize connection
for json_string in json_strings:
conn.send(body=str(json_string) ,destination='dwEmailsQueue2')
json_rdd_to_send.foreachPartition(send_to_active_mq)
Related
So I have a pandas dataframe (dfm) from which I have calculated new values from based on user input. I want to place these update/changed values into the database. I feel like I should just be able to session commit, and then I think I need to use update since the data already exists.
def report_maker(report_id):
# other code ....
if dfm['report'][r]==0:
db.session.add(report_a)
# the above works just fine.
else:
report_b= db.session.query(Standard).filter(Standard.report_id==report_id, Standard.code == dfm['code'][r])
report_b.code=dfm['code'][r]
report_b.name=dfm['name'][r]
report_b.standard=dfm['standard'][r]
report_b.level=dfm['level'][r]
report_b.report_id=report_id
db.session.update(report_b) #### the error is right here .!.!
try:
db.session.commit()
except:
print(report_a.name)
The error is
AttributeError: 'scoped_session' object has no attribute 'update'
The update() method is an attribute of the Query Object not the Session Object. If you want to update multiple rows you can pass a dictionary representation of all columns that need to be updated:
db.session.query(Foo).filter(Foo.foo == foo).update(values={'bar': bar})
If you are filtering for a unique primary_key, you could also consider to update the Row Object
foo = db.session.query(Foo).filter(Foo.foo = foo).first()
foo.bar = bar
db.session.commit()
For further reading I would recommend SQLAlchemy documention.
Thanks in advance.
When I try to display tree data from sql server, I am getting the Error:'str' object has no attribute 'children' Please help me to fix the issue.
My Code :
class Tree:
def __init__(self,name='root',children=None):
self.name=name
#self.Tid=Tid
self.children=[]
if(children):
for child in children:
self.add_child(child)
def add_child(self,node):
assert(isinstance(node,Tree))
self.children.append(node)
def __repr__(self):
return self.name
from anytree import Node, RenderTree
from anytree import ContStyle
import pyodbc
try:
cnxn = pyodbc.connect("DSN=****;database=****;trusted_connection=yes;user=****;password=****")
cursor = cnxn.cursor()
cursor.execute("select * from dbo.UserTree where Relation='self' order by Sno")
row=cursor.fetchone()
t=row[6]
print(RenderTree(t))
except Exception as e:
print("Error : ",e)
finally:
cursor.close()
cnxn.close()
FYI : output of above SQL SELECT stmt :Tree('*',[Tree('ElectronicItems',[Tree('Mobile'),Tree('TV'),Tree('Computer'),Tree('Camera'),Tree('Refrigerator')])])
Its working fine when I harcoded the value inside a variable.like this.,
t=Tree('*',[Tree('ElectronicItems',[Tree('Mobile'),Tree('TV'),Tree('Computer'),Tree('Camera'),Tree('Refrigerator')])])
print(RenderTree(t))
but throwing error when I dynamically pass the value from sql server.
If I understand you correctly, the value you are getting back from the sql statement is a string. To evaluate this string, you could use eval to interpret it during runtime:
t=eval(row[6])
Thanks Christian. it works perfect !!, Thank you Karl !!
t=eval(row[6])
I'm trying to set noCursorTimeout in the session following this instruction from MongoDB docs.
But I'm getting the error: AttributeError: 'Cursor' object has no attribute 'noCursorTimeout'
I can't wonder why I'm getting this.
from pymongo import MongoClient as Connection
from datetime import datetime
conn = Connection(get_uri())
with conn as conn:
collection = conn['db_name']['my_collection']
documents_cursor = collection.find(query).noCursorTimeout() # same with .maxTimeMS()
According to the document, you should use it like this:
cursor = collection.find({"a": 1}, no_cursor_timeout=True)
Reference: https://api.mongodb.com/python/current/migrate-to-pymongo3.html#timeout-replaced-by-no-cursor-timeout
I am new to Python and am trying to retrieve data from a MariaDB. Using the example found on the official documentation and tweaking with my own columns like this returns the following error:
AttributeError: 'NoneType' object has no attribute 'format'
Heres what my code looks like:
cursor.execute("SELECT id, name FROM playlists")
for id, name in cursor:
print("ID: {}, Name: {}").format(id,name)
Can someone explain why I'm getting this error? and whether I'll need to declare each column name in the for loop? (where it says for id, name in cursor
There's two issues here. one you need to save the output of the cursor.execute to a variable.
Second, you need to loop through the response. See below for an example:
import mysql.connector
from mysql.connector.cursor import MySQLCursor
db = mysql.connector.connect(option_files='my.conf', use_pure=True)
cursor = db.cursor()
cursor.execute("SELECT id, name FROM playlists")
# iterate over result
for row in cursor:
print("ID: {}, Name: {}").format(row[0],row[1])
cursor.close()
db.close()
I'm working with Python and JayDeBeApi to connect to a Oracle-type database.
In the SELECT's statements I need to get about 10+ thousand of records.
In the first time I done by using the "fetchAll()" method, but this loads my memory and I wouldn't like to this to happen.
I get the cursor by using the code below:
def do_select(sql, db_conn):
resultSet = None
try:
cursor = db_conn.cursor()
cursor.execute(sql)
resultSet = {
"cursor": cursor,
"columns": cursor.description
}
except Exception as error:
print("An error occurred" + str(error))
return resultSet
And instead of using this type of code:
resultSet = self.do_select(sql, self.get_db_conn())
rows = resultSet["cursor"].fetchAll()
for row in rows:
# Do something...
I would like to do something like this:
resultSet = self.do_select(sql, self.get_db_conn())
while resultSet.next():
entire_row_tuple = resultSet.getCurrent() #I don't know if this is possible in python
#Do something with entire_row_tuple...
Is this possible in python? Or, does exists a better way instead of using "fetchAll()" method?
Thank you