Import .json to Database - python

I successfully downloaded tweets into a json file. Now I try to import it in a database with this function:
def import_json(fi):
logging.warning("Loading tweets from json file {0}".format(fi))
for line in open(fi, "rb"):
data = json.loads(line.decode('utf-8'))
database.create_tweet_from_dict(data)
the json-file "keywords_BVBS04.json" lays in a folder called data which is in the current directory. The function is in a file called BVBS04.py
to start the import I type BVBS04.import_json(keywords_BVBS04.json) in ipython in the console. this is what I get back:
NameError Traceback (most recent call
last) in ()
----> 1 BVBS04.import_json(keywords_BVBS04.json)
NameError: name 'keywords_BVBS04' is not defined
Now here comes the is a beginner's question: Where/how do I have to define "keywords_BVBS04"? I tried a lot :(
Thanks!

This is what you want,
1) you need to import the function from the script, not use dot-notation on the script.
2) Quote the filename.
>>> from BVBS04 import import_json
>>> import_json("keywords_BVBS04.json")
Good luck with the rest of things

Related

How can I run a script after importing it?

Problem
I would like to import a script containing many functions and then run them, so that I can use the function. I may have misunderstood the purpose of import. I am working in Jupyter.
Reprex
#Create the script in a local folder
%%writefile test.py
c = 500
def addup(a,b,c):
return a*b + (c)
#import the file and use it
import test
addup(1,5,c)
#Error message
---------------------------------------------------------------------------
# NameError Traceback (most recent call last)
# <ipython-input-1-71cb0c70c39d> in <module>
# 1 import test
# ----> 2 addup(1,5,c)
# NameError: name 'addup' is not defined
Any help appreciated.
You have not called the function! You need a dot . to call a function from a module.
This is the correct syntax:
import test
result = test.addup(1,5,c)
Import a specific function:
from test import addup
addup(1,5,c)
Importing all of the module's functions:
from test import *
addup(1,5,c) # this way you can use any function from test.py without the need to put a dot

Python: Function: not recognizing dependencies

I am quite new to writing modules in Python.
I use Python 3.5
I have a script called describeToolbox.py that contain functions that I would like to be able to call, like this one:
#describeToolbox.py
import shelve
def getRawData(prefix):
shelfFile = shelve.open('data'+prefix)
df = shelfFile['data'+prefix]
shelfFile.close()
return df
This is meant to retrieve a dataFrame from a shelve file
In my console now, I write the following statements:
In [7]:import shelve
import describeToolbox as TB
In [8]:TB.getRawData('Myprefix')
Out [8]:
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-9-67160af666cc> in <module>()
----> 1 TB.getRawData('Myprefix')
C:\Users\Math\Documents\Docs\Commos\Notebooks\describeToolbox.py in getRawData(prefix)
1 def getRawData(prefix):
----> 2 shelfFile = shelve.open('data'+prefix)
3 df = shelfFile['data'+prefix]
4 shelfFile.close()
5 return df
NameError: name 'shelve' is not defined
It gives me an error message saying that the module 'shelve', the dependency, is not defined.
Basically I dont know where is the correct place to import all the dependencies so that my function can load them when I want to import it.
I would like to write a depository of functions I use often in one module and call them when needed.
Thank you for your help!

Import Error using cPickle in Python

I am using Pickle in Python2.7. I am getting error while using cPickle.load() method. The code and error is shown below. Can someone guide me through this?
Code:
#! usr/bin/python
import cPickle
fo = open('result','rb')
dict1 = cPickle.load(fo)
Error:
Traceback (most recent call last):
File "C:\Python27\test.py", line 7, in <module>
dicts = cPickle.load(fo)
ImportError: No module named options
It seems like you can not do
import options
but when you or someone else did
cpickle.dump(xxx, open('result', 'rb'))
there was an object with a class or function of a module options that existed at this point in time, in xxx.
Solution
You can open the file binarily and replace options with the module you replaced the old module options with.
You probably created the file in your package like in module package.main by executing the file main.py or something like it, having a module options in the same directory.
Now you do import package.main, try to read the file and options is now called package.options and the module options can not be found.
How did you create this file? How do you load it now? cPickle/pickle does not transfer source code - so if you use a function you need the module when you load it.

Fixed: Python NameError, fixed AttributeError and got this?

FIXED: turns out there is a module already called parser. Renamed it and its working fine! Thanks all.
I got a python NameError I can't figure out, got it after AttributeError. I've tried what I know, can't come up with anything.
main.py:
from random import *
from xml.dom import minidom
import parser
from parser import *
print("+---+ Roleplay Stat Reader +---+")
print("Load previous DAT file, or create new one (new/load file)")
IN=input()
splt = IN.split(' ')
if splt[0]=="new":
xmlwrite(splt[1])
else:
if len(splt[1])<2:
print("err")
else:
xmlread(splt[1])
ex=input("Press ENTER to Exit...")
parser.py:
from xml.dom import minidom
from random import *
def xmlread(doc):
xmldoc = minidom.parse(doc)
itemlist = xmldoc.getElementsByTagName('item')
for s in itemlist:
print(s.attributes['name'].value,":",s.attributes['value'].value)
def xmlwrite(doc):
print("no")
And no matter what I get the error:
Traceback (most recent call last):
File "K:\Python Programs\Stat Reader\main.py", line 10, in <module>
xmlwrite.xmlwrite(splt[1])
NameError: name 'xmlread' is not defined
The same error occurs when trying to access xmlwrite.
When I change xmlread and xmlwrite to parser.xmlread and parser.xmlwrite I get:
Traceback (most recent call last):
File "K:\Python Programs\Stat Reader\main.py", line 15, in <module>
parser.xmlread(splt[1])
AttributeError: 'module' object has no attribute 'xmlread'
The drive is K:\ because it's my personal drive at my school.
If your file is really called parser.xml, that's your problem. It needs to be parser.py in order to work.
EDIT: Okay, since that wasn't your issue, it looks like you have a namespacing issue. You import your parser module twice when you use import parser and then from parser import *. The first form of it makes "parser" the namespace and the second form directly imports it, so in theory, you should have both parser.xmlwrite and xmlwrite in scope. It's also clearly not useful to import minidom in main.py since you don't use any minidom functionality in there.
If you clear up those and still have the issue, I would suggest looking at __ init __.py. If that still does nothing, it could just plain be a conflict with Python's parser module, you could substitute a name like myxmlparser.

Writing and importing custom modules/classes

I've got a class that I'm trying to write called dbObject and I'm trying to import it from a script in a different folder. My structure is as follows:
/var/www/html/py/testobj.py
/var/www/html/py/obj/dbObject.py
/var/www/html/py/obj/__init__.py
Now, __init__.py is an empty file. Here are the contents of dbObject.py:
class dbObject:
def __init__():
print "Constructor?"
def test():
print "Testing"
And here's the contents of testobj.py:
#!/usr/bin/python
import sys
sys.path.append("/var/www/html/py")
import obj.dbObject
db = dbObject()
When I run this, I get:
Traceback (most recent call last):
File "testobj.py", line 7, in <module>
db = dbObject()
NameError: name 'dbObject' is not defined
I'm new to Python, so I'm very confused as to what I'm doing wrong. Could someone please point me in the right direction?
EDIT: Thanks to Martijn Pieters' answer I modified my testobj.py as follows:
#!/usr/bin/python
import sys
sys.path.append("/var/www/html/py")
sys.path.append("/var/www/html/py/dev")
from obj.dbObject import dbObject
db = dbObject()
However, now when I run it I get this error:
Traceback (most recent call last):
File "testobj.py", line 7, in <module>
db = dbObject()
TypeError: __init__() takes no arguments (1 given)
Is this referring to my init.py or the constructor within dbObject?
EDIT(2): Solved that one myself, the constructor must be able to take at least one parameter - a reference to itself. Simple fix. Looks like this problem is solved!
EDIT (Final): This is nice - I can cut out the import sys and sys.path.append lines and it still works in this instance. Lovely.
You need to import the class from the module:
from obj.dbObject import dbObject
This adds the class dbObject directly to your local namespace.
Your statement import obj.dbObject adds the name obj to the local namespace, so you could also do this instead:
db = obj.dbObject.dbObject()
because obj.dbObject is the dbObject.py module in your obj package directory.

Categories

Resources