I am trying to read a geojson file and insert the records into a postgres table - using the below python code.
import json
import psycopg2
conn = psycopg2.connect(host="<<ip_address>>",database="DB1", user="<<id>>", password="pwd")
cur = conn.cursor()
with open('NTA_shape.json') as f:
Geojson_data = json.load(f)
for feature in Geojson_data['features']:
type_val=feature['geometry']['type']
geom=feature['geometry']['coordinates']
ntaname=feature['properties']['NTAName']
boroname=feature['properties']['BoroName']
data = {"type":type_val,"coordinates":geom}
sql ="""Insert into <<Table_NAME> (geom,ntaname,boroname) VALUES(ST_GeomFromGeoJSON(%s),%s,%s)"""
nta_boro=(json.dumps(data),ntaname,boroname)
cur.execute(sql,nta_boro)
conn.commit()
conn.close()
But when I query the table, lot of records are missing.
If I print the json.dumps(data) variable - its showing all records.
I am not sure, what am i missing during table insert
Kindly help.
I was able to fix with below change
nta_boro=(json.dumps(data,),ntaname,boroname)
Related
I am new to working with SQL and Postgres specifically and am trying to write a simple program that stores a course id and some URLs in an SQL table with two columns. I am using the psycopg2 python library.
I am able to read from the table using:
def get_course_urls(course):
con = open_db_connection()
cur = con.cursor()
query = f"SELECT urls FROM courses WHERE course = '{course}'"
cur.execute(query)
rows = cur.fetchall()
cur.close()
close_db_connection(con)
urls = []
for url in rows:
urls.extend(url[0])
return urls
However, I am unable to insert into the table using:
def format_urls_string(urls):
return '{"' + '","'.join(urls) + '"}'
def add_course_urls(course, urls):
con = open_db_connection()
cur = con.cursor()
query = f"INSERT INTO courses (course, urls) VALUES ('{course}', '{format_urls_string(urls)}');"
print(query)
cur.execute(query)
cur.close()
close_db_connection(con)
add_course_urls("CS136", ["http://google.com", "http://wikipedia.com"])
I do not think anything is wrong with my query because when I run the same query in the SQL Shell it works as I want it to.
The locks on the columns say that the columns are READ-ONLY, however, I am able to insert through the shell. I feel like this is a very minor fix but since I am new to PostgreSQL, I am having some trouble.
Your help is appreciated!
This is the danger of doing the substitution yourself, instead of letting the db connector do it. You looked at your string, yes? You're writing
... VALUES ('CS136', '['http://google.com','http://wikipedia.com']')
which is obviously the wrong syntax. It needs to be
... VALUES ('CS136', '{"http://google.com","http://wikipedia.com"}')
which Python's formatter won't generate. So, you can either format the insertion string by hand, or put placeholders and pass the parameters to the cursor.execute call:
query = "INSERT INTO courses (course, urls) VALUES (%s,%s);"
cur.execute( query, (course, urls) )
How can we fetch details of particular row from mysql database using variable in python?
I want to print the details of particular row using variable from my database and I think I should use something like this:
data = cur.execute("SELECT * FROM loginproject.Pro WHERE Username = '%s';"% rob)
But this is showing only the index value, not the data. Please help me out.
after executing the query using cur.execute(query), you need to call fetchall function for getting data from cursor e.g.:
data = cur.fetchall()
Something like this?
cur = db.cursor()
cur.execute("SELECT * FROM loginproject.Pro WHERE Username = '%s';"% rob)
result = cur.fetchall()
for row in result:
print(row[0])
https://github.com/prosaigon/Python-Connector-Mysql
python3.6
library: pip install mysqlclient
use cursor.fetchall and cursor.description is your case
cursor.fetchall():
This will provide you all the rows fetched in a tuple format.
cursor.description:
This will provide you the list of columns in the query
And the one will go like :
cur.execute("SELECT * FROM loginproject.Pro WHERE Username = '%s';"% rob)
descriptor = cur.description
result = cur.fetchall()
I have a database named "sina2013",and the columus is Title,Content
Now I want to use pymssql module to get the data.At the same time ,using the Title as the filename of a txt file,the Content as the content of the txt file.
The strange thing is the number of files is less than the items in database.
where is the error?
the code i have tried is:
import pymssql
conn = pymssql.connect(...)
cur = conn.cursor()
cur.execute('SELECT Title,Content FROM sina2013')
count=len(cur.fetchall()) #Will return the right number :5913
for Title,Content in cur:
filename=file(str(Title)+r'.txt',r'w')
filename.write(Content )
filename.close()
cur.close()
The number of txt file is less than it should be.
what is the reason?
Perhaps changing your for loop into this:
# cursor fetchall() method returns all rows from a query
for Title,Content in cur.fetchall():
... will fix the issue?
So I have a mysql table and I am trying to take each element from one of the fields of the table. The field and table are both called "keywords". In the field there are many different random words and I am trying to take all of those and save them to a text file. Any help on how to implement this would be great, here is what I have so far.
#!/usr/bin/env python
import MySQLdb
db = MySQLdb.connect(host="", user="", passwd="", db="")
cursor = db.cursor()
sql = """SELECT DISTINCT keywords FROM keywords"""
cursor.execute(sql)
cursor.fetchall()
db.close()
for s in sql:
tweets = open("keywords.txt", "w")
What I was thinking is to turn what sql fetches into a list if possible and write that to the file. But I am open to any suggestions, thanks.
Something like this should work:
import MySQLdb
db = MySQLdb.connect(host="", user="", passwd="", db="")
cursor = db.cursor()
sql = """SELECT DISTINCT keywords FROM keywords"""
tweets = open("keywords.txt", "w")
cursor.execute(sql)
for row in cursor:
print>>tweets, row[0]
tweets.close()
db.close()
I have an access table that I am trying to add fields programmatically using Python. It is not a personal geodatabase. Just a standard Access database with some tables in it.
I have been able to access the table and get the list of field names and data types.
How do I add a new field and assign the data type to this Access table using Python.
Thanks!
SRP
Using the pyodbc module:
import pyodbc
MDB = 'c:/path/to/my.mdb'
DRV = '{Microsoft Access Driver (*.mdb)}'
PWD = 'my_password'
conn = pyodbc.connect('DRIVER=%s;DBQ=%s;PWD=%s' % (DRV,MDB,PWD))
c = conn.cursor()
c.execute("ALTER TABLE my_table ADD COLUMN my_column INTEGER;")
conn.commit()
c.close()
conn.close()
Edit:
Using win32com.client...
import win32com.client
conn = win32com.client.Dispatch(r'ADODB.Connection')
DSN = 'PROVIDER=Microsoft.Jet.OLEDB.4.0;DATA SOURCE=c:/path/to/my.mdb;'
conn.Open(DSN)
conn.Execute("ALTER TABLE my_table ADD COLUMN my_column INTEGER;")
conn.Close()