here I try to put a dictionary in an SQLite database
data1 = {
'x': x,
'y': y,
'width': width,
'height':height,
'proba': proba,
'class': label,
'id_label': id_label
}
sqliteConnection = sqlite3.connect('SQL_bdd.db')
cursor = sqliteConnection.cursor()
sqliteConnection.execute('''CREATE TABLE dic (
x INT NOT NULL,
y INT NOT NULL,
width INT NOT NULL,
height INT NOT NULL,
proba BOOLEAN NOT NULL,
class VARCHAR (255) NOT NULL,
id_label INT NOT NULL
);''')
cursor.execute('INSERT INTO dic VALUES (?,?,?,?,?,?,?)', [dict['x'], dict['y'], dict['width'], dict['height'], dict['proba'], dict['class'], dict['id_label']]);
cursor.execute("SELECT * FROM dic")
the following error this occurs and I don't know how to fix it
cursor.execute('INSERT INTO dic VALUES (?,?,?,?,?,?,?)', [dict['x'], dict['y'], dict['width'], dict['height'], dict['proba'], dict['class'], dict['id_label']]);
TypeError: 'type' object is not subscriptable
Use data1['x'], data1['y'], etc. instead of dict['x'], dict['y'], etc. to index the data in your data1 dictionary.
Related
I have a txt file below with different column names, I want to extract all the column names without the data types etc. from the table into the pandas dataframe.
create table ad.TrackData
(
track_id int unsigned auto_increment primary key,
ads_id int null,
ads_name varchar(45) null,
play_time timestamp null,
package_id int null,
package_name varchar(45) null,
company_id int null,
company_name varchar(45) null,
click_time timestamp null,
demographic varchar(300) null,
status tinyint(1) default 0 null
);
I have no idea how I am supposed to do this, it would be very much appreciated if anyone could teach me some ways to perform this.
First, you must save that text to file.txt. Then run a python script below:
with open('file.txt', 'r') as f:
txt = f.read()
lines = txt.split('\n')
columnNames = []
for line in lines:
if line.startswith(' '):
columnNames.append(line.split(' ')[1])
print(columnNames)
the output must be :
['track_id', 'ads_id', 'ads_name', 'play_time', 'package_id', 'package_name', 'company_id', 'company_name', 'click_time', 'demographic', 'status']
dict = {
'Customer': {
'token': 001,
'timestamp': datetime.datetime(2021, 7, 26, 11, 20, 40),
'last_visit_time': datetime.datetime(2021, 7,24, 11, 20, 40),
'purchase_amount': 1000,
'quantity': 4,
'items': {
'a': 200,
'b': 300,
'c': 250,
'd': 250
}
}
}
how to create table for this in sqlite python and store it in sql db
Your database structure is pretty straightforward. Simply add the higher level keys to one table and create a separate table for Item and make it a one-to-one relationship. I have attached the database diagram below.
If you want the sql query, then
CREATE TABLE `Customer` (
`token` int PRIMARY KEY,
`timestamp` timestamp,
`last_visited_time` timestamp,
`purchase_amount` int,
`quantity` int,
`item` Items
);
CREATE TABLE `Items` (
`id` int PRIMARY KEY AUTO_INCREMENT,
`a` int,
`b` int,
`c` int,
`d` int
);
ALTER TABLE `Items` ADD FOREIGN KEY (`id`) REFERENCES `Customer` (`item`);
I'm trying to get the latest bitcoin price and save it in my database. I keep getting the error NameError: name 'price_usd' is not defined when I execute my python script:
getdata.py
import requests
import urllib
import json
import pymysql
con = pymysql.connect(host = 'localhost',user = 'dbuser',passwd = 'dbpass',db = 'bitcoinprice')
cursor = con.cursor()
url = 'example.com'
urllib.urlopen(url).read()
response = urllib.urlopen(url).read()
print(response)
json_obj = str(response)
cursor.execute("INSERT INTO bitcoinprice (list_price_usd) VALUES (%s)", (price_usd))
con.commit()
con.close()
print (json_obj)
Returned JSON from API
[
{
"id": "bitcoin",
"name": "Bitcoin",
"symbol": "BTC",
"rank": "1",
"price_usd": "11117.3",
"price_btc": "1.0",
"24h_volume_usd": "9729550000.0",
"market_cap_usd": "187080534738",
"available_supply": "16827875.0",
"total_supply": "16827875.0",
"max_supply": "21000000.0",
"percent_change_1h": "0.09",
"percent_change_24h": "-0.9",
"percent_change_7d": "-4.32",
"last_updated": "1516991668"
}
]
Schema
CREATE TABLE `bitcoinprice` (
`list_id` varchar(7) CHARACTER SET utf8 DEFAULT NULL,
`list_name` varchar(7) CHARACTER SET utf8 DEFAULT NULL,
`list_symbol` varchar(3) CHARACTER SET utf8 DEFAULT NULL,
`list_rank` int(11) DEFAULT NULL,
`list_price_usd` decimal(7,6) DEFAULT NULL,
`list_price_btc` decimal(9,8) DEFAULT NULL,
`list_24h_volume_usd` decimal(10,1) DEFAULT NULL,
`list_market_cap_usd` decimal(12,1) DEFAULT NULL,
`list_available_supply` decimal(12,1) DEFAULT NULL,
`list_total_supply` bigint(20) DEFAULT NULL,
`list_max_supply` int(11) DEFAULT NULL,
`list_percent_change_1h` decimal(2,1) DEFAULT NULL,
`list_percent_change_24h` decimal(3,2) DEFAULT NULL,
`list_percent_change_7d` decimal(3,1) DEFAULT NULL,
`list_last_updated` int(11) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Assuming your "returned json from api" is correct:
Replace
cursor.execute("INSERT INTO bitcoinprice (list_price_usd) VALUES (%s)", (price_usd))
With
cursor.execute("INSERT INTO bitcoinprice (list_price_usd) VALUES (%s)",
(json.loads(json_obj)[0]['price_usd']))
For whatever reason, you seem to have imported the json module (the solution to your problem) without actually using it.
json.loads converts a json string into a python object, which in your case is a list containing one value, a dict with the data you want. [0] gets the dictionary from the list, and ['price_usd'] gets the value you were expecting to be stored in a variable named price_usd from the dict.
SOLVED
sql = "SELECT Title, Year, TmdbId, MovieFileId, Quality " \
"FROM Movies " \
"INNER JOIN MovieFiles on MovieFiles.Id = Movies.MovieFileId " \
"WHERE Quality = 7"
sql = "SELECT Title, Year, TmdbId, MovieFileId, MovieFiles.Quality " \
"FROM Movies " \
"INNER JOIN MovieFiles on MovieFiles.Id = Movies.MovieFileId " \
"WHERE MovieFiles.Quality = 7"
Neither of these work for me, I am getting no rows returned. Quality is part of the MovieFiles table. And Yes, rows exist that meet this criteria.
sql = "SELECT Title, Year, TmdbId, MovieFileId, Quality " \
"FROM Movies " \
"INNER JOIN MovieFiles on MovieFiles.Id = Movies.MovieFileId "
This works just fine, and returns the correct quality. But, if I add the where, it does not work and just returns nothing. Am I doing something wrong? (Using python)
Edit: Schema request
CREATE TABLE "Movies" ("Id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, "ImdbId" TEXT, "Title" TEXT NOT NULL, "TitleSlug" TEXT NOT NULL, "SortTitle" TEXT, "CleanTitle" TEXT NOT NULL, "Status" INTEGER NOT NULL, "Overview" TEXT, "Images" TEXT NOT NULL, "Path" TEXT NOT NULL, "Monitored" INTEGER NOT NULL, "ProfileId" INTEGER NOT NULL, "LastInfoSync" DATETIME, "LastDiskSync" DATETIME, "Runtime" INTEGER NOT NULL, "InCinemas" DATETIME, "Year" INTEGER, "Added" DATETIME, "Actors" TEXT, "Ratings" TEXT, "Genres" TEXT, "Tags" TEXT, "Certification" TEXT, "AddOptions" TEXT, "MovieFileId" INTEGER NOT NULL, "TmdbId" INTEGER NOT NULL, "Website" TEXT, "PhysicalRelease" DATETIME, "YouTubeTrailerId" TEXT, "Studio" TEXT, "MinimumAvailability" INTEGER NOT NULL, "HasPreDBEntry" INTEGER NOT NULL, "PathState" INTEGER NOT NULL, "PhysicalReleaseNote" TEXT, "SecondaryYear" INTEGER, "SecondaryYearSourceId" INTEGER)
CREATE TABLE "MovieFiles" ("Id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, "MovieId" INTEGER NOT NULL, "Path" TEXT, "Quality" TEXT NOT NULL, "Size" INTEGER NOT NULL, "DateAdded" DATETIME NOT NULL, "SceneName" TEXT, "MediaInfo" TEXT, "ReleaseGroup" TEXT, "RelativePath" TEXT, "Edition" TEXT)
I have a JSON column in a MySQL table that contains a multi-level JSON object. I can access the values at the first level using the function JSON_EXTRACT but I can't find how to go over the first level.
Here's my MySQL table:
CREATE TABLE ref_data_table (
`id` INTEGER(11) AUTO_INCREMENT NOT NULL,
`symbol` VARCHAR(12) NOT NULL,
`metadata` JSON NOT NULL,
PRIMARY KEY (`id`)
);
Here's my Python script:
import json
import mysql.connector
con = mysql.connector.connect(**config)
cur = con.cursor()
symbol = 'VXX'
metadata = {
'tick_size': 0.01,
'data_sources': {
'provider1': 'p1',
'provider2': 'p2',
'provider3': 'p3'
},
'currency': 'USD'
}
sql = \
"""
INSERT INTO ref_data_table (symbol, metadata)
VALUES ('%s', %s);
"""
cur.execute(sql, (symbol, json.dumps(metadata)))
con.commit()
The data is properly inserted into the MySQL table and the following statement in MySQL works:
SELECT symbol, JSON_EXTRACT(metadata, '$.data_sources')
FROM ref_data_table
WHERE symbol = 'VXX';
How can I request the value of 'provider3' in 'data_sources'?
Many thanks!
Try this:
'$.data_sources.provider3'
SELECT symbol, JSON_EXTRACT(metadata, '$.data_sources.provider3)
FROM ref_data_table
WHERE symbol = 'VXX';
the JSON_EXTRACT method in MySql supports that, the '$' references the JSON root, whereas periods reference levels of nesting. in this JSON example
{
"key": {
"value": "nested_value"
}
}
you could use JSON_EXTRACT(json_field, '$.key.value') to get "nested_value"