I have data on a CSV file that I have imported into the DB, and display the data into my index.html as a HTML table.
The CSV is updated frequently
Is there anyway to update the data in the DB from the CSV file every hour or day, or even every time the file is updated?
PS: As I'm new to Django, what I do now is that I delete the whole DB and than migrate and import the file again, and I don't think that is a good way to do it.
According to your requirements, you can write a management command and schedule a job using cron every hour or every day.
here is a link to read more about How to create custom django-admin commands
def dummyfunc():
tests = TEST.objects.filter(parameter = parameter)
if tests.exist():
test = tests[0]
test.field = file_name
test.save()
else:
test = Test.objects.create(
field = field_name
)
test.save()
return test
Related
first I'd like to check if the file exists, and Ive used this os.path:
def check_db_exist():
try:
file_exists = exists('games.db')
if file_exists:
file_size = os.path.getsize('games.db')
if file_size > 3000:
return True, file_size
else:
return False, 'too small'
else:
return False, 'does not exist'
except:
return False, 'error'
I have a separate file for my models, and creating the database. My concern is, if I import the class for the database it instantiates the sql file.
Moreover, pywebview when displaying my html, wipes all variables.
If I were to run this process as I load my page, then I can't access the variable for true/false sqlite exists.
db = SqliteDatabase('games.db')
class Game(Model):
game = CharField()
exe = CharField()
path = CharField()
longpath = CharField()
i_d = IntegerField()
class Meta:
database = db
This creates the table, so checking if the file exists is useless.
Then if I uncomment the first line in this file the database gest created, otherwise all of my db. variables are unusable. I must be missing a really obvious function to solve my problems.
# db = SqliteDatabase('games.db')
def add_game(game, exe, path, longpath, i_d):
try:
Game.create(game=game, exe=exe, path=path, longpath=longpath, i_d=i_d)
except:
pass
def loop_insert(lib):
db.connect()
for i in lib[0]:
add_game(i.name, i.exe, i.path, i.longpath, i.id)
db.close()
def initial_retrieve():
db.connect()
vals = ''
for games in Game.select():
val = js.Import.javascript(str(games.game), str(games.exe), str(games.path), games.i_d)
vals = vals + val
storage = vals
db.close()
return storage
should I just import the file at a different point in the file? whenever I feel comfortable? I havent seen that often so I didnt want to be improper in formatting.
edit: edit: Maybe more like this?
def db():
db = SqliteDatabase('games.db')
return db
class Game(Model):
game = CharField()
exe = CharField()
path = CharField()
file 2:
from sqlmodel import db, Game
def add_game(game, exe, path, longpath, i_d):
try:
Game.create(game=game, exe=exe, path=path, longpath=longpath, i_d=i_d)
except:
pass
def loop_insert(lib):
db.connect()
for i in lib[0]:
add_game(i.name, i.exe, i.path, i.longpath, i.id)
db.close()
I am not sure if this answers your question, since it seems to involve multiple processes and/or processors, but In order to check for the existence of a database file, I have used the following:
DATABASE = 'dbfile.db'
if os.path.isfile(DATABASE) is False:
# Create the database file here
pass
else:
# connect to database here
db.connect()
I would suggest using sqlite's user_version pragma:
db = SqliteDatabase('/path/to/db.db')
version = db.pragma('user_version')
if not version: # Assume does not exist/newly-created.
# do whatever.
db.pragma('user_version', 1) # Set user version.
from reddit:
me: To the original challenge, there's a reason I want to know whether the file exists. Maybe its flawed at the premises, I'll explain and you can fill in there.
This script will run on multiple machines I dont ahve access to. At the entry point of a first-time use case, I will be porting data from a remote location, if its the first time the script runs on that machine, its going down a different work flow than a repeated opening.
Akin to grabbing all pc programs vs appending and reading from teh last session. How would you suggest quickly understanding if that process has started and finished from a previous session.
Checking if the sqlite file is made made the most intuitive sense, and then adjusting to byte size. lmk
them:
This is a good question!
How would you suggest quickly understanding if that process
has started and finished from a previous session.
If the first thing your program does on a new system is download some kind of fixture data, then the way I would approach it is to load the DB file as normal, have Peewee ensure the tables exist, and then do a no-clause SELECT on one of them (either through the model, or directly on the database through the connection if you want.) If it's empty (you get no results) then you know you're on a fresh system and you need to make the remote call. If you get results (you don't need to know what they are) then you know you're not on a fresh system.
My project is to make a live graph according to an updating log file generated from the simpy module, but I don't know how to make the st.line_chart -> inf_chart to be a variable so that I can use add_rows to update the chart.
In mainpage.py
if st.button('Run simulator'):
...
infrastructure, application = st.columns(2)
empty = ['']
empty_dataframe = pd.DataFrame(empty,columns=[''])
...
inf_chart = infrastructure.line_chart(empty_dataframe,width=500)
app_chart = application.line_chart(empty_dataframe,width=500)
In UI.py
import mainpage
def draw_infrastructure(element,i,columns):
df = pd.DataFrame(element,index=[i],columns=[columns])
inf_chart.add_rows(df)
draw_infrastructure runs every time when the log file updates, but it becomes two same pages when I import mainpage in UI.py. Is there any solution to pass this variable inf_chart to another file?
I have an app which allows the user to upload a large datafile, process its contents into a python object (not Django model), and then present a summary of the contents to the user. In other words, the big data is present in the view and summarised to the template.
The user then selects which of the content sections to save to the database, and submits the form to do so.
I'm wrestling with how to pass the python object to the AJAX-called function without having to do all the processing again?
I've used AJAX in the past and have read the answers to suggestions for not reloading pages etc, but none of them have involved passing large objects from within a view.
# retrieve the file
storage = TemporaryParseStorage.objects.get(id=item_id)
# open the file from memory
f = open(storage.file.path, "r")
file_text = f.read()
# Parse the file:
parser = Parser()
# Process its contents to create the object - I want to store this
# object and call its member functions based on a button click in the template
objectIWantToKeep = parser.parseModel(file_text)
# Builds tree for preview
tree = build_tree_from_model(storage, model)
context = {
'storage': storage,
'model_name': model.name(),
'tree': tree
}
return render(request, 'main/upload_check.html', context)
I need to pass down the original source file name (*.md file) into the sidebar.html. How can I do that?
From this site (http://pelican.readthedocs.org/en/3.6.3/themes.html), I understand some variables are available, and all capital letter variables in pelicanconf.py files are also available, but I don't know how to get the information such as the original source file in the theme files.
I think there might be a simpler way, but using jinja filter works fine for me ( http://linkpeek.com/blog/how-to-add-a-custom-jinja-filter-to-pelican.html)
Steps to take:
Pre-setup
I make the name of the original markup file to be in the format YEAR-MONTH-DAY-NAME to be recovered from the url of the page.
Create a filter
The filter is given the url, and from the url, I can recover the original source md file path.
def tosource(url):
# example input
# posts/2014/01/26/python-unittest-structure/index.html
# posts/2014/01/26/ocaml-vs-java/index.html
# posts/2014/01/25/why-ocaml-comparison-with-python/index.html
if url.startswith("posts"):
(posts, year, month, day, name) = url.split('/')[:-1]
res = "%s/%s/%s-%s-%s-%s.md" % (year, month, year, month, day, name)
else:
res = "/" # implement later
return res
Update pelicanconf.py
Teach pelican the name and location of the filter.
import sys
sys.path.append('.')
import sourcename
JINJA_FILTERS = {'sourcename':sourcename.tosource}
OPENCONTENT = "open:///pelican/knowledge_blog/content"
As is written in http://docs.getpelican.com/en/3.5.0/themes.html#theming-pelican, all capital letter variables in the conf file are accessible in the theme files.
Update sidebar.html
I added one line of code in sidebar.html to use the Jinja filter for getting the original md file path.
Click to Edit
Generate the html
Run make html and test.
In my app,i am including a function called xml parsing.I am trying to data the data from an xml file and save it to mysql database.
I coded this with the help from google engine,but as required the data's are not saving in the database.I can run the app without any error.
Please see my codes below
views.py
def goodsdetails(request):
path = "{0}shop.xml".format(settings.PROJECT_ROOT)
xmlDoc = open(path, 'r')
xmlDocData = xmlDoc.read()
xmlDocTree = etree.XML(xmlDocData)
for items in xmlDocTree.iter('item'):
item_id = items[0].text
customername = items[1].text
itemname = items[2].text
location = items[3].text
rate = items[4].text
shop=Shop.objects.create(item_id=item_id,customername=customername,
itemname=itemname,location=location,rate=rate)
shop.save()
shops = Shop.objects.all()
context={'shops':shops}
return render(request,'index.html', context)
I am using the above logic to save the data in database from xml file.I am not getting any error but it is not saving into database
Expected answers are most welcome.
*Update:*I updated the code,really the xml data gets saved in db,but while displaying the same i am getting the following Traceback
IntegrityError at /
(1062, "Duplicate entry '101' for key 'PRIMARY'")
Thanks
Shop.objects.get loads data from the database. You want to create data, which you do by just calling Shop(item_id=item_id,customername=customername, itemname=itemname,location=location,rate=rate) and then shop.save().
If you want to update the data, you need to do something like this:
shop = Shop.objects.get(tem_id=item_id)
shop.customername = customername
...etc...
shop.save()