I am building this ecommerce app with django, and I was thinking if the default database (sqlite) was fine enough? My django app is going to have around 200 different products, and for payments I will use stripe API. I do not expect too much traffic when the app is up, since it's a website only for the country I live in with 6 million inhabitants. So can I settle with the sqlite database?
Although your answer may seem opinion based, the right answer is no anyways. SQLite3 is a lightweight database, which is commonlu used when your project is small and you don't mind to start up a server.
SQLite comes with following advantages and disadvantages (source)
Advantages of SQLite
Small footprint: As its name implies, the SQLite library is very lightweight. Although the space it uses varies depending on the system where it’s installed, it can take up less than 600KiB of space. Additionally, it’s fully self-contained, meaning there aren’t any external dependencies you have to install on your system for SQLite to work.
User-friendly: SQLite is sometimes described as a “zero-configuration” database that’s ready for use out of the box. SQLite doesn’t run as a server process, which means that it never needs to be stopped, started, or restarted and doesn’t come with any configuration files that need to be managed. These features help to streamline the path from installing SQLite to integrating it with an application.
Portable: Unlike other database management systems, which typically store data as a large batch of separate files, an entire SQLite database is stored in a single file. This file can be located anywhere in a directory hierarchy, and can be shared via removable media or file transfer protocol.
Disadvantages of SQLite
Limited concurrency: Although multiple processes can access and query an SQLite database at the same time, only one process can make changes to the database at any given time. This means SQLite supports greater concurrency than most other embedded database management systems, but not as much as client/server RDBMSs like MySQL or PostgreSQL.
No user management: Database systems often come with support for users, or managed connections with predefined access privileges to the database and tables. Because SQLite reads and writes directly to an ordinary disk file, the only applicable access permissions are the typical access permissions of the underlying operating system. This makes SQLite a poor choice for applications that require multiple users with special access permissions.
Security: A database engine that uses a server can, in some instances, provide better protection from bugs in the client application than a serverless database like SQLite. For example, stray pointers in a client cannot corrupt memory on the server. Also, because a server is a single persistent process, a client-server database cancontrol data access with more precision than a serverless database, allowing for more fine-grained locking and better concurrency.
It depends where are you going to host it. Some servers do not accept SQLite3 as database and require something more complicated as PostgreSQL (like Heroku). But if you are hosting by yourself (Rapsberry Pi for instance) then it's going to be fine and simple, if the site is going to have small traffic.
Related
I'm a noobie to Django and I'm almost at the stage of deploying a web server.
I was just having some doubts with Django's database. Currently I'm using the default sqlite3 database to store all the user models as well as the info models. I'm thinking of using AWS to deploy my web sever.
So when I get to that stage, should I continue with sqlite or should I switch to AWS's database or something like Firebase. If I continue with sqlite, where and how exactly will the information be stored? And what if I switch to something like PostgreSQL, where will the information be stored and will it be secure/fast (even if I manage to get thousands of users)?
Thanks so much, this question might be really basic but I'm super confused.
sqlite is a flat file database, it uses an exposed file in your project to save your data, this is fine in local environment, but when deploying you need to consider that the server and the database are in the same machine and using the same disk. that means if you accidentally remove the machine -and its disk space- used to serve the application, then the database itself will be deleted with all records.
Plus you will face problems if you tried to scale your servers, that is every server will have his own copy of the database and syncing all those files will be huge headache.
If your data is not that important then you can keep using sqlite, but if you are expecting high traffic and complex db structure, then I would recommend you consider a db engine like Mysql or maybe look up the databases offered by amazon here:
https://aws.amazon.com/products/databases/
For django, you will need to change the adapter when using a different db like mysql, sqlite or anything else.
https://docs.djangoproject.com/en/3.0/ref/databases/
I'm creating a pygtk app that needs a mysql connection from a remote db.
db = MySQLdb.connect("remotehost","username","password","databse", charset='utf8')
App is almost completed and going to be published. But the problem is, if anyone decompile this script they can easily read the above credentials and then there is a security issue. So how do I can protect this code or is there any way I can strongly compile this file?
Database connections are generally made from trusted computers inside a trusted network, for a variety of reasons:
As you've seen, the client needs to store access credentials to the DB.
Most of the time, such connections are made with no transport security (unencrypted), so any eavesdropper can observe and mangle requests/responses.
Latency in the path to the DB is usually a issue, so you want to minimize it, thus placing the client near to the DB
Violating this common practice means you'll have to deal with these problems.
It's very common to have a intermediary service using some other protocol (for example, HTTP/REST) to exposes an API that indirectly modifies the database. You keep the service on a host in your trusted computing base, and only that one host accesses the DB.
In this architecture, you can (and should) perform authentication and mandatory access control in the intermediary service. In turn, having different credentials for each client that accesses that service will help keep things secure.
If you can't rewrite your application at this point, you should follow patriciasz's suggestion on keeping the least priviledge possible. You may also be interested in techniques to make it harder (but not impossible) to obtain the credentials
There is no way to protect your code (compiled or not) from the owner of the machine it runs on.
In this case he will effectively have the same access restrictions your application's SQL user has.
There is no good way to protect your code, but you can use read_default_file options while using connect. The connection arguments will then be read form the file, specified with
read_default_file.
NOTE: This in no way is securing your username, password since anyone having access to the cnf file can get the information.
Build an interface between the database and the application. Only the interface will get true database access.
Give the app credentials to access the interface, and only the interface, then let the interface interact with the data base. This adds a second layer to boost security and helps to protect database integrity.
In the future develop with separate logic from the start. The app does not need to accesses the data base. Instead, it needs data from the database.
Also as a rule of database security avoid putting credentials on the client side. If you have n apps then n apps can access your data base, and controlling access points is a big part of database logic.
Separating program logic is the real deal, credentials don't need to reside on clients machine just as chip said
I have been developing a fairly simple desktop application to be used by a group of 100-150 people within my department mainly for reporting. Unfortunately, I have to build it within some pretty strict confines similar to the specs called out in this post. The application will just be a self contained executable with no need to install.
The problem I'm running into is figuring out how to handle the database need. There will probably only be about 1GB of data for the app, but it needs to be available to everyone.
I would embed the database with the application (SQLite), but the data needs to be refreshed every week from a centralized process, so I figure it would be easier to maintain one database, rather than pushing updates down to the apps. Plus users will need to write to the database as well and those updates need to be seen by everyone.
I'm not allowed to set up a server for the database, so that rules out any good options for a true database. I'm restricted to File Shares or SharePoint.
It seems like I'm down to MS Access or SQLite. I'd prefer to stick with SQLite because I'm a fan of python and SQLAlchemy - but based on what I've read SQLite is not a good solution for multiple users accessing it over the network (or even possible).
Is there another option I haven't discovered for this setup or am I stuck working with MS Access? Perhaps I'll need to break down and work with SharePoint lists and apps?
I've been researching this for quite a while now, and I've run out of ideas. Any help is appreciated.
FYI, as I'm sure you can tell, I'm not a professional developer. I have enough experience in web / python / vb development that I can get by - so I was asked to do this as a side project.
SQLite can operate across a network and be shared among different processes. It is not a good solution when the application is write-heavy (because it locks the database file for the duration of a write), but if the application is mostly reporting it may be a perfectly reasonable solution.
As my options are limited, I decided to go with a built in database for each app using SQLite. The db will only need updated every week or two, so I figured a 30 second update by pulling from flat files will be OK. then the user will have all data locally to browse as needed.
Is there any software library that provides an interface for storing and querying data like the Google App Engine Datastore, but uses a local file or service instead of running on App Engine?
The specific features I am looking for are:
Stores data as Entities with Named Properties
Query support
Atomic transactions
Python language bindings
Runs on my local machine
either stores to a single file
or connects to a local database
service
Free and open source
Thanks
You can also check MongoDB. It is an open source document-oriented database system.
You may also want to check out Appscale (http://www.appscale.com). It lets you run your App Engine apps without modification outside of Google (on your laptop, on your local cluster / behind your firewall, or in Amazon EC2). AppScale is and does each of the requirements you list here. It automatically installs/configures/manages the datastore service (and all other APIs/services) for your apps to use, so you don't have to.
Have a look at ZODB - not exactly alike but similiar http://www.zodb.org/
from the docs
Some of the features that ZODB brings to you:
Transparent persistence for Python objects
Full ACID-compatible
transaction support (including savepoints) History/undo ability
Efficient support for binary large objects (BLOBs)
Pluggable storages
Scalable architecture
which is the best back end for python applications and what is the advantage of using sqlite ,how it can be connected to python applications
What do you mean with back end? Python apps connect to SQLite just like any other database, you just have to import the correct module and check how to use it.
The advantages of using SQLite are:
You don't need to setup a database server, it's just a file
No configurations needed
Cross platform
Mainly, desktops applications are the ones that take real advantage of this. For web apps, SQLite is not recommended, since the file containing the data, is easily readable (lacks any kind of encryption), and when the web server lacks special configuration, the file is downloadable by anyone.
Django, Twisted, and CherryPy are popular Python "Back-Ends" as far as web applications go, with Twisted likely being the most flexible as far as networking is concerned.
SQLite can, as has been previously posted, be directly interfaced with using SQL commands as it has native bindings for Python, or it can be accessed with an Object Relational Manager such as SQLObject (another Python library).
As far as performance is concered, SQLite is fairly scalable and should be able to handle most use cases that don't require a seperate database server (nothing enterprise level). An additional benefit of SQLite is that the database is self-contained in a single file allowing for easy backup while remained a common enough format that multiple applications can access the data. A word of advice on using SQLite with Python, however, is that you may run into issues with threading (in the past most of the bindings for SQLite were not thread-safe, although this may have changed over time).
The language you are using at the application layer has little to do with your database choice underneath. You need to examine the advantages of other DB packages to get an idea of what you want.
Here are some popular database packages for cheap or free:
ms sql server express, pg/sql, mysql
If you mean "what is the best database?" then there's simply no way to answer this question. If you just want a small database that won't be used by more than a handful of people at a time, SQLite is what you're looking for. If you're running a database for a giant corporation serving thousands, you're probably looking for Oracle. In between those, you have MySQL, PostgreSQL, SQL Server, db2, and probably more.
If you're familiar with one of those, that may be the best to go with from a practical standpoint. If you're doing a typical webapp, my advice would be to go with MySQL or PostgreSQL as they're free and well supported by just about any ORM you could think of (my personal preference is towards PostgreSQL, but I'm not experienced enough with either of these to make a good argument one way or another). If you do go with one of those two, my recommendation is to use storm as the ORM.
(And yes, there are free versions of SQL Server and Oracle. You won't have as many choices as far as ORMs go though)