Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I have few machine learning models which I have already trained (size in 300MB - 4 GB). Now I want to query them using a rest API. I build my APIs in flask which require these models to be in memory. Is google app engine suitable for it? Or should I use compute engine instead?
it's unsuited for appengine because of reasons like:
appengine does not have instances with so much ram (unless you use flexible which is basically the same as gce option).
even if it did, the instance can go away which requires to reload data to memory constantly (thou it could be on memcached)
time restrictions on frontend instances will make it very unlikely you can even get time to load data to memory, less being able to analyze the data. a backend type of instance could do it, but will be harder than doing it from a regular VM.
good luck loading the libraries you need, as none should write to file storage, even for a temp file.
thus compute engine is the correct place.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
I want to deploy a website that uses Django on the server side and reactJs Frontend on AWS. I'm estimating traffic of about 40,000 per month. It happens that Amazon has several hosting packages. I like the EC2 instances but I don't know which particular instance will be suitable for me and being cost effective. I want to host for 6 months to get to understand the platform and later extends it if I'm comfortable with their services. The database isn't that robust. I'll also like to add updates at least two times a week.
Honestly, I haven't used AWS before and I don't even know if EC2 is the best for my project. Which of the packages will be best for my project?
Your main choice is Amazon EC2 (highly flexible) vs Amazon Lightsail (similar to a VPS).
Nobody can advise you on sizing your system, since it is totally dependent upon what your application does (video vs compute vs caching, etc) and how your users interact with the app. You should pick an Instance Type, setup a test system that simulates typical usage and then monitor how it runs.
Alternatively, pick something reasonably big, run it in production and monitor for any issues. If it all looks good, you could downsize the system.
AWS EC2 is an IaaS (Infrastructure as a Service) cloud model, and it's best suited for those who are starting to use cloud computing. We can say that it's similar to on-prem environments, you have almost full control over all aspects of computing like memory, storage, and networking, on the other hand, you need to manage them, and in some cases, you don't want or, you don't need to do. In your case, with the information provided, an EC2 seems to be the best choice, so you can safely try AWS (safely, because you will need to take care of costs, and on IaaS model you have more predictability) while learning about other services. The second step will depend on the specificities of your application like if stateful or stateless if needs to be scalable or not. Hope I have contributed.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 1 year ago.
Improve this question
I am making a rest API where users can get and post images with labels. The API will also have the ability to generate color pallets for the images. I am wondering what the best way to post images to the API is and what format the API should store the images in. I am considering using base64 encoded images but I am not sure.
Yes you can encode and decode the files(images) using base64.
Its secure and relaiable.
OR
If you meant read/write by handle of images then best way to do so is by storing images into file system and then access them URL stored in a DB. If you want to process image by editing and then used cloud services to store in file systems like AWS S3, then you can focus on the editing part with Java/JavaScript plugins for browsers. But whether you edit or write to some DB it is always better to use file system to store image file physically and locate them using some URL from DB. REST or any APIs should be written keeping in mind the basic structure of file handling. Any file of large size should not be saved into DB as CLOB or BLOB even though nobody restricts you. It is better to save files(images) into file system like S3. I’ve written APIs for writing and reading files into DB which is small in size and need to be accessed very rarely.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
We have legacy .exe solution running on industrial client's desktops that I need to port over to Google App Engine. All clients will have the same version of the app, no code customization. We need to be sure that data will not be leaked between different clients of the new GAE app or be snooped on by some external party.
So I have a couple of questions that I hope someone will be able to help me out.
Option 1: Use namespaces to separate client's data. This seems quite robust at the Datastore level; without the proper namespace data would not be served. Since we have the company name of each user, I would presume this would be quite secure against leaks.
Q1: Which precautions do I need to take to avoid someone guessing or hacking at URLs to get to unauthorized data?
Option 2: We could have a separate instance of the code and datastore for each client. This would give the optimal separation between clients.
Q2: Is there a tool or API to "clone" a given GAE app to many clients? We would feed the list of clients and update the code to all those instances in one go.
Thanks!
Which precautions do I need to take to avoid someone guessing or hacking at URLs to get to unauthorized data?
Namespaces are not a security mechanism for Datastore. A user with access to one partition in a project has access to all partitions in the project. Namespaces provide a way to organize your entities within a project.
Is there a tool or API to "clone" a given GAE app to many clients? We would feed the list of clients and update the code to all those instances in one go.
To have completely isolated instances of services, you can create new projects programmatically using the Cloud Resource Manager API and access resources across projects.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 days ago.
Improve this question
How to transfer session to another compute node with python in the following case?
case 1: If using kubernete,
case 2: Or using autoscale,
case 3: if using Amazon,
How to transfer session to another compute node with python?
So that program can run forever
Nope, none of those things can transfer a process with all of its in-memory and on-disk state across hosts.
If you’re looking at Kubernetes already, I’d encourage you to design your application so that it doesn’t have any local state. Everything it knows about lives in a database that’s maintained separately (if you’re into AWS, it could be an RDS hosted database or something else). Then you can easily run multiple copies of it (maybe multiple replicas in a Kubernetes ReplicaSet or Deployment) and easily kill one off to restart it somewhere else.
One of the high-end virtualization solutions might be able to do what you’re asking, but keeping a program running forever forever is pretty hard, particularly in a scripting language like Python. (How do you update the program? How do you update the underlying OS, when it needs to reboot to take a kernel update?)
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
i have to develop a Real Time Application that supports thousands of users with Python. The question: decide which framework and architecture use.
The first option is:
Falcon only (Admin and API)
The second:
Tornado manage request, Admin runs Django and API runs Falcon
Another problem is database. Could be one only RethinkDB (1st option), or two, with Postgresql (2nd option).
Which of these option is the best? Should try something different?
PD: One fancy implementation could be with Tornado sockets
I think your question needs to be more specific.
Without more detail about the application my general suggestion would be to not complicate things by using many frameworks, just pick one that does everything you need.
Unless you are planning to design an asynchronous API forget about Tornado. If your application is very small and has a simple API then you could consider Falcon because of it's short learning curve. Otherwise I'd suggest going with Django.
As for the DB, it's nearly impossible to provide a suggestion without understanding what type of data you are planning to store? What is your expected read/write ratio? Do you already have operational experience with any DBs?