I'm deploying my app to heroku however their free tier only allow < 500MB packages, I'm currently using tensorflow and it takes > 500 MB, I can downgrade it to lower version or use the tensorflow-cpu instead, however, I'm also using Keras, which requires at least tensorflow=2.2.0 (so size > 500MB).
I have looked into these two questions but somehow downgrading Keras causes a lot of imcompatible issues.
Deploy python app to Heroku "Slug Size too large"
Error "Keras requires TensorFlow 2.2 or higher"
What I want to achieve is to be able to use Keras while the size of my packages < 500MB so that I can push my app to heroku
Related
I'm trying to deploy a web app that has some image prediction in it using tensorflow and keras. The tensorflow package itself is like 400mb and that doesn't include my .h5 file. Is there anyway I can publish my site on Heroku or any other site that would be better. Total Heroku says it's about 620mb. Thanks!
In requirements.txt, if you specify a lower version of tensorflow like tensorflow==2.0.1 or tensorflow==1.7.0, the size of the compressed file in heroku is reduced drastically to 200 to 300 MB.Try this and check. Let me know if it works or not. Thanks and have a great day
I am trying to deploy a Python ML app (made using Streamlit) to a server. This app essentially loads a NN model that I previously trained and makes classification predictions using this model.
The problem I am running into is that because TensorFlow is such a large package (at least 150MB for the latest tensorflow-cpu version) the hosting service I am trying to use (Heroku) keeps telling me that I exceed the storage limit of 300MB.
I was wondering if anyone else had similar problems or an idea of how to fix/get around this issue?
What I've tried so far
I've already tried replacing the tensorflow requirement with tensorflow-cpu which did significantly reduce the size, but it was still too big so -
I also tried downgrading the tensorflow-cpu version to tensorflow-cpu==2.1.0 which finally worked but then I ran into issues on model.load() (which I think might be related to the fact that I downgraded the tf version since it works fine locally)
I've faced the same problem last year. I know this does not answer your Heroku specific question, but my solution was to use Docker with AWS Beanstalk. It worked out cheaper than Heroku and I had less issues with deployment. I can guide on how to do this if you are interested
You might have multiple modules downloaded. I would recommend you to open file explorer and see the actual directory of the downloaded modules.
I have been trying to install my python dependencies and it's accumulating to a size of 578 MB, not letting me push my files.
This is the first time I am trying to push python dependencies to my application. Hence cache clearing is not the expected answer
Any suggestion would be greatly appreciated
You could deploy with Docker instead. Heroku does not impose a size restriction on Docker images. Here's documentation on how to set up a Dockerfile, and then push and release a container with Heroku:
https://devcenter.heroku.com/categories/deploying-with-docker
We have a production scenario with users invoking expensive NLP functions running for short periods of time (say 30s). Because of the high load and intermittent usage, we're looking into Lambda function deployment. However - our packages are big.
I'm trying to fit AllenNLP in a lambda function, which in turn depends on pytorch, scipy, spacy and numpy and a few other libs.
What I've tried
Following recommendations made here and the example here, tests and additional files are removed. I also use a non-cuda version of Pytorch which gets its' size down. I can package an AllenNLP deployment down to about 512mb. Currently, this is still too big for AWS Lambda.
Possible fixes?
I'm wondering if anyone of has experience with one of the following potential pathways:
Cutting PyTorch out of AllenNLP. Without Pytorch, we're in reach of getting it to 250mb. We only need to load archived models in production, but that does seem to use some of the PyTorch infrastructure. Maybe there are alternatives?
Invoking PyTorch in (a fork of) AllenNLP as a second lambda function.
Using S3 to deliver some of the dependencies: SIMlinking some of the larger .so files and serving them from an S3 bucket might help. This does create an additional problem: the Semnatic Role Labelling we're using from AllenNLP also requires some language models of around 500mb, for which the ephemeral storage could be used - but maybe these can be streamed directly into RAM from S3?
Maybe i'm missing an easy solution. Any direction or experiences would be much appreciated!
You could deploy your models to SageMaker inside of AWS, and run Lambda -> Sagemaker to avoid having to load up very large functions inside of a Lambda.
Architecture explained here - https://aws.amazon.com/blogs/machine-learning/call-an-amazon-sagemaker-model-endpoint-using-amazon-api-gateway-and-aws-lambda/
I tried to have predictionio integrate with my app. I used recommendation Engine deployment as in quick start in Predictionio website.
Faced lot of issues but able to build the engine.
I tried to train the model using pio train. But it gave an error saying "java.lang.StackOverflowError". So it means memory is not enough in my server. Then I tried to increase the memory by using pio train -- --driver-memory 5g --executor-memory 5g. Still I am getting the same error
(I am using 4 cores, 6GB RAM Ubuntu 14.04 server).
SO I want to know what is the minimum server requirements have Predictionio.
Minimum Requirements can be found in here