Develop Reference

How to fix " No module named 'numpy.core._multiarray_umath'" error

I'm trying to create a df and connect it a calendar in dash. I believe everything in the code is working, however, when I ran it, I'm getting an error that said: No module named 'numpy.core._multiarray_umath'
Here's my code:
from datetime import datetime as dt
import dash
from dash.dependencies import Input, Output, State
import dash_table
import dash_core_components as dcc
import dash_html_components as html
import pandas as pd
raw = [['tom', 10, pd.to_datetime('2021-01-01')], ['nick', 15, pd.to_datetime('2021-01-01')], ['juli', 14, pd.to_datetime('2021-01-01')],['tom', 8, pd.to_datetime('2021-01-02')],
['nick', 4, pd.to_datetime('2021-01-02')], ['juli', 12, pd.to_datetime('2021-01-02')]]
df = pd.DataFrame(raw, columns=['Name', 'Apples Gathered', 'date'])
#App layout
app = dash.Dash(__name__, prevent_initial_callbacks=True, suppress_callback_exceptions=True)
app.layout = html.Div([
dcc.DatePickerRange(
id='datepicker',
display_format='DD-MM-YYYY',
first_day_of_week=1,
max_date_allowed=dt.today(),
),
])
dash_table.DataTable(
id='datatable',
columns=[
{'id': 'Name', 'name': 'Name'},
{'id': 'Apples Gathered', 'name': 'Apples Gathered'},
{'id': 'date', 'name': 'Date'}],
data = df.to_dict('records'),
)
#app.callback(
Output('datatable', 'data'),
[Input('datepicker', 'start_date'),
Input('date-picker', 'end_date')]
)
def update_table(start_date, end_date):
dff = (df.groupby(['Name', 'Apples Gathered']).sum())
return dff.to_dict('records')
if __name__ == '__main__':
app.run_server(debug=True)
Here's the version of the packages installed in the environment:
Package Version
------------------------- -------
Brotli 1.0.9
click 7.1.2
dash 1.19.0
dash-bootstrap-components 0.11.3
dash-core-components 1.15.0
dash-html-components 1.1.2
dash-renderer 1.9.0
dash-table 4.11.2
Flask 1.1.2
Flask-Compress 1.9.0
future 0.18.2
itsdangerous 1.1.0
Jinja2 2.11.3
MarkupSafe 1.1.1
numpy 1.20.1
pandas 1.2.3
pip 20.2.1
plotly 4.14.3
How do I fix the issue? I tried uninstalling and re-installing pandas.

How to hit an API Endpoint with a sklearn Model?

I am currently stuck trying to load a model to make a prediction and load the result into an API endpoint. I have this code, which contains the model and saves the model as a pickle file.
from sklearn.cluster import KMeans
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
import numpy as np
import pickle
from sklearn import datasets#Iris Dataset
iris = datasets.load_iris()
X = iris.data#KMeans
km = KMeans(n_clusters=3)
km.fit(X)
km.predict(X)
labels = km.labels_#Plotting
fig = plt.figure(1, figsize=(7,7))
ax = Axes3D(fig, rect=[0, 0, 0.95, 1], elev=48, azim=134)
ax.scatter(X[:, 3], X[:, 0], X[:, 2],
c=labels.astype(np.float), edgecolor="k", s=50)
ax.set_xlabel("Petal width")
ax.set_ylabel("Sepal length")
ax.set_zlabel("Petal length")
plt.title("K Means", fontsize=14)
with open('C:\\Users\\ryans\\kmeans.pkl', 'wb') as f:
pickle.dump(km, f)
This seems to work fine, but the last two lines, to pickle the file and save it, seems necessary ONLY the first time it is run. If I open the file, using the code below, I don't see the need to re-save the picked file. Anyway, here is the code that I am testing, to hit the API endpoint.
from flask import Flask, jsonify
import pickle
import pandas as pd
import requests
api = Flask(__name__)
#api.route('/predict', methods=['POST'])
def predict():
json_features = requests.json
query_df = pd.DataFrame(json_features)
features = pd.get_dummies(query_df)
prediction = kmeans.predict(features)
return jsonify({'prediction': list(prediction)})
if __name__ == '__main__':
try:
kmeans = pickle.load(open("C:\\Users\\ryans\\kmeans.pkl", "rb"))
api.run(debug=True, port=8000)
traceback.print_exc(file=sys.stdout)
except:
print("Exception in user code:")
print('-'*60)
traceback.print_exc(file=sys.stdout)
print('-'*60)
When I run that code, I get this error.
* Serving Flask app "untitled41" (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: on
* Restarting with windowsapi reloader
Exception in user code:
------------------------------------------------------------
Traceback (most recent call last):
File "<ipython-input-48-1749e4d56106>", line 19, in <module>
api.run(debug=True, port=8000)
File "C:\Users\ryans\Anaconda3\lib\site-packages\flask\app.py", line 990, in run
run_simple(host, port, self, **options)
File "C:\Users\ryans\Anaconda3\lib\site-packages\werkzeug\serving.py", line 1050, in run_simple
run_with_reloader(inner, extra_files, reloader_interval, reloader_type)
File "C:\Users\ryans\Anaconda3\lib\site-packages\werkzeug\_reloader.py", line 339, in run_with_reloader
sys.exit(reloader.restart_with_reloader())
SystemExit: 1
------------------------------------------------------------
It's probably easier here to modify my original post with my findings, based on what you suggested.
I added this:
api.run(debug=True, port=8000, use_reloader=False)
Now, I go to my browser and enter this: 'http://localhost:8000/predict'
I see this result:
Not Found
The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
I tested a very simple sample script:
from flask import Flask
app = Flask(__name__)
#app.route('/api', methods=['GET','POST','DELETE'])
def api():
return 'YOU MADE IT!'
if __name__ == '__main__':
app.run(debug=True, port=8000, use_reloader=False)
This works totally fine for me!
Finally, I don't understand this part:
export FLASK_APP=my_app.py
export FLASK_DEBUG=1
flask run
Does this go at the end of the 'kmeans.py' script? When I add that, I get 'Invalid Syntax' on the first of the three lines.

I assume that you are using the flask code in notebook (Ipython environment) and debug=True is forcing the process to reload, which is causing this issue, you can set use_reload to False
api.run(debug=True, port=8000, use_reloader=False)
Also, try to run it in a script with flask command (from command line /console) instead by first save the code in a file (for e.g app.py) and setting the script name and debug mode as below
export FLASK_APP=my_app.py
export FLASK_DEBUG=1
flask run

Unable to import scipy library using Flask & Elastic Beanstalk

I am trying to set up a Flask web app using Elastic Beanstalk on AWS. I have followed the tutorial here and that works fine. I am now looking to expand the Flask webapp, and this works fine, until I import scipy.spatial as spatial, when this is part of my import statements, running eb open just times out. I receive
>>>> HTTP ERROR 504
running the webapp locally works absolutely fine even with the scipy import, it is only when I try and deploy to beanstalk that it doesn't want to work. Below is my code
import os
import requests
from bs4 import BeautifulSoup
import pandas as pd
import scipy.spatial as spatial ##### Removing this and everything works!
from flask import Flask
from flask_cors import CORS
from flask_restful import Resource, Api
from flask_jsonpify import jsonify
# print a nice greeting.
def say_hello(username = "World"):
df = pd.DataFrame({"a":[1,2,3]})
return '<p>Hello %s!</p>\n' % username
# some bits of text for the page.
header_text = '''
<html>\n<head> <title>EB Flask Test</title> </head>\n<body>'''
instructions = '''
<p><em>Hint</em>: This is a RESTful web service! Append a username
to the URL (for example: <code>/Thelonious</code>) to say hello to
someone specific.</p>\n'''
home_link = '<p>Back</p>\n'
footer_text = '</body>\n</html>'
# EB looks for an 'application' callable by default.
application = Flask(__name__)
# add a rule for the index page.
application.add_url_rule('/', 'index', (lambda: header_text +
say_hello() + instructions + footer_text))
# add a rule when the page is accessed with a name appended to the site
# URL.
application.add_url_rule('/<username>', 'hello', (lambda username:
header_text + say_hello(username) + home_link + footer_text))
# run the app.
if __name__ == "__main__":
# Setting debug to True enables debug output. This line should be
# removed before deploying a production app.
application.debug = True
application.run()
I have tried increasing the command timeout for the environment from 600 to 900, although the timeout error occurs well before 600 seconds has elapsed.

Right, I am not sure why this is the case but I updated the version of scipy in my requirements.txt and the app is working as expected!
Originally I had
scipy==1.4.1
Now I have
scipy==1.2.3
I have no idea why this has fixed the deployment issue, especially given 1.4.1 works perfectly locally. If anyone has an idea, or if this a bug I should be reporting it would be good to know!

Plotly choropleth works in online but not offline mode [duplicate]

I am am trying out colaboratory with plotly notebook mode - I open a new notebook, copy and paste the following simple example from plotly's documentation, but don't see an output. There is a large blank in the output space where the plot whould normally be.
This works fine in my local notebook (which is a newer version of plotly, but per their docs offline mode should work with the google colab version)
Any ideas?
import plotly
from plotly.graph_objs import Scatter, Layout
plotly.offline.init_notebook_mode(connected=True)
plotly.offline.iplot({
"data": [Scatter(x=[1, 2, 3, 4], y=[4, 3, 2, 1])],
"layout": Layout(title="hello world")
})

plotly version 4.x
As of version 4, plotly renderers know about Colab, so the following is sufficient to display a figure in both Colab and Jupyter (and other notebooks like Kaggle, Azure, nteract):
import plotly.graph_objects as go
fig = go.Figure( go.Scatter(x=[1,2,3], y=[1,3,2] ) )
fig.show()
plotly version 3.x
Here's an example showing the use of Plotly in Colab. (Plotly requires custom initialization.)
https://colab.research.google.com/notebook#fileId=14oudHx5e5r7hm1QcbZ24FVHXgVPD0k8f
You need to define this function:
def configure_plotly_browser_state():
import IPython
display(IPython.core.display.HTML('''
<script src="/static/components/requirejs/require.js"></script>
<script>
requirejs.config({
paths: {
base: '/static/base',
plotly: 'https://cdn.plot.ly/plotly-latest.min.js?noext',
},
});
</script>
'''))
And call it in each offline plotting cell:
configure_plotly_browser_state()

simply pass "colab" as the value for the parameter renderer in fig.show(renderer="colab")
example :
import plotly.graph_objects as go
fig = go.Figure(
data=[go.Bar(y=[2, 1, 3])],
layout_title_text="A Figure Displayed with the 'colab' Renderer"
)
fig.show(renderer="colab")

Solution
import plotly.io as pio
pio.renderers.default = "colab"
You need to change the default render. Here is an excerpt from the documentation.
https://plot.ly/python/renderers/#setting-the-default-renderer
The current and available renderers are configured using the
plotly.io.renderers configuration object. Display this object to see
the current default renderer and the list of all available renderers.
>>>import plotly.io as pio
>>>pio.renderers
Renderers configuration
-----------------------
Default renderer: 'notebook_connected'
Available renderers:
['plotly_mimetype', 'jupyterlab', 'nteract', 'vscode',
'notebook', 'notebook_connected', 'kaggle', 'azure', 'colab',
'cocalc', 'databricks', 'json', 'png', 'jpeg', 'jpg', 'svg',
'pdf', 'browser', 'firefox', 'chrome', 'chromium', 'iframe',
'iframe_connected', 'sphinx_gallery']

You need to add a method in order to use Plotly in Colab.
def enable_plotly_in_cell():
import IPython
from plotly.offline import init_notebook_mode
display(IPython.core.display.HTML('''<script src="/static/components/requirejs/require.js"></script>'''))
init_notebook_mode(connected=False)
This method must be executed for every cell which is displaying a Plotly graph.
Sample:
from plotly.offline import iplot
import plotly.graph_objs as go
enable_plotly_in_cell()
data = [
go.Contour(
z=[[10, 10.625, 12.5, 15.625, 20],
[5.625, 6.25, 8.125, 11.25, 15.625],
[2.5, 3.125, 5., 8.125, 12.5],
[0.625, 1.25, 3.125, 6.25, 10.625],
[0, 0.625, 2.5, 5.625, 10]]
)
]
iplot(data)
Reference: https://colab.research.google.com/notebooks/charts.ipynb#scrollTo=WWbPMtDkO4xg

configure_plotly_browser_state() can be executed before running every cell by using IPython's pre_run_cell hook:
import IPython
IPython.get_ipython().events.register('pre_run_cell', configure_plotly_browser_state)

Heroku python console app module ImportError

I've encountered an issue while trying to access PostgreSQL from heroku python console (first attempt).
First up, I launch python using $ heroku run python
Then I try to load up the db module with >>> from app import db, this action returns:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named app
I have one web dyno running python which has app = Flask(__name__), apart from that, I don't know what other details should I provide in order to shed more light into the issue.

After experimenting with the terminal, it became clear, that there really is no way for it to know what variables do I have in my main.py, so what I did then was load the main parts of the app through the terminal like so:
>>> import os
>>> from flask import Flask, request, session, g, redirect, url_for, abort, render_template, flash
>>> from flask.ext.sqlalchemy import SQLAlchemy
>>> # create the application
>>> app = Flask(__name__)
>>> app.config.from_object(__name__)
>>> app.config['SQLALCHEMY_DATABASE_URI'] = os.environ.get('DATABASE_URL', 'some-url.com')
>>> db = SQLAlchemy(app)
And from then on I was able to access the db object as intended.
EDIT - the issue completely resolved below.
So the reason I was having an error while trying to import my apps code into the python shell was two fold:
The *.py that was the target of my import was not at the root directory, I was telling the shell to look in the wrong place.
Since the *.py I was importing was not named app.py (and I was asking to from app import db), I was also telling the shell to look for the wrong file.
The way to solve this issue was to append the path of my *.py file to the sys.path and instead of from app import db, us from my_filename_without_the_extenssion import db. For a summary, the shell received these instructions:
>>> import sys
>>> sys.path.append('path/to/my/file')
>>> from my_filename_without_the_extenssion import db
>>> # to test if everythings ok
>>> db