Develop Reference

How to plot multiple points from a list using matplotlib?

I have read a list of 3D points from a text file. The list looks like follows:
content = ['2.449,14.651,-0.992,', '6.833,13.875,-1.021,', '8.133,17.431,-1.150,', '3.039,13.724,-0.999,', '16.835,9.456,-1.031,', '16.835,9.457,-1.031,', '15.388,5.893,-0.868,', '13.743,25.743,-1.394,', '14.691,24.988,-1.387,', '15.801,25.161,-1.463,', '14.668,23.056,-1.382,', '22.378,20.268,-1.457,', '21.121,17.041,-1.353,', '19.472,13.555,-1.192,', '22.498,20.115,-1.436,', '13.344,-33.672,-0.282,', '13.329,-33.835,-0.279,', '13.147,-30.690,-0.305,', '13.097,-28.407,-0.339,', '13.251,-28.643,-0.366,', '13.527,-25.067,-0.481,', '19.433,-33.137,-0.408,', '19.445,-29.501,-0.345,', '20.592,-28.004,-0.312,', '19.109,-26.512,-0.380,', '18.521,-24.155,-0.519,', '22.837,48.245,-2.201,', '23.269,50.129,-2.282,', '23.499,46.652,-2.297,', '23.814,48.646,-2.271,', '30.377,46.501,-2.214,', '29.869,44.479,-2.143,', '29.597,41.257,-2.018,', '28.134,40.291,-2.159,', '-40.932,-0.320,-1.390,', '-36.808,0.442,-1.382,', '-30.831,0.548,-1.288,', '-29.404,1.235,-1.300,', '-26.453,1.424,-1.261,', '-30.559,2.775,-1.249,', '-27.714,3.439,-1.201,']
I want to plot all the points on a 3D plot. I have this so far:
#!/usr/bin/env python
import numpy as np
import matplotlib.pyplot as plt
with open("measurements.txt") as f:
content = f.read().splitlines()
#print content
for value in content:
x, y, z = value.split(',')
#print x, y, z
fig = plt.figure()
ax = plt.axes(projection='3d')
ax.scatter(x, y, z)
fig.savefig('scatterplot.png')
It throws an error:
Traceback (most recent call last): File "plotting.py", line 11, in
x, y, z = value.split(',')
ValueError: too many values to unpack
How do I plot these points? Thank you for your help.

First of all you need to take the values into respective arrays by spitting lines in file then pass them to the function.
content = ['2.449,14.651,-0.992,', '6.833,13.875,-1.021,', '8.133,17.431,-1.150,', '3.039,13.724,-0.999,', '16.835,9.456,-1.031,', '16.835,9.457,-1.031,', '15.388,5.893,-0.868,', '13.743,25.743,-1.394,', '14.691,24.988,-1.387,', '15.801,25.161,-1.463,', '14.668,23.056,-1.382,', '22.378,20.268,-1.457,', '21.121,17.041,-1.353,', '19.472,13.555,-1.192,', '22.498,20.115,-1.436,', '13.344,-33.672,-0.282,', '13.329,-33.835,-0.279,', '13.147,-30.690,-0.305,', '13.097,-28.407,-0.339,', '13.251,-28.643,-0.366,', '13.527,-25.067,-0.481,', '19.433,-33.137,-0.408,', '19.445,-29.501,-0.345,', '20.592,-28.004,-0.312,', '19.109,-26.512,-0.380,', '18.521,-24.155,-0.519,', '22.837,48.245,-2.201,', '23.269,50.129,-2.282,', '23.499,46.652,-2.297,', '23.814,48.646,-2.271,', '30.377,46.501,-2.214,', '29.869,44.479,-2.143,', '29.597,41.257,-2.018,', '28.134,40.291,-2.159,', '-40.932,-0.320,-1.390,', '-36.808,0.442,-1.382,', '-30.831,0.548,-1.288,', '-29.404,1.235,-1.300,', '-26.453,1.424,-1.261,', '-30.559,2.775,-1.249,', '-27.714,3.439,-1.201,']
import numpy as np
import matplotlib.pyplot as plt
#with open("measurements.txt") as f:
#content = f.read().splitlines()
#print content
#for value in content:
# x, y, z = value.split(',')
x = [float(i.split(',')[0]) for i in content]
y = [float(i.split(',')[1]) for i in content]
z = [float(i.split(',')[2]) for i in content]
#print(x, y, z)
fig = plt.figure()
ax = plt.axes(projection='3d')
ax.scatter(x, y, z)
fig.savefig('scatterplot.png')
output

It's clear ! when you do your split there is 4 values
content = ['2.449,14.651,-0.992,', '6.833,13.875,-1.021,', '8.133,17.431,-1.150,', '3.039,13.724,-0.999,', '16.835,9.456,-1.031,', '16.835,9.457,-1.031,', '15.388,5.893,-0.868,', '13.743,25.743,-1.394,', '14.691,24.988,-1.387,', '15.801,25.161,-1.463,', '14.668,23.056,-1.382,', '22.378,20.268,-1.457,', '21.121,17.041,-1.353,', '19.472,13.555,-1.192,', '22.498,20.115,-1.436,', '13.344,-33.672,-0.282,', '13.329,-33.835,-0.279,', '13.147,-30.690,-0.305,', '13.097,-28.407,-0.339,', '13.251,-28.643,-0.366,', '13.527,-25.067,-0.481,', '19.433,-33.137,-0.408,', '19.445,-29.501,-0.345,', '20.592,-28.004,-0.312,', '19.109,-26.512,-0.380,', '18.521,-24.155,-0.519,', '22.837,48.245,-2.201,', '23.269,50.129,-2.282,', '23.499,46.652,-2.297,', '23.814,48.646,-2.271,', '30.377,46.501,-2.214,', '29.869,44.479,-2.143,', '29.597,41.257,-2.018,', '28.134,40.291,-2.159,', '-40.932,-0.320,-1.390,', '-36.808,0.442,-1.382,', '-30.831,0.548,-1.288,', '-29.404,1.235,-1.300,', '-26.453,1.424,-1.261,', '-30.559,2.775,-1.249,', '-27.714,3.439,-1.201,']
Solution:
for value in content:
x, y, z,parasitic_value = value.split(',')

The element in content are:
'2.449,14.651,-0.992,'
A slightly different way to extract the data to plot from this string is to consider it as a tuple, and to use eval().
data = [eval("("+x[:len(x)-1]+")") for x in content]
Which returns:
[(2.449, 14.651, -0.992),
(6.833, 13.875, -1.021),
(8.133, 17.431, -1.15),
...
(-30.559, 2.775, -1.249),
(-27.714, 3.439, -1.201)]
EDIT: the error you got means:
You want 3 values, X, Y and Z; but when I split at ",", There are more (too many values to unpack).
content[0].split(",")
Out[4]: ['2.449', '14.651', '-0.992', '']

I see at least one error in there.
The most obvious one (because you got an error), is in splitting.
The third comma at the end is causing the string to be split into four elements
>>> l = 'a,b,c,'
>>> l.split(',')
['a', 'b', 'c', '']
you can work around that by using:
x,y,z,_ = value.split(',')
the next problem you'll run into is with your loop
for value in content:
x, y, z = value.split(',')
you are only storing the last of your values, since you overwrite them multiple times.
The easiest way to work around this is creating three lists and appending into them:
x = []
y = []
z = []
for measurement in content:
a,b,c,_ = measurement.split(',')
x.append(a)
y.append(b)
z.append(c)
This is not the most efficient way, but I think it should be easier to understand.
I recommend using it like this:
x = []
y = []
z = []
with open('measurements.txt') as file:
for line in file:
a,b,c,_ = line.split(',')
x.append(a)
y.append(b)
z.append(c)

To solve the main issue , you have to edit the list , and make it a 3d numpy array , by copying all the values , traversing the list via re.
Rather than assuming the list as multiple points , try to take the first 2 points or 3 points as a image/3D graph , and use imshow or Axes3D to plot it.

How I can link the output of the 1st program as an input for the 2nd program using python?

I have a program written in Python and need to use the output values of this program as an input values to my second program of same python folder. May I know is it possible to do; if so can you please tell me which module is used and would be great if I can get a small example.
For example below is the first program and from this how I can take the output value 'x' to plot in my second program.
from functools import partial
import numpy
import scipy.optimize
import matplotlib.pyplot as pp
def z(x, y):
return x * y - 30
x_window = 0, 15
y_window = 0, 5
xs = []
ys = []
for x in numpy.linspace(*x_window, num=200):
try:
# A more efficient technique would use the last-found-y-value as a
# starting point
y = scipy.optimize.brentq(partial(z, x), *y_window)
except ValueError:
# Should we not be able to find a solution in this window.
pass
else:
xs.append(x)
ys.append(y)
pp.plot(xs, ys)
pp.xlim(*x_window)
pp.ylim(*y_window)
pp.show()

Rearrange your code and put everything in functions:
# file my_program.py
from functools import partial
import numpy
import scipy.optimize
import matplotlib.pyplot as pp
def z(x, y):
return x * y - 30
def make_data(x_window, y_window):
xs = []
ys = []
for x in numpy.linspace(*x_window, num=200):
try:
# A more efficient technique would use the last-found-y-value as a
# starting point
y = scipy.optimize.brentq(partial(z, x), *y_window)
except ValueError:
# Should we not be able to find a solution in this window.
pass
else:
xs.append(x)
ys.append(y)
return xs, ys
def plot(xs, ys, x_window, y_window):
pp.plot(xs, ys)
pp.xlim(*x_window)
pp.ylim(*y_window)
pp.show()
if __name__ == '__main__':
x_window = 0, 15
y_window = 0, 5
xs, ys = make_data(x_window, y_window)
plot(xs, ys, x_window, y_window)
Now, in your second file, you can import it from the same folder and call make_data():
import my_program
x_window = 0, 15
y_window = 0, 5
xs, ys = my_program.make_data(x_window, y_window)

In the first program return the value needed and save it in a text file and in the second program open the file and use the value from the imported file as an input to your second program.

In e.g. bash, or even the windows command prompt, you can just pipe'em:
> python prog1.py | python prog2.py

plotting two variable function in python

I have to plot a two variable function to check model degeneracy. My code looks like this following a tutorial:
from numpy import exp,arange
from pylab import meshgrid,cm,imshow,contour,clabel,colorbar,axis,title,show
import math
# the function that I'm going to plot
def z_func(x1,x2):
L = exp(-(1-x1)**2 - 100((x2-x1**2)**2))
return L
x1 = arange(-5.0,5.0,0.1)
x2 = arange(-5.0,5.0,0.1)
X1,X2 = meshgrid(x1, x2) # grid of point
Z = z_func(X1, X2) # evaluation of the function on the grid
im = imshow(Z,cmap=cm.RdBu) # drawing the function
# adding the Contour lines with labels
cset = contour(Z,arange(-1,1.5,0.2),linewidths=2,cmap=cm.Set2)
clabel(cset,inline=True,fmt='%1.1f',fontsize=10)
colorbar(im) # adding the colobar on the right
# latex fashion title
title('$z=exp(-(1-x1)^2 - 100(x2-x1^2)^2)$')
show()
and I am getting the following error:
File "multi.py", line 14, in <module>
Z = z_func(X1, X2) # evaluation of the function on the grid
File "multi.py", line 8, in z_func
L = exp(-(1-x1)**2 - 100((x2-x1**2)**2))
TypeError: 'int' object is not callable
I think there is problem how i define my function, How to fix this error and plot the function?

IndentationError in python code

I am trying to run a small python code for data-mining and getting following error.
ERROR:
File "prediction.py", line 10
data=pd.read_csv(file_name)
^
IndentationError: expected an indented block
I am a beginner pls help.
My Code:
# Required Packages
import csv
import sys
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from sklearn import datasets, linear_model
# Function to get data
def get_data(file_name):
data = pd.read_csv(file_name)
flash_x_parameter = []
flash_y_parameter = []
arrow_x_parameter = []
arrow_y_parameter = []
for x1,y1,x2,y2 in zip(data[‘flash_episode_number’],data[‘flash_us_viewers’],data[‘arrow_episode_number’],data[‘arrow_us_viewers’]):
flash_x_parameter.append([float(x1)])
flash_y_parameter.append(float(y1))
arrow_x_parameter.append([float(x2)])
arrow_y_parameter.append(float(y2))
return flash_x_parameter,flash_y_parameter,arrow_x_parameter,arrow_y_parameter
# Function to know which Tv show will have more viewers
def more_viewers(x1,y1,x2,y2):
regr1 = linear_model.LinearRegression()
regr1.fit(x1, y1)
predicted_value1 = regr1.predict(9)
print predicted_value1
regr2 = linear_model.LinearRegression()
regr2.fit(x2, y2)
predicted_value2 = regr2.predict(9)
#print predicted_value1
#print predicted_value2
if predicted_value1 > predicted_value2:
print "The Flash Tv Show will have more viewers for next week"
else:
print "Arrow Tv Show will have more viewers for next week"
x1,y1,x2,y2 = get_data(‘input.csv’)
#print x1,y1,x2,y2
more_viewers(x1,y1,x2,y2)`

Indentation matters in python.
You're getting the error because you need to indent at the 10th line around your function definition:
# Function to get data
def get_data(file_name):
data = pd.read_csv(file_name)
flash_x_parameter = []
flash_y_parameter = []
...

In python you have to indent code for any nested block so python can recognize what belongs to global code, function code and any inner block code:
def get_data(file_name):
data = pd.read_csv(file_name)
....
global code again

This should take care of the indenting issue. Please read the docs for PEP8 [docs].
Use 4 spaces per indentation level.
Continuation lines should align wrapped elements either vertically
using Python's implicit line joining inside parentheses, brackets and
braces, or using a hanging indent [6] . When using a hanging indent
the following should be considered; there should be no arguments on
the first line and further indentation should be used to clearly
distinguish itself as a continuation line.
# Required Packages
import csv
import sys
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from sklearn
import datasets, linear_model
# Function to get data
def get_data(file_name):
data = pd.read_csv(file_name)
flash_x_parameter = []
flash_y_parameter = []
arrow_x_parameter = []
arrow_y_parameter = []
for x1, y1, x2, y2 in zip(data[‘flash_episode_number’], data[‘flash_us_viewers’], data[‘arrow_episode_number’], data[‘arrow_us_viewers’]):
flash_x_parameter.append([float(x1)])
flash_y_parameter.append(float(y1))
arrow_x_parameter.append([float(x2)])
arrow_y_parameter.append(float(y2))
return flash_x_parameter, flash_y_parameter, arrow_x_parameter, arrow_y_parameter
# Function to know which Tv show will have more viewers
def more_viewers(x1, y1, x2, y2):
regr1 = linear_model.LinearRegression()
regr1.fit(x1, y1)
predicted_value1 = regr1.predict(9)
print predicted_value1
regr2 = linear_model.LinearRegression()
regr2.fit(x2, y2)
predicted_value2 = regr2.predict(9)
# print predicted_value1# print predicted_value2
if predicted_value1 > predicted_value2:
print "The Flash Tv Show will have more viewers for next week"
else :
print "Arrow Tv Show will have more viewers for next week"
x1, y1, x2, y2 = get_data(‘input.csv’)# print x1, y1, x2, y2
more_viewers(x1, y1, x2, y2)

Pandas error with basemap/proj for map plotting

I ran the Python code below that is an example of "Plotting Maps: Visualizing Haiti Earthquake Crisis Data" on a book, Python for Data Analysis. Page 242-246
The code is supposed to create a plot map of Haiti but I got an error as below:
Traceback (most recent call last):
File "Haiti.py", line 74, in <module>
x, y = m(cat_data.LONGITUDE, cat_data.LATITUDE)
File "/usr/local/lib/python2.7/site-packages/mpl_toolkits/basemap/__init__.py", line 1148, in __call__
xout,yout = self.projtran(x,y,inverse=inverse)
File "/usr/local/lib/python2.7/site-packages/mpl_toolkits/basemap/proj.py", line 286, in __call__
outx,outy = self._proj4(x, y, inverse=inverse)
File "/usr/local/lib/python2.7/site-packages/mpl_toolkits/basemap/pyproj.py", line 388, in __call__
_proj.Proj._fwd(self, inx, iny, radians=radians, errcheck=errcheck)
File "_proj.pyx", line 122, in _proj.Proj._fwd (src/_proj.c:1571)
RuntimeError
I checked if mpl_toolkits.basemap and proj module were installed okay on my machine. Basemap was installed from source as instructed and proj was installed by Homebrew and they looks fine to me.
If you have basemap and proj installed, does this code run successfully? If not, do you think if it's a module installation issue, the code itself, or any other?
Haiti.csv file can be downloaded from https://github.com/pydata/pydata-book/raw/master/ch08/Haiti.csv
import pandas as pd
import numpy as np
from pandas import DataFrame
data = pd.read_csv('Haiti.csv')
data = data[(data.LATITUDE > 18) & (data.LATITUDE < 20) &
(data.LONGITUDE > -75) & (data.LONGITUDE < -70)
& data.CATEGORY.notnull()]
def to_cat_list(catstr):
stripped = (x.strip() for x in catstr.split(','))
return [x for x in stripped if x]
def get_all_categories(cat_series):
cat_sets = (set(to_cat_list(x)) for x in cat_series)
return sorted(set.union(*cat_sets))
def get_english(cat):
code, names = cat.split('.')
if '|' in names:
names = names.split(' | ')[1]
return code, names.strip()
all_cats = get_all_categories(data.CATEGORY)
english_mapping = dict(get_english(x) for x in all_cats)
def get_code(seq):
return [x.split('.')[0] for x in seq if x]
all_codes = get_code(all_cats)
code_index = pd.Index(np.unique(all_codes))
dummy_frame = DataFrame(np.zeros((len(data), len(code_index))),
index=data.index, columns=code_index)
for row, cat in zip(data.index, data.CATEGORY):
codes = get_code(to_cat_list(cat))
dummy_frame.ix[row, codes] = 1
data = data.join(dummy_frame.add_prefix('category_'))
from mpl_toolkits.basemap import Basemap
import matplotlib.pyplot as plt
def basic_haiti_map(ax=None, lllat=17.25, urlat=20.25, lllon=-75, urlon=-71):
# create polar stereographic Basemap instance.
m = Basemap(ax=ax, projection='stere',
lon_0=(urlon + lllon) / 2,
lat_0=(urlat + lllat) / 2,
llcrnrlat=lllat, urcrnrlat=urlat,
llcrnrlon=lllon, urcrnrlon=urlon,
resolution='f')
# draw coastlines, state and country boundaries, edge of map. m.drawcoastlines()
m.drawstates()
m.drawcountries()
return m
fig, axes = plt.subplots(nrows=2, ncols=2, figsize=(12, 10))
fig.subplots_adjust(hspace=0.05, wspace=0.05)
to_plot = ['2a', '1', '3c', '7a']
lllat=17.25; urlat=20.25; lllon=-75; urlon=-71
for code, ax in zip(to_plot, axes.flat):
m = basic_haiti_map(ax, lllat=lllat, urlat=urlat,
lllon=lllon, urlon=urlon)
cat_data = data[data['category_%s' % code] == 1]
# compute map proj coordinates.
print cat_data.LONGITUDE, cat_data.LATITUDE
x, y = m(cat_data.LONGITUDE, cat_data.LATITUDE)
m.plot(x, y, 'k.', alpha=0.5)
ax.set_title('%s: %s' % (code, english_mapping[code]))

This is resolved by changing m(cat_data.LONGITUDE, cat_data.LATITUDE) to m(cat_data.LONGITUDE.values, cat_data.LATITUDE.values), thanks to Alex Messina's finding.
With a little further study of mine, pandas changed that Series data of DataFrame (derived from NDFrame) should be passed with .values to a Cython function like basemap/proj since v0.13.0 released on 31 Dec 2013 as below.
Quote from github commit log of pandas:
+.. warning::
+
+ In 0.13.0 since ``Series`` has internaly been refactored to no longer sub-class ``ndarray``
+ but instead subclass ``NDFrame``, you can **not pass** a ``Series`` directly as a ``ndarray`` typed parameter
+ to a cython function. Instead pass the actual ``ndarray`` using the ``.values`` attribute of the Series.
+
+ Prior to 0.13.0
+
+ .. code-block:: python
+
+ apply_integrate_f(df['a'], df['b'], df['N'])
+
+ Use ``.values`` to get the underlying ``ndarray``
+
+ .. code-block:: python
+
+ apply_integrate_f(df['a'].values, df['b'].values, df['N'].values)
You can find the corrected version of the example code here.