In this paper, a very simple model is described to illustrate how the ant colony algorithm works. In short, it assumes two nodes which are connected via two links one of which is shorter. Then, given a pheromone increment and a pheromone evaporation dynamics, one expects that all ants eventually pick the shorter path.
Now, I'm trying to replicate the simulation of this paper corresponding to scenario above whose result should be (more or less) like below.
Here is an implementation of mine (taking the same specification as that of the test above).
import random
import matplotlib.pyplot as plt
N = 10
l1 = 1
l2 = 2
ru = 0.5
Q = 1
tau1 = 0.5
tau2 = 0.5
epochs = 150
success = [0 for x in range(epochs)]
def compute_probability(tau1, tau2):
return tau1/(tau1 + tau2), tau2/(tau1 + tau2)
def select_path(prob1, prob2):
if prob1 > prob2:
return 1
if prob1 < prob2:
return 2
if prob1 == prob2:
return random.choice([1,2])
def update_accumulation(link_id):
global tau1
global tau2
if link_id == 1:
tau1 += Q / l1
return tau1
if link_id == 2:
tau2 += Q / l2
return tau2
def update_evapuration():
global tau1
global tau2
tau1 *= (1-ru)
tau2 *= (1-ru)
return tau1, tau2
def report_results(success):
plt.plot(success)
plt.show()
for epoch in range(epochs-1):
temp = 0
for ant in range(N-1):
prob1, prob2 = compute_probability(tau1, tau2)
selected_path = select_path(prob1,prob2)
if selected_path == 1:
temp += 1
update_accumulation(selected_path)
update_evapuration()
success[epoch] = temp
report_results(success)
However, what I get is fairly weird as below.
It seems that my understanding of how pheromone should be updated is flawed.
So, can one address what I am missing in this implementation?
Three problems in the proposed approach:
As #Mark mentioned in his comment, you need a weighted random choice. Otherwise the proposed approach will likely always pick one of the paths and the plot will result in a straight line as you show above. However, I think this was part of the solution, because even with this, you will likely still get a straight line because of early convergence, which led two problem two.
Ant Colony Optimization is a metaheuristic that needs several (hyper) parameters configured to guide the search for a certain solution (e.g., tau from above or number of ants). Fine tuning this parameters is important because you can converge early on a particular result (which is fine to some extent - if you want to use it as an heuristic). But the purpose of a metaheuristic is to provide you with some middle ground between the exact and heuristic algorithms, which makes the continous exploration/exploitation an important part of its workings. This means the parameters need to be careful optimised for your problem size/type.
Given that the ACO uses a probabilistic approach for guiding the search (and as the plot from the referenced paper is showing), you will need to run the experiment several times and compute some statistic on those numbers. In my case below, I computed the average over 100 samples.
import random
import matplotlib.pyplot as plt
N = 10
l1 = 1.1
l2 = 1.5
ru = 0.05
Q = 1
tau1 = 0.5
tau2 = 0.5
samples = 10
epochs = 150
success = [0 for x in range(epochs)]
def compute_probability(tau1, tau2):
return tau1/(tau1 + tau2), tau2/(tau1 + tau2)
def weighted_random_choice(choices):
max = sum(choices.values())
pick = random.uniform(0, max)
current = 0
for key, value in choices.items():
current += value
if current > pick:
return key
def select_path(prob1, prob2):
choices = {1: prob1, 2: prob2}
return weighted_random_choice(choices)
def update_accumulation(link_id):
global tau1
global tau2
if link_id == 1:
tau1 += Q / l1
else:
tau2 += Q / l2
def update_evaporation():
global tau1
global tau2
tau1 *= (1-ru)
tau2 *= (1-ru)
def report_results(success):
plt.ylim(0.0, 1.0)
plt.xlim(0, 150)
plt.plot(success)
plt.show()
for sample in range(samples):
for epoch in range(epochs):
temp = 0
for ant in range(N):
prob1, prob2 = compute_probability(tau1, tau2)
selected_path = select_path(prob1, prob2)
if selected_path == 1:
temp += 1
update_accumulation(selected_path)
update_evaporation()
ratio = ((temp + 0.0) / N)
success[epoch] += ratio
# reset pheromone values here to evaluate new sample
tau1 = 0.5
tau2 = 0.5
success = [x / samples for x in success]
for x in success:
print(x)
report_results(success)
The code above should return something close to the desired plot.
Related
The following code does exactly what I want; however, the for loop is far too slow. On my machine, the wall time for the for loop is 1min 5s. I'm looking for an alternative to the for loop that is much faster.
# Imports
from sympy.solvers.solveset import solveset_real
from sympy import Symbol, Eq
# Define variables
initial_value = 1
rate = Symbol('r')
decay_obs_window = 1480346
target_decay = .15
# Solver to calculate decay rate
decay_rate = solveset_real(Eq((initial_value - rate * decay_obs_window), target_decay), rate).args[0]
# Generate weights
weights = []
for i in range(5723673):
# How to handle data BEYOND decay_obs_window
if i > decay_obs_window and target_decay == 0:
# Record a weight of zero
weights.append(0)
elif i > decay_obs_window and target_decay > 0:
# Record the final target weight
weights.append(decayed_weight)
# How to handle data WITHIN decay_obs_window
else:
# Calculate the new slightly decayed weight
decayed_weight = 1 - (decay_rate * i)
weights.append(decayed_weight)
weights[0:10]
I wrote this list comprehension with the hope of improving the execution time. While it works perfectly, it does not yield any appreciable runtime improvement over the for loop 😞:
weights = [0 if i > decay_obs_window and target_decay == 0 else decayed_weight if i > decay_obs_window and target_decay > 0 else (decayed_weight := 1 - (decay_rate * i)) for i in range(len(weights_df))]
I'm interested in any approaches that would help speed this up. Thank you 🙏!
FINAL SOLUTION:
This was the final solution that I settled on. On my machine, the wall time to execute the entire thing is only 425 ms. It's a slightly modified version of Aaron's proposed solution.
import numpy as np
from sympy.solvers.solveset import solveset_real
from sympy import Symbol, Eq
# Define variables
initial_value = 1
rate = Symbol('r')
decay_obs_window = 1480346
target_decay = .15
# Instantiate weights array
weights = np.zeros(5723673)
# Solver to calculate decay rate
decay_rate = solveset_real(Eq((initial_value - rate * decay_obs_window), target_decay), rate).args[0]
# Fix a bug where numpy doesn't like sympy floats :(
decay_rate = float(decay_rate)
# How to weight observations WITHIN decay_obs_window
weights[:decay_obs_window + 1] = 1 - np.arange(decay_obs_window + 1) * decay_rate
# How to weight observations BEYOND decay_obs_window
weights[decay_obs_window + 1 : 5723673] = target_decay
weights
TLDR; None of the variables you test against in your if statements ever change during the loop, so you can easily kick the conditional logic out of the loop, and decide beforehand. I also am a huge proponent of numpy and vectorization.
Looking at the logic there aren't too many possible outcomes of what weights ends up looking like. As RufusVS mentioned, you can separate out the first section where no additional logic is being calculated. It is also a simple linear function, so why not compute it with numpy which is great for linear algebra:
import numpy as np
weights = np.zeros(5723673)
#Fix a bug where numpy doesn't like sympy floats :(
decay_rate = float(decay_rate)
weights[:decay_obs_window + 1] = 1 - np.arange(decay_obs_window + 1) * decay_rate
Then you can decide what to do with the remaining values based on the value of target_decay outside of any loops because it never changes:
if target_decay == 0:
pass #weights array started out filled with 0's so we don't need to do anything
elif target_decay > 0:
#fill the rest of the array with the last value of the window
weights[decay_obs_window + 1 : 5723673] = weights[decay_obs_window + 1]
pass
else: #target_decay < 0:
#continue calculating the linear function
weights[decay_obs_window + 1 : 5723673] = 1 - np.arange(decay_obs_window + 1, 5723673) * decay_rate
By breaking it into two loops, you eliminate a lot of comparison to the break point:
# Imports
from sympy.solvers.solveset import solveset_real
from sympy import Symbol, Eq
# Define variables
initial_value = 1
rate = Symbol('r')
decay_obs_window = 1480346
target_decay = .15
# Solver to calculate decay rate
decay_rate = solveset_real(Eq((initial_value - rate * decay_obs_window), target_decay), rate).args[0]
# Generate weights
weights = []
for i in range(decay_obs_window+1):
# Calculate the new slightly decayed weight
decayed_weight = 1 - (decay_rate * i)
weights.append(decayed_weight)
for i in range(decay_obs_window+1, 5723673):
# How to handle data BEYOND decay_obs_window
if target_decay == 0:
weights.append(0)
elif target_decay > 0:
# Record the final target weight
weights.append(decayed_weight)
else:
# Calculate the new slightly decayed weight
decayed_weight = 1 - (decay_rate * i)
weights.append(decayed_weight)
weights[0:10]
Modified to include #MarkSouls comment, and my own further observation:
# Imports
from sympy.solvers.solveset import solveset_real
from sympy import Symbol, Eq
# Define variables
initial_value = 1
rate = Symbol('r')
decay_obs_window = 1480346
target_decay = .15
# Solver to calculate decay rate
decay_rate = solveset_real(Eq((initial_value - rate * decay_obs_window), target_decay), rate).args[0]
TOTAL_ENTRIES = 5723673
# Generate weights
weights = [0]* TOTAL_ENTRIES
for i in range(decay_obs_window+1):
# Calculate the new slightly decayed weight
decayed_weight = 1 - (decay_rate * i)
weights[i]=decayed_weight
if target_decay == 0:
pass
elif target_decay > 0:
for i in range(decay_obs_window+1, TOTAL_ENTRIES):
# Record the final target weight
weights[i]=decayed_weight
else:
for i in range(decay_obs_window+1, TOTAL_ENTRIES):
decayed_weight = 1 - (decay_rate * i)
weights[i]=decayed_weight
weights[0:10]
I have what I believe to be an absolutely optimal method here:
# Imports
from sympy.solvers.solveset import solveset_real
from sympy import Symbol, Eq
# Define variables
initial_value = 1
rate = Symbol('r')
decay_obs_window = 1480346
target_decay = .15
# Solver to calculate decay rate
decay_rate = solveset_real(Eq((initial_value - rate * decay_obs_window), target_decay), rate).args[0]
# Generate weights
wLength = 5723673
weights = [1 - (decay_rate * i) for i in range(decay_obs_window + 1)]
extend_length = wLength - decay_obs_window - 1
if target_decay == 0:
weights.extend(0 for _ in range(extend_length))
elif target_decay > 0:
decayed_weight = weights[-1]
weights.extend(decayed_weight for _ in range(extend_length))
This brings all the branching logic out of the loop, so it's only calculated once instead of ~ 1.5 million times.
That said, this still represents nearly no improvement in speed over what you already have. The fact of the matter is that most of your time is spent calculating 1 - (decay_rate * i), and there's nothing you can do in python to speed this up.
If you really need more performance you're probably at the point of needing to figure out how to call a C (or Rust) library.
Numpy is perfect for this. We can use the fromfunction method to create an array. First import the function:
from numpy import fromfunction
Then replace
weights = [1 - (decay_rate * i) for i in range(decay_obs_window + 1)]
with
weights = fromfunction(lambda i: 1 - (decay_rate * i), (decay_obs_window + 1, )).tolist()
This is likely to represent the fastest you can do this in Python.
I've been tinkering with neural networks and have some simple code that almost works. The only problem is that my network will not mutate properly. I've tested the network class on its own and it will mutate but it doesn't seem to want to mutate when used as a subclass.
%matplotlib inline
#for jupyter
import matplotlib #import for plotting results
import matplotlib.pyplot as plt
import numpy as np #np for random and exp
from datetime import datetime as dt #for time seed
#GLOBALS
sp = 100.0 #y setpoint
seconds = 120 #simulation length
timescale = 0.1 #timestep
generations = 10000 #generations to simulate
population = 20 #number of ships per generation
debug = False #unused
#NN class
class Network:
#create array of neuron placeholder values for feedforward function
def initNeurons(self):
neuronList = []
for i in range(len(self.layers)):
neuronList.append([])
for j in range(self.layers[i]):
neuronList[i].append(0)
self.neurons = neuronList
#print(self.neurons)
#randomly generate weights for each neuron based on number of neurons in previous layers
def initWeights(self):
weightsList = []
#for each layer
for i in range(1,len(self.layers)):
layerWeights = []
neuronsInPrevLayer = self.layers[i-1] #number of neurons in previous layer
#for each neuron in layer
for j in range(len(self.neurons[i])):
neuronWeights = []
#for each neuron in previous layer
for k in range(neuronsInPrevLayer):
neuronWeights.append(2*(np.random.rand()-0.5)) #generate random weight (-1-1)
layerWeights.append(neuronWeights)
weightsList.append(layerWeights)
self.weights = weightsList
#calculate the values of each neuron and return output neurons
def feedForward(self, netinputs):
for i in range(len(netinputs)):
#print(self.neurons[0])
self.neurons[0][i] = netinputs[i]
for i in range(1,len(self.layers)):
for j in range(len(self.neurons[i])):
value = 0.25
for k in range(len(self.neurons[i-1])):
value += self.weights[i-1][j][k] * self.neurons[i-1][k]
self.neurons[i][j] = (1/(1+np.exp(-value)))
return self.neurons[-1]
#randomly mutate weights while iterating through them
def mutate(self):
change = False
for i in range(len(self.weights)):
for j in range(len(self.weights[i])):
for k in range(len(self.weights[i][j])):
weight = self.weights[i][j][k]
#print(weight)
randnum = np.random.rand() * 1000
if randnum <= 20:
weight *= -1
change = True
elif randnum <= 40:
weight = np.random.rand() - 0.5
change = True
elif randnum <= 80:
weight *= np.random.rand()
change = True
self.weights[i][j][k] = weight
if change and debug:
#print('mutation!')
pass
#iterate through weights and copy
def copyWeights(self, copyWeight):
for i in range(len(self.weights)):
for j in range(len(self.weights[i])):
for k in range(len(self.weights[i][j])):
self.weights[i][j][k] = copyWeight[i][j][k]
#copies the weights from a passed NN
def Network(self, copyNetwork):
self.layers = []
self.neurons = []
self.weights = []
self.fitness = -9000.0
for i in range(len(copyNetwork.layers)):
self.layers.append(copyNetwork.layers[i])
np.random.seed(dt.now().microsecond)
self.initNeurons()
self.initWeights()
self.copyWeights(copyNetwork.weights)
#INITIALIZATION FUNCTION
#initializes NN given an array of neuron counts EX: [3,5,3,1] 3 input neurons 1 output neuron with 2 hidden layers
def __init__(self, inLayers ):
self.layers = [] #array with neurons per layer ex: [2,4,2]
self.neurons = [] #placeholder array for neuron values for feedforward
self.weights = [] #weight values for each layer and neuron
self.fitness = -9000.0 #initial fitness set to nonsense value
for i in range(len(inLayers)):
self.layers.append(inLayers[i])
np.random.seed(dt.now().microsecond) #seed for RNG
self.initNeurons() #create arrays for sotring neuron values
self.initWeights() #create weights for calculating neuron values
#environment for ship simulation
class environment(object):
def __init__(self): # initialize self when created
self.objects = []
self.t = 0
self.dt = timescale
self.seconds = seconds
def init(self): #initialize ships
for p in self.objects:
p.init()
def start(self): #iterate through time and call step for each ship
for i in range(0, self.seconds*100, int(self.dt*100)):
self.t += self.dt
#if i % 10 == 0:
#print(self.t)
for p in self.objects:
p.step()
class ship():
def __init__(self, m, x, y, v, thrust_max, throttle, env):
self.brain = Network([3,8,4,3,2]) #create NN for throttle control with 3 inputs and 2 outputs
self.deltasp = [] #array for difference from setpoint per step for plotting and analysis
self.yset = [] #array for y coord (height) per step for plotting and analysis
self.mass = m #mass for force and acceleration calculation
self.x = x #initial x value
self.y = y #initial y value
self.velocity = v #initial velocity
self.thrust_max = thrust_max #maximum thrust
self.throttle = throttle/100.0 #initial throttle
self.a = 0.0 #initial acceleration
env.objects.append(self) #add ship to environment objects
self.env = env
def init(self):
self.deltasp = [] #array for difference from setpoint per step for plotting and analysis
self.yset = [] #array for y coord (height) per step for plotting and analysis
self.x = 0 #x coord for plotting (unused)
self.y = 0 #y coord (height)
self.velocity = 0 #ship velocity
self.a = 0 #ship acceleration
#calculate acceleration based on thrust throttle and mass minus acceleration due to gravity
def acc(self):
return ((self.thrust_max*self.throttle)/(self.mass))-9.8
def step(self): #ship step
#get outputs from NN
self.outputs = self.brain.feedForward([self.throttle,self.a,((self.y)-sp)])
#if output 1 is high increase thrust by variable amount based on certainty
if self.outputs[0] >= 0.6:
self.throttle += self.outputs[0]-0.5
#print('throttle up')
#if output 2 is high decrease thrust by variable amount based on certainty
elif self.outputs[1] >= 0.6:
self.throttle -= self.outputs[1]-0.5
#throttle limiting between 0% and 100% (0-1)
if self.throttle <= 0:
self.throttle = 0
elif self.throttle >= 1:
self.throttle = 1
#store delta from setpoint to array for plotting and analysis
self.deltasp.append(abs(self.y-sp))
#increase x arbitrarily (legacy code from turtle version could be useful later)
self.x += 0.1
#increase y (height) by timestep times velocity
self.y += self.env.dt*self.velocity
#calculate new acceleration
self.a = self.acc()
#limit height to 0
#floor collision detection
if self.y < 0:
self.a = 0
self.velocity = 0
self.y = 0
#store y coord (height) for plotting and analysis
self.yset.append(self.y)
#calculate new velocity based on acceleration and timestep
self.velocity = self.velocity + self.env.dt*self.a
#calculate fitness as summation of difference from setpoint
self.brain.fitness = sum(self.deltasp)
#mutate NN for evolution
def evolve(self):
self.brain.mutate()
def bubble_sort(seq): #modified bubblesort borrowed from http://python3.codes/popular-sorting-algorithms/
for ob in seq:
#print(ob.brain.weights)
pass
changed = True
while changed:
changed = False
for i in range(len(seq) - 1):
if abs(seq[i].brain.fitness) < abs(seq[i+1].brain.fitness):
seq[i], seq[i+1] = seq[i+1], seq[i]
changed = True
return None
def reproduce(ships): #make new ships based on fitness
mute_ships = []
return_ships = []
for o in ships:
mute_ships.append(o)
bubble_sort(mute_ships) #sort ships by fitness
for i in range(len(ships)): #create array of mutated best ship
mute_ships[-1].evolve()
return_ships.append(mute_ships[-1])
return return_ships #array of mutated ships
def main(): #main loop
new_ships = [] #mutated ships container
for gen in range(generations): #loop for generations
ships = []
env = environment()
if gen == 0:# if first generation generate initial population
for i in range(population):
np.random.seed(dt.now().microsecond)
shp = ship(500.0, -100.0, 0.0, 0.0, 9800.0, 0.0, env)
ships.append(shp)
else: #if not first generation copy ships from mutated ships
ships = new_ships
for o in ships:
o.env.objects.append(o)
new_ships = []
env.init() #initialize environment
env.start() #start environment simulation
for o in reproduce(ships):#mutate ships
new_ships.append(o)
del env
### DEBUGGING ###
#print(len(new_ships))
#print(ships[0].brain.weights == new_ships[0].brain.weights)
#if ships[0].brain.weights == new_ships[0].brain.weights:
#print('no mutations')
#print("generation: ", gen + 1)
#for o in ships:
#print(o.brain.fitness)
#plt.plot(range(len(o.yset)),o.yset)
#print(ships[0].brain.fitness)
for o in ships: #plot different statistics
#print(o.brain.fitness)
plt.plot(range(len(o.yset)),o.yset)
#plt.plot(range(len(o.deltasp)),o.deltasp)
return "done"
if __name__ == '__main__':
main()
At this point I'm pretty stuck. Sorry for some of the spaghetti code. I've tried to clean it up a bit.
Ok, here's what I see:
In reproduce() you are doing a bunch of weird things:
you sort the list (which has no effect on the mutation), then mutate them (which probably destroys the sorted-ness);
you iterate with an index into the array, which is usually unPythonic, but operate on mute_ships[-1] each time - which means that, with 20 ships, you are mutating the last ship 20 times and the others not at all!
.evolve() seems to be an in-place operation, but you copy the result into a new list to return; then in the calling function, main(), you copy the result into a new list again using a loop (the slow way) instead of using the list() constructor (simpler and faster)
that makes no difference anyway, because the list only contains references to the same ship instances anyway!
Instead, try
def reproduce(ships):
for ship in ships:
ship.evolve()
Naming conventions: PEP8 says classes should be capitalized, methods should be lowercase. Also, some of the variable names are nasty (o.env.objects.append(o)?)
You don't need bubblesort; you can just do mute_ships = sorted(ships, key = lambda ship: ship.brain.fitness. That replaces about 16 lines of code.
You aren't evolving the ships or environments at all, so separate classes for them are kind of overkill. I would probably rename Network to ShipController, and stick the whole simulation into a ShipController.evaluate() method.
np.random seeds itself quite happily; the only good reason to seed it yourself is if you want to be able to repeat a run by giving it the same seed again. Also, np.random.seed() expects a value in 0 .. 4.3 billion; you are giving it a value in 0 .. 1 million. By doing so, you are greatly reducing the actual randomness of your algorithm.
You have a global sp but also an sp in main() which never gets used, which will confuse you if you ever try to change it.
You are not really using numpy properly; it works best on vectorized blocks of calculations, not lots of little split-up calculations inside Python loops.
I am trying Bayesian regression using Metropolis-Hastings. The test data is generated as follows (Python code, I didn't copy the entire code):
trueA = 5 ; trueB = 7 ;trueSD = 10 ; sample_size = 261
x = np.arange(-sample_size/8, sample_size/8, (sample_size*2/8)/sample_size)
y = trueA *x + trueB + npr.normal(loc=0, scale=trueSD, size=sample_size)
I defined log likelihood as follows:
def likelihood(param):
a = param[0][0] ; b = param[0][1] ; sd = param[0][2] ; pred = a*x + b
sumSqError = np.power((y - pred), 2).sum()
likelihoodsum = ((sample_size/2)*(np.log(1)-np.log(np.power(sd,2)))) + (- 1/(2*np.power(sd,2)) * sumSqError)
return likelihoodsum
To make next points, I prepared the following function:
def next_param(param, param_index):
a_next = param[0][0] ; b_next = param[0][1] ; sd_next = param[0][2]
if param_index == 0:
a_next = param[0][0] + npr.normal(0, 0.1)
elif param_index == 1:
b_next = param[0][1] + npr.normal(0, 0.1)
elif param_index == 2:
sd_next = param[0][2] + npr.normal(0, 0.1)
return np.array([[a_next, b_next, sd_next]])
This code works well (acceptance rate is high enough and I can estimate the parameters), though I know sd_next can go negative in the above code, which is weird.
So, I decided to use log for sd_next:
elif param_index == 2:
sd_next = np.log(param[0][2]) + npr.normal(0, 0.1)
return np.array([[a_next, b_next, np.exp(sd_next)]])
However, the estimated parameters are far from the true values. How can I make a standard deviation always positive in Metropolis-Hastings?
JFI, here is MCMC part:
num_sampling = 1000
chain = np.zeros((num_sampling, 1, 3))
chain[0][0][0] = 20 # starting value for a
chain[0][0][1] = 15 # starting value for b
chain[0][0][2] = 15 # starting value for sd
num_accepted = 0
for i in range(num_sampling-1):
chain_previous = chain[i][:]
chain_new = np.zeros((1, 1, 3))
for p in range(3):
proposal = next_param(chain_previous, p)
probab = likelihood(proposal) - likelihood(chain_previous)
if 0 < probab:
chain_new[0][0][p] = proposal[0][p]
num_accepted += 1
else:
chain_new[0][0][p] = chain[i][0][p]
chain[i+1] = chain_new[0][:]
It is not weird at all that you get a negative standard deviation $\sigma$ when your proposal is a Normal distribution, with support $(-\infty,+\infty)$.
And the Metropolis-Hastings accept-reject step should also include the prior distribution on the three parameters. Including the Jacobian when the proposal is on $\log\sigma$.
As written the Metropolis-Hastings accept-reject step is incorrect!
if 0 < probab:
is not the right condition for accepting a move to the proposed value: one should compared the (log-)probability with a (log-)uniform. In the current format, you converge to a maximum of the likelihood.
I am trying to implement gradient descent on a dataset. Even though I tried everything, I couldn't make it work. So, I created a test case. I am trying my code on a random data and try to debug.
More specifically, what I am doing is, I am generating random vectors between 0-1 and random labels for these vectors. And try to over-fit the training data.
However, my weight vector gets bigger and bigger in each iteration. And then, I have infinities. So, I do not actually learn anything. Here is my code:
import numpy as np
import random
def getRandomVector(n):
return np.random.uniform(0,1,n)
def getVectors(m, n):
return [getRandomVector(n) for i in range(n)]
def getLabels(n):
return [random.choice([-1,1]) for i in range(n)]
def GDLearn(vectors, labels):
maxIterations = 100
stepSize = 0.01
w = np.zeros(len(vectors[0])+1)
for i in range(maxIterations):
deltaw = np.zeros(len(vectors[0])+1)
for i in range(len(vectors)):
temp = np.append(vectors[i], -1)
deltaw += ( labels[i] - np.dot(w, temp) ) * temp
w = w + ( stepSize * (-1 * deltaw) )
return w
vectors = getVectors(100, 30)
labels = getLabels(100)
w = GDLearn(vectors, labels)
print w
I am using LMS for loss function. So, in all iterations, my update is the following,
where w^i is the ith weight vector and R is the stepSize and E(w^i) is the loss function.
Here is my loss function. (LMS)
and here is how I derivated the loss function,
,
Now, my questions are:
Should I expect good results in this random scenario using Gradient Descent? (What is the theoretical bounds?)
If yes, what is my bug in my implementation?
PS: I tried several other maxIterations and stepSize parameters. Still not working.
PS2: This is the best way I can ask the question here. Sorry if the question is too specific. But it made me crazy. I really want to learn the problem.
Your code has a couple of faults:
In GetVectors() method, you did not actually use the input variable m;
In GDLearn() method, you have a double loop, but you use the same variable i as the loop variables in both loops. (I guess the logic is still right, but it's confusing).
The prediction error (labels[i] - np.dot(w, temp)) has the wrong sign.
Step size does matters. If I am using 0.01 as step size, the cost is increasing in each iteration. Changing it to be 0.001 solved the problem.
Here is my revised code based on your original code.
import numpy as np
import random
def getRandomVector(n):
return np.random.uniform(0,1,n)
def getVectors(m, n):
return [getRandomVector(n) for i in range(m)]
def getLabels(n):
return [random.choice([-1,1]) for i in range(n)]
def GDLearn(vectors, labels):
maxIterations = 100
stepSize = 0.001
w = np.zeros(len(vectors[0])+1)
for iter in range(maxIterations):
cost = 0
deltaw = np.zeros(len(vectors[0])+1)
for i in range(len(vectors)):
temp = np.append(vectors[i], -1)
prediction_error = np.dot(w, temp) - labels[i]
deltaw += prediction_error * temp
cost += prediction_error**2
w = w - stepSize * deltaw
print 'cost at', iter, '=', cost
return w
vectors = getVectors(100, 30)
labels = getLabels(100)
w = GDLearn(vectors, labels)
print w
Running result -- you can see the cost is decreasing with each iteration but with a diminishing return.
cost at 0 = 100.0
cost at 1 = 99.4114482617
cost at 2 = 98.8476022685
cost at 3 = 98.2977744556
cost at 4 = 97.7612851154
cost at 5 = 97.2377571222
cost at 6 = 96.7268325883
cost at 7 = 96.2281642899
cost at 8 = 95.7414151147
cost at 9 = 95.2662577529
cost at 10 = 94.8023744037
......
cost at 90 = 77.367904046
cost at 91 = 77.2744249433
cost at 92 = 77.1823702888
cost at 93 = 77.0917090883
cost at 94 = 77.0024111475
cost at 95 = 76.9144470493
cost at 96 = 76.8277881325
cost at 97 = 76.7424064707
cost at 98 = 76.6582748518
cost at 99 = 76.5753667579
[ 0.16232142 -0.2425511 0.35740632 0.22548442 0.03963853 0.19595213
0.20080207 -0.3921798 -0.0238925 0.13097533 -0.1148932 -0.10077534
0.00307595 -0.30111942 -0.17924479 -0.03838637 -0.23938181 0.1384443
0.22929163 -0.0132466 0.03325976 -0.31489526 0.17468025 0.01351012
-0.25926117 0.09444201 0.07637793 -0.05940019 0.20961315 0.08491858
0.07438357]
I am trying to make a training set of data points by making a line (perceptron) f and making the points on one side +1 and -1 on the other. Then making a new line g and trying to get it as close to f as possible by updating with w = w+ y(t)x(t) where w is weights and y(t) is +1,-1 and x(t) is coordinates of a missclassified point. after implementing this tho i am not getting a very good fit from g to f. here is my code and some sample outputs.
import random
random.seed()
points = [ [1, random.randint(-25, 25), random.randint(-25,25), 0] for k in range(1000)]
weights = [.1,.1,.1]
misclassified = []
############################################################# Function f
interceptf = (0,random.randint(-5,5))
slopef = (random.randint(-10, 10),random.randint(-10,10))
point1f = ((interceptf[0] + slopef[0]),(interceptf[1] + slopef[1]))
point2f = ((interceptf[0] - slopef[0]),(interceptf[1] - slopef[1]))
############################################################# Function G starting
interceptg = (-weights[0],weights[2])
slopeg = (-weights[1],weights[2])
point1g = ((interceptg[0] + slopeg[0]),(interceptg[1] + slopeg[1]))
point2g = ((interceptg[0] - slopeg[0]),(interceptg[1] - slopeg[1]))
#############################################################
def isLeft(a, b, c):
return ((b[0] - a[0])*(c[1] - a[1]) - (b[1] - a[1])*(c[0] - a[0])) > 0
for i in points:
if isLeft(point1f,point2f,i):
i[3]=1
else:
i[3]=-1
for i in points:
if (isLeft(point1g,point2g,i)) and (i[3] == -1):
misclassified.append(i)
if (not isLeft(point1g,point2g,i)) and (i[3] == 1):
misclassified.append(i)
print len(misclassified)
while misclassified:
first = misclassified[0]
misclassified.pop(0)
a = [first[0],first[1],first[2]]
b = first[3]
a[:] = [x*b for x in a]
weights = [(x + y) for x, y in zip(weights,a)]
interceptg = (-weights[0],weights[2])
slopeg = (-weights[1],weights[2])
point1g = ((interceptg[0] + slopeg[0]),(interceptg[1] + slopeg[1]))
point2g = ((interceptg[0] - slopeg[0]),(interceptg[1] - slopeg[1]))
check = 0
for i in points:
if (isLeft(point1g,point2g,i)) and (i[3] == -1):
check += 1
if (not isLeft(point1g,point2g,i)) and (i[3] == 1):
check += 1
print weights
print check
117 <--- number of original missclassifieds with g
[-116.9, -300.9, 190.1] <--- final weights
617 <--- number of original missclassifieds with g after algorithm
956 <--- number of original missclassifieds with g
[-33.9, -12769.9, -572.9] <--- final weights
461 <--- number of original missclassifieds with g after algorithm
There are at least few problems with your algorithm:
Your "while" conditions is wrong - the perceptron learning is not about iterating once through all misclassified points as you do now. The algorithm should iterate through all the points for as long as any of them is missclassified. In particular - each update can make some correctly classified point as the wrong one, so you have to always iterate through all of them and check if everything is fine.
I am pretty sure that what you actually wanted is update rule in form of (y(i)-p(i))x(i) where p(i) is predicted label and y(i) is a true label (but this obviously degenrates to your method if you only update misclassifieds)