Develop Reference

Suggesting the next available IP network block

I was wondering if there's a good way to find the next available gap to create a network block given a list of existing ones?
For example, I have these networks in my list:
[
'10.0.0.0/24',
'10.0.0.0/20',
'10.10.0.0/20',
]
and then someone comes along and ask: "Do you have have enough space for 1 /22 for me?"
I'd like to be able to suggest something along the line:
"Here's a space: x.x.x.x/22" (x.x.x.x is something that comes before 10.0.0.0)
or
"Here's a space: x.x.x.x/22" (x.x.x.x is something in between 10.0.0.255 and 10.10.0.0)
or
"Here's a space: x.x.x.x/22" (x.x.x.x is something that comes after 10.10.15.255)
I'd really appreciate any suggestions.

The ipaddress library is good for this sort of use case. You can use the IPv4Network class to define subnet ranges, and the IPv4Address objects it can return can be converted into integers for comparison.
What I do below:
Establish your given list as a list of IPv4Networks
Determine the size of the block we're looking for
Iterate through the list, computing the amount of space between consecutive blocks, and checking if our wanted block fits.
You could also return an IPv4Network with the subnet built into it, instead of an IPv4Address, but I'll leave that as an exercise to the reader.
from ipaddress import IPv4Network, IPv4Address
networks = [
IPv4Network('10.0.0.0/24')
IPv4Network('10.0.0.0/20')
IPv4Network('10.0.10.0/20')
]
wanted = 22
wanted_size = 2 ** (32 - wanted) # number of addresses in a /22
space_found = None
for i in range(1, len(networks):
previous_network_end = int(networks[i-1].network_address + int(networks[i-1].hostmask))
next_network_start = int(networks[i].network_address)
free_space_size = next_network_start - previous_network_end
if free_space_size >= wanted_size:
return IPv4Address(networks[i-1] + 1) # first available address

Hex code using Python

I want to retrieve hexadecimal data from user, using python. How to retrieve the data from user and convert it to hex.
#to read varibales from Python
STX = '\xF7' #hex(input("enter STX Value"))
Deviceid = hex(input("enter device id"))
subid = hex(input("enter address of the Device and load details"))
Comnd = hex(41)
Data = hex(01)
EorCode = input("enter EOR Code")
ADD_sum = '\xF2' #hex(input("Enter Add sum value"))
tuple = (STX, Deviceid,subid,Comnd,Data,EorCode,ADD_sum)
print tuple
i am reading the above data from user,but i am getting output as follows
enter device id03
enter address of the Device and load details81
enter EOR Code32
('\xf7', '0x3', '0x51', '0x29', '0x1', '0x20', '\xf2')
But i need to be printed as 0x03 and 0x01.
I am very new to PYTHON please help.

You're looking for string formatting:
>>> "0x{0:04x}".format(42)
'0x002a'
So you'll want to modify your lines like so:
Deviceid = "0x{0:2x}".format((input("enter device id"))
Also, if any other Python developer will be looking at this code you may want to look at the Python style guide, PEP8.
Following the style guide, your code might look like this:
stx = '\xF7' # hex(input("enter STX Value"))
device_id = hex(input("enter device id")) # deviceid might also be fine
sub_id = hex(input("enter address of the Device and load details"))
comnd = hex(41)
data = hex(01)
eor_code = input("enter EOR Code")
add_sum = '\xF2' # hex(input("Enter Add sum value"))
values = (stx, device_id, sub_id, comnd, data, eor_code, add_sum)
print values # tuple is a keyword - it's best to *not* override them if possible
Of course,
A style guide is about consistency. Consistency with this style guide is important. Consistency within a project is more important. Consistency within one module or function is most important.
But most importantly: know when to be inconsistent -- sometimes the style guide just doesn't apply. When in doubt, use your best judgment. Look at other examples and decide what looks best. And don't hesitate to ask!

It seems to me that all you really need is to specify how to print the numbers, but hex function returns a string.
Because in python, '10' is a string and this is different from 10, which is an int. Python is dynamicaly, but strongly typed language.
So in order to have output you want, you may choose from 2 options:
write your own function to convert numbers to hexaxecimal numbers in a format you want and use it instead of hex:
def myhex(num):
return '0x%02x' % num
this 0x%02x means - first, 0x is just normal text which you probably want to prefix all your hexadecimal numbers, %02x means: print argument as hexadecimal number of length 2, prefixed with 0 if it's too short (one-digit hexadecimal number).
do not convert numbers to hex when reading values (it's probably a good thing to work with numbers represented as numbers) and print them formated to your specification at the end:
print '(' + ', '.join('%0x02x' % x for x in tuple) + ')'
which creates list of all values in tuple (btw, avoid using keywords as your variable names when possible) converted to correct 2-digit hexadecimal numbers with 0x prefixes, joins them using ', ' and surrounds them with parentheses. But feel free to change it - I'm just building on your example and trying to duplicate your output.

Converting an imperative algorithm into functional style

I wrote a simple procedure to calculate the average of the test coverage of some specific packages in a Java project. The raw data in a huge html file is like this:
<body>
package pkg1 <line_coverage>11/111,<branch_coverage>44/444<end>
package pkg2 <line_coverage>22/222,<branch_coverage>55/555<end>
package pkg3 <line_coverage>33/333,<branch_coverage>66/666<end>
...
</body>
Given the specified packages "pkg1" and "pkg3", for example, the average line coverage is:
(11+33)/(111+333)
and average branch coverage is:
(44+66)/(444+666)
I wrote the follow procedure to get the result and it works well. But how to implement this calculation in a functional style? Something like "(x,y) for x in ... for b in ... if...". I know a little Erlang, Haskell and Clojure, So solutions in these languages are also appreciated. Thanks a lot!
from __future__ import division
import re
datafile = ('abc', 'd>11/23d>34/89d', 'e>25/65e>13/25e', 'f>36/92f>19/76')
core_pkgs = ('d', 'f')
covered_lines, total_lines, covered_branches, total_branches = 0, 0, 0, 0
for line in datafile:
for pkg in core_pkgs:
ptn = re.compile('.*'+pkg+'.*'+'>(\d+)/(\d+).*>(\d+)/(\d+).*')
match = ptn.match(line)
if match is not None:
cvln, tlln, cvbh, tlbh = match.groups()
covered_lines += int(cvln)
total_lines += int(tlln)
covered_branches += int(cvbh)
total_branches += int(tlbh)
print 'Line coverage:', '{:.2%}'.format(covered_lines / total_lines)
print 'Branch coverage:', '{:.2%}'.format(covered_branches/total_branches)

Down below you can find my Haskell solution. I will try to explain the important points I went through as I wrote it.
First you will find that I created a data structure for coverage data. It's generally a good idea to create data structures to represent whatever data you want to handle. This is in part because it makes it easier to design your code when you can think in terms of whatever you are designing – closely related to functional programming philosophies, and in part because it can eliminate a few bugs where you think you are doing something but are in actuality doing something else.
Related to the point before: The first thing I do is to convert the string-represented data into my own data structure. When you are doing functional programming, you are often doing things in "sweeps." You don't have a single function that converts data to your format, filters out the unwanted data and summarises the result. You have three different functions for each of those tasks, and you do them one at a time!
This is because functions are very composable, i.e. if you have three different ones, you can stick them together to form a single one if you want to. If you start with a single one, it is very difficult to take it apart to form three different ones.
The actual workings of the conversion function is actually quite uninteresting unless you are specifically doing Haskell. All it does is try to match each string with a regex, and if it succeeds, it adds the coverage data to the resulting list.
Again, mad composition is about to happen. I don't create a function to loop over a list of coverages and sum them up. I create a single function to sum two coverages, because I know I can use it together with the specialised fold loop (which is sort of like a for loop on steroids) to summarise all coverages in a list. There's no need for me to reinvent the wheel and create a loop myself.
Besides, my sumCoverages function works with a lot of specialised loops, so I don't have to write a ton of functions, I just stick my single function into a ton of pre-made library functions!
In the main function you will see what I mean by programming in "sweeps" or "passes" over the data. First I convert it to the internal format, then I filter out the unwanted data, then I summarise the remaining data. These are completely independent computations. That's functional programming.
You will also notice that I use two specialised loops there, filter and fold. This means that I don't have to write any loops myself, I just stick in a function to those standard library loops and let those take it from there.
import Data.Maybe (catMaybes)
import Data.List (foldl')
import Text.Printf (printf)
import Text.Regex (matchRegex, mkRegex)
corePkgs = ["d", "f"]
stats = [
"d>11/23d>34/89d",
"e>25/65e>13/25e",
"f>36/92f>19/76"
]
format = mkRegex ".*(\\w+).*>([0-9]+)/([0-9]+).*>([0-9]+)/([0-9]+).*"
-- It might be a good idea to define a datatype for coverage data.
-- A bit of coverage data is defined as the name of the package it
-- came from, the lines covered, the total amount of lines, the
-- branches covered and the total amount of branches.
data Coverage = Coverage String Int Int Int Int
-- Then we need a way to convert the string data into a list of
-- coverage data. We do this by regex. We try to match on each
-- string in the list, and then we choose to keep only the successful
-- matches. Returned is a list of coverage data that was represented
-- by the strings.
convert :: [String] -> [Coverage]
convert = catMaybes . map match
where match line = do
[name, cl, tl, cb, tb] <- matchRegex format line
return $ Coverage name (read cl) (read tl) (read cb) (read tb)
-- We need a way to summarise two coverage data bits. This can of course also
-- be used to summarise entire lists of coverage data, by folding over it.
sumCoverage (Coverage nameA clA tlA cbA tbA) (Coverage nameB clB tlB cbB tbB) =
Coverage (nameA ++ nameB ++ ",") (clA + clB) (tlA + tlB) (cbA + cbB) (tbA + tbB)
main = do
-- First we need to convert the strings to coverage data
let coverageData = convert stats
-- Then we want to filter out only the relevant data
relevantData = filter (\(Coverage name _ _ _ _) -> name `elem` corePkgs) coverageData
-- Then we need to summarise it, but we are only interested in the numbers
Coverage _ cl tl cb tb = foldl' sumCoverage (Coverage "" 0 0 0 0) relevantData
-- So we can finally print them!
printf "Line coverage: %.2f\n" (fromIntegral cl / fromIntegral tl :: Double)
printf "Branch coverage: %.2f\n" (fromIntegral cb / fromIntegral tb :: Double)

Here are some quickly-hacked, untested ideas applied to your code:
import numpy as np
import re
datafile = ('abc', 'd>11/23d>34/89d', 'e>25/65e>13/25e', 'f>36/92f>19/76')
core_pkgs = ('d', 'f')
covered_lines, total_lines, covered_branches, total_branches = 0, 0, 0, 0
for pkg in core_pkgs:
ptn = re.compile('.*'+pkg+'.*'+'>(\d+)/(\d+).*>(\d+)/(\d+).*')
matches = map(datafile, ptn.match)
statsList = [map(int, match.groups()) for match in matches if matches]
# statsList is a list of [cvln, tlln, cvbh, tlbh]
stats = np.array(statsList)
covered_lines, total_lines, covered_branches, total_branches = stats.sum(axis=1)
Well, as you can see I haven't bothered to finish off the remaining loop, but I think the point is made by now. There's certainly a lot more than one way to do this; I elected to show off map() (which some will say makes this less efficient, and it probably does), as well as NumPy to get the (admittedly light) math done.

This is the corresponding Clojure solution:
(defn extract-data
"extract 4 integer from a string line according to a package name"
[pkg line]
(map read-string
(rest (first
(re-seq
(re-pattern
(str pkg ".*>(\\d+)/(\\d+).*>(\\d+)/(\\d+)"))
line)))))
(defn scan-lines-by-pkg
"scan all string lines and extract all data as integer sequences
according to package names"
[pkgs lines]
(filter seq (for [pkg pkgs
line lines]
(extract-data pkg line))))
(defn sum-data
"add all data in valid lines together"
[pkgs lines]
(apply map + (scan-lines-by-pkg pkgs lines)))
(defn get-percent
[covered all]
(str (format "%.2f" (float (/ (* covered 100) all))) "%"))
(defn get-cov
[pkgs lines]
{:line-cov (apply get-percent (take 2 (sum-data pkgs lines)))
:branch-cov (apply get-percent (drop 2 (sum-data pkgs lines)))})
(get-cov ["d" "f"] ["abc" "d>11/23d>34/89d" "e>25/65e>13/25e" "f>36/92f>19/76"])

Translate ruby to python

I'm rewriting some code from Ruby to Python. The code is for a Perceptron, listed in section 8.2.6 of Clever Algorithms: Nature-Inspired Programming Recipes. I've never used Ruby before and I don't understand this part:
def test_weights(weights, domain, num_inputs)
correct = 0
domain.each do |pattern|
input_vector = Array.new(num_inputs) {|k| pattern[k].to_f}
output = get_output(weights, input_vector)
correct += 1 if output.round == pattern.last
end
return correct
end
Some explanation: num_inputs is an integer (2 in my case), and domain is a list of arrays: [[1,0,1], [0,0,0], etc.]
I don't understand this line:
input_vector = Array.new(num_inputs) {|k| pattern[k].to_f}
It creates an array with 2 values, every values |k| stores pattern[k].to_f, but what is pattern[k].to_f?

Try this:
input_vector = [float(pattern[i]) for i in range(num_inputs)]

pattern[k].to_f
converts pattern[k] to a float.

I'm not a Ruby expert, but I think it would be something like this in Python:
def test_weights(weights, domain, num_inputs):
correct = 0
for pattern in domain:
output = get_output(weights, pattern[:num_inputs])
if round(output) == pattern[-1]:
correct += 1
return correct
There is plenty of scope for optimising this: if num_inputs is always one less then the length of the lists in domain then you may not need that parameter at all.
Be careful about doing line by line translations from one language to another: that tends not to give good results no matter what languages are involved.
Edit: since you said you don't think you need to convert to float you can just slice the required number of elements from the domain value. I've updated my code accordingly.

Pretty printing a list of list of floats?

Basically i have to dump a series of temperature readings, into a text file. This is a space delimited list of elements, where each row represents something (i don't know, and it just gets forced into a fortran model, shudder). I am more or less handling it from our groups side, which is extracting those temperature readings and dumping them into a text file.
Basically a quick example is i have a list like this(but with alot more elements):
temperature_readings = [ [1.343, 348.222, 484844.3333], [12349.000002, -2.43333]]
In the past we just dumped this into a file, unfortunately there is some people who have this irritating knack of wanting to look directly at the text file, and picking out certain columns and changing some things (for testing.. i don't really know..). But they always complain about the columns not lining up properly, they pretty much the above list to be printed like this:
1.343 348.222 484844.333
12349.000002 -2.433333
So those wonderful decimals line up. Is there an easy way to do this?

you can right-pad like this:
str = '%-10f' % val
to left pad:
set = '%10f' % val
or in combination pad and set the precision to 4 decimal places:
str = '%-10.4f' % val
:
import sys
rows = [[1.343, 348.222, 484844.3333], [12349.000002, -2.43333]]
for row in rows:
for val in row:
sys.stdout.write('%20f' % val)
sys.stdout.write("\n")
1.343000 348.222000 484844.333300
12349.000002 -2.433330

The % (String formatting) operator is deprecated now.
You can use str.format to do pretty printing in Python.
Something like this might work for you:
for set in temperature_readings:
for temp in set:
print "{0:10.4f}\t".format(temp),
print
Which prints out the following:
1.3430 348.2220 484844.3333
12349.0000 -2.4333
You can read more about this here: http://docs.python.org/tutorial/inputoutput.html#fancier-output-formatting

If you also want to display a fixed number of decimals (which probably makes sense if the numbers are really temperature readings), something like this gives quite nice output:
for line in temperature_readings:
for value in line:
print '%10.2f' % value,
print
Output:
1.34 348.22 484844.33
12349.00 -2.43

In Python 2.*,
for sublist in temperature_readings:
for item in sublist:
print '%15.6f' % item,
print
emits
1.343000 348.222000 484844.333300
12349.000002 -2.433330
for your example. Tweak the lengths and number of decimals as you prefer, of course!