Develop Reference

Why does my C code and Python code take more time just for itration in for loop

I have a Python code and a C code.
My Python code is this:
import os
import time
count = 0
number_of_itration =100000000
begin = time.time()
for i in range(number_of_itration):
count += 1
end = time.time()
print(f'Sum is {count}')
print(f"Total runtime of the program is {end - begin} Seconds")
This code takes 12 seconds to execute.
I have a similar C Code:
int main (int argc, char *argv[])
{
if (InitCVIRTE (0, argv, 0) == 0)
return -1;
start_t = clock();
for(i=0; i< 100000000; i++)
{
count= count+1;
}
end_t = clock();
total_t = (double)(end_t - start_t) / CLOCKS_PER_SEC;
printf("Total time taken by CPU: %f\n", total_t );
RunUserInterface ();
return 0;
}
This takes 0.395 seconds.
And I have a labview code:
Which takes just 0.093 seconds.
Where am I going wrong? I was expecting C to run faster.
My C IDE is Lab windows CVI, which is from National Instruments.
My system configuration is:
How can I optimize C code for processor?

If you have debugging turned off, LabVIEW will constant fold that For Loop during compilation. It won't take any time at all to execute because it'll all reduce to a single pre-computed constant value. Change the input to the For Loop's shift register from a constant to a control and add the control to the connector pane. You should see the time increase to be about equal to the C code. I suspect if you change your compile options on the C code, you'd see the time there decrease.
Python doesn't do those kinds of optimizations.

How to print a string as an input for a stdin stream?

How can I pass a stdin stream a custom string using for example python with printf? I can't do it before starting the executable because it's something i need to do after some other inputs.
The code is :
void play(){
int guess=0;
char buf[CHOICEBUFSIZE];
long long int attempt=0;
time_t timer;
int value;
time(&timer);
timer /= 60;
timer *= 60;
//printf("<<<<<< Time: %t\n", timer);
srand(timer);
do {
attempt += 1;
value = rand();
puts(" > Gimme the magic number or 0 to gtfo!");
printf(" < ");
//RIGHT DOWN HERE
fgets(buf, CHOICEBUFSIZE, stdin);
guess = atoi(buf);
printf("%d\n", value);
if (guess == value){
win(attempt);
return;
}
else {
puts(" > Not even close!\n > Try again!");
}
} while (guess);
}
Here's a screen showing where I need to input it.
Thanks in advance. I'm sure the answer is simple but I couldn't find anything similiar to my case.

stdin is an input stream, so you cannot write to it. Since it sounds like you're trying to perform two tasks in parallel and have one communicate with the other, you may want to look into using pthreads or fork().

Python File Generator in Perl [duplicate]

I am learning Perl at my work and enjoying it. I usually do my work in Python but boss wants Perl.
Most of the concepts in Python and Perl match nicely: Python dictionary=Perl hash; Python tuple=Perl list; Python list=Perl array; etc.
Question: Is there a Perl version of the Python form of an Iterator / Generator?
An example: A Classic Python way to generate the Fibonacci numbers is:
#!/usr/bin/python
def fibonacci(mag):
a, b = 0, 1
while a<=10**mag:
yield a
a, b = b, a+b
for number in fibonacci(15):
print "%17d" % number
Iterators are also useful if you want to generate a subsection of a much larger list as needed. Perl 'lists' seem more static - more like a Python tuple. In Perl, can foreach be dynamic or is only based on a static list?
The Python form of Iterator is a form that I have gotten used to, and I do not find it documented in Perl... Other than writing this in loops or recursively or generating a huge static list, how do I (for ex) write the Fibonacci subroutine it in Perl? Is there a Perl yield that I am missing?
Specifically -- how do I write this:
#!/usr/bin/perl
use warnings; use strict; # yes -- i use those!
sub fibonacci {
# What goes here other than returning an array or list?
}
foreach my $number (fibonacci(15)) { print $number . "\n"; }
Thanks in advance to being kind to the newbie...

The concept of an iterator is a little different in Perl. You basically want to return a one-use subroutine "closed" over the persistent variables.
use bigint;
use strict;
use warnings;
sub fibonacci {
my $limit = 10**( shift || 0 );
my ( $a, $b ) = ( 0, 1 );
return sub {
return if $a > $limit;
( my $r, $a, $b ) = ( $a, $b, $a + $b );
return $r;
};
}
my $fit = fibonacci( 15 );
my $n = 0;
while ( defined( my $f = $fit->())) {
print "F($n): $f\n";
$n++;
}
And if you don't like the while loop, then here is two shots at some syntactic sugar, which basically accomplish an each-item loop.:
sub iterate ($$) {
my $iter = shift;
my $action = shift;
while ( defined( my $nextval = $iter->())) {
local *_ = \$nextval;
$action->( $_ );
}
return;
}
iterate fibonacci( 15 ) => sub { print "$_\n"; };
sub iter (&$) {
my $action = shift;
my $iter = shift;
while ( defined( my $nextval = $iter->())) {
local *_ = \$nextval;
$action->( $_ );
}
return;
}
iter { print "$_\n" } fibonacci( 15 );

For an even more flexible solution than Python's generators, I have written the module List::Gen on CPAN which provides random access lazy generator arrays:
use List::Gen;
my $fib; $fib = cache gen {$_ < 2 ? $_ : $$fib[$_ - 1] + $$fib[$_ - 2]};
say "#$fib[0 .. 15]"; # 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610
Since generators pretend to be arrays, they can mix seamlessly with normal perl code. There is also an object oriented approach:
my $fib; $fib = cache gen {$_ < 2 ? $_ : $fib->get($_ - 1) + $fib->get($_ - 2)};
say join ' ' => $fib->slice(0 .. 15);
In each case, the generator is lazy, calculating nothing upon creation, and then calculating only those values required to satisfy the slices. The recursive definition of the Fibonacci sequence calls itself many times, so the cache function is used to make sure each value is only calculated once.
You can also use generators as iterators:
while (my $num = $fib->next) {
last if $num > 10**15;
print "$_\n";
}
$fib->next can also be written $fib->(). Since the generator is still random access, you can $fib->reset() or $fib->index = 10;
Let me know if you have any questions.
Update:
I have released a new version of the module (0.80) that makes it easier to use iterative algorithms in generators. Here is an example that closely mirrors the OP's example:
use List::Gen '*';
sub fibonacci {
my $limit = 10**shift;
my ($x, $y) = (0, 1);
While {$_ < $limit} gather {
($x, $y) = ($y, take($x) + $y)
}
}
say for #{fibonacci 15};
if you use bigint; before or at the top of the sub, you can of course:
say for #{fibonacci 400}; # or more

The excellent Higher-Order Perl book (available for free at the specified link) contains a lot of information on related topics, and in particular has a whole chapter on iterators. By "higher order" the author implies using Perl's abilities as a functional language with first-class functions to implement all kinds of cool stuff. It really is a very good book - I read most of it, and the chapters on iterators and streams are terrific. I highly recommend to at least skim through it if you plan to write Perl code.

There is a similar method to produce a Iterator / Generator, but it is not a "first class citizen" as it is on Python.
In Perl, if you do not see what you want (after a MANDATORY trip to CPAN FIRST!), you can roll your own that is similar to a Python iterator based on Perl closures and an anonymous subroutine.
Consider:
use strict; use warnings;
sub fibo {
my ($an, $bn)=(1,0);
my $mag=(shift || 1);
my $limit=10**$mag;
my $i=0;
return sub {
($an, $bn)=($bn, $an+$bn);
return undef if ($an >=$limit || wantarray );
return $an;
}
}
my $num;
my $iter=fibo(15);
while (defined($num=$iter->()) ) { printf "%17d\n", $num; }
The sub fibo maintains a Perl closure that allows persistent variables to be maintained. You can do the same by having a module, similar to C / C++. Inside fibo an anonymous subroutine does the work of returning the next data item.
To quote from the Perl Bible "You will be miserable until you learn the difference between scalar and list context" -- p 69 (A highly recommended book btw...)
In this case, the annon sub only returns a single value. The only looping mechanism that I know of in Perl that can work in scalar context is while; The others try to fill the list before proceeding I think. Therefor, if you called the anon sub in list context, it will dutifully return the next fibonacci number, unlike Python's for iterators, and the loop would terminate. That is why I put the return undef if .... wantarray because it does not work in list context as written.
There are ways to fix that. Indeed, you can write subroutines that act like map foreach etc but it is not as straightforward as Python's yield. You will need an additional function to use inside a foreach loop. The tradeoff is the Perl approach has tremendous power and flexibility.
You can read more about Perl iterators in Mark Jason Dominus' excellent book "Higher Order Perl" Chapter 4 is all about Interators brian d foy also has an excellent article on Interators in the Perl Review.

There's a good practical example here and a PDF article here... but I'm too rusty in Perl to try to implement your challenge directly (as you'll see, both the example and the approach in the PDF use a less direct approach).

In this case, memoization can be used.
use strict;
use warnings;
use Memoize;
memoize('fib');
foreach my $i (1..15) {
print "$i -> ",fib($i),"\n";
}
sub fib {
my $n = shift;
return $n if $n < 2;
fib($n-1) + fib($n-2);
}

Here is a response tailored to conform closely to the question as originally posed.
Any perl module that implements lazy lists (e.g. List::Gen, Memoize, etc.) and also lets you supply your own generator subroutine (I don't mean 'generator' as in Python) will allow you to do as shown in this example. Here the module that lazily produces the list is called Alef.
#!/usr/bin/perl -w
use strict; use warnings;
use Alef;
my $fibo;
BEGIN {
my ($a, $b) = (0, 1);
$fibo = sub {
($a, $b) = ($b, $a+$b);
$a;
}
}
my $fibonacci = new Alef($fibo);
foreach my $number ($fibonacci->take(15)){ print $number . "\n"; }
Here is the output:
[spl#briareus ~]$ ./fibo.pl
0
1
1
2
3
5
8
13
21
34
55
89
144
233
377
There is nothing magical happening behind the scenes with the lazy list module used here. This is what Alef's take subroutine looks like.
sub take {
my ($self,$n) = (#_);
my #these = ();
my $generator = $self->{'generator'};
for (1..$n){
push(#these,$self->{'this'});
$self->{'this'} = &$generator($self->{'this'});
}
#these;
}

There are a few iterator/generator modules on CPAN that would help here. Here is your example directly translated to the Coro::Generator module:
use 5.016;
use warnings;
use Coro::Generator;
sub gen_fibonacci {
my $mag = shift;
generator {
my ($a, $b) = (0, 1);
while ($a <= 10 ** $mag) {
yield $a;
($a, $b) = ($b, $a + $b);
}
yield undef; # stop it!
};
}
my $fibonacci = gen_fibonacci(15);
while (defined (my $number = $fibonacci->())) {
printf "%17d\n", $number;
}

Better way to parse and search a string?

I have been looking to speed up a basic Python function which basically just takes a line of text and checks the line for a substring. The Python program is as follows:
import time
def fun(line):
l = line.split(" ", 10)
if 'TTAGGG' in l[9]:
pass # Do nothing
line = "FCC2CCMACXX:4:1105:10758:14389# 81 chrM 1 32 10S90M = 16151 16062 CATCACGATGGATCACAGGTCTATCACCCTATTAACCACTCACGGGAGCTTTCCATGCATTTGGTATTTTCGTCTGGGGGGTGTGCACGCTTAGGGGATAGCATTG bbb^Wcbbbbccbbbcbccbba]WQG^bbcdcb_^_c_^`ccdddeeeeeffggggiiiiihiiiiihiiihihiiiihghhiihgfgfgeeeeebbb NM:i:1 AS:i:85 XS:i:65 RG:Z:1_DB31"
time0 = time.time()
for i in range(10000):
fun(line)
print time.time() - time0
I wanted to see if I could use some of the high level features of Rust to possibly gain some performance, but the code runs considerably slower. The Rust conversion is:
extern crate regex;
extern crate time;
use regex::Regex;
fn main() {
let line = "FCC2CCMACXX:4:1105:10758:14389# 81 chrM 1 32 10S90M = 16151 16062 CATCACGATGGATCACAGGTCTATCACCCTATTAACCACTCACGGGAGCTTTCCATGCATTTGGTATTTTCGTCTGGGGGGTGTGCACGCTTAGGGGATAGCATTG bbb^Wcbbbbccbbbcbccbba]WQG^bbcdcb_^_c_^`ccdddeeeeeffggggiiiiihiiiiihiiihihiiiihghhiihgfgfgeeeeebbb NM:i:1 AS:i:85 XS:i:65 RG:Z:1_DB31";
let substring: &str = "TTAGGG";
let time0: f64 = time::precise_time_s();
for _ in 0..10000 {
fun(line, substring);
}
let time1: f64 = time::precise_time_s();
let elapsed: f64 = time1 - time0;
println!("{}", elapsed);
}
fn fun(line: &str, substring: &str) {
let l: Vec<&str> = line.split(" ")
.enumerate()
.filter(|&(i, _)| i==9)
.map(|(_, e) | e)
.collect();
let re = Regex::new(substring).unwrap();
if re.is_match(&l[0]) {
// Do nothing
}
}
On my machine, Python times this at 0.0065s vs Rusts 1.3946s.
Just checking some basic timings, the line.split() part of the code takes around 1s, and the regex step is around 0.4s. Can this really be right, or is there an issue with timing this properly?

As a baseline, I ran your Python program with Python 2.7.6. Over 10 runs, it had a mean time of 12.2ms with a standard deviation of 443μs. I don't know how you got the very good time of 6.5ms.
Running your Rust code with Rust 1.4.0-dev (febdc3b20), without optimizations, I got a mean of 958ms and a standard deviation of 33ms.
Running your code with optimizations (cargo run --release), I got a mean of 34.6ms and standard deviation of 495μs. Always do benchmarking in release mode.
There are further optimizations you can do:
Compiling the regex once, outside of the timing loop:
fn main() {
// ...
let substring = "TTAGGG";
let re = Regex::new(substring).unwrap();
// ...
for _ in 0..10000 {
fun(line, &re);
}
// ...
}
fn fun(line: &str, re: &Regex) {
// ...
}
Produces an average of 10.4ms with a standard deviation of 678μs.
Switching to a substring match:
fn fun(line: &str, substring: &str) {
// ...
if l[0].contains(substring) {
// Do nothing
}
}
Has a mean of 8.7ms and a standard deviation of 334μs.
And finally, if you look at just the one result instead of collecting everything into a vector:
fn fun(line: &str, substring: &str) {
let col = line.split(" ").nth(9);
if col.map(|c| c.contains(substring)).unwrap_or(false) {
// Do nothing
}
}
Has a mean of 6.30ms and standard deviation of 114μs.

A direct translation of the Python would be
extern crate time;
fn fun(line: &str) {
let mut l = line.split(" ");
if l.nth(9).unwrap().contains("TTAGGG") {
// do nothing
}
}
fn main() {
let line = "FCC2CCMACXX:4:1105:10758:14389# 81 chrM 1 32 10S90M = 16151 16062 CATCACGATGGATCACAGGTCTATCACCCTATTAACCACTCACGGGAGCTTTCCATGCATTTGGTATTTTCGTCTGGGGGGTGTGCACGCTTAGGGGATAGCATTG bbb^Wcbbbbccbbbcbccbba]WQG^bbcdcb_^_c_^`ccdddeeeeeffggggiiiiihiiiiihiiihihiiiihghhiihgfgfgeeeeebbb NM:i:1 AS:i:85 XS:i:65 RG:Z:1_DB31";
let time0 = time::precise_time_s();
for _ in 0..10000 {
fun(line);
}
println!("{}", time::precise_time_s() - time0);
}
Using cargo run --release on stable (1.2.0), I get about 0.0267 as compared to about 0.0240 for Python (CPython, 2.7.10). Given Python's in on strings is just a C routine, this is reasonable.
Impressively, on beta (1.3.0) and nightly (1.4.0) this decreases to about just 0.0122, or about twice the speed of CPython!

How do I debug code that segfaults unless run through gdb?

That's a single threaded code.
In particular: ahocorasick Python extension module (easy_install ahocorasick).
I isolated the problem to a trivial example:
import ahocorasick
t = ahocorasick.KeywordTree()
t.add("a")
When I run it in gdb, all is fine, same happens when I enter these instructions into Python CLI. However, when I try to run the script regularily, I get a segfault.
To make it even weirder, the line that causes segfault (identified by core dump analysis) is a regular int incrementation (see the bottom of the function body).
I'm completely stuck by this moment, what can I do?
int
aho_corasick_addstring(aho_corasick_t *in, unsigned char *string, size_t n)
{
aho_corasick_t* g = in;
aho_corasick_state_t *state,*s = NULL;
int j = 0;
state = g->zerostate;
// As long as we have transitions follow them
while( j != n &&
(s = aho_corasick_goto_get(state,*(string+j))) != FAIL )
{
state = s;
++j;
}
if ( j == n ) {
/* dyoo: added so that if a keyword ends up in a prefix
of another, we still mark that as a match.*/
aho_corasick_output(s) = j;
return 0;
}
while( j != n )
{
// Create new state
if ( (s = xalloc(sizeof(aho_corasick_state_t))) == NULL )
return -1;
s->id = g->newstate++;
debug(printf("allocating state %d\n", s->id)); /* debug */
s->depth = state->depth + 1;
/* FIXME: check the error return value of
aho_corasick_goto_initialize. */
aho_corasick_goto_initialize(s);
// Create transition
aho_corasick_goto_set(state,*(string+j), s);
debug(printf("%u -> %c -> %u\n",state->id,*(string+j),s->id));
state = s;
aho_corasick_output(s) = 0;
aho_corasick_fail(s) = NULL;
++j; // <--- HERE!
}
aho_corasick_output(s) = n;
return 0;
}

There are other tools you can use that will find faults that does not necessarily crash the program.
valgrind, electric fence, purify, coverity, and lint-like tools may be able to help you.
You might need to build your own python in some cases for this to be usable. Also, for memory corruption things, there is (or was, haven't built exetensions in a while) a possibility to let python use direct memory allocation instead of pythons own.

Have you tried translating that while loop to a for loop? Maybe there's some subtle misunderstanding with the ++j that will disappear if you use something more intuitive.