Writing Apache Storm Spouts and Bolts using Python and Pandas [closed] - python

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
I want to turn an existing python-based implementation (a data analysis tool on a stream of events) into a storm topology.
During the research phase my team and I used python pandas a lot to develop a prototype of our tool and found it to be very helpful in terms of programmer productivity. Now we want to create a storm topology that does the same thing and aim to reuse our existing python modules as bolts or at least make an informed decision about whether it is a good idea performance-wise to do so.
Any restrictions on using a python script depending on external libs as a Storm Bolt on a cluster? Also, does anybody have a feeling for what the performance penalty will be for using an interpreted and non-JVM language like Python instead of Java for our bolts? The pandas library itself is designed with high-performance in mind.
Thanks

Related

Deploying python scripts [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I am a python beginner and I am a little experienced in OO-programming in Java and PHP and also fucntional programming in R . Thus, my question is considering the general usage of python scripts in everyday use-cases.
I want to "learn" how to think/approach a problem that I do experience when facing a situation with my software where a "script" could help me out or improve something.
For instance, I've heard friends talking about their self-made python scripts to evenly mute the audio of movies to avoid loud outliers in explosive scenes, etc. Another example, in my case righ tnow, is to filter out certain pictures with no GPS-time meta information for the timezone in order to sort these fotos in accordance with the others.
I really want to get the essence and recipe based on the aforementioned examples to better integrate Python in my everyday life and get an intuitive feeling for it. (i.e. how would a simple script look like that takes a picture, filters out its meta data, and does something -> where do I have to run the script so I can call the function with these .JPG files as its arguments?).
I would also be glad if some of you could recommend some practical tutorials or literature.
Thank you in advance :)
P.S. I know it is not a concrete question but rather it is intended to get a glimpse on a wide field of usage and thinking - but I want to get this essential take away that motivates me and shows me the direction.

Controlling computer hardware using Python [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I'm a beginner programmer . I want to build an OS using Python and assembly . And for that I guess I will have to control the hardware. My question is " Is it possible to control computer's hardware parts using python ."
Unfortunately Python is classified as a very high level programming
language. It cannot be used, for example, to directly access
hardware and perform low-level data structure manipulation. It is
completely dependent on something to abstract the hardware from it,
and that is the Kernel. It is, however, technically possible to create
an operating system centered on Python, that is; have only the very
low level stuff in written in C and assembly and have most of the rest
of the operating system written in Python.
This article discusses with more detail what languages are
suitable for writing operating system kernels.
More

Policy based design in Python [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
I was very impressed by the policy based design described in Modern C++ Design by Andrei Alexandrescu and tried it successfully in some light weight programs. Now I have to write a real world system in Python and I think that approach would be very useful here. However, I can't find a single example of this approach in Python. Is it not recommended in Python or are there better alternatives? Can someone point me to an example of policy based design in Python? My aim is to develop an auctioning system and I want to be able to choose the auction strategy - English, Dutch, Silent, etc - at run time.
Since Python and Ruby are so similar, I guess an example in Ruby will also do.
My aim is to develop an auctioning system and I want to be able to choose the auction strategy - English, Dutch, Silent, etc - at run time.
You could just use strategy pattern (also known as the policy pattern) to accomplish exactly this.
You could also use mixins to provide the strategy instead of composition.

Python - First Interface with a Program [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I have spent the last six months learning python as a way to automate my working environment. So far I have automated data extraction and report downloading from various web-based sources through the use of webcrawlers, interacted with excel files, created visual representations of data through matplotlib, and removed almost all the monotony from bank reconciliation.
I now come to a new task which takes up a large amount of my daily workload. We use an accounts program called Sage 50 Accounts. I effectively want to begin to learn how to manipulate the data contained within this program so that my daily thought patterns can be put into Python code.
Because this hasn't been done, there's no pre-made API. So my question is:
When wishing to interact with a new program through Python, how does a programmer begin such an inquiry?
Please accept that this question is only vague and general because I'm incredibly new to such a task.
SData is Sage's general data access API layer and should suit your purposes.
Otherwise you might need to invest in or obtain a Sage Development SDK.

web.py and flask [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
For these pythonic web frameworks;
They looks similar, except a few minor differences like their url handling schemes. So it is very hard to choose one of them.
I want to learn from your experiences that which one more suits for a web application that is small in size, but need long term support (initial application is small but it is supposed to became larger over time) and why?
Flask is a new and a very active project with good documentation and guidelines for new development and plugins. Its community is quite large and Armin (the lead) is an A grade programmer. Hence it is definitely the better contender.
Web.py is an older and more mature project which is more a library than a framework. It's what we use for the Internet Archives Open Library website. The documentation is spotty but the code is rock solid and it's over the years accrued a lot of useful utilities into it.
The final decision is yours but for a new project, I'd go with Flask.

Categories

Resources