Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
I am working on a project where there is a necessity to store considerable data. I was wondering what is the difference between using SQL and the datascience library in python. I intend to use SQL from its python based libraries too or use a csv file to store info if I am going to use "datascience". I am leaning very much towards "datascience" as I find the following advantages:
It is subjectively very easy to use for me. I make much less mistakes.
With my limited knowledge in runtime, I think the datascience library will be more efficient.
Most importantly, it has many inbuilt functions that could allow me to make easier functions.
However, since so many people are using SQL, I was wondering if I am missing something major, particularly in scalability.
Some people online said that SQL allows us to store files on a database, but I do not see how that makes a difference. I can simply store the file in a folder on a system and save the link in the "datascience" table.
The "datascience library" is only intended to be a tool for teaching basic concepts in an academic entry level class. Unless you are taking such a class, you should ignore it and learn more standard tools.
If it helps you, you can learn Data Science using Pandas starting just from flat data files, such as CSV and JSON. You will absolutely need to learn to interface with SQL and NoSQL servers eventually. The advantages of a database over flat files are numerous and well described elsewhere.
It's up to you whether you want to learn Pandas first and SQL second, or SQL first. Many people in the real world would have learned SQL before Python/Pandas/Data Science, so you may want to go that route.
If you go ahead and study that datascience library, you will learn some concepts, but will then have to re-learn everything in there "for real." Maybe this is best for your learning style, maybe it isn't. We don't know you well enough. Do you want academic hand holding or do you want to do things the real way?
Good luck and enjoy your journey.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
I'm writing some system which uses a lot of rules. It's time for me to organize them and make them efficient. Main requirements are - business friendly, easy to understand, easy to find, easy to maintain, testable.
This question is not about how to create rule engine. I'm not writing one. My goal is to find way to maintain lot of rules in one place. and make it easy. I need some expertise advice how to do so, what approach to take. Below are examples what I did already to show that I'm working on this task and not simply ask somebody to do my job.
So far I have 3 approaches:
1) Array typed:
item=context.GetNextItem()
if ['banana','apple','orange'].Contains(item): EatRaw(item)
if ['banana','apple','potato'].Contains(item): BakeAndEat(item)
if ['meat','egg','potato','fish'].Contains(item): FryAndEat(item)
if ['pasta','egg','potato'].Contains(item): BoilAndEat(item)
2) Separated file for each item:
item=context.GetNextItem()
execfile(str(item)+'.py')
#banana.py:
EatRaw(item)
BakeAndEat(item)
#potato.py:
BakeAndEat(item)
FryAndEat(item)
BoilAndEat(item)
3) Database stored:
item=context.GetNextItem()
SQL = "SELECT rule FROM rules where item='"+str(item)+"';"
for row in cursor.execute(SQL):
eval( row.rule+"(str("+item+"))" )
Table RULES
banana,EatRaw
banana,BakeAndEat
potato,BakeAndEat
potato,FryAndEat
potato,BoilAndEat
3.a) Data in file
File RULES.txt
banana,EatRaw
banana,BakeAndEat
potato,BakeAndEat
potato,FryAndEat
potato,BoilAndEat
This file could be considered as a UI.
Each approach has it's own cons and pros but, to be honest, I'm not satisfied with any. Files grow, became bulky, hard to search, maintain and understand. Any other approach or suggestion is welcome.
Lets zoom in on this statement:
Main requirements are - business friendly, easy to understand
However, your approaches so far are easy for programmers to understand, but not particularly easy for business users to understand.
You are approaching this problem from the wrong direction, you are starting with the "which datastructure is good for ergonomics", rather than "How business will users view or modify the 'rules'"
Start with a good few rounds of UI design. Once you've got those, put it in front of potential users (if you have any), then the implementation will follow naturally, whichever closely models or supports the way the resulting UI works and is used.
edit:
a "ui" need not be a fancy single page javascript application, it can be a text file on a particular network share that gets read every day by a cron job, that's still a "user interface". Design that in a way that is compatible with both the business users needs and the available budget.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I am trying to create a virus scanner in Python, and I know that signature based detection is possible, but is heuristic based detection possible in Python, ie. run a program in a safe environment, or scan the program's code, or check what the program behaves like, and then decide if the program is a virus or not.
Python is described as a general purpose programming language so yes, this is defiantly possible but not necessarily the best implementation. In programming, just like a trade, you should use the best tools for the job.
It could be recommended prototyping your application with Python and Clamd and then consider moving to another language if you want a closed source solution, which you can sell and protect your intellectual property.
Newb quotes:
Anything written in python is typically quite easy to
reverse-engineer, so it won't do for real protection.
I disagree, in fact a lot but it is up for debate I suppose. I really depends how the developer packages the application.
Yes, it is possible.
...and...
No, it is probably not the easiest, fastest, best performing, or most efficient way to accomplish the task.
Well, sure it's possible. Python is turing-complete, so you can use it to the same ends as you can use other programming languages like C++. And you can certainly do a primitive signature-based or code-inspecting check in Python without great difficulty. So the answer to that question is yes.
Now for the deeper question: are you asking whether Python is a good tool for this job? I don't think so, primarily because Python Code is Hard to Obscure, which means that if you develop an anti-virus in Python, it becomes weak the moment you give it to other people. That's because a virus developer will find it easy to inspect your anti-virus engine, since you will not be able to obscure your python code. That means that they can find vulnerabilities in your virus scanner easily.
Indeed, one of the key components of a good anti-virus is making it as hard to reverse-engineer as possible, so that virus developers won't figure out what the weaknesses of your anti-virus engine are. Anything written in python is typically quite easy to reverse-engineer, so it won't do for real protection.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 9 years ago.
Improve this question
I am a newbie to python and its framework Django. I am learning both the technologies parallel from its documentation, so I thought if there are some projects which I can use as reference for learning then it will help me learn the things fast. Can someone suggest me any such links.
I would suggest that you learn Python well, really well, since django uses Python's myriad of features to almost their fullest extent. I would start off with any random Python tutorial, but Zed Shaw's Learn Python the Hard Way is the best, and you can find a free HTML version of what he wrote. You can then get a book that teaches you to make games (great way to learn); Invent your own games with Python.
Well since you're a newbie, you'll probably want to start off by making a blog or something and then slowly moving up and making more complicated websites. The best approach to learning django is to first make something simple, then look at other people's code and learn from that. You an pick up a lot from reading other people's code and how they've structured their logic.
For a free resource, I would check out Mike Hibbert's Django tutorials on YouTube. He teaches you pretty much everything. Don't worry about the IDE he is using, just use a text editor like Sublime Text 2.
After you've made your simple website, its best that you start reading a book called Two Scoops of Django, by Daniel Greenfield. Really good tips and tricks on how to use django right. Great for newbies.
After that, start reading code. Read other people's code and understand how they made their sites. You can take a look at many sample django projects on the web. I would suggest that you go over to github and just randomly search for django projects. Thats what I did, and I think looking at other people's code is the best way to learn.
Lastly, if you want to get even more into django, join the IRC. Great bunch of guys there, not as lively as the Rails bunch, but they're pretty awesome.
Seconding the tutorial. It really is one of the most comprehensive and easy-to-follow framework tutorials out there. By the time you've finished it you'll have a much greater understanding of how all the django pieces fit together. It also has plenty of room to expand so you can experiment with new ideas.
Completing the tutorial has an added benefit: the django documentation frequently refers back to the tutorial app in its examples. So having the tutorial project available means you can quickly try out the features you're reading up on in an actual project.
One of the things that I (and possibly most people that are new to python as well as django) struggled with was separating "what bits are python?" from "what bits are django?". There's no real quick-fix for this other than gaining experience but like another answer mentioned, learning python on the side (maybe doing a few projects just using python) can help you with writing clean code within django too, and also see what is "missing" without django.
First start with Python as indicated.
Then, with regard to Django: I learnt a lot from the http://www.gettingstartedwithdjango.com/ video series, especially on set up and interesting packages to use.
No need to watch the video if you don't want to, as everything is outlined below the video as well.
Next to that, http://www.effectivedjango.com is a great help.
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 11 years ago.
Improve this question
I am new to programming. I decided to stick with Python after messing around with objective-c.
I'm beginning to get a grasp on most of the basic concepts, but as an artist, I think visually. This said, I would like to attempt to work in reverse. In other words, I'd like to start out with a complete, functional piece of software and look at the code and mess around with it to see what does what. This is kind of a similar approach to CAD modeling, where you can see which features in the model affect it, and how. This may not make sense to those who are not familiar with CAD, but I hope someone out there can relate?
Exploring existing projects is an excellent way to learn programming. Breaking existing code and fixing it is an even better way.
There are many open source projects out there that you might want to explore. Some of them are code libraries, while others are entire applications. As you are new to programming, you may want to consider looking at projects that are not too gigantic as it might be overwhelming.
Additionally, it may be even more helpful for you to dissect a program that you would actually use. I have not used this program, but PythonCAD might interest you:
http://sourceforge.net/projects/pythoncad/
This most likely does not fall into the small or simple category of open source projects but it sounds like you may already understand the intended functionality which should be of considerable aid to you in your exploration.
Browse through Google Code for open source Python projects that are of interest to you:
http://code.google.com/query/#q=python
I recommend to take a look at the PyRoom project, a no non-sense text editor for writing without distractions. It's barely 2K lines of code and seems simple to understand. You can access the repository here.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I am learning Python because it appeals to me as a mathematician but also has many useful libraries for scientific computing, image processing, web apps, etc etc.
It is frustrating to me that for certain of my interests (eletronic music or installation art) there are very specific programming languages which seem better suited to these purposes, such as Max/MSP, PureData, and ChucK -- all quite fascinating.
My question is, how should one approach these different languages? Should I simply learn Python and manage the others by using plugins and Python interpreters in them? Are there good tools for integrating the languages, or is the proper way simply to learn all of them?
I would say learn them all. While it's true that many languages can do many things, specialised languages are usually more expressive and easier to use for a particular task. Case-in-point is while most languages allow shell interaction and process control very few are as well suited to the task as bash scripts.
Plugins and libraries can bridge the gap between general and specialised languages but in my experience this is not always without drawbacks - be they speed, stability or complexity. It isn't uncommon to have to compile additional libraries or apply patches or use untrusted and poorly supported modules. It also isn't uncommon that the resulting interface is still harder to use than the original language.
I know about 15 languages well and a few of those very well. I do not use my prefered languages when another is more suitable.
This thread is a little old, but I wanted to point out that the majority of the mature audio development environments e.g. supercollider/max-msp/pure data can be controlled via open sound control. You can google up a better description of OSC, but suffice it to say that it allows you to send control data to synths built in these environments similar to how MIDI works, but way more extensive. This does not solve the problem of actually building synths in python per se but it allows you to "drive" these other environments without having to know the ins and outs of the language.
It's perfectly possible to build good interfaces from Python to such specialized languages: one example in point is RPy, which lets you drive R (for statistics) from Python (for all sort of general-purpose stuff).
Of course, one has to be competent in both languages - and such bridges, unfortunately, will not already exist for every given pair of one general purpose language and one specialized one. "Learning all of them", if you want to use all of them, remains the royal road!
Python would be a great language to learn, since it works well with a lot of other languages. It makes a great general purpose language as well as a "glue" language. Spend time learning the languages you are interested in, and keep Python knowledge around for it's flexibility and power. I don't think I would recommend trying to learn them all unless you really have the time.
You may interested to know that PureData has a python extension.