I'm exploring many technologies, but I would like your input on which web framework would make this the easiest/ most possible. I'm currently looking to JSP/JSF/Primefaces, but I'm not sure if that is capable of this app.
Here's a basic description of the app:
Users log in with their username and password (maybe I can somehow incorporate OPENID)?
With a really nice UI, they will be presented a large list of questions specific to a certain category, for example, "Cooking". (I will manually compile this list and make it available.)
When they click on any of these questions, a little input box opens up below it to allow the user to put in a link/URL.
If the link they enter has the same question on that webpage the URL points to, they will be awarded one point. This question then disappears and gets added to a different page that has a list of all correctly linked questions.
On the right side of the screen, there will be a leaderboard with the usernames of the people with the top ten points.
The idea is relatively simple - to be able to compile links to external websites for specific questions by allowing many people to contribute.
I know I can build the UI easily with Primefaces. [B]What I'm not sure is if JSP/JSF gives the ability to parse HTML at a certain URL to see if it contains words.[/B] I can do this with python easily by using urllib, but I can't use python for web GUI building (it is very difficult). What is the best approach?
Any help would be appreciated!!! Thanks!
The best approach is whatever is best for you. If Python isn't your strength but Java is, then use Java. If you're a Python expert and know little Java, use Python.
There are so many resources on the Internet supporting so many platforms that the decision really comes down to what works best for you.
For starters, forget about JSP/JSF. This is an old combination that had many problems. Please consider Facelets/JSF. Facelets is the default templating language in the current version of JSF, while JSP is there only for backwards compatibility.
What I'm not sure is if JSP/JSF gives the ability to parse HTML at a certain URL to see if it contains words.
Yes it does, although the actual fetching of data and parsing of its content will be done by plain Java code. This itself has nothing to do with the JSF APIs.
With JSF you create a Facelet containing your UI (input fields, buttons, etc). Then still using JSF you bind this to a so-called backing bean, which is primarily a normal Java class with only one or two JSF specific annotations applied to it (e.g. #ManagedBean).
When the user enters the URL and presses some button, JSF takes care of calling some action method in your Java class (backing bean). In this action method you now have access to the URL the user entered, and from here on plain Java coding starts and JSF specifics end. You can put the code that fetches the URL and does the parsing you require in a separate helper class (separation of concerns), or at your discretion directly in the backing bean. The choice is yours.
Incidentally we had a very junior programmer at our office use JSF for something not unlike what you are requesting here and he succeeded in doing it in a short time. It thus really isn't that hard ;)
No web technology does what you want. Parsing documents found at certain urls is out of the scope of building web interfaces.
However, each of Java's web technologies will give you, without limits, access to a rich and varied (if not too rich and much too varied) set of libraries and frameworks running on JVM. You could safely say that if there is a library for doing something, there will be a Java version available. Downloading and parsing a document will not require more than what is available in the standard library (unless you insist on injecting your dependencies or crosscutting your concerns), so no problems with doing your project with JSP, or JSF/Primefaces, or whatever.
Since you claim to already know Python, and since you will have to add some HTML/CSS anyway, I suggest you try Django. It's dead simple, has a set of OpenID plugins to choose from, will give you admin interface for free (so you can prime the pump with the first set of links).
Related
I am looking for advice on how to automate submitting data directly to a Siebel application at work using Python. Currently I enter the data into an Autohotkey GUI and when a button is selected it enters the data into Siebel for me using mouse moves and mouse clicks to select the right entries for each piece of data. Obviously this is prone to errors and I would like to make the application better if possible. Using an object oriented programming language would improve this greatly. Just to clarify, this is NOT for automation testing. The data and account/page that I am submitting too changes quite often. So, modules like Selenium, Mechanize, and BeautifulSoup won't work for this as far as I can tell. Since not everything has a form or a friendly label that I can submit data to. If anyone has experience with Siebel and knows a way to copy data from and submit data directly to different entries that would be great.
Right now my best option is to use modules like Pyautogui and Pywinauto to perform mouse moves and clicks to copy what my Autohotkey script does. But this seems inefficient and potentially prone to errors. There has to be a better way to accomplish the same thing using Python. I am just not certain how and I would appreciate any advice you guys may have. Even if that is "no there is no other way" it would help me figure out what to do next. Thanks in advance!
Interacting with the Siebel CRM Application can be done in a large number of ways (SOAP, REST, COM, Java, UI, to name a few supported) and the use-case and environment typically define the preferred approach. A decent Siebel developer/consultant will be able to help you make the right choice.
The ease and available tooling for automating the UI is largely dependent on the version of Siebel you run. Prior to OpenUI this was mainly the domain of large Test Automation vendors (HP, Mercury, Oracle) and required a separate license module to be purchased.
Post Open UI the web UI itself became a single DOM object and much more suitable to automation using open source test tooling like Selenium. With the Test Automation license module activate it will also introduce additional HTML attributes that help to create stable locators.
If interacting with the the UI is just a means to change data, then I would advise an alternate approach: directly interact with the business layer. The added advantage is that there is much more information in the data objects than is typically available in a single UI screen and is more structure.
The easiest approach is probably using the web services. The older versions support mainly SOAP but the latest version also support REST. Most programming languages have support for these approaches and will allow you to import their WSDL files. Keep in mind that you're dependent on the DEV team to extend these interfaces too when they add fields to the UI.
Another approach that gives the most flexibility is to directly interact with the business layer using Java (Bean) or COM. The java approach only requires two JAR files and Google has enough examples on this approach to explain how to use it. When Python is your preferred approach, then the COM interface is an interesting approach. This GitHub project has some good examples to get you on your way.
First off, I am very new to programming and I have a relatively basic understanding of python, and average understanding of html.
Using what I know in python, I am trying to create a basic strategy game, a bit like the likes of Age of Empires, or Command and Conquer, based on collecting resources and using it to build things, except using a simple text or button-clicking type interface. I can do a text interface fine, but its a bit boring, and I would like to use some images. I have had a 1 hour lecture on tkinter, but I have tried and failed to make anything remotely 'usable' from it. What I can do, is make decent looking html pages which would serve my purpose very well.
What I am wondering is if there is a simple way of executing python functions and calling/displaying python variables through a html page? The python functions do all the logic and present variables which represent current resource levels, production, storage capability, levels of buildings, etc. At the most basic level all I need is a way of displaying these variables, and having buttons which execute a function to say, upgrade a building, which recalculates production and all that, and returns the new set of values.
As a really simple example:
<p> Wheat production: *python integer representing production*</p>
<button type="button" onclick="*execute python variable*">Upgrade Wheat</button>
There would also, I imagine, be a need to somehow update the variables which are changed on the html page. So the button executes a function to upgrade wheat production, python now has a new value for the wheat production variable, and this needs to be updated on the page, whether this is automatic, or by some other method. I guess the simple way would be if pressing the button could also reload the page, but that seems a little clumsy.
Does anyone know of a simple way of doing this? Or perhaps a python library which might help me here?
Yes, there is a way of doing this.
There are two approaches to this. One is to have all the work done on the server, the other approach is to use Javascript.
The first approach is this: write a python script that generates your HTML. If you use Django, you will get a lot of work done for you, but you will also get a lot of stuff you don't want. Django does have a built-in template language. Django is beyond the scope of this answer. You will get to do exactly what you describe above; an example of a template might be <p> Wheat production: {{wheat_production}}</p> - your python code will set up a dict mydict={"wheat_production":10} and you will pass the name of your template file and the dict to a function which will spit out your page. You will also have to learn about HTML forms, if you haven't done so yet.
The other approach is to use Ajax - Javascript that, when your page is displayed (and, perhaps, when buttons are clicked, or at regular intervals) will send/receive some data to allow you to update your page. I suggest looking into JQuery to do some of the lifting for you. This means that you can update bits of the page without having to reload the entire thing. You will still have to write some code on the server to talk to the database, and send the output, usually as JSON, back to the client.
When writing this sort of thing, make sure all of your security is on the server side, and don't trust anything the user tells you. For example, if you store the number of gold pieces in a field on your form, it's going to take someone about 10 seconds to give themselves as much gold as they want. Similarly, if a player can sell a diamond for 20 gold pieces, make sure they have the diamond before giving them the gold pieces - you don't want to end up with a player with 1,000,000 gold pieces and negative a thousand diamonds. Javascript is 100% insecure, anything that Javascript can do, the player can also do.
Take a look at http://pyjs.org/
What is pyjs?
pyjs is a Rich Internet Application (RIA) Development Platform for
both Web and Desktop. With pyjs you can write your JavaScript-powered
web applications entirely in Python.
pyjs contains a Python-to-JavaScript compiler, an AJAX framework and a
Widget Set API. pyjs started life as a Python port of Google Web
Toolkit, the Java-to-JavaScript compiler.
You can compile Python programs to javascript, and also use their Python libraries to generate HTML. Here's an example from their getting started guide:
from pyjamas import Window
from pyjamas.ui import RootPanel, Button
def greet(sender):
Window.alert("Hello, AJAX!")
class Hello:
def onModuleLoad(self):
b = Button("Click me", greet)
RootPanel().add(b)
How to generate a random yet valid website link, regardless of languages. Actually, the more diverse the language of the website it generates, the better it is.
I've been doing it by using other people's script on their webpage, how can i not rely on these random site forwarding script and make my own?. I've been doing it as such:
import webbrowser
from random import choice
random_page_generator = ['http://www.randomwebsite.com/cgi-bin/random.pl',
'http://www.uroulette.com/visit']
webbrowser.open(choice(random_page_generator), new=2)
I've been doing it by using other people's script on their webpage, how can i not rely on these random site forwarding script and make my own?
There are two ways to do this:
Create your own spider that amasses a huge collection of websites, and pick from that collection.
Access some pre-existing collection of websites, and pick from that collection. For example, DMOZ/ODP lets you download their entire database;* Google used to have a customized random site URL;** etc.
There is no other way around it (short of randomly generating and testing valid strings of arbitrary characters, which would be a ridiculously bad idea).
Building a web spider for yourself can be a fun project. Link-driven scraping libraries like Scrapy can do a lot of the grunt work for you, leaving you to write the part you care about.
* Note that ODP is a pretty small database compared to something like Google's or Yahoo's, because it's primarily a human-edited collection of significant websites rather than an auto-generated collection of everything anyone has put on the web.
** Google's random site feature was driven by both popularity and your own search history. However, by feeding it an empty search history, you could remove that part of the equation. Anyway, I don't think it exists anymore.
A conceptual explanation, not a code one.
Their scripts are likely very large and comprehensive. If it's a random website selector, they have a huge, huge list of websites line by line, and the script just picks one. If it's a random URL generator, it probably generates a string of letters (e.g. "asljasldjkns"), plugs it between http:// and .com, tries to see if it is a valid URL, and if it is, sends you that URL.
The easiest way to design your own might be to ask to have a look at theirs, though I'm not certain of the success you'd have there.
The best way as a programmer is simply to decipher the nature of URL language. Practice the building of strings and testing them, or compile a huge database of them yourself.
As a hybridization, you might try building two things. One script that, while you're away, searches for/tests URLs and adds them to a database. Another script that randomly selects a line out of this database to send you on your way. The longer you run the first, the better the second becomes.
EDIT: Do Abarnert's thing about spiders, that's much better than my answer.
The other answers suggest building large databases of URL, there is another method which I've used in the past and documented here:
http://41j.com/blog/2011/10/find-a-random-webserver-using-libcurl/
Which is to create a random IP address and then try and grab a site from port 80 of that address. This method is not perfect with modern virtual hosted sites, and of course only fetches the top page but it can be an easy and effective way of getting random sites. The code linked above is C but it should be easily callable from python, or the method could be easily adapted to python.
I want to update/rewrite a small (10 page), simple website; 8 pages are entirely static and could be written in html, 1 page has a contact form and the other has to display a filterable list of clubs. At the moment the site is written in classic asp and uses dreamweaver templates for consistent pages.
My requirements are
A "masterpage" / Templating system, so all shared page elements are written in only 1 place.
Lightweight / low overhead framework
To learn a new language
I could use ASP.NET Webforms or ASP.NET MVC to get the masterpage, but they both come with overhead that isn't necessary for such a small site and on my godaddy hosting spinning up a site from cold is noticeably slower than a pure html page.
The clubs page will show a list of clubs filterable by location, but I don't want to use a database to store this list - there is another site that has the official list of clubs, but the system isn't capable of providing this as a service or other consumable resource so I would need to scrape the details periodically and cache them locally or use an iframe or something
I thought maybe Python or django might be good candidates but don't know enough to know. I now think that what I'm looking for is a "micro web framework". I've taken a quick look at the Mercurial Web Server which is written in python and that looks quite straightforward, but I don't have access to the hosted web server on Go Daddy, so can't install python...
Edit
I need this to run on my current shared hosting with GoDaddy on (IIS7)
Edit2
The list of clubs is maintained by the official HQ website, they occasionally add / remove clubs. I just need to keep my list up to date with theirs. I have been checking every few months (if I remember) and updating an MS SQL database, but that's hugely over the top. I was thinking of just pulling the details down into a json format and persisting it in a text file (once a month, or something) which I could then use as the basis of a table with jQuery filtering on. The club details are just text; Name of Club, Main contact, phone number, address and email address.
I would also like publishing to be simple, commit the code to Mercurial (or git) and have that run the site. I know bitbucket (and github) both serve static page sites (I'm not sure how I would get a contact us form to work in that environment - but it's the deployment model I would like)
The site I am looking to update is Seika Dojo
There is no need to run monsters to serve 10 almost static pages. If you plan to pull and cache some data out of the web, it is a way to go to update static HTML.
As another author mentioned HTML5 can help you. Take a look at jQuery for table filtration. As for page regeneration with common elements consider either jekyll/hyde or org-mode (using batch processing mode with emacs). You have a plenty of languages to choose from.
Well, I don't know about the other frameworks, but I have good experiences building a small site in NancyFx.
NancyFx supports multiple view engines. You could use the SuperSimpleViewEngine; masterpages come out of the box.
Getting started with Nancy is super easy.
I think you already know .NET/C#, but Nancy takes a lot of advantage of new dynamic features which are fun to play with.
Python is my favorite language, but I wouldn't use it for creating a simple website. I would recommend you to go with ready made CMS solutions, like Wordpress.
You would learn something new.
You won't have to implement any features (CMS + plug-ins will provide all you need).
You will get all the support you need.
It is easy to deploy on any hosting (since it's PHP based -yeap, sorry, php-).
Cactus:
https://github.com/koenbok/Cactus
is my current favourite static site generator - it uses Django templates to create a set of pages (in the 'build' directory) from a set of templates (in the 'pages' directory) and all the usual images and css in a 'static' directory.
Do your filterable table with Javascript on the client - it doesn't sound too complex. This lovely table grid component:
http://datatables.net/
might be just the ticket.
Pelican wasn't mentioned. It produces static pages and is done in Python. It meets all of the stated criteria.
See http://wiki.python.org/moin/WebFrameworks and play with a couple of the choices that tickle your fancy. I would tend toward the lighter frameworks based on what you described.
Although not exactly on the topic, since you don't need entirely static website, StaticMatic
may be of interest as one of the best lightweight tools for static content generation.
Flask and web.py would both be good choices. I don't think that Django is really suited for such a small project but you can definitely use it.
Perhaps look at cherrypy or webpy? They are very minimalist, python web frameworks designed for something like this. (I think django is too big for this small of an app, IMHO)
Also, take a look at Sinatra if you want to learn some ruby.
Express is good for some javascript experience.
For single-page apps, backbone.js is popular and really powerful, but might not be what you want.
But most importantly, have fun learning a new language!
I've been looking around further and the goal of a templating system can be met by just using xml includes, or Embedded JavaScript to pull in the header / footer / menu sections. All the 'work' is then done on the client browser, so the web server only needs to serve up static files which is about as lightweight as you can get.
Based on my experience, any 8 page static website someday needs dynamic features so it is always a good idea to start from something extensible.
Then the first decision is to use a Framework or a CMS. This depends on your expectation of the site getting bigger in the future, your ability to develop custom codes to achieve dynamic features and the structure and the requirements of the project in hand.
When we need a CMS we use,
Orchard if ASP .NET MVC 3: Take a look at our site www.dreamrain.com which looks like a static website but actually uses Orchard themes and modules for the dynamic features. If you don't need themes and modules, you can still create a 8 page website and then you can extend it in the future. Btw, we built this site in 5 days with 4 hours of development effort
Note: Orchard may need Full Trust so check its documentation and ask GoDaddy if they allow it.
WordPress if PHP,
When we need a Framework we use,
ASP .NET MVC 3 for .NET,
Django or Pyramid for Python,
Zend for PHP,
Grails for Groovy/Java/J2EE,
Play for SCALA
I want to make a Python script available as a service on the net. The script, which is my first 'proper' Python program, takes a txt file as argument and writes an image into the work directory. So:
How difficult is it for somebody who is new to Python and web development?
How much work is it?
Do I need a framework (Django, cherryPy, web2py)?
Are there good tutorials?
How do I avoid the server to be compromised?
What are my next steps?
==> What is the easiest way?
In the end it is enough, if it is a white page, with some text, and a button, which when clicked, opens a file dialog. After the txt is processed, the server should just return the image, which was written on the hard drive. Already I have access to a server which has Ubuntu installed through a friend.
[update]
Thanks for all your answers. After reading them I want to stress again, that I want to have it as minimal as possible. Srikar's suggestion sounds like the easiest one:
Put it in executable directory of your OS (commonly known as CGI
path). Provide a simple HTML form & upon form submission hit this
script which executes & returns back the image you want to display.
Any objections or comments? Do you know any tutorials for that?
[udpate2]
I found this SO answer: File Sharing Site in Python Is this a sensible approach?
It's not too difficult. Actually, it sounds like a good first project.
That too subjective to answer. An hour to days.
No, you don't need one, but I'd use one if I were you. They abstract away some of the stuff you really don't care about, and you'll learn a tool you can use again in the future.
Plenty. If you want a real rundown of how Python works for the web, read the HOWTO from Python.org. If you just want to learn how to do this one project, pick a framework and do their tutorial.
This question is so broad and complex that I'm not going to try to answer it. Search this site, or Google, for questions like that.
Your next step should be to pick a framework; I've used Django successfully. Just download it, follow the installation instructions, and work your way through their tutorial; it should tell you everything you need to know to do what you want. If you still have questions once you've learned how to do the basics, come back and ask again!
Edit: The answer to that other question will certainly work for you. There, they just receive a GET request and respond with data from a Python file. You need to receive a GET request, respond with an HTML page (easy enough), then respond to a POST request that includes an uploaded file (slightly more complicated) and run your python routine on the uploaded file and then respond with the created image (or a link to it).
Take a look at this page which includes a simple Python script to do file uploads. You should easily be able to modify it to do what you want.
How difficult is it for somebody who is new to Python and web development?
Depends on your level of knowledge.
How much work is it?
Depends on which method you choose to solve the problem.
Do I need a framework (Django, cherryPy, web2py)?
Not necessarily - you could get started by using the CGI (http://docs.python.org/library/cgi.html)
Are there good tutorials?
Yes, there are plenty. The Python docs are an excellent place to start.
How do I avoid the server to be compromised?
Again, depends on the method you choose to solve the problem, although there are commonalities.
What are my next steps?
Dare I say it again, choose a method, read the docs, have a play!
If its just as simple as you have described it. Then you might not even need Django. You could simply use CGI scripting. All of these design decisions, depend on whether
You need (or foresee) a SQL storage?
or a Content-Management-System?
Will you need multiple-user support?
Do you need tight security?
Do you need different privileges for different users?
Do you need an Admin to manage your site?
If the answer to above questions is atleast 60% correct, then you might consider Django. otherwise, just write a python script. Put it in executable directory of your OS (commonly known as CGI path). Provide a simple HTML form & upon form submission hit this script which executes & returns back the image you want to display. So, it all depends on the features you need...
In the end, I created what I needed with Flask.
They have a well documented pattern / tutorial on Uploading Files. The tutorial is understandable even for people with little python and web expericence.
To get a first working version it took me 2h and the resulting code was only 50 lines. This includes, starting the webserver, having a html file/form with file upload and serving a file back to the user.