python, using a twisted based proxy and keep trace of initial url - python

i have found on StackOverflow and manage to use a skeleton proxy based on twisted, but i wonder if it can help me to solve my problem :
i would like to be able to remember the dependencies bewteen the data that pass through this proxy, (i'am new to this network behaviour and not know how to do it) that is i would like to be able to determine that this particular ajax answer is following i call made by this javascript which was itself loaded from this url/html page and so on.
is it possible ?
(as another example, i would like to be able to record/log in a similar way as the tree structure that is available when using Firebug, so that the proxy can log and say "i was first asked to go to this url", then, i see "this particular xyz.js file" and "this azerty.css" both where loaded as they depended on "this first url". Another example is for Ajax answer/response being able to tknow that it is linked to the initial page url page.à
(an so on, on a eventually recursive basis... asked in another I mean i need to determine which are the external files/data/ajax response loaded after by the initial html page passing through the proxy.)
can i do this kind of tracing from a proxy based on twisted ?
maybe i am wrong by thinking that a proxy can be able to do this without parsing/understanding the complete initial url. IF so my trouble wil lbe that i must handle Javascript and thus be able to parse and executing it :/
thanks
first edit : removing my attempts as asked. Sorry i have some difficulties in my question because i am not an expert in internet communication.
EDIT : Following Monoid comment, is it at least possible to be able to pair a particular Ajax answer with a javascript file or call from the proxy point of view ?

Related

How to access array values in Postman using python

I came across this question that was asked a while back to send array data in postman (Is it possible to send an array with the Postman Chrome extension?).
My problem is though, I want to be able to access all those variables sent. Currently, I'm only receiving the first one because I say:
key=shareholder, value=100
If I try
key=shareholder[0], value=100
key=shareholder[1], value=200
or even without the indexes,then I get a NONETYPE error for shareholder
code:
request_data={
"shareholder": []
}
request_data["shareholder"].append(int(request.form.get("shareholder")))
Would definitely appreciate the help. Sorry if anything I said is unclear
To access a variable in Postman is incredibly easy. BUT, Postman is Javascript application. You need to use python nowhere with postman. Javascript is a comprehensive one.
I didn't get what kind of variable you want to access, but if you give me a clear example I will tell you how to reach any level of variable in postman
MORE INFO:
https://www.getpostman.com/docs/v6/postman/environments_and_globals/variables

What approach to use for the below scenario?

Please help me with best/easiest approach for the below scenario:
Url A has 'n' number of stacks and under each stack many user would be added and removed in a regular phase. And, I have built a python script using API call (json) which added all the stacks and host information to Url B.
Now, whenever an event (adding/removing hosts under stacks) occurs in url A has to be reflected in url B. What would be the best approach to accomplish this?

How do I find out what I can do with the Pelican signals? I'm trying to write a plugin

The signals are sort-of described here but, then what?
For example, one of the signals is desribed:
invoked before writing each article, the article is passed as content
How do I change that content? How do I access it? What functions are available?
I've been looking at examples in the pelican plugins repo on github, but I'm still confused. (How did those people even learn how to write those plugins?)
I hardly know where to start.
You have to look at the source code of pelican. I think there is no better way.
For example, search for the signal you are interested in, e.g. article_generator_write_article: https://github.com/getpelican/pelican/search?utf8=%E2%9C%93&q=article_generator_write_article
Then, look into the search result, e.g. generators.py and click on the line number containing your signal. Of course, you could also create a clone and do all this locally. This depends on your way of working.
Surrounding code:
def generate_articles(self, write):
"""Generate the articles."""
for article in chain(self.translations, self.articles):
signals.article_generator_write_article.send(self, content=article)
write(article.save_as, self.get_template(article.template),
self.context, article=article, category=article.category,
override_output=hasattr(article, 'override_save_as'), blog=True)
As you can see, the signal call provides you with an article object. You can now 1) look in the source code to find the respective python class of this object to find out about its inner workings, methods and attributes or 2) go the hacky path and simply print the object's members print(article.__dict__).
I suppose, without having looked in the code, that article has an attribute content which contains the HTML code generated from your source file. This is where your desired change comes in.
Note that if you want to change the source code before processing this is not that easy. I just wrote a little plugin which is capable of doing this.
There you can also see the signal API in action. You simply have to connect a handler function to the desired signal.
I hope this helps :)

Python search script in an HTML webpage

I have an HTML webpage. It has a search textbox. I want to allow the user to search within a dataset. The dataset is represented by a bunch of files on my server. I wrote a python script which can make that search.
Unfortunately, I'm not familiar with how can I unite the HTML page and a Python script.
The task is to put a python script into the html file so, that:
Python code will be run on the server side
Python code can somehow take the values from the HTML page as input
Python code can somehow put the search results to the HTML webpage as output
Question 1 : How can I do this?
Question 2 : How the python code should be stored on the website?
Question 3 : How it should take HTML values as input?
Question 4 : How can it output the results to the webpage? Do I need to install/use any additional frameworks?
Thanks!
There are too many things to get wrong if you try to implement that by yourself with only what the standard library provides.
I would recommend using a web framework, like flask or django. I linked to the quickstart sections of the comprehensive documentation of both. Basically, you write code and URL specifications that are mapped to the code, e.g. an HTTP GET on /search is mapped to a method returning the HTML page.
You can then use a form submit button to GET /search?query=<param> with the being the user's input. Based on that input you search the dataset and return a new HTML page with results.
Both frameworks have template languages that help you put the search results into HTML.
For testing purposes, web frameworks usually come with a simple webserver you can use. For production purposes, there are better solutions like uwsgi and gunicorn
Also, you should consider putting the data into a database, parsing files for each query can be quite inefficient.
I'm sure you will have more questions on the way, but that's what stackoverflow is for, and if you can ask more specific questions, it is easier to provide more focused answers.
I would look at the cgi library in python.
You should check out Django, its a very flexible and easy Python web-framework.

Getting a URL with Python

I'm trying to do something similar to placekitten.com, wherein a user can input two strings after the base URL and have those strings alter the output. I'm doing this in Python, and I cannot for the life of me figure out how to grab the URL. In PHP I can do it with query string and $_REQUEST. I can't find a similar method in Python that doesn't rely on CGI.
(I know I could do this with Django, but that's serious overkill for this project.)
This is just by looking at the docs but have you tried it?
cherrypy.request.path_info
The docs say:
The ‘relative path’ portion of the Request-URI. This is relative to the script_name (‘mount point’) of the application which is handling this request.
http://docs.cherrypy.org/stable/refman/_cprequest.html#cherrypy._cprequest.Request.path_info

Categories

Resources