Develop Reference

Example of a working OWASP Zap script with authenticated scan using API

Can someone please show a script that is capable of doing the above? I have found a good amount of instruction on the web and tried a lot of different things but still can't get Zap to login to the page to perform a full scan.
The best I get is something like this:
'http://XXX',
'http://XXX/robots.txt',
'http://XXX/sitemap.xml',
'http://XXX/webui',
'http://XXX/webui/index.html',
'http://XXX/webui/index.html?Password=ZAP&Username=ZAP',
'http://XXX/webui/login',
'http://XXX/webui/login/assets',
'http://XXX/webui/login/assets/images',
'http://XXX/webui/login/assets/images/companylogo.png',
'http://XXX/webui/login/assets/styles',
'http://XXX/webui/login/assets/styles/login.css',
'http://XXX/webui/login/login.js',
'http://XXX/webui/login/redirect.js',
'http://XXX/webui?Password=ZAP&Username=ZAP'
Many thanks
from zapv2 import ZAPv2
from random import randint
import socket
zap_ip = 'zap' #name of a Docker container running Zap
target = 'http://example.com'
auth_url = target + "webui/index.html"
scanners = ['90020', '90029']
# authorized Web UI user
username = test
password = test
auth_data = 'password={%password%}&username={%username#%}'
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
zap = ZAPv2(proxies={'http': 'http://' + zap_ip + ':' + str(port),
'https': 'http://' + zap_ip + ':' + str(port)})
new_context = randint(1, 100000000000)
session = zap.core.session_location
session_name = 'session_1.session' if zap.core.session_location == \
'session_0.session' else 'session_0.session'
zap.core.new_session(name=session_name)
zap.core.load_session(session_name)
context_id = zap.context.new_context(new_context)
zap.context.include_in_context(new_context, '.*')
zap.ascan.disable_all_scanners()
for scanner in scanners:
zap.ascan.enable_scanners(scanner)
all_rules = [scanner for scanner in \
zap.ascan.scanners() if scanner['enabled'] == 'true']
start_url = auth_url if auth_url else target
zap.urlopen(start_url)
auth_method_name = 'formBasedAuthentication'
authmethod_configparams = 'loginUrl=%s&loginRequestData=%s' % (auth_url, auth_data)
authcred_configparams = 'username=%s&password=%s' % (username, password)
zap.authentication.set_authentication_method(contextid=context_id,
authmethodname=auth_method_name,
authmethodconfigparams=authmethod_configparams)
user_id = zap.users.new_user(contextid=context_id, name=username)
zap.users.set_authentication_credentials(contextid=context_id,
userid=user_id,
authcredentialsconfigparams=authcred_configparams)
zap.users.set_user_enabled(contextid=context_id, userid=user_id, enabled=True zap.forcedUser.set_forced_user(context_id, user_id)
zap.forcedUser.set_forced_user_mode_enabled('true')
spider = zap.spider.scan_as_user(url=target, contextid=context_id,
userid=user_id, recurse='false')
while (int(zap.spider.status()) < 100):
time.sleep(2)
zap.ascan.scan(target)
zap.ascan.remove_all_scans()
zap.core.delete_all_alerts()
zap.context.remove_context(new_context)

Authentication is, in general, a pain. There are so many different ways authentication can be implemented its really difficult to provide anything other than very generic advice.
However the fact that you've got a URL like 'http://XXX/webui?Password=ZAP&Username=ZAP' implies you have not configured something correctly as these are the default values supplied by the ZAP spider.
If you can supply more details about what your application appears to expect and what you are doing then we should be able to help some more.

timeout and performance issues on redirecting inside django

I am currently having problems with Timeouts and performance on Django redirection. The issue was not visible until I was surfing to my locally hosted application with 2 devices and only one worker enabled on my localhost, timeout set to 30 seconds.
I have a views.py function that redirects a page, based on that is given the URL. I do a lookup for the pk in a table and return the url. I also have a counter that keeps track of the amount of forwards.
urls.py here:
url(r'^i/(?P<pk>[-\w]+)/$', frontendapp_views.item_view, name="item_view"),
The page redirects instantly to the "desired_url_forward", however, the connection stays open with the user, while in fact, the user has left my Django environment. This somehow leaves my worker waiting for 30 seconds while I was already forwarded to an external page, not allowing to process any other request with one worker.
I could increase the number of workers or shorten the timeout time, but that doesn't feel right as it is not fixing the core issue.
This is the only thing I found out on this topic but I am not skilled enough to understand this: https://github.com/requests/requests/issues/520
This is how the views.py looks like:
def item_view(request,pk):
pk_binairy = urlsafe_base64_decode(pk)
pk_int = int.from_bytes(pk_binairy, byteorder='little')
desired_url_forward_object = get_object_or_404(forwards,pk = pk_int)
channel_cleaned_utm = re.sub(' +',' ',"".join([request.GET.get('utm_source', ''),' ',request.GET.get('utm_medium', ''),' ',request.GET.get('utm_campaign', ''),' ',request.GET.get('utm_term', ''),' ',request.GET.get('utm_content', '')]))
channel_cleaned = request.META.get('HTTP_REFERER')
if channel_cleaned is None:
channel_cleaned = 'Direct Traffic'
visitor_ip_request = get_client_ip(request)
location_request = get_client_location(request, visitor_ip_request)
clickstat = clickstats(
urlid = pk_int,
user = desired_url_forward_object.user,
channel = channel_cleaned,
visitor_ip = visitor_ip_request,
city = location_request['city'],
region = location_request['region'],
country = location_request['country'],
device_type = request.user_agent.device.family,
browser = request.user_agent.browser.family,
browser_version = request.user_agent.browser.version_string,
operating_system = request.user_agent.os.family ,
operating_system_version = request.user_agent.os.version_string
)
clickstat.save()
if desired_url_forward_object.counterA <= desired_url_forward_object.counterB:
desired_url_forward = desired_url_forward_object.urlA
desired_url_forward_object.counterA = F('counterA') + 1
else:
desired_url_forward = desired_url_forward_object.urlB
desired_url_forward_object.counterB = F('counterB') + 1
desired_url_forward_object.save()
return redirect(desired_url_forward)
Anyone suggestions? Thanks for the help!

Python / Django - Local variable designed before assignment

I know this topic is talked many times in Stackoverflow but it concerns many different methods and I need help. I'm stuck since four hours ^^'
Here is the message : local variable 'menuItem' referenced before assignment
def B2BpartnerMenuDetailModify(request, partnerId, menuId, menuItemId):
message = ''
e = B2BpartnerUser(request, partnerId)
try:
menuDetail = Menu.objects.get(id=menuId)
except Menu.DoesNotExist:
return logoutUser(request)
if request.method == 'POST':
form = MenuDetailForm(request.POST, mySelf=partnerId)
if form.is_valid():
descrShort = form.cleaned_data['descrShort']
paragraph = form.cleaned_data['paragraph']
producteur = form.cleaned_data['producteur']
position = MenuItem.objects.filter(menuId = menuDetail).filter(paragraph = paragraph).count() + 1
menuItem = MenuItem(menuId = menuDetail)
menuItem.descrShort = descrShort
menuItem.paragraph = paragraph
menuItem.producteur = producteur
menuItem.save()
if producteur > 0:
menuItemProd = MenuItemProd(menuItemId = menuItem)
menuItemProd.entrepriseId = producteur
menuItemProd.save()
message = _('Details modified successfuly')
else:
data = {'descrShort': menuItem.descrShort, 'paragraph': menuItem.paragraph, 'producteur': menuItem.producteur}
form = MenuDetailForm(initial=data)
menuItems = MenuItem.objects.filter(menuId = menuDetail).select_related()
menus = Menu.objects.filter(entrepriseId=e)
menuParagraph = MenuParagraph.objects.filter(actif=1)
modifier = True
#detail = False
return render (request, 'front/B2Bmenu.html', {'MenuDetailForm': form, 'menus': menus, 'message': message, 'partnerId': partnerId, 'modifier': modifier, 'detail': detail, 'menuDetail': menuDetail, 'menuParagraph': menuParagraph, 'menuId': menuId, 'menuItems': menuItems})
I'm sure I can get my page when this error is resolved. I'm sure it's a little error, I'm a beginner at Python but I love the language :)
If you want I can give you more details but I don't think it's necessary ^^
Have a nice day and thank you for your help :)

I found it !
I just forgot to add another try for this variable, just after the first try.
try:
menuItem = MenuItem.objects.get(id=menuItemId)
except MenuItem.DoesNotExist:
return logoutUser(request)

script to serve from url, for requests matching regular expression

I am a complete n00b in Python and am trying to figure out a stub for mitmproxy.
I have tried the documentation but they assume we know Python so i am at a stalemate.
I've been working with a script:
original_url = 'http://production.domain.com/1/2/3'
new_content_path = '/home/andrepadez/proj/main.js'
body = open(new_content_path, 'r').read()
def response(context, flow):
url = flow.request.get_url()
if url == original_url:
flow.response.content = body
As you can predict, the proxy takes every request to 'http://production.domain.com/1/2/3' and serves the content of my file.
I need this to be more dynamic:
for every request to 'http://production.domain.com/*', i need to serve a correspondent URL, for example:
http://production.domain.com/1/4/3 -> http://develop.domain.com/1/4/3
I know i have to use a regular expression, so i can capture and map it correctly, but i don't know how to serve the contents of the develop url as "flow.response.content".
Any help will be welcome

You would have to do something like this:
import re
# In order not to re-read the original file every time, we maintain
# a cache of already-read bodies.
bodies = { }
def response(context, flow):
# Intercept all URLs
url = flow.request.get_url()
# Check if this URL is one of "ours" (check out Python regexps)
m = re.search('REGEXP_FOR_ORIGINAL_URL/(\d+)/(\d+)/(\d+)', url)
if None != m:
# It is, and m will contain this information
# The three numbers are in m.group(1), (2), (3)
key = "%d.%d.%d" % ( m.group(1), m.group(2), m.group(3) )
try:
body = bodies[key]
except KeyError:
# We do not yet have this body
body = // whatever is necessary to retrieve this body
= open("%s.txt" % ( key ), 'r').read()
bodies[key] = body
flow.response.content = body

Reportlab: header with data from page

I'm using the on page function and a page template to make headers for a subset of the pages in my document:
templates.append(PageTemplate(id='Overview', frames=frame, onPage=HeaderOverview))
The header function for this template:
################################
# Function HeaderOverview - header for overview page
def HeaderOverview(canvas,doc):
canvas.saveState()
headboxh = 15
headboxx = 20
headboxy = 730
headboxw = 570
canvas.rect(headboxx, headboxy, headboxw, headboxh, fill=1)
canvas.setFillColor(colors.black)
canvas.setFont("Helvetica", 14)
canvas.setFillColor(colors.white)
canvas.drawString(headboxx + 15,headboxy+.25*headboxh,"Mathematics")
textWidth = stringWidth("Mathematics", "Helvetica", 12)
canvas.setFont("Helvetica", 12)
canvas.drawString(headboxw - 15 - textWidth,headboxy+.25*headboxh,course)
canvas.restoreState()
This works great, except that the course variable that's passed (which changes with each page in the section) is the last one in the sequence, since this function's not really called until the final build (I think that's how it works). What I need is to do this so that the value is the value that's on the page. If I could draw it as I write the page itself, that'd be fine, too. Here's my attempt at that:
####################################################################################
# Function makeGradeOverview(course): makes Overview chart for grade
#
def makeGradeOverview(canvas, course):
report.append(NextPageTemplate("Overview"))
report.append(PageBreak())
headboxh = 50
headboxx = 20
headboxy = 600#730
headboxw = 540
canvas.saveState()
canvas.setFont("Helvetica", 12)
textWidth = stringWidth(course, "Helvetica", 12)
canvas.drawString(headboxw - 15 - textWidth,headboxy+.25*headboxh,course)
canvas.restoreState()
# put course name as title
if len(course)<=2:
headerrow = ''.join(['Grade ', course, ' Overview'])
else:
headerrow = ''.join([course, ' Overview'])
report.append(Paragraph(headerrow, styles["Overview Title"]))
report.append(Spacer(1, 16))
GridInfo = []
topics = topiclist(course)
for topic in topics:
report.append(Paragraph(topic, styles["Overview Sub"]))
report.append(Spacer(1, 8))
subtopics = subtopiclist(course, topic)
sublist = []
for subtopic in subtopics:
report.append(Paragraph(''.join([r'<bullet>&bull</bullet>',subtopic]), styles["Overview Table"]))
This doesn't throw an error or anything, but it doesn't seem to actually draw anything, either.
Thanks for the help!

Here's another idea...
Perhaps it would work to use specific flowables that can be identified to update the course. You can add custom attributes to flowables if necessary to help identify them (see this post).
For example, you might be able to do something like this:
...
report.append(some_content)
report.append(PageBreak())
report[-1].new_course = True # gives that PageBreak flowable a custom attribute
report.append(some_more_content)
...
And set up some variables:
course_list = [...]
course_iter = iter(course_list)
current_course = next(course_iter)
Then you can check each flowable after it is rendered to see if it has that attribute and update the current course if it does.
def afterFlowable(flowable):
global current_course
if hasattr(flowable, 'new_course'):
current_course = next(course_iter)
doc.afterFlowable = afterFlowable
HeaderOverview will be able to use the current_course variable to get the right course, since both HeaderOverview and afterFlowable are called at various points during the final build.