Im using masscan and the example script but when I print the results only the command is printed because the json output for scan is broken(atleast one issue mentiont that). How can I fix that/still acces the json data?
The example code:
import masscan
mas = masscan.PortScanner()
mas.scan('172.0.8.78/24', ports='22,80,8080', arguments='--max-rate 1000')
print(mas.scan_result)
Masscan should return the scan results.
Instead it returns: {"command_line": "masscan -oJ - 0.0.0.0/24 -p 25565 --rate=1000", "scan":{}
I am trying to use the gitlab projects api to edit multiple project MR templates. The problem is, that it only sends the first line of the markdown template.
While messing around with the script, I was toying with converting it to html when I found that it sent the whole template when converted to html.
I am probably missing something super simple but for the life of me, I cant figure out why it would be able to send the entire template in html but only send the first line of it natively in markdown.
I have been searching for a solution for a bit now so I apologize if my googlefu missed an obvious answer here.
Here is the script...
#! /usr/bin/env python3
import argparse
import requests
gitlab_addr = "https://gitlab.com/api/v4"
# Insert your project IDs into the array below.
project_IDs = [xxxx, yyyy, zzzz]
# Insert your MR template info below.
with open('/.gitlab/merge_request_templates/DefaultMRTemplate.md', 'r') as file:
MR_template = file.read()
#print(MR_template)
def getArgs():
parser = argparse.ArgumentParser(
description='This tool updates the default template for a single '
'or multiple program\'s MRs. \n\nYou will need to edit '
'the script to input your MR template and projects IDs.'
'\nYou will also need to pass in your API Token via '
' command line.\n\nYou want to see "200 OK" on the '
' command line as confirmation.',
formatter_class=argparse.RawTextHelpFormatter)
parser.add_argument("token", type=str,
help="API Token. Create one at User Settings / Access Tokens")
return parser.parse_args()
def ChangeTemplate():
token = getArgs().token
headers = {"PRIVATE-TOKEN": token, }
for x in project_IDs:
addr = f"{gitlab_addr}/projects/{x}/?merge_requests_template={MR_template}"
response = requests.put(addr, headers=headers)
# You want to see "200 OK" on the command line.
print(response.status_code, response.reason)
def main():
ChangeTemplate()
if __name__ == '__main__':
main()
Here is a sample template...
See guidance here: https://example.com/Gitlab+MR+Guide
## Description
%% Put a description here %%
%% Add an issue link here %%
## Tests
%% Include test listing here %%
## Checklists
**Author Checklist**
- [ ] A: Did you fill out the description, add an issue link (in title or desc) and fill out the test section?
- [ ] A: Add a peer to the MR
**Assignee 1 Checklist:**
- [ ] P: Verify the description field is filled out, issue link is included (in title or desc) and the test section is filled out
- [ ] P: Add a code owner to the MR
**Assignee 2 (Code Owner) Checklist:**
- [ ] O: Verify the description field is filled out, issue link is included (in title or desc) and the test section is filled out
- [ ] O: Verify unit test coverage is at least 40% line coverage with a goal of 90%
problem output...
See guidance here: https://example.com/Gitlab MR Guide
Your data needs to be properly encoded in the request. Trying to format the literal contents of the file into the query string won't work here.
Use the data keyword argument to requests.put, which will pass the data in the request body (or use params to set query params). requests will handle the proper encoding of the data.
addr = f"{gitlab_addr}/projects/{x}/"
payload = {'merge_requests_template': MR_template}
response = requests.put(addr, headers=headers, data=payload)
# or params=payload to use query string
How to replace string after specific character using python
I have a file with below contents
The Test file contents are as below "Test1"{
Serial = 12345
IP = 12.10.23.10
User = user1
}
how do i replace the contents after the = symbol using python ?
for example i want to replace with below contents.
The Test file contents are as below "Test1"{
Serial = 22330011
IP = 1.1.1.1
User = User_11
}
The contents after = symbols are not pre defined, hence substituting 12345 with 22330011 is not required here.
need a logic to find what is there after = symbol and replace it with the user defined value.
Lets say i have above data in temp.txt
file=open('temp.txt','r')
data=file.readlines()
outdata=[]
for line in data:
try:
lhs,rhs=line.split('=')
rhs=input()
outdata.append('='.join([lhs,' '+rhs,'\n']))
except:
outdata.append(line)
file.close()
file=open('temp.txt','w')
for line in outdata:
file.write(line)
This code read from the file and ask the input from user for rhs and updates in the file again
I added a line in the python code “speedtest.py” that I found at pimylifeup.com. I hoped it would allow me to track the internet provider and IP address along with all the other speed information his code provides. But when I execute it, the code only grabs the next word after the find all call. I would also like it to return the IP address that appears after the provider. I have attached the code below. Can you help me modify it to return what I am looking for.
Here is an example what is returned by speedtest-cli
$ speedtest-cli
Retrieving speedtest.net configuration...
Testing from Biglobe (111.111.111.111)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by GLBB Japan (Naha) [51.24 km]: 118.566 ms
Testing download speed................................................................................
Download: 4.00 Mbit/s
Testing upload speed......................................................................................................
Upload: 13.19 Mbit/s
$
And this is an example of what it is being returned by speediest.py to my .csv file
Date,Time,Ping,Download (Mbit/s),Upload(Mbit/s),myip
05/30/20,12:47,76.391,12.28,19.43,Biglobe
This is what I want it to return.
Date,Time,Ping,Download (Mbit/s),Upload (Mbit/s),myip
05/30/20,12:31,75.158,14.29,19.54,Biglobe 111.111.111.111
Or may be,
05/30/20,12:31,75.158,14.29,19.54,Biglobe,111.111.111.111
Here is the code that I am using. And thank you for any help you can provide.
import os
import re
import subprocess
import time
response = subprocess.Popen(‘/usr/local/bin/speedtest-cli’, shell=True, stdout=subprocess.PIPE).stdout.read().decode(‘utf-8’)
ping = re.findall(‘km]:\s(.*?)\s’, response, re.MULTILINE)
download = re.findall(‘Download:\s(.*?)\s’, response, re.MULTILINE)
upload = re.findall(‘Upload:\s(.*?)\s’, response, re.MULTILINE)
myip = re.findall(‘from\s(.*?)\s’, response, re.MULTILINE)
ping = ping[0].replace(‘,’, ‘.’)
download = download[0].replace(‘,’, ‘.’)
upload = upload[0].replace(‘,’, ‘.’)
myip = myip[0]
try:
f = open(‘/home/pi/speedtest/speedtestz.csv’, ‘a+’)
if os.stat(‘/home/pi/speedtest/speedtestz.csv’).st_size == 0:
f.write(‘Date,Time,Ping,Download (Mbit/s),Upload (Mbit/s),myip\r\n’)
except:
pass
f.write(‘{},{},{},{},{},{}\r\n’.format(time.strftime(‘%m/%d/%y’), time.strftime(‘%H:%M’), ping, download, upload, myip))
Let me know if this works for you, it should do everything you're looking for
#!/usr/local/env python
import os
import csv
import time
import subprocess
from decimal import *
file_path = '/home/pi/speedtest/speedtestz.csv'
def format_speed(bits_string):
""" changes string bit/s to megabits/s and rounds to two decimal places """
return (Decimal(bits_string) / 1000000).quantize(Decimal('.01'), rounding=ROUND_UP)
def write_csv(row):
""" writes a header row if one does not exist and test result row """
# straight from csv man page
# see: https://docs.python.org/3/library/csv.html
with open(file_path, 'a+', newline='') as csvfile:
writer = csv.writer(csvfile, delimiter=',', quotechar='"')
if os.stat(file_path).st_size == 0:
writer.writerow(['Date','Time','Ping','Download (Mbit/s)','Upload (Mbit/s)','myip'])
writer.writerow(row)
response = subprocess.run(['/usr/local/bin/speedtest-cli', '--csv'], capture_output=True, encoding='utf-8')
# if speedtest-cli exited with no errors / ran successfully
if response.returncode == 0:
# from the csv man page
# "And while the module doesn’t directly support parsing strings, it can easily be done"
# this will remove quotes and spaces vs doing a string split on ','
# csv.reader returns an iterator, so we turn that into a list
cols = list(csv.reader([response.stdout]))[0]
# turns 13.45 ping to 13
ping = Decimal(cols[5]).quantize(Decimal('1.'))
# speedtest-cli --csv returns speed in bits/s, convert to bytes
download = format_speed(cols[6])
upload = format_speed(cols[7])
ip = cols[9]
date = time.strftime('%m/%d/%y')
time = time.strftime('%H:%M')
write_csv([date,time,ping,download,upload,ip])
else:
print('speedtest-cli returned error: %s' % response.stderr)
$/usr/local/bin/speedtest-cli --csv-header > speedtestz.csv
$/usr/local/bin/speedtest-cli --csv >> speedtestz.csv
output:
Server ID,Sponsor,Server Name,Timestamp,Distance,Ping,Download,Upload,Share,IP Address
Does that not get you what you're looking for? Run the first command once to create the csv with header row. Then subsequent runs are done with the append '>>` operator, and that'll add a test result row each time you run it
Doing all of those regexs will bite you if they or a library that they depend on decides to change their debugging output format
Plenty of ways to do it though. Hope this helps
I am trying to run a python3 program file and am getting some unexpected behaviors.
I'll start off first with my PATH and env setup configuration. When I run:
which Python
I get:
/c/Program Files/Python36/python
From there, I cd into the directory where my python program is located to prepare to run the program.
Roughly speaking this is how my python program is set up:
import modulesNeeded
print('1st debug statement to show program execution')
# variables declared as needed
def aFunctionNeeded():
print('2nd debug statement to show fxn exe, never prints')
... function logic...
if __name__ == '__main__':
aFunctionNeeded() # Never gets called
Here is a link to the repository with the code I am working with in case you would like more details as to the implementation. Keep in mind that API keys are not published, but API keys are in local file correctly:
https://github.com/lopezdp/API.Mashups
My question revolves around why my 1st debug statements inside the files are printing to the terminal, but not the 2nd debug statements inside the functions?
This is happening in both of the findRestaurant.py file and the geocode.py file.
I know I have written my if __name__ == '__main__': program entry point correctly as this is the same exact way I have done it for other programs, but in this case I may be missing something that I am not noticing.
If this is my output when I run my program in my bash terminal:
$ python findRestaurant.py
inside geo
inside find
then, why does it appear that my aFunctionNeeded() method shown in my pseudo code is not being called from the main?
Why do both programs seem to fail immediately after the first debug statements are printed to the terminal?
findRestaurant.py File that can also be found in link above
from geocode import getGeocodeLocation
import json
import httplib2
import sys
import codecs
print('inside find')
sys.stdout = codecs.getwriter('utf8')(sys.stdout)
sys.stderr = codecs.getwriter('utf8')(sys.stderr)
foursquare_client_id = "..."
foursquare_client_secret = "..."
def findARestaurant(mealType,location):
print('inside findFxn')
#1. Use getGeocodeLocation to get the latitude and longitude coordinates of the location string.
latitude, longitude = getGeocodeLocation(location)
#2. Use foursquare API to find a nearby restaurant with the latitude, longitude, and mealType strings.
#HINT: format for url will be something like https://api.foursquare.com/v2/venues/search?client_id=CLIENT_ID&client_secret=CLIENT_SECRET&v=20130815&ll=40.7,-74&query=sushi
url = ('https://api.foursquare.com/v2/venues/search?client_id=%s&client_secret=%s&v=20130815&ll=%s,%s&query=%s' % (foursquare_client_id, foursquare_client_secret,latitude,longitude,mealType))
h = httplib2.Http()
result = json.loads(h.request(url,'GET')[1])
if result['response']['venues']:
#3. Grab the first restaurant
restaurant = result['response']['venues'][0]
venue_id = restaurant['id']
restaurant_name = restaurant['name']
restaurant_address = restaurant['location']['formattedAddress']
address = ""
for i in restaurant_address:
address += i + " "
restaurant_address = address
#4. Get a 300x300 picture of the restaurant using the venue_id (you can change this by altering the 300x300 value in the URL or replacing it with 'orginal' to get the original picture
url = ('https://api.foursquare.com/v2/venues/%s/photos?client_id=%s&v=20150603&client_secret=%s' % ((venue_id,foursquare_client_id,foursquare_client_secret)))
result = json.loads(h.request(url, 'GET')[1])
#5. Grab the first image
if result['response']['photos']['items']:
firstpic = result['response']['photos']['items'][0]
prefix = firstpic['prefix']
suffix = firstpic['suffix']
imageURL = prefix + "300x300" + suffix
else:
#6. if no image available, insert default image url
imageURL = "http://pixabay.com/get/8926af5eb597ca51ca4c/1433440765/cheeseburger-34314_1280.png?direct"
#7. return a dictionary containing the restaurant name, address, and image url
restaurantInfo = {'name':restaurant_name, 'address':restaurant_address, 'image':imageURL}
print ("Restaurant Name: %s" % restaurantInfo['name'])
print ("Restaurant Address: %s" % restaurantInfo['address'])
print ("Image: %s \n" % restaurantInfo['image'])
return restaurantInfo
else:
print ("No Restaurants Found for %s" % location)
return "No Restaurants Found"
if __name__ == '__main__':
findARestaurant("Pizza", "Tokyo, Japan")
geocode.py File that can also be found in link above
import httplib2
import json
print('inside geo')
def getGeocodeLocation(inputString):
print('inside of geoFxn')
# Use Google Maps to convert a location into Latitute/Longitute coordinates
# FORMAT: https://maps.googleapis.com/maps/api/geocode/json?address=1600+Amphitheatre+Parkway,+Mountain+View,+CA&key=API_KEY
google_api_key = "..."
locationString = inputString.replace(" ", "+")
url = ('https://maps.googleapis.com/maps/api/geocode/json?address=%s&key=%s' % (locationString, google_api_key))
h = httplib2.Http()
result = json.loads(h.request(url,'GET')[1])
latitude = result['results'][0]['geometry']['location']['lat']
longitude = result['results'][0]['geometry']['location']['lng']
return (latitude,longitude)
The reason you're not seeing the output of the later parts of your code is that you've rebound the standard output and error streams with these lines:
sys.stdout = codecs.getwriter('utf8')(sys.stdout)
sys.stderr = codecs.getwriter('utf8')(sys.stderr)
I'm not exactly sure why those lines are breaking things for you, perhaps your console does not expect utf8 encoded output... But because they don't work as intended, you're not seeing anything from the rest of your code, including error messages, since you rebound the stderr stream along with the stdout stream.