I have a postman collection which has multiple folders containing around 100 requests with all methods like GET, POST, DELETE, PUT.
I need to parse the postman.json collection file in python and create requests and write to a txt file.
These request need to be passed to another tool. Can you help me with it. Any pointers would be helpful.
I am stuck in parsing collection JSON file, which is very difficult.
HOW TO GUIDE
Read the doc about JSON encoder/decoder
To parse a JSON string json_str:
import json
obj = json.loads(json_str)
To parse a JSON file:
import json
import io
with io.open("path/to/file.json", mode-"rb") as fd:
obj = json.load(fd)
Read the doc about Requests
There is a requests.request() function, where you can pass the method (GET/OPTIONS/POST/PUT/PATCH/DELETE/HEAD) in parameter. Usage:
import requests
response = requests.request('GET', 'http://httpbin.org/get')
Write into a binary file
You can write the result of your request into a binary file. You can also use a text file but some response may be binary, for instance if you download an image.
with io.open("path/to/file.dat", mode="wb") as fd:
fd.write(response.content)
The code below will help you understand how to parse the Collection JSON file so that you can recursively accumulate all the requests in that collection. I have used the Postman Collection SDK along with a helper utility called Lodash.
You can either use this snippet below to get the request information you want to be consumed by using Python requests. Or to keep things simple, just use Javascript if you can.
var fs = require('fs'), // needed to read JSON file from disk
sdk = require('postman-collection'),
Collection = sdk.Collection,
Request = sdk.Request,
Item = sdk.Item,
ItemGroup = sdk.ItemGroup,
_ = require('lodash'),
myCollection,
requests = [],
dfs = function (item, requests) { // fn -> Depth first search
// Check if this is a request
if (Item.isItem(item)) {
if (item.request && Request.isRequest(item.request)) {
requests.push(item.request);
}
}
// Check if this is a nested folder
else if (ItemGroup.isItemGroup(item)) {
// Check if this is an empty folder
if (item.items && (item.items.count() === 0)) {
return requests;
}
// Do a depth first search for requests on the nested folder
item.each(function (item) {
requests.push(dfs(item, []));
})
}
return requests;
};
// Load a collection to memory from a JSON file on disk
myCollection = new Collection(JSON.parse(
fs.readFileSync('<path_to_your_collection_json_file>').toString()));
myCollection.items.each(function (item) {
// Check if this is a request at the top level
if (Item.isItem(item)) {
if (item.request && Request.isRequest(item.request)) {
requests.push(item.request);
}
}
// Check if this is a folder at the top level
else if (ItemGroup.isItemGroup(item)) {
item.items.each(function (item) {
requests.push(dfs(item, []));
})
}
});
// Flatten. After flattening requests will an array of PostmanRequest objects
requests = _.flattenDeep(requests)
// Serialize each PostmanRequest to it's JSON representation
requests = _.map(requests, (r) => { return r.toJSON(); })
_.each(requests, (request) => {
console.log(request.url); // The request URL
console.log(request.method); // The HTTP Verb of your request Eg. GET, POST
_.each(request.header, (header) => {
console.log(header.key, header.value); // Eg. key -> 'Content-Type', value -> 'application/json'
});
// You can also access the request body and the auth, certificate and proxy used by the request
// Your PostmanRequest description is also available
});
Related
How can I download attachments from a list (not the documents of the page) in SharePoint using an API in Python?
I have a list with 60+ rows and the contents is added from a Power App form. Each row is a different entry and may or may not have an attachment.
I'm working on an automation process that will read each line of the list and enter it into SAP.
I can't seem to find an API that can get attachments from a list, they are all about getting them from the documents folder of the page.
Below is the site URL I'm using, and what the list looks like.
https://{site url}/sites/{group name}/Lists/{listname}/AllItems.aspx?e=3%3A32b71e288cd841f5b13422c0b99ffe89&at=9
function GetListItemAttachments() {
// Specify the Id of the Item that you want to fetch
var Itemid = 1;
$.ajax
({
// _spPageContextInfo.webAbsoluteUrl - will give absolute URL of the site where you are running the code.
// You can replace this with other site URL where you want to apply the function
url: _spPageContextInfo.webAbsoluteUrl + "/_api/web/lists/getByTitle('List Name')/items(" + Itemid + ")/AttachmentFiles",
method: "GET",
headers:
{
// Accept header: Specifies the format for response data from the server.
"Accept": "application/json;odata=verbose"
},
success: function (data, status, xhr) {
var dataresults = data.d.results;
for (var i = 0; i < dataresults.length; i++) {
alert(dataresults[i]["FileName"]);
}
},
error: function (xhr, status, error) {
console.log("Failed");
}
});
}
Read more: Get All Attachments From List Item in SharePoint using REST API.
First post here, so ya know, be nice?
I'm setting up a dashboard in Dashing (http://dashing.io/) using a JSON feed on a server, which looks like:
{
"error":0,
"message_of_the_day":"Welcome!",
"message_of_the_day_hash":"a1234567890123456789012345678901",
"metrics":{
"daily":{
"metric1":"1m 30s",
"metric2":160
},
"monthly":{
"metric1":"10m 30s",
"metric2":"3803"
}
},
I have been experimenting with grabbing the data from the feed, and have managed to do so by Python with no issues:
import json
import urllib2
data = {
'region': "Europe"
}
req = urllib2.Request('http://192.168.1.2/info/handlers/handler.php')
req.add_header('Content-Type', 'application/json')
response = urllib2.urlopen(req, json.dumps(data))
print response.read()
However I haven't yet been successful, and get numerous errors in Ruby.
Would anyone be able to point me in the right direction in parsing this in Ruby?
My attempts to write a basic script, (keeping it simple and outside of Dashing) don't pull through any data.
#!/usr/bin/ruby
require 'httparty'
require 'json'
response = HTTParty.get("http://192.168.1.2/info/handlers/handler.php?region=Europe")
json = JSON.parse(response.body)
puts json
In python code you are sending a JSON and adding a header. I bet it makes sense to do that in ruby as well. The code below is untested, since I can’t test it, but it should lead you into the right direction:
#!/usr/bin/ruby
require 'httparty'
require 'json'
response = HTTParty.post(
"http://192.168.1.2/info/handlers/handler.php",
headers: {'Content-Type', 'application/json'},
query: { data: { 'region' => "Europe" } }
# or maybe query: { 'region' => "Europe" }
)
puts response.inspect
When I look at the documentation for passing parameters to the Jasper Report REST 2 API here: http://community.jaspersoft.com/documentation/jasperreports-server-web-services-guide/v550/running-report-asynchronously I see that I need to have a "parameters" dict. The example in the link shows the XML which is not all that useful since it's unclear exactly what the equivalent JSON should look like. The closest I could find is in this link: http://community.jaspersoft.com/documentation/jasperreports-server-web-services-guide/v56/modifying-report-parameters. Now, I am sending the equivalent of that to the server (and every other permutation I can think of), and I continue to get a "400 Client Error: Bad Request" back. I could really use an exact example of the python code to generate the required "parameters" parameter for say "my_parameter_1="test_value_1".
Here is my current POST data (with a few params missing for brevity). I know this is correct since the report works fine if I omit the "parameters" parameter:
{
'outputFormat': 'pdf',
'parameters': [{'name': 'ReportID', 'value': ['my_value_1']}],
'async': 'true',
'pages': '',
'interactive': 'false'
}
Nice Job there Staggart. I got it now. Because I wasn't reading with max. scrutinity, I wasted some additional time. So the interested coder is not only advised to be aware of the nested, syntactictally interesting reportParameter-property, but especially that the value-property inside that is an array. I suppose one could pass some form of Lists/Arrays/Collections here?
What irritated me was, if I should construct more than one "reportParameter" property, but that would be nonsense according to
Does JSON syntax allow duplicate keys in an object.
So just for the record, how to post multiple parameters:
{
"reportUnitUri": "/reports/Top10/Top10Customers",
"async": true,
"freshData": true,
"saveDataSnapshot": false,
"outputFormat": "pdf",
"interactive": false,
"ignorePagination": true,
"parameters": {
"reportParameter": [
{
"name": "DATE_START_STRING",
"value": ["14.07.2014"]
},
{
"name": "DATE_END_STRING",
"value": ["14.10.2014"]
}
]
}
}
If someone accidently is struggling with communicating with jasper via REST and PHP. Do yourself a favour and use the Requests for PHP instead of pure CURL. It even has a fallback for internally using Sockets instead of CURL, when latter isn't available.
Upvote for you Staggart.
OK, thanks to rafkacz1 # http://community.jaspersoft.com/questions/825719/json-equivalent-xml-post-reportexecutions-rest-service who posted an answer, I figured it out. As he report there, the required format is:
"parameters":{
"reportParameter":[
{"name":"my_parameter_1","value":["my_value_1"]}
]
}
Pay particular attention to the plurality of "reportParameter".
Here is an example that worked for me. Im using Python 2.7, and the community edition of Jaspersoft. Like the C# example above, this example also uses the rest v2 which made it very simple for me to download a pdf report quickly
import requests
sess = requests.Session()
auth = ('username', 'password')
res = sess.get(url='http://your.jasper.domain:8080/jasperserver/', auth=auth)
res.raise_for_status()
url = 'http://your.jasper.domain:8080/jasperserver/rest_v2/reports/report_folder/sub_folder/report_name.pdf'
params = {'Month':'2', 'Year':'2017','Project': 'ProjectName'}
res = sess.get(url=url, params=params, stream=True)
res.raise_for_status()
path = '/path/to/Downloads/report_name.pdf'
with open(path, "wb") as f:
f.write(res.content)
Here's a full example about generate a report using Rest V2, in my case it's running on C#:
try {
var server = "http://localhost:8080/jasperserver";
var login = server + "/rest/login";
var report = "/rest_v2/reports/organization/Reports/report_name.pdf";
var client = new WebClient();
//Set the content type of the request
client.Headers[HttpRequestHeader.ContentType] = "application/x-www-form-urlencoded";
//Set the username and password
NameValueCollection parametros = new NameValueCollection();
parametros.Add("j_username", "jasperadmin");
parametros.Add("j_password", "123456");
//Request to login
client.UploadValues(login, "POST", parametros);
//Get session cookie
string session = client.ResponseHeaders.Get("Set-Cookie");
//Set session cookie to the next request
client.Headers.Add("Cookie", session);
//Generate report with parameters: "start" and "end"
var reporte = client.DownloadData(server + report + "?start=2015-10-01&end=2015-10-10");
//Returns the report as response
return File(reporte, "application/pdf", "test.pdf");
}catch(WebException e){
//return Content("There was a problem, status code: " + ((HttpWebResponse)e.Response).StatusCode);
return null;
}
Is there any way to retrieve message body in html form using GMail api ?
I have already gone through the message.get docs. Tried changing the format param to full, minimal & raw. But it did not help. It's returning the plain text of mail body.
Description of format values:
"full": Returns the parsed email message content in the payload field and the raw field is not used. (default)
"minimal": Only returns email message metadata such as identifiers and labels, it does not return the email headers, body, or payload.
"raw": Returns the entire email message content in the raw field as a string and the payload field is not used. This includes the identifiers, labels, metadata, MIME structure, and small body parts (typically less than 2KB).
Can't we simply get message body in html form or is there any other way to do this so that mail is displayed on the screen with very minimal difference when they see in my app or GMail ?
Email messages that have both HTML and plain text content will have multiple payload parts, and the part with the mimeType "text/html" will contains the HTML content. You can find it with logic like:
var part = message.parts.filter(function(part) {
return part.mimeType == 'text/html';
});
var html = urlSafeBase64Decode(part.body.data);
Both FULL and RAW will return you any text/html parts depending on how you'd like it. If you use FULL you'll get a parsed representation which will be nested json dictionaries that you'll have to walk over looking for the text/html part. If you opt for the RAW format you'll get the entire email in RFC822 format in the Message.raw field. You can pass that to the mime libraries in your chosen language and then use that to find the part you're interested in. Mime is complicated, you'll likely have a top-level "multipart" type with text/html as one of the direct children of it but no guarantees, it's an arbitrarily deep tree structure! :)
Here is the full tutorial:
1- Assuming you already went through all credentials creation here
2- This is how you retrieve a Mime Message:
public static String getMimeMessage(String messageId)
throws Exception {
//getService definition in -3
Message message = getService().users().messages().get("me", messageId).setFormat("raw").execute();
Base64 base64Url = new Base64(true);
byte[] emailBytes = base64Url.decodeBase64(message.getRaw());
Properties props = new Properties();
Session session = Session.getDefaultInstance(props, null);
MimeMessage email = new MimeMessage(session, new ByteArrayInputStream(emailBytes));
return getText(email); //getText definition in at -4
}
3- This is the piece that creates the Gmail instance:
private static Gmail getService() throws Exception {
final NetHttpTransport HTTP_TRANSPORT = GoogleNetHttpTransport.newTrustedTransport();
// Load client secrets.
InputStream in = SCFManager.class.getResourceAsStream(CREDENTIALS_FILE_PATH);
if (in == null) {
throw new FileNotFoundException("Resource not found: " + CREDENTIALS_FILE_PATH);
}
GoogleClientSecrets clientSecrets = GoogleClientSecrets.load(JSON_FACTORY, new InputStreamReader(in));
// Build flow and trigger user authorization request.
GoogleAuthorizationCodeFlow flow = new GoogleAuthorizationCodeFlow.Builder(
HTTP_TRANSPORT, JSON_FACTORY, clientSecrets, SCOPES)
.setDataStoreFactory(new FileDataStoreFactory(new java.io.File(TOKENS_DIRECTORY_PATH)))
.setAccessType("offline")
.build();
LocalServerReceiver receiver = new LocalServerReceiver.Builder().setPort(8888).build();
Credential credential = new AuthorizationCodeInstalledApp(flow, receiver).authorize("user");
return new Gmail.Builder(HTTP_TRANSPORT, JSON_FACTORY, credential)
.setApplicationName(APPLICATION_NAME)
.build();
}
4- And this is how you parse Mime Messages:
public static String getText(Part p) throws
MessagingException, IOException {
if (p.isMimeType("text/*")) {
String s = (String) p.getContent();
return s;
}
if (p.isMimeType("multipart/alternative")) {
// prefer html text over plain text
Multipart mp = (Multipart) p.getContent();
String text = null;
for (int i = 0; i < mp.getCount(); i++) {
Part bp = mp.getBodyPart(i);
if (bp.isMimeType("text/plain")) {
if (text == null) {
text = getText(bp);
}
continue;
} else if (bp.isMimeType("text/html")) {
String s = getText(bp);
if (s != null) {
return s;
}
} else {
return getText(bp);
}
}
return text;
} else if (p.isMimeType("multipart/*")) {
Multipart mp = (Multipart) p.getContent();
for (int i = 0; i < mp.getCount(); i++) {
String s = getText(mp.getBodyPart(i));
if (s != null) {
return s;
}
}
}
return null;
}
5- If you were wondering how to get the email id, this is how you list them:
public static List<String> listTodayMessageIds() throws Exception {
ListMessagesResponse response = getService()
.users()
.messages()
.list("me")
.execute();
if (response != null && response.getMessages() != null && !response.getMessages().isEmpty()) {
return response.getMessages().stream().map(Message::getId).collect(Collectors.toList());
} else {
return null;
}
}
Note:
If after this you want to query that html body in the "kind of Java Script way", I recommend you to explore jsoup library.. very intuitive and easy to work with:
Document jsoup = Jsoup.parse(body);
Elements tds = jsoup.getElementsByTag("td");
Elements ps = tds.get(0).getElementsByTag("p");
I hope this helps :-)
I am using NodeJS to send notifications to my clients using NowJS but the data that I need to send in the notifications has to come from my database.
Since I am using Django backend, I can make HTTP requests to my Node.js server and send the required data. But I need to be able to accept this data using Node.js. How can I do this?
How can I do this?
require("http").createServer(function (req, res) {
doStuff();
}).listen(PORT);
I'm a fan of using Connect/Express, so here's a trivial example you could use:
var express = require('express');
var app = express.createServer();
var nowjs = require("now");
var everyone = nowjs.initialize(app);
app.use(express.bodyParser()); //If you want nice JSON parsing
app.post('/sendNotification', function(req, res){
console.log(req.params) //Look at the POST params
console.log(req.body) //Look at the POST body
everyone.now.receiveMessage(req.body.dataYouCareAbout);
res.send('Notification Sent!');
});
You could then use that "sendNotification" endpoint to accept POST data and send it (or some piece of it, or anything really) down to NowJS.
Use formidable. Here's a short example from their docs:
var formidable = require('formidable'),
http = require('http'),
util = require('util');
http.createServer(function(req, res) {
if (req.url == '/upload' && req.method.toLowerCase() == 'post') {
// parse a file upload
var form = new formidable.IncomingForm();
form.parse(req, function(err, fields, files) {
res.writeHead(200, {'content-type': 'text/plain'});
res.write('received upload:\n\n');
res.end(util.inspect({fields: fields, files: files}));
});
return;
}