I want to make google trasnlate script.
I am making a request to translate.google.com and google return an array but the array contains undefined items.You can imagine response comes as string.
I can remove commas if there is more than one consecutive with regex etc. but I am looking best solution :)
How can I convert this javascript array to python list?
["a","b",,,"e"]
My script : http://ideone.com/jhjZe
JavaScript part - encoding
In Javascript you do:
var arr = ["a","b",,,"e"];
var json_string = JSON.stringify(arr);
then you somehow pass json_string (now equal to "["a","b",null,null,"e"]" string) from JavaScript to Python.
Python part - decoding
Then, on Python side do:
json_string = '["a","b",null,null,"e"]' # passed from JavaScript
try:
import simplejson as json
except (ImportError,):
import json
result = json.loads(json_string)
As a result you get [u'a', u'b', None, None, u'e'] in Python.
More links
See below:
demo of JavaScript part,
Documentation on JSON.stringify() at Mozilla Developer Network,
demo of Python part,
Dependencies
The above solutions require:
JSON.stringify() in JavaScript, which is in all mobile browsers, Chrome, Firefox, Opera, Safari and in IE since version 8.0 (more detailed list of compatible browsers is here),
json Python library (the above code will use simplejson optionally, if available, but is not required), which comes in standard library,
So, in short there are no external dependencies.
Related
https://yeastmine.yeastgenome.org/yeastmine/customQuery.do
The above webpage has something like this. As far as I understand, JSON does not support single quote, only double quote is allowed. So the things in {} is not a valid JSON object. What is the best way to extract this object from the resulted HTML page and convert it to JSON? Thanks.
var helpMap = {'NcRNAGene': ...
This one mentions JSON.stringify. But I am not sure how to first get helpMap as JS object in the first place in python or nodejs.
Convert JS object to JSON string
In the console of that website you can write javascript. In this case you are right that JSON.Stringify is what you want here, you use it by passing the javascript object helpMap into it as a parameter, the result is the JSON-encoded string:
jsonString = JSON.stringify(helpMap)
console.log(jsonString)
You should be able to copy that json string out of your console (in chrome there will be a "Copy" button at the end of it).
Suppose the webpage is downloaded to x.html, run the following.
grep '^ \+var helpMap' < x.html | ./main.js
main.js has the following code.
fs=require('fs');
data = fs.readFileSync(process.stdin.fd);
eval(data.toString());
console.log(helpMap);
Then use JSON.stringify() on helpMap if necesssary.
Quick overview: I am writing a a very simple script using Python and Selenium to view Facebook Metrics for multiple Facebook pages.
I am trying to find a clean way to loop through the pages and output their results (it's only one number that I am collecting).
Here is what I have right now but it is not working.
# Navigate to metrics page
pages = ["page_example_1", "page_example_2", "page_example_3"]
for link in pages:
browser.get(('https://www.facebook.com/{link}/insights/?section=navVideos'))
# Navigate to metrics page
pages = ["page_example_1", "page_example_2", "page_example_3"]
for link in pages:
browser.get('https://www.facebook.com/'+ link + '/insights/?section=navVideos')
its just string concatenation
or if you are so much inclined to use that syntax, have a look at the comment by #heather
It didn't work for you, because you aimed to use Python 3.6's f-strings, but forgot to prepend your string with the f char - crucial for this syntax. E.g. your code should be (only the relevant part):
browser.get(f'https://www.facebook.com/{link}/insights/?sights/?section=navVideos')
Alternatively you could use string formatting (e.g. the established approach before 3.6):
browser.get('https://www.facebook.com/{}/insights/?sights/?section=navVideos'.format(link))
In general, string concatenation - 'string1' + variable + 'string2' - is discouraged in python for performance and readability reasons.
BTW, in your sample code you had brackets around the get()'s argument - it is browser.get((arg)), which essentially turned it to a tuple, and might've caused error in the call. Not sure was it a typo or on purpose, as you can see I and the other responders have removed it.
I am trying to write some functions to connect to IBM Cloud Object Storage using AWS Version 4 signing protocols. There's some boilerplate code for doing this that works in Python over here, but when I try to convert the code to R I find myself in a conundrum.
When using HMAC functions in R & Python the .hexdigest() method yields identical outputs for the same string in both languages:
# Python
import hmac
import hashlib
>>> hmac.new("key", "message".encode('utf-8'), hashlib.sha256).hexdigest()
'6e9ef29b75fffc5b7abae527d58fdadb2fe42e7219011976917343065f58ed4a
# R
library(digest)
> hmac("key", enc2utf8("message"), "sha256")
[1] "6e9ef29b75fffc5b7abae527d58fdadb2fe42e7219011976917343065f58ed4a"
If you use the .digest() method in Python from the crypto library - which the code from AWS/IBM suggests - then you get ASCII output. I am able to approximate this ASCII output in R by changing the hmac() output to raw and then converting to character. Below you can see it is not exactly the same - though it is close.
# Python
>>> hmac.new("key", "message".encode('utf-8'), hashlib.sha256).digest()
"n\x9e\xf2\x9bu\xff\xfc[z\xba\xe5'\xd5\x8f\xda\xdb/\xe4.r\x19\x01\x19v\x91sC\x06_X\xedJ"
# R
> rawToChar(hmac("key", "message", "sha256", raw = T))
[1] "n\x9e\xf2\x9bu\xff\xfc[z\xba\xe5'Տ\xda\xdb/\xe4.r\031\001\031v\x91sC\006_X\xedJ"
What is causing the discrepancy here? The source code and docs for .digest() says that it returns a string of 8-bit data that may contain non-ASCII bytes. How can I replicate this output in R?
Ultimately I believe this is what is causing my signatures to fail when trying to authenticate with IBM Cloud Object Storage. Any help or alternative suggestions for HMAC libraries in Python/R would be greatly appreciated!
I am trying to use the requests library in Python to push data (a raw value) to a firebase location.
Say, I have urladd (the url of the location with authentication token). At the location, I want to push a string, say International. Based on the answer here, I tried
data = {'.value': 'International'}
p = requests.post(urladd, data = sjson.dumps(data))
I get <Response [400]>. p.text gives me:
u'{\n "error" : "Invalid data; couldn\'t parse JSON object, array, or value. Perhaps you\'re using invalid characters in your key names."\n}\n'
It appears that they key .value is invalid. But that is what the answer linked above suggests. Any idea why this may not be working, or how I can do this through Python? There are no problems with connection or authentication because the following works. However, that pushes an object instead of a raw value.
data = {'name': 'International'}
p = requests.post(urladd, data = sjson.dumps(data))
Thanks for your help.
The answer you've linked is a special case for when you want to assign a priority to a value. In general, '.value' is an invalid name and will throw an error.
If you want to write just "International", you should write the stringified-JSON version of that data. I don't have a python example in front of me, but the curl command would be:
curl -X POST -d "\"International\"" https://...
Andrew's answer above works. In case someone else wants to know how to do this using the requests library in Python, I thought this would be helpful.
import simplejson as sjson
data = sjson.dumps("International")
p = requests.post(urladd, data = data)
For some reason I had thought that the data had to be in a dictionary format before it is converted to stringified JSON version. That is not the case, and a simple string can be used as an input to sjson.dumps().
I am so frustrated trying to use Ruby to fetch a specific url content.
I've tried many different ways like open-uri, standard request none worked so far. I always get empty html. I also tried to use python to fetch the same url which always returned the correct html content. I am really not sure why... Please help as I am newbiew to both Ruby and Python... I want to use Ruby (prefer the tidy syntax and human friendly function names, easier to install libs using gem and homebrew (on mac) than python easy_install) but I am now considering Python because it just works (yet still trying to get my head around 2.x and 3.x issue). I may be doing something really stupid but I think is very unlikely.
ruby 1.9.2p136 (2010-12-25 revision 30365) [i386-darwin10.6.0]
Implementation 1:
url = URI.parse('http//:www.stackoverflow.com/') req = Net::HTTP::Get.new(url.path)
res = Net::HTTP.start(url.host, url.port) {|http| http.request(req) }
puts res.body #empty
Implementation 2:
doc = Nokogiri::HTML(open("http//:www.stackoverflow.com/", "User-Agent" => "Safari"))
#empty
#I tried to use without user agent, without Nokogiri none worked.
Python Implementation which worked every time perfectly
f = urllib.urlopen("http//:www.stackoverflow.com/")
# Read from the object, storing the page's contents in 's'.
s = f.read()
f.close()
print s
If that is your exact code it is invalid for several reasons.
http: should be http://
URL needs a path. if you want the root page of example.com it needs to be http://example.com/ the trailing slash is significant.
if you put 2 lines of code on one line you need to use ; to denote the end of the first line
SO
require 'net/http'
url = URI.parse('http://www.yellowpages.com.au/search/listings?clue=plumber&locationClue=Australia')
req = Net::HTTP::Get.new(url.path)
res = Net::HTTP.start(url.host, url.port) {|http| http.request(req) }
puts res.body
Same is true with using open in nokogiri
EDIT: that site is returning bad results many times:
counter = 0
20.times do
url = URI.parse('http://www.yellowpages.com.au/search/listings?clue=plumber&locationClue=Australia')
req = Net::HTTP::Get.new(url.path)
res = Net::HTTP.start(url.host, url.port) {|http| http.request(req) }
sleep 1
counter +=1 unless res.body.empty?
end
puts counter
for me this only returned once a non empty body. If you substitute in another site it works all the time
curl "http://www.yellowpages.com.au/search/listings?clue=plumber&locationClue=Australia"
Yields the same inconsistent results.
Two examples with openURI (standard lib), a wrapper for (among others) the rather cumbersome Net::HTTP :
require 'open-uri'
open("http://www.stackoverflow.com/"){|f| puts f.read}
puts URI::parse("http://www.google.com/").read