This question already has answers here:
Python basics - request data from API and write to a file
(2 answers)
Closed 2 years ago.
In requests library, if you need to execute
curl http://www.example.com/file.xlsx
The command is
response = requests.get('http://www.example.com/file.xlsx')
What If I want to execute the command with -O option?
curl http://www.example.com/file.xlsx -O
How do I achieve that?
There's no explicit "-O" = write to the same filename.
You if you need that, you can store the url in a variable
and use several ways of getting it. A lazy way is using rpartition('/')[2] on
the url string.
The rest of the code to quickly save the result is here:
import requests
from pathlib import Path
response = requests.get('http://www.example.com/file.xlsx')
Path('file.xlsx').write_bytes(response.content)
# or if you want to have the file name extracted use this less readable line
# Path(response.url.rpartition('/')[2]).write_bytes(response.content)
To achieve the use case of CURL's -O option in Python, you have to write few lines of code like,
import requests
r = requests.get('https://your.url.com/path')
with open('path.txt','w') as fd:
fd.write(r.text)
I'm not sure requests supports a flag for this purpose. However, you can try doing something like:
from pathlib import Path
p = Path('http://www.example.com/file.xlsx')
r = requests.get(str(p))
Path(p.name).write_bytes(r.content)
Related
when I execute the command:
curl http://cs-service:5000/swdpconfig/swdp_templateConfigData/robot_framework
the output is:
{
"adm_ts_path": "/data/ngxp_test_automation/bin/admin",
"be_ts_path": "/data/ngxp_test_automation/bin/backend",
"fe_ts_path": "/data/ngxp_test_automation/bin/frontend",
"ip": "40.124.25.232",
"password": "Er1c550n#123",
"port": "22",
"user": "ngxpcdd"
}
Now I want to use the key-value pairs as parameters in my python script. How can I do that
You could pipe it directly to python if you want it executed in cli, and if not as someone mentioned you can save it as a file and load it in python with json.loads
Here is already answered stackoverflow question that might peak your interest
Pipe output from shell command to a python script
why would you mix python and curl ... that basically doesnt make much sense ... but you easily can
import subprocess,json
data = json.loads(subprocess.check_output('curl ...'))
it would be a much better idea to just use python (either requests or urllib)
import requests
url = "http://cs-service:5000/swdpconfig/swdp_templateConfigData/robot_framework"
data = requests.get(url).json()
if you wanted to use urllib instead i think its just
import urllib.request, json
data = json.loads(urllib.request.urlopen(urlData).read())
This question already has answers here:
How should I write a Windows path in a Python string literal?
(5 answers)
Closed 6 months ago.
I have a JSON file called data.json and I am trying to print the data inside that JSON file. The JSON file got created by the command:
git log --pretty="format:{"commit":"%h", "merge":"%p", "author":"%an", "title":"%s", "body":"%b"}",>"C:\test_temp\data.json"
I am trying to print the data inside the file with the function parse_json but I am getting an error that says IOError: [Errno 22] invalid mode ('r') or filename "C:\test_temp\data.json"
json_directory = "C:\test_temp\data.json"
def parse_json_file(json_directory):
with open(json_directory) as f:
data = json.load(f)
print(data)
The json file is already there but I am not sure why it cannot read that file.
Also the data that got generate from the JSON file does not have proper formatting as the dictionary is not surrounded by the " " even though I indicated it in the executed git log command. Will that cause a problem if I try to parse the json file.
Maybe try:
json_directory = "C:\\test_temp\\data.json"
Your command is producing invalid json, so your json.load method call will never succeed.
You need to escape the quotes-- what you have supplied (as you can see from stack overflow's syntax highlighting) is actually a series of strings which your shell is concatenating together.
In BASH on OSX, escaping the strings looks like:
git log --pretty="format:{\"commit\":\"%h\", \"merge\":\"%p\", \"author\":\"%an\", \"title\":\"%s\", \"body\":\"%b\"}"
You could also enclose the entire argument to pretty with single quotes, as follows:
git log --pretty='format:{"commit":"%h", "merge":"%p", "author":"%an", "title":"%s", "body":"%b"}',>"C:\test_temp\data.json"
Once your json generation command is corrected, I suspect your script will succeed so long as the paths are correct.
If you try to correct the command as I have recommended and it does not work, please post the JSON file you are generating, as well as the shell you are using.
This question already has answers here:
How to use youtube-dl from a python program?
(7 answers)
Closed 4 years ago.
I want to create a python program that allows me to input search terms and return the url of the first result
def search(search):
#Gets the youtube url of the first result
return url
Something along those lines.
I don't know too much about youtube-dl nor how to implement it into python as I couldn't find much info on how to do this through google.
If you are using python 3.5 then this is pretty simple. I don't know if this works in previous versions of python 3.x.
The result that is returned here is a list of the URLs. So if you want the first result use search(text)[0]
import subprocess
def search(text):
command=['youtube-dl', 'ytsearch:"' + text+'"', '-g']
result=subprocess.run(command,stdout=subprocess.PIPE,stderr=subprocess.PIPE,universal_newlines=True).stdout.split()
return result
You must know that every good linux programs has some instructions built-in. To see them, try youtube-dl --help or man youtube-dl
For your question, from the command line:
youtube-dl "ytsearch:Rick Astley" -g
No idea about python, but it should be pretty close to the following equivalent in bash.
#!/bin/bash
youtube-dl "ytsearch: $1" -g
just save this in a file, give it the «execution permision» «allow file to be executed as program» or from the command line chmod +x myprogram.
Then to call:
./myprogram 'rick astley'
This will output the first link. The procedure is exactly the same in many languages: python, php, bash (etc). This is a bash script.
Good luck ;)
I have a list containing url of images. I want to read the images in each url line by line using python. I have tried different ways, but could only read one line.
Not having seen your code, but I would recommend using Requets.
In a shell I did:
pip install --user requests
to get the above module.
If you have an url you would be able to perform in an interactive Python
import requests
r = requests.get("http://docs.python-requests.org/en/master/_static/requests-sidebar.png")
And to examine the content of the image:
print r.content
Beware the above prints the binary content to your console.
Hope it helps.
I am trying to edit Chrome's preferences using command line. The file is a JSON file and I want to edit the data below:
{"browser":{"last_redirect_origin":""}}
to
{"browser":{"enabled_labs_experiments":["ssl-version-max#2"],"last_redirect_origin":""}}
I was using sed command earlier to accomplish this but want to know how this can be done using python. This was the command I was using:
sed -i '.bak' -e 's|\(\"browser\"\):{\(\".*origin\":\"\"\)}|\1:{\"enabled_labs_experiments\":[\"ssl-version-max#2\"],\2}|' ~/Library/Application\ Support/Google/Chrome/Local\ State
The reason I can't use jq is that it is not native to macs and will need installation. I am not able to understand how to do this with python.
I will really appreciate it if someone could help me with this or point me in the right direction.
EDIT
This is what my python script looks like:
import json
jsonData = json.loads(open('/Users/username/Library/Application Support/Google/Chrome/Local state').read())
if 'enabled_labs_experiments' in jsonData['browser']:
if 'ssl-version-max#2' in jsonData['browser']['enabled_labs_experiments']:
print('Exist')
else:
jsonData['browser']['enabled_labs_experiments'] = ['ssl-version-max#2']
print('Added')
After the changes are made, I would like to commit the changes to the file.
Why are you using regex why are you not using the built in json module in python?
import json
d = json.loads(path_to_your_file)
d["browser"]["enabled_labs_experiments"] = ["ssl-version-max#2"]
import json
with open(path_to_json_file) as f:
data = f.read()
d = json.loads(data)
d["browser"]["enabled_labs_experiments"] = ["ssl-version-max#2"]
with open(path_to_json_file, 'w') as f:
f.write(json.dumps(d))