i have made a simple python file which want to submit in Livy.Livy is currently running in local mode.Also I have mentioned following property in livy.conf file.
Property name: livy.file.local-dir-whitelist,
value "/usr/local/livy/scripts"
My file is kept in following path "/usr/local/livy/scripts"
import json, pprint, requests, textwrap
host = 'http://localhost:8998'
data = {'kind': 'spark'}
headers = {'Content-Type': 'application/json'}
r = requests.post(host + '/sessions', data=json.dumps(data), headers=headers)
r.json()
I am submitting it using curl as follows:
curl -X POST --data '{"file": "/usr/local/livy/scripts/pi.py"}' -H "Content-Type: application/json" 10.140.178.24:8999/batches
It is giving me following error:
requirement failed: Local path /usr/local/livy/scripts/pi.py cannot be added to user sessions.
My Ubuntu system only have following things:
a)Spark
b)Livy
c)Java
What am I doing wrong here?
For people using incubating mode of livy for first time,kindly check that the template file is renamed with stripping off .template in livy.conf.template.Then make sure that the following configurations are present in it.
livy.spark.master = local
livy.file.local-dir-whitelist = /path/to/script/folder/
Kindly make sure that forward slash is present in end of path
Then write url in following manner for
Python:
curl -v -X POST --data '{"file": "/path/to/script/folder/name-of-python-file.py"}' -H "Content-Type: application/json" localhost:8998/batches
Note:It will not accept relative path,whole absolute path needs to be defined in it.
curl -X POST --data '{"file": "/usr/local/livy/scripts/pi.py"}' -H "Content-Type: application/json" 10.140.178.24:8999/batches
{"id":2,"state":"starting","log":[]}
Related
Trying to convert curl below to the Python requests, below is the curl command which is working fine.
curl -k -H "Authorization: Token token=\"$token\""https://conjur.com/secret
Above curl works fine and gives expected output but when I turn that into python requests it is giving me trouble the header is weird not sure how to pass that, i tried below
token="abcdef"
token_header = {'Authorization': f'Token token={token}'}
requests.get("https://conjur.com/secret", headers=token_header).text
Above code gives error and looks like header is not working as expected,let me know how can i solve it ?
Already mentioned above
In your curl command you have quotes around the token:
curl -k -H "Authorization: Token token=\"$token\"" https://conjur.com/secret
Note the token=\"$token\"
To have quotes in your Python code do:
token_header = {"Authorization": f'Token token="{token}"'}
I am sending text by cUrl
curl -X POST -d "Separate account charge and opdeducted fr" http://192.168.50.8/text
and try to get
#application.route("/text",methods=['POST'])
def clausIE():
content = request.data
text = str(content, encoding="utf-8")
But get empty string, what I am doing wrong?
Note: I use Python3.6
This is not really a Flask problem, you are using the wrong curl options.
The -d switch should only be used for form data. curl automatically will set the Content-Type header to application/x-www-form-urlencoded, which means that Flask will load the raw body content and parse it as a form. You'll have to set a different Content-Type header manually, using -H 'Content-Type: application/octet-stream' or another mime-type more appropriate to your data.
You also want to use --data-binary, not -d (--data), as the latter also tries to parse the content into key-value fields and will remove newlines:
curl -X POST -H 'Content-Type: application/octet-stream' \
--data-binary "Separate account charge and opdeducted fr" \
http://192.168.50.8/text
The complete answer seems to be scattered around some comments and the accepted reply. So to summarize this, the Python Flask code should look like
#application.route("/text",methods=['POST'])
def clausIE():
content = request.get_data()
text = str(content, encoding="utf-8")
return text
and this is what you should have at your terminal
curl -X POST --data-binary "Hello World!" http://192.168.50.8/text
At my own setup (OS X) I am allowed to drop -X POST and to wrap the URL around quotes, so that gives
curl --data-binary "Hello World!" "http://192.168.50.8/text"
I am trying to make a post request using curl in python but the below script throws error
import os
first_name1 = "raj"
last_name1 = "kiran"
full_name = "raj kiran"
headline = "astd"
location1 = "USA"
current_company1 = "ss"
curl_req = 'curl -X POST -H "Content-Type: application/json" -d '{"first_name":"{0}","last_name":"{1}","current_company":"{2}","title":"{3}","campus":"","location":"{4}","display_name":"{5}","find_personal_email":"true"}' http://localhost:8090'.format(first_name1,last_name1,current_company1,headline,location1,full_name)
os.popen(curl_req)
Error:
SyntaxError: invalid syntax
How to make above program work?
The problem in your code is the quotes. Change it to:
curl_req = '''curl -X POST -H "Content-Type: application/json" -d '{"first_name":"{0}","last_name":"{1}","current_company":"{2}","title":"{3}","campus":"","location":"{4}","display_name":"{5}","find_personal_email":"true"}' http://localhost:8090'''.format(first_name1,last_name1,current_company1,headline,location1,full_name)
But, as mentioned in the comments, requests will always be a better choice.
Requests syntax:
import requests
post_data = {
# all the data you want to send
}
response = requests.post('http://localhost:8090', data=post_data)
print response.text
One good resource I've used for converting a cURL request to Python requests is curlconverter. You can enter your cURL request and it will format it for Python requests.
As a side note, it can also convert for PHP and Node.js.
Hope that helps!
I'm sending a POST request via terminal to my flask backend server by doing the following:
curl -v -H "Accept: application/json" -H "Content-type: application/json" -X POST -d '{“Email”:”email#email.com”,”Password”:”testpass”}' http://127.0.0.1:5000/auth/login
I can print out the request data in flask by doing the following:
print(request.data)
However, when I try to input the data into a form to validate the information, the process comes to a halt and fails.
I'm pretty sure the issue is caused by this code, since nothing gets executed after this line:
data = MultiDict(mapping=request.json)
Any idea why?
I'm not sure that your quotes are all the same in your curl command. I tried adapting that to one of my apis and it returned an error. Are you using the right quotes in your json? Are those smart quotes? I was able to replace them and it worked.
curl -v -H "Accept: application/json" -H "Content-type: application/json" -X POST -d '{"email":"email#email.com","password":"testpass"}' http://127.0.0.1:5000/auth/login
If you are then taking the dictionary and then loading them into a form (form = LoginForm(**data)) you also need to make sure your keys match the naming of your form. I see in your json you have Email with E uppercase. Make sure that is the name of the form field in your form. If it is email = StringField(...) it won't map.
Am using curl exe in windows, to communicate with my Django backend.
Following is the command am using.
curl --dump-header - -H "Accept: application/json" -H "Content-Type: application/json" -X POST --data "{\"uid\":12,\"token\":\"asdert\"}" http://localhost:8000/restapi/v1/foo/
Now this give the data in wrong format. i.e. in the view the post is showing this data
print request.POST
{"{\"uid\":12,\"access_token\":\"asdert\"}": [""]}
What is the correct way to post json data ?
Edit:
I have tried several other methods for e.g.
I am trying to communiate with my rest api using http://slumber.in/.
Even here am getting the same result as above.
import slumber
api = slumber.API("http://localhost/restapi/v1/"
api.foo.post({"uid":"100"})
Excerpts from the view
print request.POST
{u'{"uid": "100"}': [u'']}
P.S. - curl --dump-header - -H "Accept: application/json" -H "Content-Type: application/json" -X POST --data "uid=12&token=asdert" http://localhost:8000/restapi/v1/foo/
This works. But this is not Json format.
I tried your command with http://httpbin.org/post and it worked fine.
Now, your problem is that you should access the incoming JSON data from
request.raw_post_data
instead of request.POST.
(Or if you are using Django 1.4+, use request.body instead as request.raw_post_data is being deprecated)
Detailed code should be something like this:
import json
if request.method == "POST":
data = json.loads(request.raw_post_data)
print data