$client = new Client();
$a = 'https://scholar.google.com/citations?user=';
$gscID = 'EnegzCwAAAAJ';//for eg
$b = '&hl=en&oi=ao';
$url = $a . $gscID . $b;
$crawler = $client->request('GET', $url);
//python script
$process = new Process(['python', public_path() . '\ext.py'], null, ['SYSTEMROOT' => getenv('SYSTEMROOT'), 'PATH' => getenv("PATH")]);
$process->run();
// executes after the command finishes
if (!$process->isSuccessful()) {
throw new ProcessFailedException($process);
}
$out[] = $process->getOutput();
dd($out);
Extracting the tables from the given url webpage
Now I want to pass the $url variable to the python script stored in my public folder because everytime my url changes with different person's ID.
Any help would be really appreciated
Found out an easy solution
$output = shell_exec("python ext.py $url");//ext.py is name of my script and $url is my variable which I want to pass
And then in python script write
import sys
url = sys.argv[1]
You can add script parameters in the array that you are passing to Process.
Here is the doc.https://symfony.com/doc/current/components/process.html#using-features-from-the-os-shell
Related
I've been getting this problem for a little while now.
The thing is that i have an HTML form proceed by PHP which looks like that:
<?php
if (isset($_POST)) {
$link = mysqli_connect("localhost", "root", "password", "tablename");
$url = $_POST['url'];
$shorturl = str_replace(['/', 'https:', 'http:'], '', $url);
$iconpath = shell_exec("python3 /home/favicon.py {$url}");
$sql = "INSERT INTO links (url, shorturl, iconpath) values ('{$url}', '{$shorturl}', '{$iconpath}')";
$result = mysqli_query($link, $sql);
}
/*
header('Location: /');
exit;
*/
?>
This code is supposed to extract the given URL from the form, execute python file that downloads favicon of the site and insert the data into a MySQL table.Python code:
import sys
import requests
url = sys.argv[1]
img_data = requests.get(f'{url}/favicon.ico').content
iconpath = '/home/favicon.png'
with open(iconpath, 'wb') as handler:
handler.write(img_data)
print(iconpath)
It all works fine except for the favicon part. When i run the python file from shell, it works fine. When i run it from PHP interactive console by the exact same code written in the form handler, it works fine. But it just won't run properly in the handler, because even when i try to debug it like
echo $iconpath;
it doesn't actually show anything in there. What can i do with it?
P.S. the code actually inserts $url and $shorturl variables into the database, so i said it all works fine except the shell_exec part
I'm trying to utilize a database from another program in a php based website tool, and apparently the original was built in python and puts some of it's data into a python tuple and serializes it to store it as a blob in the sql table.
I'm not a python programmer so I'm not sure how to even see what is in this blob, but I do know that some of the 'type' indicators for the data field are stored in there and I want to extract them and anything else useful.
Is there any way to 'unserialize' a python tuple in php?
The blob data turned out to be a pickled tuple (part of the reason I despise python - both data types that only python can read! Python programmers: 'standardized conventions? Who needs standardized conventions?!?!')
I came up with a cludgy way to 'unpickle' the data and json serialize it using a command line. To get the binary blob data into the command line, I base64 encode it. It's janky but it works for what I need:
/**
* use a python exec call to 'unpickle' the blob_data
* to get the binary blob into a command line argument, base64 encode it
* to get the data back out of python, json serialize it
* #param string $blob binary blob data
* #return mixed
*/
public static function unpickle($blob) {
$cmd = sprintf("import pickle; import base64; import json; print(json.dumps(pickle.loads(base64.b64decode('%s'))))", base64_encode($blob));
$pcmd = sprintf("python -c \"%s\"", $cmd);
$result = exec($pcmd);
$resdec = json_decode($result);
return $resdec;
}
With a little more playing on this concept, I gave myself a few more alternatives. First is, I took the command line version above and made it into a little more functional python script:
unpickle.py:
#!/usr/bin/env python3
import pickle
import json
import sys
import base64
import select
def isBase64(s):
try:
return s == base64.b64encode(base64.b64decode(s)).decode('ascii')
except Exception:
return False
bblob = None
if (len(sys.argv) > 1) and isBase64(sys.argv[1]):
bblob = base64.b64decode(sys.argv[1])
elif select.select([sys.stdin, ], [], [], 0.0)[0]:
try:
with open(0, 'rb') as f:
bblob = f.read()
except Exception as e:
err_unknown(e)
if bblob != None:
unpik = pickle.loads(bblob)
jsout = json.dumps(unpik)
print(jsout)
This script allows you to either specify the blob data from the pickled tuple 'byte' as a base64 encoded string on the command line, or you can pipe raw blob data into the script. Both variations will output json if the data is valid and formatted properly. (null if not)
You can convert this to a self-contained binary to plop on systems without python using pyinstaller -F if need be. To play with it in the event I am running it on systems with the pyinstaller binary vs one with the python script vs one with just python, I created the following static methods in my laravel model. (I'll eventually move it into a service module)
/**
* call either a pyinstaller binary or python script with raw blob data to be unpickled
*
* #param string $b binary data of blob
* #return false|mixed
*/
public static function unpickle($b)
{
$cmd = base_path(env('UNPICKLE_BINARY', 'bin/unpickle'));
if(!(is_file($cmd) && is_executable($cmd))) { // make sure unpickle cmd exists
// check for UNPICKLE_BINARY with .py after and python binary
$pyExe = env('PYTHON_EXE', '/usr/bin/python');
if (is_file($cmd.".py") && (is_file($pyExe) && is_executable($pyExe))) {
$cmd = sprintf("%s %s.py", $pyExe, $cmd);
} else
return static::unpyckle($b); // try direct python call
}
$descriptorspec = [
["pipe", "r"],
["pipe", "w"],
["pipe", "w"]
];
$cwd = dirname($cmd);
$env = [];
$process = proc_open($cmd, $descriptorspec, $pipes, $cwd, $env);
if (is_resource($process)) {
fwrite($pipes[0], $b);
fclose($pipes[0]);
$output = stream_get_contents($pipes[1]);
fclose($pipes[1]);
$return_value = proc_close($process);
if(static::isJson($output))
return json_decode($output);
else
return false;
}
return false;
}
/**
* use a python exec call to 'unpickle' the blob_data
* to get the binary blob into a command line argument, base64 encode it
* to get the data back out of python, json serialize it
* #param string $blob binary blob data
* #return mixed
*/
public static function unpyckle($blob) {
$pyExe = env('PYTHON_EXE', '/usr/bin/python');
if (!(is_file($pyExe) && is_executable($pyExe)))
throw new Exception('python executable not found!');
$bblob = base64_encode($blob);
$cmd = sprintf("import pickle; import base64; import json; print(json.dumps(pickle.loads(base64.b64decode('%s'))))", $bblob);
$pcmd = sprintf("%s -c \"%s\"", $pyExe, $cmd);
$result = exec($pcmd);
$resdec = json_decode($result);
return $resdec;
}
/**
* try to detect if a string is a json string
*
* #param $str
* #return bool
*/
public static function isJson($str) {
if(is_string($str) && !empty($str)) {
json_decode($str);
return (json_last_error() == JSON_ERROR_NONE);
}
return false;
}
example .env values:
UNPICKLE_BINARY=bin/unpickle
PYTHON_EXE=/usr/bin/python3
basically showing three different ways to call python to do essentially the same thing...
I am working in Qt 5 and struggling with a multipart upload. My script is as close to the docs as possible:
QUrl testUrl("http://localhost/upload/test.php");
QNetworkRequest request(testUrl);
QHttpMultiPart *multiPart = new QHttpMultiPart(QHttpMultiPart::FormDataType);
QString preview_path = "C:/preview.jpg";
QHttpPart previewPathPart;
previewPathPart.setHeader(QNetworkRequest::ContentDispositionHeader, QVariant("form-data; name=\"preview_path\""));
previewPathPart.setBody(preview_path.toLatin1());
QHttpPart previewFilePart;
previewFilePart.setHeader(QNetworkRequest::ContentTypeHeader, QVariant( "image/jpeg"));
previewFilePart.setHeader(QNetworkRequest::ContentDispositionHeader, QVariant("form-data; name=\"preview_file\""));
QFile *file = new QFile(preview_path);
if (!file->exists()) {
emit error(tr("Upload Error. File does not exist: ") + preview_path);
return;
}
file->open(QIODevice::ReadOnly);
previewFilePart.setBodyDevice(file);
file->setParent(multiPart); // we cannot delete the file now, so delete it with the multiPart
multiPart->append(previewPathPart);
multiPart->append(previewFilePart);
reply = networkManager->post(request, multiPart);
multiPart->setParent(reply); // delete the multiPart with the reply
connect(reply, SIGNAL(finished()),
this, SLOT (uploadReply()));
connect(reply, SIGNAL(error(QNetworkReply::NetworkError)),
this, SLOT (uploadError(QNetworkReply::NetworkError)));
connect(reply, SIGNAL(uploadProgress(qint64, qint64)),
this, SLOT (uploadProgress(qint64, qint64)));
then my uploadReply() slot just prints the reply:
QString data = (QString) reply->readAll();
qDebug() << data;
I have made the upload script is as simple as possible and running in XAMPP with Apache/2.2.21 and PHP 5.3.8. My upload_max_filesize and post_max_size are 1000M.
echo "preview_path: " . (isset($_POST['preview_path']) ? $_POST['preview_path'] : "not set") . "\r\n";
echo "preview_file exists: " . (isset($_POST['preview_file']) ? "true" : "false" ). "\r\n";
echo '$_FILES: ';
print_r($_FILES);
echo "preview_file content: " . $_POST['preview_file'];
my progress slot shows the roughly the correct number of bytes being uploaded.
But the output shows:
preview_path: C:/preview.jpg
preview_file exists: true
$_FILES: Array
(
)
preview_file content: ????
It seems like the bytes are being submitted as $_POST variable rather than a $_FILE as they should be? There is no error in the apache log. How can I debug this?
Solved!
Thanks to this question for the hint - I added 'filename' to the content disposition for the QHttpPart and it now uploads as expected. I thought filename was optional but seems to work in this case. Hope this helps someone else!
QString preview_name = "preview.jpg";
QHttpPart previewFilePart;
previewFilePart.setHeader(QNetworkRequest::ContentTypeHeader, QVariant("image/jpeg"));
previewFilePart.setHeader(QNetworkRequest::ContentDispositionHeader, QVariant("form-data; name=\"preview_file\"; filename=\""+ preview_name + "\""));
I want to print out an invoice(pdf) automatically, what's recently saved on the server. And also making manual saving possible
I'm using prestashop 1.6.1 and the invoices are mostly downloaded from the prestashop admin page, but I needed more easier way to print out these invoices, so I made an adminpage for myself it looks like this:
The printer button has a href of the invoice generated address
like:"http://www.example.com/admin/index.php?controller=AdminPdf&submitAction=generateInvoicePDF&id_order=3230"
From the link I can download it and then print it when it's opened in pdf reader, but I want to do this in one click.
Soo... I made an script for automatically printing the pdf when its saved on some specific location
#! /usr/bin/python import os
import os
import time
import os.path
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
class ExampleHandler(FileSystemEventHandler):
def on_created(self, event):
output=str(event.src_path.replace("./",""))
print(output)
#print event.src_path.replace("./","")
print "Got event for file %s" % event.src_path
os.system("lp -d HL2250DN %s" % output)
observer = Observer()
event_handler = ExampleHandler()
observer.schedule(event_handler, path='.',recursive=False)
observer.start()
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
observer.stop()
observer.join()
There is two options to download it automatically to the server
1. Override PDF.php and PDFGenerator.php files like this:
PDF.php
class PDF extends PDFCore
{
public function render($display = true)
{
if($this->template == PDF::TEMPLATE_INVOICE)
parent::render('F', true);
return parent::render($display);
}
}
?>
PDFGenerator.php
<?php
class PDFGenerator extends PDFGeneratorCore
{
public function render($filename, $display = true)
{
if (empty($filename)) {
throw new PrestaShopException('Missing filename.');
}
$this->lastPage();
if ($display === true) {
$output = 'D';
} elseif ($display === false) {
$output = 'S';
} elseif ($display == 'D') {
$output = 'D';
} elseif ($display == 'S') {
$output = 'S';
} elseif ($display == 'F') {
$output = 'F';
$filename = '/folder/for/print_it/'.str_replace("#", "", $filename);
} else {
$output = 'I';
}
return $this->output($filename, $output);
}
}
?>
2. Use script to download
First attempt
The first option worked for automatic saving, but when I tried to save invoices manually I got an blank or broken pdf file. I also tried to change the pdf.php, but it dident work out for me. Also made an post about this: Prestashop saving invoices manually and automatically. No answers were given and I moved on second option.
Second attempt
I tried to download invoices with python script and it worked, but how can I know which one to download?
#!/usr/bin/env python
import requests
import webbrowser
url = "http://www.example.com/admin/index.php?controller=AdminLogin&token=5a01dc4e606bca6c26e95ddea92d3d15"
url2 = "http://www.example.com/admin/index.php?controller=AdminPdf&token=35b276c05aa6f5eb516737a8d534eb66&submitAction=generateInvoicePDF&id_order=3221"
payload = {'example': 'example',
'example': 'example',
'stay_logged_in':'2',
'submitLogin':'1',}
with requests.session() as s:
# fetch the login page
s.get(url)
# post to the login form
r = s.post(url, data=payload)
print(r.text)
response = s.get(url2)
with open('/tmp/metadataa.pdf', 'wb') as f:
f.write(response.content)
Soo the problem with this option is that.. How can I pass the href(what was clicked from the printer button) to url?
Solving this has been really frustrating and I know there is an simple and easy option for this, but I'm still looking for this.
Everytime you generate invoice PDF you are forcing it to be saved as local file.
What you want to do is add an extra GET parameter to the print button and check for its presence in the overriden class so that the PDF only gets stored as local file when you want to print directly.
So first add a GET parameter to print buttons eg. &print=1. Either in your template or wherever you are generating these buttons so that the button's href looks like this:
http://www.example.com/admin/index.php?controller=AdminPdf&submitAction=generateInvoicePDF&id_order=3230&print=1
Now you can check if parameter exists in PDF class and only then force the PDF to be outputted to local file.
class PDF extends PDFCore
{
public function render($display = true)
{
if($this->template == PDF::TEMPLATE_INVOICE && Tools::getValue('print') == 1) {
// Output PDF to local file
parent::render('F');
// Redirect back to the same page so you don't get blank page
Tools::redirectAdmin(Context::getContext()->link->getAdminLink('AdminMyController'));
}
else {
return parent::render($display);
}
}
}
You can keep the overriden PDFGenerator class as is.
I have an Adobe Air Program that calls a python script. I dont' think the actionscript 3.0 is making the proper call. Code:
var file:File;
var args:Vector.<String> = new Vector.<String>;
file = new File().resolvePath("/usr/bin/python");
var pyScript:File;
pyScript = File.applicationDirectory.resolvePath("python/mac/merge.py");
var tempOutPath:String = File.applicationStorageDirectory.resolvePath("out.pdf").nativePath;
args.push(pyScript.nativePath, "-w", "-o", tempOutPath, "-i");
for(var x:int; x < numFilesToProcess; x++){
var pdfPath:String = File(pdfs.getItemAt(x)).nativePath;
args.push(pdfPath);
}
callNative(file, args);
In terminal (Mac), the following works fine:
python merge.py -w -o out.pdf -i file1.pdf file2.pdf
The args.push(pyScript.native.... line is the problematic one. I'd appreciate some assistance.
I have faced a similar problem using Air. I needed to print to a receipt printer from an Air app. I couldn't do it from the app itself so I used a python RPC server to do the work for me and talked to it over http. below is a simplified version to give you an idea:
The python RPC server
from SimpleXMLRPCServer import SimpleXMLRPCServer
from SimpleXMLRPCServer import SimpleXMLRPCRequestHandler
class RequestHandler(SimpleXMLRPCRequestHandler):
rpc_paths = ('/','/RPC2')
server = SimpleXMLRPCServer(('localhost', 8123), requestHandler=RequestHandler)
def myService( arg0, arg1 ):
#do the stuff
return 0
server.register_function(myService)
server.serve_forever()
In Air I create the call as an XML string and make my request.
I haven't shown all the details as I was using javascript not actionscript so please treat this as pseudocode.
// XML as a string
// possibly create the XML and toXMLString() it?
var data:String = '
<?xml version="1.0"?>
<methodCall>
<methodName>myService</methodName>
<params>
<param>
<string>file1.pdf</string>
</param>
<param>
<string>file2.pdf</string>
</param>
</params>
</methodCall>';
var req:URLRequest = new URLRequest('localhost:8123');
rec.method = 'POST';
rec.data = data;
var loader:URLLoader = new URLLoader( req );
//etc