I'm trying to call a Go function from Python using c-shared (.so) file. In my python code I'm calling the function like this:
website = "https://draftss.com"
domain = "draftss.com"
website_ip = "23.xxx.xxx.xxx"
website_tech_finder_lib = cdll.LoadLibrary("website_tech_finder/builds/websiteTechFinder.so")
result_json_string: str = website_tech_finder_lib.FetchAllData(website, domain, website_ip)
On Go side I'm converting the strings to Go strings based on this SO post (out of memory panic while accessing a function from a shared library):
func FetchAllData(w *C.char, d *C.char, dIP *C.char) *C.char {
var website string = C.GoString(w)
var domain string = C.GoString(d)
var domainIP string = C.GoString(dIP)
fmt.Println(website)
fmt.Println(domain)
fmt.Println(domainIP)
.... // Rest of the code
}
The website domain and domainIP just have the first characters of the strings that I passed:
fmt.Println(website) // -> h
fmt.Println(domain) // -> d
fmt.Println(domainIP) // -> 2
I'm a bit new to Go, so I'm not sure if I'm doing something stupid here. How do I get the full string that I passed?
You need to convert the parameters as UTF8 bytes.
website = "https://draftss.com".encode('utf-8')
domain = "draftss.com".encode('utf-8')
website_ip = "23.xxx.xxx.xxx".encode('utf-8')
lib = cdll.LoadLibrary("website_tech_finder/builds/websiteTechFinder.so")
result_json_string: str = website_tech_finder_lib.FetchAllData(website, domain, website_ip)
Related
Hi every one I try to use BAC0 package in python 3 to get value of multiple point in bacnet network.
I user something like following:
bacnet = BAC0.lite(ip=x.x.x.x)
tmp_points = bacnet.readRange("11:2 analogInput 0 presentValue");
and it seems not OK :(
error is:
BAC0.core.io.IOExceptions.NoResponseFromController: APDU Abort Reason : unrecognizedService
And in document I just can find
def readRange(
self,
args,
range_params=None,
arr_index=None,
vendor_id=0,
bacoid=None,
timeout=10,
):
"""
Build a ReadProperty request, wait for the answer and return the value
:param args: String with <addr> <type> <inst> <prop> [ <indx> ]
:returns: data read from device (str representing data like 10 or True)
*Example*::
import BAC0
myIPAddr = '192.168.1.10/24'
bacnet = BAC0.connect(ip = myIPAddr)
bacnet.read('2:5 analogInput 1 presentValue')
Requests the controller at (Network 2, address 5) for the presentValue of
its analog input 1 (AI:1).
"""
To read multiple properties from a device object, you must use readMultiple.
readRange will read from a property acting like an array (ex. TrendLogs objects implements records as an array, we use readRange to read them using chunks of records).
Details on how to use readMultiple can be found here : https://bac0.readthedocs.io/en/latest/read.html#read-multiple
A simple example would be
bacnet = BAC0.lite()
tmp_points = bacnet.readMultiple("11:2 analogInput 0 presentValue description")
I am trying to use python script in the Android studio using chaquopy. But I have 2 problems.
I am not able to import python random.
How to retrieve a list.
Here is the Python Script.
import random
def getPlayers(wk, batsman, bowler, allRounder):
chosen = batsman[:3] + bowler[:3] + allRounder[:1] + wk[:1]
remainder = batsman[3:] + bowler[3:] + allRounder[1:] + wk[1:]
random.shuffle(remainder)
chosen.extend(remainder[:3])
players = {'Batsman': [x for x in chosen if x in batsman],
'Bowler': [x for x in chosen if x in bowler],
'AllRounder': [x for x in chosen if x in allRounder],
'Wk': [x for x in chosen if x in wk]}
for key in players:
for name in players[key]:
return f'{key}: {name}'
It is showing no module named random found.
And The activity file
private String getTeam() {
Python python = Python.getInstance();
PyObject file = python.getModule("getTeam");
return file.callAttr("getPlayers", wk, batsman, bowler, ar);
}
I am calling getTeam method in onCreate. So, How can I get the list of keys and values from python script?
EDIT
I have used this code to access the data but It is showing com.chaquo.python.PyException: TypeError: jarray does not support slice syntax.
Here is the Code
String[] team = getTeam();
for (String s : team) {
Log.d("Player", s);
}
}
private String[] getTeam() {
Python python = Python.getInstance();
PyObject file = python.getModule("getTeam");
return file.callAttr("getPlayers", wk.toArray(), batsman.toArray(), bowler.toArray(), ar.toArray()).toJava(String[].class);
}
It is showing no module named random found.
I assume you're talking about an error shown in the Android Studio editor, rather than one which happened at runtime. As the documentation says, errors in the editor are harmless: just go ahead and run your app, and if there really is an error, the details will be displayed in the Logcat.
How can I get the list of keys and values from python script?
You can write the Python code like this:
return [f'{key}: {name}' for key in players for name in players[key]]
And the Java code like this:
file.callAttr("getPlayers", wk, batsman, bowler, ar).toJava(String[].class)
That will give you a Java String[] array, which you can then process however you want.
EDIT for jarray does not support slice syntax:
This issue is fixed in Chaquopy 9.0.0. With older versions, you can work around it by converting the array to a Python list.
To convert in Python:
def getPlayers(wk, batsman, bowler, allRounder):
wk = list(wk)
batsman = list(batsman)
# etc.
Or to convert in Java:
PyObject list = python.getBuiltins().get("list");
return file.callAttr("getPlayers",
list.call(wk.toArray()),
list.call(batsman.toArray()),
// etc.
I'm wondering if there is an easy to to initialize BPF maps from python userspace. For my project, I'll have a scary looking NxN 2d array of floats for each process. For simplicity's sake, lets assume N is constant across processes (say 5). To achieve kernel support for this data, I could do something like:
b = BPF(text = """
typedef struct
{
float transMat[5][5];
} trans_struct;
BPF_HASH(trans_mapping, char[16], trans_struct);
.....
""")
I'm wondering if theres an easy way to initialize this map from python. Something like:
for ele in someDictionary:
#asume someDitionary has mapping (comm -> 5x5 float matrix)
b["trans_mapping"].insert(ele, someDictionary[ele])
I suppose at the crux of my confusion is -- 1) are all map methods available to the user, 2) how do we ensure type consistenty when going from python objects to c structures
Solution based on pchaigno's comment -- The key things to note are the use of c_types to ensure type consistency across environments, and extracting the table by indexing the BPF program object. Due to our ability to get maps by indexing, the get_table() function is now considered out of date. This format provides the structure of loading data into the map from the python front-end, but doesn't completely conform with the specifics of my question.
from time import sleep, strftime
from bcc import BPF
from bcc.utils import printb
from bcc.syscall import syscall_name, syscalls
from ctypes import *
b = BPF(text = """
BPF_HASH(start, u32, u64);
TRACEPOINT_PROBE(raw_syscalls, sys_exit)
{
u32 syscall_id = args->id;
u32 key = 1;
u64 *val;
u32 uid = bpf_get_current_uid_gid();
if (uid == 0)
{
val = start.lookup(&key); //find value associated with key 1
if (val)
bpf_trace_printk("Hello world, I have value %d!\\n", *val);
}
return 0;
}
""")
thisStart = b["start"]
thisStart[c_int(1)] = c_int(9) #insert key-value part 1->9
while 1:
try:
(task, pid, cpu, flags, ts, msg) = b.trace_fields()
except KeyboardInterrupt:
print("Detaching")
exit()
print("%-18.9f %-16s %-6d %s" % (ts, task, pid, msg))
I am following the Machine Learning with Spark Book and trying to convert the python code to scala code and using Beaker notebook to share variables in order to pass values to python to plot with matplotlib as described in the book. Most of the code so far I have been able to convert but I am having some issues with the try-catch conversion with data cleansing with the u.item dataset. Below code ends in a infinite loop without a clear issue what the error is.
val movieData = sc.textFile("/Users/minHenry/workspace/ml-100k/u.item")
val movieDataSplit = movieData.first()
val numMovies = movieData.count()
def convertYear(x:String):Int = x.takeRight(4) match {
case x => x.takeRight(4).toInt
case _ => 1900
}
val movieFields = movieData.map(lines => lines.split('|'))
print(movieData.first())
val years1 = movieFields.map(fields => fields(2))
val years = movieFields.map(fields => fields(2).map(x=>convertYear(x.toString())))
val filteredYears = years.filter(x => x!=1900)
years.take(2).foreach(println)
I suspect my problem is with my pattern match but I am not exactly sure what's wrong with it. I think the takeRight() works because it doesn't complain about the type that this function is being applied to.
UPDATE
I have updated the code as follows, per advice from the answer provided thus far:
import scala.util.Try
val movieData = sc.textFile("/Users/minHenry/workspace/ml-100k/u.item")
def convertYear(x:String):Int = Try(x.toInt).getOrElse(1900)
val movieFields = movieData.map(lines => lines.split('|'))
val preYears = movieFields.map(fields => fields(2))
val years = preYears.map(x => x.takeRight(4))//.map(x=>convertYear(x))
println("=======> years")
years.take(2).foreach(println) //--output = 1995/n1995
println("=======> filteredYears")
val filteredYears = years.filter(x => x!=1900)
filteredYears.take(2).foreach(println)
//val movieAges = filteredYears.map(yr => (1998-yr)).countByValue()
I commented out the map following the takeRight(4) because its easier to comment than putting x=>convertYear(x.takeRight(4)) and should produce the same output. When I apply this convertYear() function i still end up in an infinite loop. the values print as expected in the few print statements shown. The problem is if i cannot remove the data point that cannot be easily converted to Int then I am unable to run the countByValue() function in the last line.
Here is the link to my public beaker notebook for more context:
https://pub.beakernotebook.com/#/publications/56eed31d-85ad-4728-a45d-14b3b08d673f
movieData: RDD[String]
movieFields: RDD[Array[String]]
years1: RDD[String]
val years = movieFields.map(fields => fields(2).map(x=>convertYear(x.toString()))) - fields(2) is String and so x is Char, because String is treated as Seq[Char]. All inputs to convertYear(x: String) have only one letter string.
Your error is types incompatability hiding (convertYear(x.toString())). It's alarm bell. Always use type system in scala, don't hide problem with toString() or isInstanceOf or something else. Then compiler shows error before running.
P.S.
Second call of takeRight is useless.
def convertYear(x:String):Int = x.takeRight(4) match {
case x => x.takeRight(4).toInt
case _ => 1900
}
Pattern matching is about checking type or conditions (with if statement). Your first partial function doesn't check anything. All inputs go to x.takeRight(4).toInt. Also there is no defence against toInt exception.
Use instead def convertYear(x: String): Int = Try(x.toInt).getOrElse(1900).
Update
scala> import scala.util.Try
import scala.util.Try
scala> def convertYear(x:String):Int = Try(x.toInt).getOrElse(1900)
convertYear: (x: String)Int
scala> List("sdsdf", "1989", "2009", "1945", "asdf", "455")
res0: List[String] = List(sdsdf, 1989, 2009, 1945, asdf, 455)
scala> res0.map(convertYear)
res1: List[Int] = List(1900, 1989, 2009, 1945, 1900, 455)
With RDD all the same, because it is a functor as List.
val filteredYears = years.filter(x => x!=1900) Wouldn't work as you expect. x is a String not Int. Scala doesn't implicitly convert types for comparision. So you always get true.
I'm using a matlab-function in simulink to call a python script, that do some calculations from the input values. The python-script gives me a string back to the matlab-function, that I split to an array. The splitted string has always to be a cell array with 6 variable strings:
dataStringArray = '[[-5.01 0.09785429][-8.01 0.01284927]...' '10.0' '20.0' '80.0' '80.0' '50.0'
To call the functions like strsplit or the python-script itself with a specific m-file, I'm using coder.extrinsic('*') method.
Now I want to index to a specific value for example with dataStringArray(3) to get '20.0' and define it as an output value of the matlab-function, but this doesn't work! I tried to predefine the dataStringArray with dataStringArray = cell(1,6); but get always the same 4 errors:
Subscripting into an mxArray is not supported.
Function 'MATLAB Function' (#23.1671.1689), line 42, column 24:
"dataStringArray(3)"
2x Errors occurred during parsing of MATLAB function 'MATLAB Function'
Error in port widths or dimensions. Output port 1 of 's_function_Matlab/MATLAB Function/constIn5' is a one dimensional vector with 1 elements.
What do I'm wrong?
SAMPLE CODE
The commented code behind the output definitions is what I need.:
function [dataArrayOutput, constOut1, constOut2, constOut3, constOut4, constOut5] = fcn(dataArrayInput, constIn1, constIn2, constIn3, constIn4, constIn5)
coder.extrinsic('strsplit');
% Python-Script String Output
pythonScriptOutputString = '[[-5.01 0.088068861]; [-4.96 0.0]]|10.0|20.0|80.0|80.0|50.0';
dataStringArray = strsplit(pythonScriptOutputString, '|');
% Outputs
dataArrayOutput = dataArrayInput; % str2num(char((dataStringArray(1))));
constOut1 = constIn1; % str2double(dataStringArray(2));
constOut2 = constIn2; % str2double(dataStringArray(3));
constOut3 = constIn3; % str2double(dataStringArray(4));
constOut4 = constIn4; % str2double(dataStringArray(5));
constOut5 = constIn5; % str2double(dataStringArray(6));
SOLUTION 1
Cell arrays are not supported in Matlab function blocks, only the native Simulink datatypes are possible.
A workaround is to define the whole code as normal function and execute it from the MATLAB-Function defined with extrinsic. It`s important to initialize the output variables with a known type and size before executing the extrinsic function.
SOLUTION 2
Another solution is to use the strfind function, that gives you a double matrix with the position of the splitter char. With that, you can give just the range of the char positions back that you need. In this case, your whole code will be in the MATLAB-Function block.
function [dataArrayOutput, constOut1, constOut2, constOut3, constOut4, constOut5] = fcn(dataArrayInput, constIn1, constIn2, constIn3, constIn4, constIn5)
coder.extrinsic('strsplit', 'str2num');
% Python-Script String Output
pythonScriptOutputString = '[[-5.01 0.088068861]; [-4.96 0.0]; [-1.01 7.088068861]]|10.0|20.0|80.0|80.0|50.0';
dataStringArray = strfind(pythonScriptOutputString,'|');
% preallocate
dataArrayOutput = zeros(3, 2);
constOut1 = 0;
constOut2 = 0;
constOut3 = 0;
constOut4 = 0;
constOut5 = 0;
% Outputs
dataArrayOutput = str2num(pythonScriptOutputString(1:dataStringArray(1)-1));
constOut1 = str2num(pythonScriptOutputString(dataStringArray(1)+1:dataStringArray(2)-1));
constOut2 = str2num(pythonScriptOutputString(dataStringArray(2)+1:dataStringArray(3)-1));
constOut3 = str2num(pythonScriptOutputString(dataStringArray(3)+1:dataStringArray(4)-1));
constOut4 = str2num(pythonScriptOutputString(dataStringArray(4)+1:dataStringArray(5)-1));
constOut5 = str2num(pythonScriptOutputString(dataStringArray(5)+1:end));
When using an extrinsic function, the data type returned is of mxArray, which you cannot index into as the error message suggests. To work around this problem, you first need to initialise the variable(s) of interest to cast them to the right data type (e.g. double). See Working with mxArrays in the documentation for examples of how to do that.
The second part of the error message is a dimension. Without seeing the code of the function, the Simulink model and how the inputs/outputs of the function are defined, it's difficult to tell what's going on, but you need to make sure you have the correct size and data type defined in the Ports and Data manager.