I have 2 services that are defined in the same thrift file and share a port. I can use any method from serviceA no problem but whenever i try to call any of ServiceB's methods i get the exception.
this is my thrift file (service-a.thrift):
service ServiceA extends common.CommonService {
list<i64> getByIds(1: list<i64> ids)
...
}
service ServiceB extends common.CommonService {
list<i64> getByIds(1: list<i64> ids)
...
}
notes:
I'm working with a python client
Thrift version 0.8.0
Any ideas?
We had this need as well and solved by writing a new implementation of TProcessor that creates a map of multiple processors. The only gotcha is that with this implementation you need to ensure no method names overlap - i.e. don't use nice generic names like Run() in different servers. Apologies on not converting C# to Python...
Example Class:
using System;
using System.Collections;
using System.Collections.Generic;
using System.IO;
using System.Reflection;
using Thrift;
using Thrift.Protocol;
/// <summary>
/// Processor that allows for multiple services to run under one roof. Requires no method name conflicts across services.
/// </summary>
public class MultiplexProcessor : TProcessor {
public MultiplexProcessor(IEnumerable<TProcessor> processors) {
ProcessorMap = new Dictionary<string, Tuple<TProcessor, Delegate>>();
foreach (var processor in processors) {
var processMap = (IDictionary) processor.GetType().GetField("processMap_", BindingFlags.NonPublic | BindingFlags.Instance).GetValue(processor);
foreach (string pmk in processMap.Keys) {
var imp = (Delegate) processMap[pmk];
try {
ProcessorMap.Add(pmk, new Tuple<TProcessor, Delegate>(processor, imp));
}
catch (ArgumentException) {
throw new ArgumentException(string.Format("Method already exists in process map: {0}", pmk));
}
}
}
}
protected readonly Dictionary<string, Tuple<TProcessor, Delegate>> ProcessorMap;
internal protected Dictionary<string, Tuple<TProcessor, Delegate>> GetProcessorMap() {
return new Dictionary<string, Tuple<TProcessor, Delegate>>(ProcessorMap);
}
public bool Process(TProtocol iprot, TProtocol oprot) {
try {
TMessage msg = iprot.ReadMessageBegin();
Tuple<TProcessor, Delegate> fn;
ProcessorMap.TryGetValue(msg.Name, out fn);
if (fn == null) {
TProtocolUtil.Skip(iprot, TType.Struct);
iprot.ReadMessageEnd();
var x = new TApplicationException(TApplicationException.ExceptionType.UnknownMethod, "Invalid method name: '" + msg.Name + "'");
oprot.WriteMessageBegin(new TMessage(msg.Name, TMessageType.Exception, msg.SeqID));
x.Write(oprot);
oprot.WriteMessageEnd();
oprot.Transport.Flush();
return true;
}
Console.WriteLine("Invoking service method {0}.{1}", fn.Item1, fn.Item2);
fn.Item2.Method.Invoke(fn.Item1, new object[] {msg.SeqID, iprot, oprot});
}
catch (IOException) {
return false;
}
return true;
}
}
Example Usage:
Processor = new MultiplexProcessor(
new List<TProcessor> {
new ReportingService.Processor(new ReportingServer()),
new MetadataService.Processor(new MetadataServer()),
new OtherService.Processor(new OtherService())
}
);
Server = new TThreadPoolServer(Processor, Transport);
As far as I know, there's no straightforward way to bind several services to a single port without adding this field to TMessage and recompiling thrift. If you want to have two services using the same Server, you should reimplement Thrift Server, which it doesn't seem an easy task
Related
I'm trying to pass data from a Raspberry Pi device to a firebase database. I've been successful on making the Pi side of things workout. But so far I've been unsuccessful on making the data flow to my pc using WinForms and C#.
The expected result would be the data inside the database flow to the GUI in WinForms.
Firebase profile has a parent and that parent has 2 children. The parent is named values and the two children named gyroData and sarjData. One equal to "10" and the other equal to "5". I have had a suspicion that the string values could be the issue but it was not. I've tried changing them to integer values which resulted in another error.
Here is my C# program:
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;
using Newtonsoft.Json;
using FireSharp;
using FireSharp.Config;
using FireSharp.Response;
using FireSharp.Interfaces;
namespace databaseGUI
{
public partial class Form1 : Form
{
IFirebaseConfig config = new FirebaseConfig
{
// double checked these in my program, all proper.
BasePath = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
AuthSecret = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
};
IFirebaseClient client;
public Form1()
{
InitializeComponent();
}
private void Form1_Load(object sender, EventArgs e)
{
client = new FireSharp.FirebaseClient(config);
if (client != null)
{
MessageBox.Show("Connected");
}
}
private void RTBButton_Click(object sender, EventArgs e)
{
FirebaseResponse response = client.Get(#"values");
Dictionary<string, valuesClass> data = JsonConvert.DeserializeObject<Dictionary<string, valuesClass>>(response.Body.ToString());
PopulateRTB(data);
}
private void dataView_Click(object sender, EventArgs e)
{
FirebaseResponse response = client.Get(#"values");
Dictionary<string, valuesClass> data = JsonConvert.DeserializeObject<Dictionary<string, valuesClass>>(response.Body.ToString());
PopulateDataView(data);
}
private void PopulateRTB(Dictionary<string, valuesClass> record)
{
richTextBox1.Clear();
foreach (var item in record)
{
richTextBox1.Text += item.Key + "\n";
richTextBox1.Text += item.Value.gyroData + "\n";
richTextBox1.Text += item.Value.sarjData + "\n";
}
}
private void PopulateDataView(Dictionary<string, valuesClass> record)
{
dataGridView1.Rows.Clear();
dataGridView1.Columns.Clear();
dataGridView1.Columns.Add("gyrodata", "Gyro");
dataGridView1.Columns.Add("sarjdata", "Sarj");
foreach (var item in record)
{
dataGridView1.Rows.Add(item.Key, item.Value.gyroData, item.Value.sarjData);
}
}
}
}
My data class:
namespace databaseGUI
{
class degerlerClass
{
public string gyroData { get; set; }
public string sarjData { get; set; }
}
}
And my python program:
import smbus
import time
from pyrebase import pyrebase
import sys
config = {
"apiKey": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"authDomain": "xxxxxxxxxxxxxxxxxxxxxxx.com",
"databaseURL": "https://xxxxxxxxxxxxxxx.firebaseio.com",
"storageBucket":""
}
firebase = pyrebase.initialize_app(config)
db = firebase.database()
data = {"sarj": str(10), "gyro": str(5) }
db.child("values").set(data)
print(data)
When the main app is running but not active the service notification click brings it to front, that's Ok.
Another case when I close the app by swiping it out from the recent apps list, the service is restarted and when I click the notification just black screen is shown and that's all.
The one interesting thing I noticed, after some time (an hour or two) it gets working - the main app is started on service notification click!
How can I achieve that once the service restarted?
The app is created with Python+Kivy and built with Buildozer and I run it on Android 10.
I tried setFlags(Intent.FLAG_ACTIVITY_NEW_TASK) for the Intent but it didn't help.
Here is some code.
PythonService.java:
package org.kivy.android;
import android.os.Build;
import java.lang.reflect.Method;
import java.lang.reflect.InvocationTargetException;
import android.app.Service;
import android.os.IBinder;
import android.os.Bundle;
import android.content.Intent;
import android.content.Context;
import android.util.Log;
import android.app.Notification;
import android.app.PendingIntent;
import android.os.Process;
import java.io.File;
//imports for channel definition
import android.app.NotificationManager;
import android.app.NotificationChannel;
import android.graphics.Color;
public class PythonService extends Service implements Runnable {
// Thread for Python code
private Thread pythonThread = null;
// Python environment variables
private String androidPrivate;
private String androidArgument;
private String pythonName;
private String pythonHome;
private String pythonPath;
private String serviceEntrypoint;
// Argument to pass to Python code,
private String pythonServiceArgument;
public static PythonService mService = null;
private Intent startIntent = null;
private boolean autoRestartService = false;
public void setAutoRestartService(boolean restart) {
autoRestartService = restart;
}
public int startType() {
return START_NOT_STICKY;
}
#Override
public IBinder onBind(Intent arg0) {
return null;
}
#Override
public void onCreate() {
super.onCreate();
}
#Override
public int onStartCommand(Intent intent, int flags, int startId) {
if (pythonThread != null) {
Log.v("python service", "service exists, do not start again");
return START_NOT_STICKY;
}
startIntent = intent;
Bundle extras = intent.getExtras();
androidPrivate = extras.getString("androidPrivate");
androidArgument = extras.getString("androidArgument");
serviceEntrypoint = extras.getString("serviceEntrypoint");
pythonName = extras.getString("pythonName");
pythonHome = extras.getString("pythonHome");
pythonPath = extras.getString("pythonPath");
boolean serviceStartAsForeground = (
extras.getString("serviceStartAsForeground").equals("true")
);
pythonServiceArgument = extras.getString("pythonServiceArgument");
pythonThread = new Thread(this);
pythonThread.start();
if (serviceStartAsForeground) {
doStartForeground(extras);
}
return startType();
}
protected int getServiceId() {
return 1;
}
protected void doStartForeground(Bundle extras) {
String serviceTitle = extras.getString("serviceTitle");
String serviceDescription = extras.getString("serviceDescription");
Notification notification;
Context context = getApplicationContext();
Intent contextIntent = new Intent(context, PythonActivity.class);
contextIntent.setFlags(Intent.FLAG_ACTIVITY_NEW_TASK); // my attempt but it didn't helped
PendingIntent pIntent = PendingIntent.getActivity(context, 0, contextIntent, PendingIntent.FLAG_UPDATE_CURRENT);
if (Build.VERSION.SDK_INT < Build.VERSION_CODES.O) {
notification = new Notification(
context.getApplicationInfo().icon, serviceTitle, System.currentTimeMillis());
try {
// prevent using NotificationCompat, this saves 100kb on apk
Method func = notification.getClass().getMethod(
"setLatestEventInfo", Context.class, CharSequence.class,
CharSequence.class, PendingIntent.class);
func.invoke(notification, context, serviceTitle, serviceDescription, pIntent);
} catch (NoSuchMethodException | IllegalAccessException |
IllegalArgumentException | InvocationTargetException e) {
}
} else {
// for android 8+ we need to create our own channel
// https://stackoverflow.com/questions/47531742/startforeground-fail-after-upgrade-to-android-8-1
String NOTIFICATION_CHANNEL_ID = "org.kivy.p4a"; //TODO: make this configurable
String channelName = "Background Service"; //TODO: make this configurable
NotificationChannel chan = new NotificationChannel(NOTIFICATION_CHANNEL_ID, channelName,
NotificationManager.IMPORTANCE_NONE);
chan.setLightColor(Color.BLUE);
chan.setLockscreenVisibility(Notification.VISIBILITY_PRIVATE);
NotificationManager manager = (NotificationManager) getSystemService(Context.NOTIFICATION_SERVICE);
manager.createNotificationChannel(chan);
Notification.Builder builder = new Notification.Builder(context, NOTIFICATION_CHANNEL_ID);
builder.setContentTitle(serviceTitle);
builder.setContentText(serviceDescription);
builder.setContentIntent(pIntent);
builder.setSmallIcon(context.getApplicationInfo().icon);
notification = builder.build();
}
startForeground(getServiceId(), notification);
}
#Override
public void onDestroy() {
super.onDestroy();
pythonThread = null;
if (autoRestartService && startIntent != null) {
Log.v("python service", "service restart requested");
startService(startIntent);
}
Process.killProcess(Process.myPid());
}
/**
* Stops the task gracefully when killed.
* Calling stopSelf() will trigger a onDestroy() call from the system.
*/
#Override
public void onTaskRemoved(Intent rootIntent) {
super.onTaskRemoved(rootIntent);
stopSelf();
}
#Override
public void run(){
String app_root = getFilesDir().getAbsolutePath() + "/app";
File app_root_file = new File(app_root);
PythonUtil.loadLibraries(app_root_file,
new File(getApplicationInfo().nativeLibraryDir));
this.mService = this;
nativeStart(
androidPrivate, androidArgument,
serviceEntrypoint, pythonName,
pythonHome, pythonPath,
pythonServiceArgument);
stopSelf();
}
// Native part
public static native void nativeStart(
String androidPrivate, String androidArgument,
String serviceEntrypoint, String pythonName,
String pythonHome, String pythonPath,
String pythonServiceArgument);
}
and the class extending PythonService:
package org.test.schedules;
import android.content.Intent;
import android.content.Context;
import org.kivy.android.PythonService;
public class ServiceService extends PythonService {
#Override
protected int getServiceId() {
return 1;
}
static public void start(Context ctx, String pythonServiceArgument) {
Intent intent = new Intent(ctx, ServiceService.class);
String argument = ctx.getFilesDir().getAbsolutePath() + "/app";
intent.putExtra("androidPrivate", ctx.getFilesDir().getAbsolutePath());
intent.putExtra("androidArgument", argument);
intent.putExtra("serviceTitle", "Schedules1106fgs_sdk28");
intent.putExtra("serviceDescription", "Service");
intent.putExtra("serviceEntrypoint", "service/main.py");
intent.putExtra("pythonName", "service");
intent.putExtra("serviceStartAsForeground", "true");
intent.putExtra("pythonHome", argument);
intent.putExtra("pythonPath", argument + ":" + argument + "/lib");
intent.putExtra("pythonServiceArgument", pythonServiceArgument);
ctx.startService(intent);
}
static public void stop(Context ctx) {
Intent intent = new Intent(ctx, ServiceService.class);
ctx.stopService(intent);
}
}
Also here is a piece of AndroidManifest.xml
<activity android:name="org.kivy.android.PythonActivity"
android:label="#string/app_name"
android:configChanges="mcc|mnc|locale|touchscreen|keyboard|keyboardHidden|navigation|orientation|screenLayout|fontScale|uiMode|uiMode|screenSize|smallestScreenSize|layoutDirection"
android:screenOrientation="portrait"
android:launchMode="singleTask"
>
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
</activity>
<service android:name="org.kivy.android.PythonService"
android:process=":pythonservice" />
<service android:name="org.test.schedules.ServiceService"
android:process=":service_service" />
Will be much appreciated for your help
Update
I haven't found the solution yet but noticed but seems the issue is related to background cached process. So if I go to Developer options -> Running services -> Show cached processes and stop the process there manually the app restarts fine.
I guess the solution could be somehow to prevent creating the cached process in the background or to kill that process at some specific moment, on app exit or service restart, I'm not an expert in Android, can anyone help with this?
I was trying to write a simple program to setup a connection between python and java using py4j. I wrote the following two lines hoping that everything would run since I'm not making any changes
from py4j.java_gateway import JavaGateway, GatewayParameters
gateway = JavaGateway(gateway_parameters=GatewayParameters(port=25335))
random = gateway.jvm.java.util.Random()
which resulted in the following error
py4j.protocol.Py4JNetworkError: An error occurred while trying to connect to the Java server (127.0.0.1:25335)
I looked around for a while and read that this might happen if java is listening on to a different port. So I wrote a for block to see what happens
for i in range(65535+1):
try:
gateway = JavaGateway(gateway_parameters=GatewayParameters(port=i))
random = gateway.jvm.java.util.Random()
print("passed port num", str(i))
except:
pass
The above block yielded nothing. None of the ports could connect. I can't figure out where I am going wrong.
How do I find the port number which the java side is using?
I am using py4j version 0.10.7 and python 3.6.0
EDIT
I have used the same java code as used in the py4j tutorial
I have a file called java_stack.java and py4j_gs.java. Both are in the same directory. I'm calling `java py4j_gs.java from the terminal. These are the contents of the two files
java_stack.java
package py4j.examples;
import py4j.GatewayServer;
public class StackEntryPoint {
private Stack stack;
public StackEntryPoint() {
stack = new Stack();
stack.push("Initial Item");
}
public Stack getStack() {
return stack;
}
public static void main(String[] args) {
GatewayServer gatewayServer = new GatewayServer(new StackEntryPoint());
gatewayServer.start();
System.out.println("Gateway Server Started");
}
}
py4j_gs.java
package py4j.examples;
import py4j.GatewayServer;
public class StackEntryPoint {
private Stack stack;
public StackEntryPoint() {
stack = new Stack();
stack.push("Initial Item");
}
public Stack getStack() {
return stack;
}
public static void main(String[] args) {
GatewayServer gatewayServer = new GatewayServer(new StackEntryPoint());
gatewayServer.start();
System.out.println("Gateway Server Started");
}
}
I also faced something same like
py4j.java_gateway:An error occurred while trying to connect to the Java server
while working on pyspark shell.
To solve this first we need to start the java geteway server.
After executing following java code my problem got resolve.
import py4j.GatewayServer;
public class Test {
public static void main(String[] args) {
Test app = new Test ();
// app is now the gateway.entry_point
GatewayServer server = new GatewayServer(app);
server.start();
}
}
I am trying to read data from Kafka internal topic __consumer_offsets using kafka-python client. I create consumer and successfully fetched data but the problem is data is serialised and look like it's in wire format , I want to deserialize this data into some readable format, i figured out that there are key_deserialize and value_deserialize options available in kafka consumer api but the problem is i couldn't figure out what value should to give to these fields , Could anyone please help me in this ?
my consumer code looks like
consumer = KafkaConsumer(bootstrap_servers=Settings.instance().kafka_server,
consumer_timeout_ms=2000,
enable_auto_commit="False",
exclude_internal_topics="False",
value_deserializer = bytes.decode, # not working•
group_id=self._group_id
)
and the consumed message looks like ::
ConsumerRecord(topic='__consumer_offsets', partition=26, offset=12983, timestamp=1520765864606, timestamp_type=0, key=b'
\x00\x01\x00\x16console-consumer-56707\x00\x06events\x00\x00\x00\x00', value=b'\x00\x01\x00\x00\x00\x00\x00\x00\x00\xb6\x00\x00\x00\x00\x01b\x14\xb5\x8a\x9d\x00\x00\x01b\x19\xdb\
xe6\x9d', checksum=-1872169212, serialized_key_size=38, serialized_value_size=28)
Well, you need to implement custom Serializer(at producer end) and Deserializer (at consumer end). Make sure to put same custom value class on class path at consumer end as that of producer.
public class CustomDeserializer implements Deserializer<ValueClass> {
public CustomDeserializer() {
}
public void configure(Map<String, ?> map, boolean b) {
}
public ValueClass deserialize(String s, byte[] MessageBytes) {
ValueClass eEventMessage = null;
ObjectMapper objectMapper = new ObjectMapper();
try {
eEventMessage = (ValueClass)objectMapper.readValue(MessageBytes, ValueClass.class);
} catch (IOException ex) {
// your stuffs
}
return eEventMessage;
}
public void close() {
}
}
Set this custom class at consumer end properties.
I have the following code
public static class Post {
public String author;
public String title;
public Post(String author, String title) {
}
}
// Get a reference to our posts
final FirebaseDatabase database = FirebaseDatabase.getInstance();
DatabaseReference ref = database.getReference("server/saving-data/fireblog/posts");
// Attach a listener to read the data at our posts reference
ref.addValueEventListener(new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot dataSnapshot) {
Post post = dataSnapshot.getValue(Post.class);
System.out.println(post);
}
#Override
public void onCancelled(DatabaseError databaseError) {
System.out.println("The read failed: " + databaseError.getCode());
}
});
I would like to write it in python. But it seems that Firebase library for python doesn't support it? Any ideas?