Send message to a kafka topic using java - python

After several weeks looking for some information here and google, I've decided to post it here to see if anyone with the same problem can raise me a hand.
I have a java application developed in Eclipse Ganymede using tomcat to connect with my local database. The problem is that I want to send a simple message ("Hello World") to a Kafka Topic published on a public server. I've imported the libraries and developed the Kafka function but something happens when I run in debug mode. I have no issues or visible errors when compiling, but when I run the application and push the button to raise this function it stops in KafkaProducer function because there is NoClassDefFoundError kafka.producer..... It seems like it is not finding the library properly, but I have seen that it is in the build path properly imported.
I am not sure if the problem is with Kafka and the compatibility with Eclipse or Java SDK (3.6), it could be?. Anyone knows the minimum required version of Java for Kafka?
Also, I have found that with Kafka is really used Scala but I want to know if I can use this Eclipse IDE version for not change this.
Another solution that I found is to use a Python script called from the Java application, but I have no way to call it from there since I follow several tutorials but then nothing works, but I have to continue on this because it seems an easier option. I have developed the .py script and works with the Kafka server, now I have to found the solution to exchange variables from Java and Python. If anyone knows any good tutorial for this, please, let me know.
After this resume of my days and after hitting my head with the walls, maybe someone has found this error previously and can help me to find the solution, I really appreciate it and sorry for the long history.

Please include the Kafka client library within the WAR file of the Java application which you are deploying to Tomcat

Please use org.apache.kafka.clients.producer.KafkaProducer rather than kafka.producer.Producer (which is the old client API) and make sure you have the Kafka client library on the classpath. The client library is entirely in Java. It's the old API that's written in scala, as is the server-side code. You don't need to import the server library in your code or add it to the classpath if you use the new client API.

At the end the problem was related with the library that was not well added. I had to add it in the build.xml file, importing here the library. Maybe this is useful for the people who use an old Eclipse version.
So now it finds the library but I have to update Java version, other matter. So it is solved

Related

FirebaseMessaging.getToken() using python

I'm building a kivy app and almost got firebase cloud messaging working using the pyfcm
package from pypi.
What i'm stuck on is how to get the "registration token" that the firebase SDK creates on app install so that i have a registration token to send push notifications to.
There seems to be lots of info scattered around the net in regards to doing it using some other language other than "python", but python is all i know so i'm trying to work it out for that language.Any one have tips on how i go about that please ?
I can't find a python package that has the FirebaseMessaging.getToken() in it which is what the firebase documentation keeps referring to, so i'm lost and my brain is going mushy now due to researching this for hours and hours so far :)
any help appreciated
To answer my own question.
For those wondering the same question as i had. I came across this package https://github.com/Fox520/pushyy
In that package the guy has incorporated the ability to get the device token and also update the token when it changes. It takes a little manipulating to get the package to work properly with the latest gradle version but it is do able.
There doesnt seem to be a direct "python" method as such to get the token so using this package makes use of the required java and python code to achieve the task. Using this and using pyfcm from pypi to send messages, i'm now able to send firebase push notifications from my kivy app :)

Dynamically updating web app to monitor progress of python script? Through Web.py perhaps?

I've got a python script that I want to build a locally-hosted web gui app for, so I can use the modern styling and tools available to web apps.
The scripts I'm running take a while to process, and I want to update the web app with visual updates, or at least something akin to what the console sees when using print() in python.
My initial hosting efforts have been based on this tutorial, and I tried out the methods in this answer to try and get data to update in a streamed fashion, but the pages only showed once the entire script was finished.
I'm wondering whether web.py could help me?
Any guidance, or even the right terms to google would be appreciated. Thanks.
--
Update: I've been reading up on node.js (something I've failed to do for years..) and, please correct me if I'm wrong, but it seems like it could be the answer. I'm even considering re-writing my original functions into node.js given the existence of this serial comms library

Access Hadoop on server from python

First of all, I need to say that I am a girl who knows very little about remote server. A lot of similar questions asked here is very difficult for me to understand. So I come to ask.
My task is to generate a script which helps me fetch some data from a server.
The data is stored in Hadoop. Usually I log in the server with a user name and a temporary password. I run 'hive' clauses on the server. After I get all the data on the server, I download it. Then on my computer, I manipulate the data on my own computer with Python.
Now I hope to do this with one Python script.
I find thrift package, but don't know how to begin to understand.
I wonder should I install hive on my computer, then use sys to run hive. Or should I log in the server and run hive on the server in my script?
In any case, can thrift help me log in the server?
Thanks very much!
Although Thrift can surely help you, it is recommended to use a more higher-level client. They are usually well-tested and will keep all (or most of) the low level stuff away from you. In particular, HBase looks promising in your case. I'd recommend to have a look at that one, and to compare it with the Hadoop Python Thrift Client described in this tutorial.

RESTful Web service or API for a Python program in WebFaction

I have developed a few python programs that I want to make available online.
I am new to web services, and I am not sure what I need to do in order to create a service where somebody makes a request to an URL (for example), and the URL triggers a Python program that displays something in the user's browser, or a set of inputs are given to the program via browser, and then python does whatver it is supposed to do.
I was playing with the google app engine, which runs fine with the tutorial, and was planning to use it becuase it looks easy, but the problem with GAE is that it does not work well (or does not work at all) with some libraries that I plan to use.
I guess what I am trying to do is some sort of API using my WebFaction account.
Can anybody point me in the right directions? What choices do I have in WebFaction? What are the easiest tools available?
Thank you very much for your help in advance.
Cheers
Well, your question is a little bit generic, but here are a few pointers/tips:
Webfaction allows you to install pretty much anything you want (you need to compile it / or ask the admins to install some CentOS package for you).
They provide some default Apache server with mod_wsgi, so you can run web2py, Django or any other wsgi frameworks.
Most popular Python web frameworks have available installers in Webfaction (web2py, django...), so I would recommend you to go with one of them.
I would also install supervisord to keep your service running after some reboot/crash/problem.
I would be glad to help you if you have any specific question...

libspotify Python package for calling Spotify api?

I'm building an app that needs to get artwork information out of the libspotify Spotify API.
I'm building the app in python on google appengine. Does anyone know of a package that will enable me to access the libspotify API? The official page is C and I've googled around to try and find a suitable wrapper but can't seem to find one.
Thanks
Tom
There is also https://github.com/mopidy/pyspotify which is actively used in some applications and up to date.
I don't think you will be able to call libspotify at all. From the docs:
"The Python interpreter runs in a secured "sandbox" environment to isolate your application for service and security. The interpreter can run any Python code, including Python modules you include with your application, as well as the Python standard library. The interpreter cannot load Python modules with C code; it is a "pure" Python environment."
Check Spotimeta
http://pypi.python.org/pypi/spotimeta/

Categories

Resources