We are engaged in microservices that exchange JSON messages with each other. Somehow they themselves came to use JSONSchema for generation. The JAVA was used for generation and everything was good if the developer was nearby. But what about the analytics that coordinate the fields with the customer?
He needs an operational tool to generate a message and show it to the customer. Well...the first thing that comes to mind is the good old AltovaXMLSpy. A great remedy, but unfortunately paid. After working for a month, it stops working. The second remedy is conditionally free LiquidStudio2020. Not Altova, of course, but still - generates and visualizes. But.. after 15 days, basic functions, without paying for a license, become unavailable.
And so the idea arose to write a script in Python. Found the JSONSchema library, even...but nowhere is it described how it has a scheme to generate a message. It feels like it's impossible or we're using the wrong library. I'll tell you right away, online generators should not be offered. We have a complex structure of objects in JSONSchema linked through "$ ref." Does anyone have a similar experience in generating JSON messages?
Related
Firstly, apologies for the very basic question. I have looked into other answers but they haven't quite answered what I'm after. I'm confident designing a site in HTML/CSS and have very very basic knowledge of Python.
I want to run a very basic Python script on my website. It analyses tweets about a specific topic, and then posts a sentiment analysis score. I want it to run this sentiment analysis every hour and cache the score.
I have a working Python script which does this in Jupyter Notebook. Could you give me an overview of how I would make this script function online and cache the results? I've read into using Python web frameworks, but from my limited understanding, they seem like overkill?
Thank you for your help!
Could you give me an overview of how I would make this script function online
The key thing would be to uncouple the two parts of your system:
Producing the data
Showing it in a website.
So the first thing to do is have your sentiment-analysis script push its value to a database. The database could be something as simple as a csv file, or it could be a key/value store, or something like MySQL or CouchDB (or hundreds of other choices).
Over on the website you have to make a decision between:
Server-side
Client-side
If the former, you could program in Python if that is what you are most familiar with. Whatever language/framework combination you go for, there will an example tutorial of how to read a value from a database and display it: it is just about the most fundamental thing.
If client-side you will usually be programming in JavaScript. Again you need to choose a framework, but again you should easily be able to find a tutorial to follow.
(Unless you have a good reason to prefer server-side, such as familiarity with an existing framework, or security issues with accessing your database, I'd go with a client-side approach.)
I've read into using Python web frameworks... overkill?
Yes and no. You are going to need some kind of database, and some kind of framework. It would be good to understand the basics of web security, too. If the sentiment analysis is your major goal, all that is going to be a distraction, and it might be better to find a friend who already knows web programming to work with. Or just find a tutorial that is very close to what you want to do, and adapt that.
(P.S. I was going to flag your question as "too broad", but you did ask for an overview, so I hope this helps.)
I am looking for advice on how to automate submitting data directly to a Siebel application at work using Python. Currently I enter the data into an Autohotkey GUI and when a button is selected it enters the data into Siebel for me using mouse moves and mouse clicks to select the right entries for each piece of data. Obviously this is prone to errors and I would like to make the application better if possible. Using an object oriented programming language would improve this greatly. Just to clarify, this is NOT for automation testing. The data and account/page that I am submitting too changes quite often. So, modules like Selenium, Mechanize, and BeautifulSoup won't work for this as far as I can tell. Since not everything has a form or a friendly label that I can submit data to. If anyone has experience with Siebel and knows a way to copy data from and submit data directly to different entries that would be great.
Right now my best option is to use modules like Pyautogui and Pywinauto to perform mouse moves and clicks to copy what my Autohotkey script does. But this seems inefficient and potentially prone to errors. There has to be a better way to accomplish the same thing using Python. I am just not certain how and I would appreciate any advice you guys may have. Even if that is "no there is no other way" it would help me figure out what to do next. Thanks in advance!
Interacting with the Siebel CRM Application can be done in a large number of ways (SOAP, REST, COM, Java, UI, to name a few supported) and the use-case and environment typically define the preferred approach. A decent Siebel developer/consultant will be able to help you make the right choice.
The ease and available tooling for automating the UI is largely dependent on the version of Siebel you run. Prior to OpenUI this was mainly the domain of large Test Automation vendors (HP, Mercury, Oracle) and required a separate license module to be purchased.
Post Open UI the web UI itself became a single DOM object and much more suitable to automation using open source test tooling like Selenium. With the Test Automation license module activate it will also introduce additional HTML attributes that help to create stable locators.
If interacting with the the UI is just a means to change data, then I would advise an alternate approach: directly interact with the business layer. The added advantage is that there is much more information in the data objects than is typically available in a single UI screen and is more structure.
The easiest approach is probably using the web services. The older versions support mainly SOAP but the latest version also support REST. Most programming languages have support for these approaches and will allow you to import their WSDL files. Keep in mind that you're dependent on the DEV team to extend these interfaces too when they add fields to the UI.
Another approach that gives the most flexibility is to directly interact with the business layer using Java (Bean) or COM. The java approach only requires two JAR files and Google has enough examples on this approach to explain how to use it. When Python is your preferred approach, then the COM interface is an interesting approach. This GitHub project has some good examples to get you on your way.
I'm working on a project to try and create a more streamlined process to enter data into our database.
Currently, we're just using raw_input("Question: "), but this seems very outdated and is prone to mistakes. Given that there are over 100 questions we need to answer, this can take quite some time, loading only one question at a time.
I was interested in creating a web-based form to do this instead, as individuals wouldn't need to install python and various libraries on their own computers, but connect to an IP address on our network instead.
However, they're going to be using this to input medical data into our database, so I need some sort of secure-login feature, and only after being "validated", is a technician redirected to our medical form. I saw that using FlaskWTF might be able to solve this issue, but their documentation is a little confusing to me.
I'm wondering exactly how to do this. I've been looking at FlaskWTF (recommended on another post), but I don't need the ability to upload documents, only get data from a text box, or to see if someone has selected multiple boxes (i.e., if the person indicates they have both cancer AND diabetes).
Likewise, I'm wondering if I can create a google form, download it, and host it on an internal server. However, I'm curious on how I can retrieve the data with Python. I saw this post, but its only for a one question form, and I have at least 100 questions on this form.
If you think creating a webform is too difficult for someone who has no major python web experience, nor experience with HTML/PHP (I've mostly worked with databases using elasticsearch and some, albeit very basic, python-based AI/Chatterbots), would you recommend creating a form using TKinker instead? If so, how to I save form inputs as variables, and make it look a little prettier?
My apologies that I don't have code here, but rather a series of questions. As I continue to work on this project, I will probably post snippits of code to this site.
Best!
I'm looking for a tool/library written in python similar with logstash (ruby + java).
My goals are:
parse all system logs from syslog
parse application specific logs (apache, django, mysql etc.)
store results in something like elasticsearch
graph results based on different criteria
thanks!
ps: regexes are a way to go but I feel will be quite of work to start from scratch
Shameless plug (I am the author of the library) -
logtools does everything you mentioned and much, much more. I try to keep the documentation up to date and show alot of examples, similar to use cases you describe, in the README file. hopefully it would fit what you have in mind, give it a try, and any feedback is welcome - I try to add/fix any issues brought up by users. Check it out at http://github.com/adamhadani/logtools or download latest stable release at https://pypi.python.org/pypi/logtools
I'm exploring many technologies, but I would like your input on which web framework would make this the easiest/ most possible. I'm currently looking to JSP/JSF/Primefaces, but I'm not sure if that is capable of this app.
Here's a basic description of the app:
Users log in with their username and password (maybe I can somehow incorporate OPENID)?
With a really nice UI, they will be presented a large list of questions specific to a certain category, for example, "Cooking". (I will manually compile this list and make it available.)
When they click on any of these questions, a little input box opens up below it to allow the user to put in a link/URL.
If the link they enter has the same question on that webpage the URL points to, they will be awarded one point. This question then disappears and gets added to a different page that has a list of all correctly linked questions.
On the right side of the screen, there will be a leaderboard with the usernames of the people with the top ten points.
The idea is relatively simple - to be able to compile links to external websites for specific questions by allowing many people to contribute.
I know I can build the UI easily with Primefaces. [B]What I'm not sure is if JSP/JSF gives the ability to parse HTML at a certain URL to see if it contains words.[/B] I can do this with python easily by using urllib, but I can't use python for web GUI building (it is very difficult). What is the best approach?
Any help would be appreciated!!! Thanks!
The best approach is whatever is best for you. If Python isn't your strength but Java is, then use Java. If you're a Python expert and know little Java, use Python.
There are so many resources on the Internet supporting so many platforms that the decision really comes down to what works best for you.
For starters, forget about JSP/JSF. This is an old combination that had many problems. Please consider Facelets/JSF. Facelets is the default templating language in the current version of JSF, while JSP is there only for backwards compatibility.
What I'm not sure is if JSP/JSF gives the ability to parse HTML at a certain URL to see if it contains words.
Yes it does, although the actual fetching of data and parsing of its content will be done by plain Java code. This itself has nothing to do with the JSF APIs.
With JSF you create a Facelet containing your UI (input fields, buttons, etc). Then still using JSF you bind this to a so-called backing bean, which is primarily a normal Java class with only one or two JSF specific annotations applied to it (e.g. #ManagedBean).
When the user enters the URL and presses some button, JSF takes care of calling some action method in your Java class (backing bean). In this action method you now have access to the URL the user entered, and from here on plain Java coding starts and JSF specifics end. You can put the code that fetches the URL and does the parsing you require in a separate helper class (separation of concerns), or at your discretion directly in the backing bean. The choice is yours.
Incidentally we had a very junior programmer at our office use JSF for something not unlike what you are requesting here and he succeeded in doing it in a short time. It thus really isn't that hard ;)
No web technology does what you want. Parsing documents found at certain urls is out of the scope of building web interfaces.
However, each of Java's web technologies will give you, without limits, access to a rich and varied (if not too rich and much too varied) set of libraries and frameworks running on JVM. You could safely say that if there is a library for doing something, there will be a Java version available. Downloading and parsing a document will not require more than what is available in the standard library (unless you insist on injecting your dependencies or crosscutting your concerns), so no problems with doing your project with JSP, or JSF/Primefaces, or whatever.
Since you claim to already know Python, and since you will have to add some HTML/CSS anyway, I suggest you try Django. It's dead simple, has a set of OpenID plugins to choose from, will give you admin interface for free (so you can prime the pump with the first set of links).