I'm researching a good way to get my SPSS model logic into my website in real time. Currently we built a shody python script that mimics what the SPSS model does with the data. The problem is whenever we make an update or optimization to the SPSS model we have to go in and adjust the python code. How do people solve for this usually?
I've gotten a suggestion to create a config file for all the frequently updated functions in SPSS to translate over to the current python script. We're open to completely generating the python script from the SPSS model though, if there's a way to do that.
I've looked into cursor method but these seem have their main value in automating SPSS with python, which isn't really what we need.
You may want to look into the use of the Watson Machine Learning service on IBM Bluemix.
https://console.bluemix.net/catalog/services/machine-learning?taxonomyNavigation=data
The SPSS Streams Service that is part of this allows you to take your stream, deploy it, and then call it via REST API.
Note the Streams Service does not support legacy R nodes, Extension nodes (R or Python code), or Predictive Extensions. However, it does support Text Analytics.
The service allows up to 5000 calls a month for free and then can be purchased on a per prediction basis.
Related
I am having a hard time to deploy an R model and expose it as a web service using azuremlsdk for R. The Python side of Azure machine learning appears to be more mature as Python was deemed more important by Microsoft. Anyway, I was wondering if one score an R model, persisted as .rds file, in Python. I understand R can talk to Python via reticulate. Any input from Python experts would be very much appreciated. Thanks.
We have purchased Sage 100 partner account. I have also set up Sage ERP 100 on windows server 2016. But, I am stuck at the following points.
Where to add business
How to set up web services and access REST APIs
How I will make server configuration
Any help in Sage 100 setup will be appreciated.
Typically you would work with a Sage partner or reseller to set up your Sage 100 environment. Depending on your location, there should be several available. You would typically check the Sage website to see the Sage partners in your area.
With that said, I used to do a lot of programming against Sage 100 and I can tell you that there is no REST or web services API. What you would typically do is deploy your own API that reads from Sage 100 as a database. There is an ODBC connection that is included by default with the product, called SOTAMAS90, that will allow you read-only access to all the Sage 100 tables. The 32-bit connector is installed automatically when you install the program. There is a 64 bit version as well, but that takes more work to set up. The 32 bit version is easiest, but it does require that your API code be running as a 32 bit service or program.
I would typically write C# programs that consume the SOTAMAS90 data and serve it via REST. ASP.NET Web API or Core are both good choices for doing this.
Since the SOTAMAS90 ODBC client is read-only, you will have to do something else if you need to write data back to Sage 100. The two interfaces that I'm familiar with are VI and BOI.
VI, or Visual Integrator is basically a utility for importing data from a source file (typically a CSV). It has some limitations, but it does work. You can launch it programmatically, which makes it usable on-demand. If doesn't throw error messages, however. If a row can't be written, it just skips it. You can view a report after the fact to see what wrote and what didn't.
BOI, or the Business Object Interface, is a COM component that you can code against. It provides more robust data validation, and does throw errors on a per-record (and sometimes a per-field) basis so you can respond to those in your code accordingly. Unfortunately, while most of the modules are exposed the BOI, not all of them are. Every year, Sage is porting more and more functionality to "the new framework" which also means it is available via BOI.
Finally, you can also set up a linked server in SQL Server to serve the ODBC data that way. Any way you hit that SOTAMAS90 DSN though, it's slow. Some developers like to copy all of the data to SQL Server and serve it from there. If you do that, be sure to add foreign keys and indexes. And run a nightly ETL to keep the data fresh. There are also solutions via User Defined Scripts that will allow you to respond to individual row CRUD events.
Hope that helps.
Aaron
I would like to do some machine learning tasks on data as it comes in through stream analytics from event hub. However, much of my data processing pipeline and prediction service is in python. Is there a way to send time chunked data into the python script for processing?
The Azure ML studio function does not suit my need because it appears to work on single rows of data, and the aggregation functions available in Stream Analytics don't seem to work for this data.
With the recently rolled out integration with Azure functions, you might be able to do that. Try that route.
This link describes creating an azure function. You will have to create a Http trigger and choose python as the language. There are also templates for several languages.
This question also has additional details about function.
Per my experience,you could put your data into Azure Storage.Then configure the Import Data component in Azure ML and connect Execute Python Script as input data.
Or you could use Azure Storage Python SDK to query data in your Execute Python Script directly.
However,the two methods mentioned above could only process part of the data at one time, so they should be used only at the experimental stage.
If you need to continue processing data, I suggest you use the web service component.
You could put logical code of querying data and processing result into web service.Please refer to this official tutorial to deploy your web service.
Hope it helps you.
I am working on collecting system metrics, running custom scripts to get the application related performance data and storing the data in time-series database(kairosdb with cassandra). I know there is collectd and telegraf framework for the same purpose. collectd seems to satisfy my requirements but not sure about the performance and also we need to run the custom scripts at different interval using exec plugin. I am not sure whether we can achieve this using collectd.
Also i came across telegraf. It is written in go language. It is tag based, so it is easy for me to store the data using kairosdb. But I am not sure how efficient it is and whether it will server my purpose.
Is there any other opensource collection framework available in perl or python to collect system metrics,run custom scripts and store the data in time-series database(kairosdb)?
Just use Kairosdb REST API and some HTTP client e.g. HTTP::Tiny.
Here an TFTP based example Kairosdb Stress Yaml
This year me and a friend have to make a project for the final year of university. The plan is to make a proxy/sever that allows to store ontologies and RDF's, by this way this data is "chained" to a web, so you can make a request for that web and the proxy will send you the homepage with metadata.
We have been thinking to use python and rdflib, and for the web we don't know which framework is the best. We thought of django, but we think that is very big for our purpose, and we decided that webpy or web2py is a better option.
We don't have any python coding experience, this will be our very first time. We have been always programming in c++ and java.
So taking into account everything we've mentioned our question is, which would be the best web framework for our project? And will rdflib suit fine with this framework?
Thanks :)
I have developed several Web applications with Python framworks consuming RDF data. The choice always depends on the performance needed and the amount of data you'll have to handle.
If the number of triples you'll handle is in the magnitude of few thousands then you can easily put together a framework with RDFlib + Django. I have used this choice with toy applications but as soon as you have to deal with lots of data you'll realise that it simply doesn't scale. Not because of Django, the main problem is RDFlib's implementation of a triple store - it is not great.
If you're familiar with C/C++ I recommend you to have a look at Redland libraries. They are written in C and you have bindings for Python so you can still develop your Web layer with Django and pull out RDF data with Python. We do this quite a lot and it normally works. This option will scale a bit more but won't be great either.
In case your data grows to millions of triples then I recommend you to go for a Scalable Triple store. You can access them through SPARQL and HTTP. My choice is always 4store. Here you have a Python client to issue queries and assert/remove data 4store Python Client