How do I dynamically visualize large and complex data sets (stored in Elasticsearch) with dozens of sub-graphs and drill-downs? I would like some kind of dynamic control over this, but would prefer to not roll my own (in Python). Is it possible to use Kibana for this kind of thing? Or is there some better tool for the task?
Best would be if I could have rudimentary control over layout, then be able to show a number of bar charts if the user wants to see time series, but if she wants to slice it perpendicular I could show for instance pie charts. User should be able to interactively klick her way down, generating AND/OR lucene expressions, etc.
The more I rubber duck, the more I feel like I need to build this myself in Bokeh or something of the kind. If I need to create all such business logic manually, what would be my best HTML/graphing library? Or are there plugins for Kibana that does this perhaps? If I have to create things manually, it does not necessarily need to be in Python, but it needs to be back-end (for Elasticsearch security).
I wrote my own, took around 500 LoC and a week or two of devopsing.
Related
I'm working on a project to breakdown 3D models but I'm quite lost. I hope you can help me.
I'm getting a 3D model from Autodesk BIM and the format could be native or generic CAD formats (.stp, .igs, .x_t, .stl). Then, I need to "measure" somehow the maximum dimensions to model a raw material body, it will always have the shape of a huge panel. Once I get both bodies, I will get the difference to extract the solids I need to analyze; and, on each of these bodies, I need to extract the faces, and then the lines or curves of each face.
This sounds something really easy to do on a CAD software, but the idea is to automate this process. I was looking into openSCAD, but seems that works only to model geometry and it doesn't handle well imported solids. I'm leaving a picture with the idea of what I need to do in the link below.
So, Any idea how can I do this? which langue and library can help in this project?
I can see this automation possible with a few in between steps:
OpenSCAD can handle differences well, so your "Extract Bodies" seems plausible
1.5 Before going further, you'll have to explain how you "filtered out" the cylinder. Will you do this manually? If you don't, you will have it considered for analysis and have a lot of faces as a result.
I don't think openSCAD provides you a vertex array. However, it can save to .STL, which is kinda easy to parse with the programming language of your choice, you'll have to study .stl file structure a bit (this sounds much more frightening than it is - if you open an stl with an editor you will probably immediately realize what's happening).
Since you've parsed the file, you can now calculate lines with high school math.
This is not an easy, GUI way to do what you ask, but if you have a few skills you'll have your automation, and depending on the amount of your projects it may be worth it.
I have been working in this project, and foundt the library "trimesh" is better to solve this concern. Give it a shot, and save some time.
I began to fall in love with a Python Visualization library called Altair, and i use it with every small data science project that ive done.
Now, in terms of Industry use cases, Does it make sense to visualize Big Data or should we just take a random sample?
Short answer: no, if you're trying to visualize data with tens of thousands of rows or more, Altair is probably not the right tool. But there are efforts in progress to add support for larger datasets in the vega ecosystem; see https://github.com/vega/scalable-vega.
Situation
Our company generates waste from various locations in US. The waste is taken to different locations based on suppliers' treatment methods and facilities placed nationally.
Consider a waste stream A which is being generated from location X. Now the overall costs to take care of Stream A includes Transportation cost from our site as well treatment method. This data is tabulated.
What I want to achieve
I would like my python program to import excel table containing this data and plot the distance between our facility and treatment facility and also show in a hub-spoke type picture just like airlines do as well show data regarding treatment method as a color or something just like on google maps.
Can someone give me leads on where should I start or which python API or module that might best suite my scenario?
This is a rather broad question and perhaps not the best for SO.
Now to answer it: you can read excel's csv files with the csv module. Plotting is best done with matplotlib.pyplot.
After running a bunch of simulations I'm going to be outputting the results into a table created using SQLAlchemy. I plan to use this data to generate statistics - mean and variance being key. These, in turn, will be used to generate some graphs - histograms/line graphs, pie-charts and box-and-whisker plots specifically.
I'm aware of the Python graphing libraries like matplotlib. The thing is, I'm not sure how to have this integrate with the information contained within the database tables.
Any suggestions on how to make these two play with each other?
The main problem is that I'm not sure how to supply the information as "data sets" to the graphing library.
It looks like matplotlib takes simple python data types -- lists of numbers, etc, so you'll be need to write custom code to massage what you pull out of mysql/sqlalchemy for input into the graphing functions...
I searched around for ways to make rose diagrams (circular histograms) in Google Chart. The API has only radar diagrams, so it seems not technically possible (am I correct?). This wind rose example was the closest I came to a solution.
Because I needed them, I figured out a way to fake them quickly using the Radar plots, Python and the Google-Chartwrapper library. There's a (non-technical) write-up available and the code is on Github.
Before I take this further (i.e. clean code, abstract, waste more time, etc.), has anyone else seen examples of Rose diagrams in Google Chart that might be useful?
(By the way, I know about matplotlib, etc. I'm using Python 3.x out of necessity and, as yet, the graphing libraries haven't caught up enough to use as I need them. See also SO question 418835))
Well, it looks like no-one has any other examples. I DID decide to play around with this to see what else is possible and created a quick proof of concept for time-based rose diagrams. It's just a silly thing to show your relative Twitter posting amount by time of day, but shows how Google Chart can be used for rose diagrams. The problem here, and a problem in general that this has illustrated with Google Chart, is that you cannot have embedded data contained in the chart, with drill-down capabilities, popups, etc. (as you could with a Javascript/Flex/other solution).
It'd be nice if there were a Google Chart-style API that was javascript based to allow embedded data.