asked on February 20, 2024
Hi,
We are using FlexMonster to power a SaaS Pivot Table Analysis solution as part of our SaaS financial services software. While we have gotten it working at a pretty large data scale by using the .net server library and leveraging Parquet to load data quickly and in a low memory utilization way, the one thing we aren't able to solve is the fact that some of the data we are rolling up is duplicated data associated with entities (things like customer name, contract value, etc). It would be far more efficient to just include an ID to those entities and then post aggregation join to get the additional data. It would also shrink the raw data load down since we wouldn't need to duplicate that data across many records in the underlying data set loaded into Flexmonster.
Is there any way to marry up a mutable data set to the aggregated result before that result is brought back to the client without us rolling our own implementation?
⋅ February 20, 2024
Semi related to Bills request. Elasticsearch seemed to work well for us when we tested it in terms of performance at the scale we need. The only thing preventing us from moving to Elasticsearch is some of the missing features for it here. Would it be possible to add support for things like "flat table", "Selecting sublevels from a multilevel hierarchy" and "calculated values" to Elasticsearch?
⋅ February 21, 2024
Hi!
Thank you for reaching out to us.
We understand your request and agree that it would be useful to link several data instances as done in relational databases. Kindly note that there is no such functionality in Flexmonster Data Server out of the box.
Regarding the Elasticsearch, we are glad to hear the updates about your research with this data source. Please note that for now, we are not planning to add support for the mentioned features to the Elasticseacrch data source.
Our team is also wondering if you had some time to check MongoDB and our MongoDB Сonnector that we have suggested earlier. This approach has all the functionality you are asking for, and it is also possible to use built-in aggregations on the MongoDB side if needed.
Hope our answer has addressed your request well.
Looking forward to hearing from you soon.
Kind regards,
Solomiia
⋅ February 21, 2024
I did investigate/test mongoDB. At the scale of data we have, it did not seem to perform nearly as well as Elasticsearch did.
⋅ February 22, 2024
Hello, Brian!
Thank you for getting back to us.
We appreciate the provided details about your research on MongoDB and agree that Elasticsearch performs better when working with big data.
Feel free to reach out to us in case of any other questions.
Kind regards,
Solomiia
