Maintenance - ElasticSearch vs OpenSearch
We have reviewed OpenSearch to see whether we can support it as Data Store to Germain (and in addition to ElasticSearch, Postgres, MySQL, and Oracle). Here is a comparison and the main take-aways of OpenSearch (OS) vs. ElasticSearch (ES).
Comparison and the main take-aways:
OS is a fork of ES that claims to be compatible at the API level; this is good for Germain
OS on AWS automates backups and management of the underlying hardware, which could significantly reduce the maintenance overhead compared to running ES directly on EC2 instances
Unfortunately, the OS and ES code bases have already diverged to a point where we would need to support separate code paths for each of them (~1-2 weeks of engineering work for us at this point).
OS is based on ES 7.10.2. The latest ES 7 release as of this writing is 7.17, with version 8 now available as well
Despite the fairly recent fork, there are already a number of changes in the API between ES 7.17 and the latest OS release. Further, the latest ES client actively prevents us from connecting to OS, so we would have to include an older version of the ES client (7.10.2)
The pace of development of OS looks to be a lot slower than ES, judging by the number of new features added to each of them since the fork
Notes on compatibility changes:
Data stream support is limited in 7.10, specifically the templates need to be created manually because the high level client doesn't support them
Point-in-time queries are not supported, need to use scroll instead
Limited support for transform options
Options for lifecycle policies have changed slightly
Less maintenance needed for OS in some cases (e.g. if used on AWS without any other tech like Puppet) vs ES.