Implement indexer, support update by schedule mode

[unicoder88]

Hi!

We noticed number of issues concerning real-time sync:

• When we set Stock status to item as "out of stock" in admin panel, this item will not be shown at recommendation section, if index refresh is "By Save" . But if index refresh is "By Schedule", "out of stock" items still appears in recommendation section
• When all items was bought, product became as "out of stock". And if it was as recommended product, it's still there, until next synchronization from my.clerk.io

... and possibly more. This is because sync event works only when saving a product in admin panel.

I suggest making a step towards indexer and support update by schedule mode.

Please review :)

[unicoder88]

Well, our original difficulties were:

• when Magento indexes are in update by schedule mode, at the moment of product saving some changed data is outdated in indexes (inventory, prices) and send incorrectly to Clerk
• controller save event is too narrow, it doesn't account for API product save - we have product updates from external ERP
• stock changes without saving product are missed out as well - selling last product on frontend

Using update by schedule appeals to us as it has number of advantages:

• faster product saving in admin and REST API
• detects modifications in database - mysql triggers automatically schedule changes for reindex
• allows for some optimizations for bulk operations. For example when in need to save product attribute in nightly cron on big database (150-200k products), we benefit from calling $product->addAttributeUpdate() instead or spinning slow $product->save()

With that being said, catalog inventory index is also in update by schedule mode. This means that it is not updated at the moment when product is saved, and Clerk receives incorrect in-stock/out-of-stock values.

Lastly, I peeked for reference at other similar project Algolia https://github.com/algolia/algoliasearch-magento-2. It's not perfect but over the time has covered a lot of special cases.

What do you think, do we need more specific example?

[unicoder88]

It's becoming harder to make improvements in both this indexer and original observer :)
New feature - reindex parent products when child product changes - was added in this branch only.

[pikulsky]

Seeing how we're not using an indexer right now I'm not entirely sure what the benefits to this commit would be, could you help me understand? :)

@RBech ,
Do you still have concerns about supporting "Update by Schedule" indexing mode?

As a reference about benefits, I would suggest to check "Indexers" part of Magento's "Performance Best Practices" article:
https://devdocs.magento.com/guides/v2.3/performance-best-practices/configuration.html

Since now the module has Adapters, so it's more easy now to support "Update by Schedule" mode without duplicating code.
What do you think?