Troubleshoot indexing process

📘

Note

The information below is intended to be used by advanced users and developers

Before troubleshooting indexing process make sure that Magento cron is stopped and not running. Messages queue consumer hawksearch.indexing has to be terminated.

Full re-index

1. (Optional) Clean queue_message and magento_operation tables
   SQL

   DELETE FROM queue_message WHERE topic_name like "hawksearch.%";
   DELETE FROM magento_operation where topic_name like "hawksearch.%";
   

2. Get the list of indexes via API (Postman)
   Request:

   curl --location --request GET 'https://indexing-dev.hawksearch.net/api/v2/indexing/' \
   --header 'Content-Type: application/json' \
   --header 'Accept: application/json' \
   --header 'X-HawkSearch-ApiKey: <API_KEY>' \
   --data-raw ''
   

   Result:

   {
       "IndexNames": [
           "magentoelastic.20220427.111354",
           "magentoelastic.20220426.153846"
       ]
   }
   

3. Get current index name (Postman)
   Request:

   curl --location --request GET 'https://indexing-dev.hawksearch.net/api/v2/indexing/current' \
   --header 'Content-Type: application/json' \
   --header 'Accept: application/json' \
   --header 'X-HawkSearch-ApiKey: <API_KEY>' \
   --data-raw ''
   

   Result:

   {
       "IndexName": "magentoelastic.20220426.153846"
   }
   

4. Re-index via CLI command
   bin/magento indexer:reindex hawksearch_entities
5. Review changes in indexes list via API (Postman). Make that non-current index "magentoelastic.20220427.111354" has been removed and a new one "magentoelastic.20220504.143505" has been created
   Request:

   curl --location --request GET 'https://indexing-dev.hawksearch.net/api/v2/indexing/' \
   --header 'Content-Type: application/json' \
   --header 'Accept: application/json' \
   --header 'X-HawkSearch-ApiKey: <API_KEY>' \
   --data-raw ''
   

   Result:

   {
       "IndexNames": [
           "magentoelastic.20220504.143505",
           "magentoelastic.20220426.153846"
       ]
   }
   

6. Review changes in queue_message and magento_operation tables
   1. queue_message
      
   2. queue_message table
      
      Ensure that a bulk of new messages is created in the queue_message table and corresponding operations are created in the magento_operation table. The queue_message.body and magento_operation.serialized_data columns contain the newly created index name "magentoelastic.20220504.143505" in their text values.
7. Process messages one by one. The number of messages handled at the same time can be controlled via --max-messages parameter.
   bin/magento queue:consumers:start --max-messages=1 hawksearch.indexing
8. After the last message is processed check via API (Postman) that the current index is changed to magentoelastic.20220504.143505
   Request:

   curl --location --request GET 'https://indexing-dev.hawksearch.net/api/v2/indexing/current' \
   --header 'Content-Type: application/json' \
   --header 'Accept: application/json' \
   --header 'X-HawkSearch-ApiKey: <API_KEY>' \
   --data-raw ''
   

   Result:

   {
       "IndexName": "magentoelastic.20220504.143505"
   }
   

Partial re-index in “Update by Schedule” mode

1. (Optional) Clean queue_message and magento_operation tables
   SQL

   DELETE FROM queue_message WHERE topic_name like "hawksearch.%";
   DELETE FROM magento_operation where topic_name like "hawksearch.%";
   

2. Demonstrate the list of indexes via API (Postman)
3. Get current index name (Postman)
4. Update any product data
5. Demonstrate that no messages in queue_message table yet
6. Run cron to trigger indexer_update_all_views job
   bin/magento cron:run --group=index
7. Demonstrate the created messages in queue_message table
8. Process messages:
   bin/magento queue:consumers:start --max-messages=1 hawksearch.indexing
9. Demonstrate index is not changed
10. Demonstrate that index is triggered in Hawk API Logs
11. Demonstrate a new item created in the index (run API search)

Partial re-index in “Update on Save” mode

1. Clean queue_message table
2. Demonstrate the list of indexes via API (Postman)
3. Get current index name (Postman)
4. Update any product data
5. Demonstrate the created messages in queue_message table
6. Process messages:
   bin/magento queue:consumers:start --max-messages=1 hawksearch.indexing
7. Demonstrate index is not changed
8. Demonstrate that index is triggered in Hawk API Logs
9. Demonstrate a new item created in the index (run API search)