JSON error during Smart Indexing

Hi,We decided on your suggestion to further investigate application of Smart Searching for our website. Our web developer found a Smart Search plugin for Virtuemart which paved the way.
The first step was to index the Joomla website articles, what went smooth. The second step was to activate the Edocman Smart Search plugin and continue to index all our scanned .pdf documents. It looked as if this was OK and the first results were visible as search hits, until an error occurred with the message: The server returned the following message: An error occurred while retrieving the JSON data: HTTP status code 500
You can cancel the message and continue indexing, however the error message pops up frequently.
Any advice?         

Hi,
Thank you for sharing the details of the error you encountered while indexing PDF documents with the Edocman Smart Search plugin.
To help troubleshoot this issue, please try the following steps:
1. Check the Server Error Log:
Please access your server’s error log (usually located at /logs/error.log or via your hosting control panel) and look for any entries related to HTTP 500 errors or the Smart Search/Edocman plugins. These logs often provide more specific information about what caused the error.

2. Enable Joomla Debug Mode:
In your Joomla administrator panel, go to System > Global Configuration > System, and turn on Debug System. This will display more detailed error messages on your site and can help identify the root cause of the issue.

3. Sometimes, HTTP 500 errors occur due to PHP limitations. Please check your server’s PHP settings, especially the following parameters:
- memory_limit (increase if your PDF files are large)
- max_execution_time (raise if the indexing process takes longer)
- file_uploads and post_max_size (ensure these are sufficient for your files)

Thanks
Dam

Hi,
Thank you for your recommendations. They have lead to the following observations:

1. The indexing halts after a while, so no indexed results are added, despite the bar is showing that indexing is active but not moving and is currently at about 25%
2. There we a number of orphans, after removing, the indexing continued, again for a while
3. This morning indexing contnued surprisingly, again for a while. So new indexed results were added
4. Every time the PSON error is shown the Server error log indicates: IP = 84.80.44.251 End of script output before headers: index.php, referer: www.dedecanije.nl/administrator/index.ph...dexer&tmpl=component
5. Optimizing the indexed results leads repeatedly to the following error message: 1205 Lock wait timeout exceeded; try restarting transaction

Hopefully this brings a solution a step further.
Thank you.

Hi,
Please re-activate the super admin account on your site, i will check the problem for you.
Thanks
Dam

I just did.
Can you also check why the indexed documents are protected with user name and password?

Hi,
Currently, your documents are not being indexed correctly because the file plugins/edocman/indexer/binaries/linux/pdftotext does not have the proper permissions (777).

Please change the permission of this file to 777. After that, go to the document management section in Edocman and click on the buttons in the "Indexed content" column to index (read and store) the content of your PDF files.

Once this process is complete, you can run the indexing again in Joomla Smart Search.

At this moment, I am not yet certain about the cause of the error when indexing in Joomla Smart Search, as the error log does not clearly indicate where the issue is occurring.

Dam

Hi,
What makes you believe the documents are not indexed as you can see from the screenshot the setting is indeed 777. In de document management section all the documents have a green tickmark.
The 11 non-indexed documents about Migrants have been deleted in the meantime.

Hi,
We have now set the permission for the pdftotext file to 777.
To re-index your documents correctly, please click on each icon in the "Indexed content" column to index the documents again. Previously, the documents were not indexed properly because the pdftotext file did not have the correct 777 permission.
Regarding the green tick marks in the "Indexed content" column:
These appear because the indexed data for each document contains the message:
sh: 1: /var/www/vhosts/dedecanije.nl/httpdocs/plugins/edocman/indexer/binaries/linux/pdftotext: Permission denied
Since this message is not empty, Edocman marks the document as indexed.
As for the 11 Migrants documents that were not indexed and have been deleted, I am not aware of the details regarding their deletion.
Dam

I followed your conclusion, deleted all the documunts and started from scratch. I uploaded in batch mode about 300 documents in one go, in total over 2000 documents.
Initially it went error-free, but gradually the indicated PSON error message appeared, always related to the error log message: IP = 84.80.44.251 End of script output before headers: index.php, referer: www.dedecanije.nl/administrator/index.ph...dexer&tmpl=component
I left the last indexing unfinished, so you can observe the situation yourself.
Thank you for your kind attention.

Hi, We have reviewed the error logs related to your recent indexing activities in Joomla Smart Search. Based on the information provided, it appears that the issue may be related to insufficient PHP execution time or memory limits on your server.To help resolve this, we kindly ask you to adjust your PHP configuration as follows:

• Increase the value of
  Code:

  max_execution_time
  to the maximum allowed by your hosting environment.
• Increase the value of
  Code:

  memory_limit
  to the highest possible setting.

These changes will ensure that your scripts have sufficient resources to complete the indexing process without interruption.If you need assistance making these adjustments, please let us know your hosting environment (e.g., cPanel, Plesk, etc.), and we will be happy to provide step-by-step instructions.Thank you for your cooperation.Best regards,
Dam