Search result shows hidden content parts

[mariofischer]

We updated flowpack/elasticsearch-contentrepositoryadaptor to 7.0.6 (from 6.0.6). In the search result there are now content-parts (elements from the content tree) shown which are hidden in Neos (Before that the result did not show the hidden parts).
These elements are part of the indexed fulltext.

[mariofischer]

Additional clarification: The problem seems to be that the content of a node is being indexed into the page fulltext even if the parent node is hidden.

[dfeyer]

This issue duplicate this one:
#214 (comment)

[kdambekalns]

Am no longer sure this is a duplicate. Even though #214 is quite old, I am seeing this issue now after an upgrade of ESCR from 6 to 7 – now hidden nodes are returned that were not found before.

Fact: Both indexes (old and new) do contain data about the nodes, and the data itself does not differ (both are not marked as hidden in the index.)

The question thus is: Did the code fetching the nodes from the ES result change in that regard? It does not look like it for me: 6.0.6...7.0.9#diff-5310f1e603b84f1f7463df8670be1896e0cf5f7c3a32b4aa0a8f2a374c5eb6c4L833-L872

Does that mean the CR changed it's behaviour? But I did not update that… 🤔

[daniellienert]

Hey @kdambekalns, thats quite interesting. Obviously we have several related problems here:

• Hidden nodes, found by ES are resolved by the CR (The hack, why it kind of worked before)
• Nodes, hidden by there parent are found in general
• Fulltext extracts of nodes, hidden by there parent is indexed and even worse, displayed in result snippets

I am also pretty sure, that this can be solved with current CR as somehow flowquery is also capable to handle it correctly.

Wat do you think about a brainstorming session to discuss some approaches?

[kitsunet]

I would also be interested then :)

[kdambekalns]

Wat do you think about a brainstorming session to discuss some approaches?

It would be a pleasure. Suggest a date/time starting this Friday that suits you. :)

[TheLalaMan]

Any news on this? Our editors also noticed this recently...

[mficzel]

I wonder wether the issue is in the elasticsearch or in the convertHitsToNodes method.

In older versions we often had the issue that we wanted the first 10 results but only got 8 because during mapping of the elastic result to a list of nodes the nodes with hidden parents were thrown out. This was a problem aswell but way harder to spot.

Checked further ... the method used internally is the $contextNode->getNode($nodePathFromElasticSearch) ... imho getNode should return null for nodes that are not accessible and am pretty sure it did in the past.

[mficzel]

As a hotfix the aspect here https://gist.github.com/mficzel/292d6dab09ea0ef9017b804ad49f97a2 removes the items with hidden parents from the results. Totally not nice but it kinda works ...

[dlubitz]

Is there already some solution discussed? Our customers also complain about "hidden (by parent) content nodes" which appear in search results.

I had another look into the diff 6.0.6...7.0.9#diff-5310f1e603b84f1f7463df8670be1896e0cf5f7c3a32b4aa0a8f2a374c5eb6c4L833-L872 as @kdambekalns already had. It might be the case that before these changes the hidden nodes never were indexed, because the context was using the default of invisibleContentShown instead of (now) setting it to invisibleContentShown => true.

See:

• 6.0.6...7.0.9#diff-c274e5453204ef30dfca697d508fa7b4a8491ef8f8ad81faf371e6fcee060c7aR83
• 6.0.6...7.0.9#diff-f6816b294141205c0501d37b20f662e24ab8ab3656971828c657933ce1198d52R87

I guess this was necessary to handle the time controlled visibility of nodes. But then IMHO it misses some check if some node in the parent path is hidden.