Manual indexes do not persist during export/import

[john-bodley]

I've been using manual indexes via APOC procedures in order to perform full-text search.

Part of our pipeline includes using the efficient export-binary and import-binary commands to create backups and share the graph between various development environments, i.e., testing, staging, production.

The issue I ran into is if I export a database which includes a manual index, delete the graph and re-import it the manual index does not persist. Note that isn't true for legacy indexes which also leverage lucene (in our case this is used by the GraphAware UUID which are correctly imported, i.e.,

> cypher-shell 
Connected to Neo4j 3.2.1 at bolt://localhost:7687.
Type :help for a list of available commands or :exit to exit the shell.
Note that Cypher queries must end with a semicolon.
neo4j> CALL apoc.index.list();
+----------------------------------------------------------------------------------------------------------------------------------------------------+
| type   | name   | config                                                                                                                           |
+----------------------------------------------------------------------------------------------------------------------------------------------------+
| "NODE" | "id"   | {autoUpdate: "true", `keysForLabel:Entity`: "id", type: "fulltext", to_lower_case: "true", provider: "lucene", labels: "Entity"} |                                                                                   |
| "NODE" | "uuid" | {type: "exact", provider: "lucene"}                                                                                              |
+----------------------------------------------------------------------------------------------------------------------------------------------------+
2 rows available after 6 ms, consumed after another 0 ms

The graph is then exported, then the graph store is deleted prior to re-importing the graph, i.e.,

/usr/bin/neo4j-shell -c "export-binary -o /tmp/graph.db.bin"
neo4j stop
rm -rf /var/lib/neo4j/data/databases/graph.db
neo4j start
neo4j-shell -c "import-binary -c -i /tmp/graph.db.bin"

Inspecting the non-schema indexes shows that the full-text index didn't survive during either the export or import phase,

> cypher-shell 
Connected to Neo4j 3.2.1 at bolt://localhost:7687.
Type :help for a list of available commands or :exit to exit the shell.
Note that Cypher queries must end with a semicolon.
neo4j> CALL apoc.index.list();
+-------------------------------------------------------+
| type   | name   | config                              |
+-------------------------------------------------------+
| "NODE" | "uuid" | {type: "exact", provider: "lucene"} |
+-------------------------------------------------------+

1 row available after 8 ms, consumed after another 0 ms

[jexp]

The GraphAware plugin recreates and repopulates the index on startup if it doesn't exist.

As the manual indexes (and their content) are out of control of the database, they are not exported, you'd have to extract and export lucene content, because node-ids can change after the import which makes the indexes incorrect.

So it's easier to just recreate them after import.

Please also note that I'm trying to move all the functionality from here to apoc and then sunset these tools.

[john-bodley]

Thanks @jexp for the response and explanation. I gathered that this tool is being sunsetted and in favor of APOC which is developing into a fairly powerful suite of procedures.