[master < ] Add batched and parallel import

[gitbuda]

Resolves #42 -> mgconsole ~1000 lines/s (VERY SLOW)

DATASET
D1) Playground Cora Scientific Publications, N: 2708, E:5278, T:7986
D2) Playground Marve Cinematic Universe, N:21732 E: 682943, T: 704675

HARDWARE
H1) Ubuntu 20.04, Ryzen 7, 8 cores, 64GB RAM
H2) MacBook Pro M1, 16GB RAM

MEASUREMENTS

context	nodes	edges	serial (n+e)/s	parallel (n+e)/s	batch	workers
D1 + H1	2708	5278	1198.37	2642.62	1000	32
D2 + H1	21732	682943	5655.13	43820.34	1000	32
D1 + H2	2708	5278	736.51	2252.75	1000	16
D2 + H2	21732	682943	1060.32	7939.91	1000	16

NOTE
a) Parsing depends on the size of each line/node/edge, for smaller nodes/edges, query parsing time on H2) is >10k/s, but for bigger nodes (e.g. with large properties) can be much slower, e.g. ~3k/s

TASKS

• Add STORAGE MODE to the parser
• Improve (make it correct) the query parsing/analysis part
• Update the readme page
• hacked batching -> measure -> more-less the same as single-thread execution because the same calls are still there
• hacked parallelization -> measure
  • ensuring all nodes+edges are properly created will be complex -> IDEA: use summary
• proper batched and parallel execution -> measure
  • implement a limited number of batches + shuffling to minimize serialization errors
• Add hacked version of query detection type state machine
• in STORAGE MODE IN_MEMORY_ANALYTICAL mode, redundant edges are created for some reason
• Measure the parsing and execution performance with the state machine
• try a bigger dataset (500k+ edges)
• implement benchmark test based on the lab quick start datasets
• every character is processed + query parsing overhead is low -> use that budget to learn more about queries
• if backoff is not there -> "mg_raw_transport_recv: Bad file descriptor" OUTCOME: Created issue Debug and improve mgclient session inside mgconsole #53
• create proper .cpp implementation files for all runners (interactive, serial_import, ...)
• somehow make the std::getline part faster -> measure OUTCOME: Abandoned for now
• move line parsing to be parallel as well (NOTE: not easy because a query can span over many lines) -> measure OUTCOME: Abandoned for now

[gitbuda]

[gitbuda]

On M1, execution is completely removed, just the "single thread batching" overhead -> a lot of room for parallel execution

[gitbuda]

Single-threaded batched execution but with the same calls, without index creation and __vertex_id prop removal because it's impossible to create indexes in a multi-query transaction -> very similar results.

[gitbuda]

Still just a bit faster 🏃