Shard MySQL for connect four context (#164)

Applies schema-based sharding for the connect four database #119.
The application itself only knows about logical shards and relies
on a proxy, such as ProxySQL, to forward queries to physical shards.

* The env variable APP_CONNECT_FOUR_DOCTRINE_DBAL_SHARDS
defines a comma-separated list of active shards. As games can
be sharded across multiple physical databases, the transactional scope
(technical-wise) is moved to the repository layer, and removed from the
application layer. The changes to the repository, which is now very
persistence oriented, aim to make this explicit. The shard selection
happens based on the game id.
* The env variable APP_CONNECT_FOUR_DOCTRINE_DBAL_DATABASE
is introduced to allow selecting a shard for database creation and
migration, as doctrine commands currently don't allow passing
the database name via cli arguments.
* Besides the repository, the event store access must also be
aware of sharding, since this is read when the query model is not
yet populated. For that matter, to query the events from the
write model, the repository is used, which encapsulates the
EventStore.
* Remove query model's GameNotFoundException
This is an aside refactoring, not related. This indirection
wasn't needed in the past. Additionally, an instance of
GameId is required to find a game, not a loose string.
* The current implementation doesn't take resharding into
consideration.

Author: marein

.env

@@ -16,7 +16,9 @@ APP_CHAT_RUN_MIGRATIONS=1
 ############################
 #   Connect Four Context   #
 ############################
-APP_CONNECT_FOUR_DOCTRINE_DBAL_URL=mysqli://root:password@mysql:3306/connect-four?persistent=1
+APP_CONNECT_FOUR_DOCTRINE_DBAL_URL=mysqli://root:password@mysql:3306?persistent=1
+APP_CONNECT_FOUR_DOCTRINE_DBAL_DATABASE=connect-four
+APP_CONNECT_FOUR_DOCTRINE_DBAL_SHARDS=connect-four
 APP_CONNECT_FOUR_RUN_MIGRATIONS=1
 APP_CONNECT_FOUR_PREDIS_CLIENT_URL=redis://redis:6379?persistent=1
 

README.md

@@ -308,18 +308,23 @@ In the next step we've to scale the databases. We've to divide this into two par
 1. First we want to scale the databases for reading purposes.  
 Since there should not be a concurrency problem in this application, we can add replicas for the MySQL and Redis stores.
 2. Then we want to scale the databases for writing purposes.  
-__Example for connect four__: I've put much effort to allow scaling the Connect Four context properly.
-Because we use the CQRS pattern to decouple the queries that span multiple games,
-such as counting running games or listing open games, we can ignore this queries in this case.
-To fetch the command model we exclusively need a game id.
-With this in mind, we can leverage a technique called
+__Example for connect four__: This context already can be scaled-out by
 [sharding](https://en.wikipedia.org/wiki/Shard_(database_architecture))
-for this. The shard key is in our case the game id.  
+the database, since queries that span multiple games have been offloaded. How this is made possible is described
+[in this section](#connect-four).
+Only the game id is needed for the execution of the command model,
+which is why it's well suited for the sharding key.
+Sharding is done at the application level, more specifically in the
+[repository](/src/ConnectFour/Port/Adapter/Persistence/Repository/DoctrineJsonGameRepository.php).
+The application uses schema-based sharding and is aware of all existing logical shards,
+while it's only aware of one physical connection. To actually forward queries to separate physical shards,
+a proxy such as ProxySQL can be used. An example will be added with
+[#118](https://github.com/marein/php-gaming-website/issues/118).  
 __Example for chat__: Currently there shouldn't be queries that span multiple chats.
 To invoke a chat operation (either writing or reading) we exclusively need a chat id.
 As in connect four context, we can use
 [sharding](https://en.wikipedia.org/wiki/Shard_(database_architecture))
-for the chat context.
+for the chat context, where the chat id is the sharding key.
 
 You may have seen that all contexts uses only one MySQL and one Redis instance.
 This could be different for the production environment depending on the scale.

bin/connect-four/onEntrypoint

@@ -4,12 +4,16 @@ set -e
 
 if [ "${APP_CONNECT_FOUR_RUN_MIGRATIONS}" = "1" ]
 then
-    bin/console doctrine:database:create \
-        --connection=connect_four \
-        --if-not-exists
-    bin/console doctrine:migrations:migrate \
-        --configuration=config/connect-four/migrations.yml \
-        --conn=connect_four \
-        --allow-no-migration \
-        --no-interaction
+    IFS=, read -ra values <<< "${APP_CONNECT_FOUR_DOCTRINE_DBAL_SHARDS}"
+    for value in "${values[@]}"
+    do
+        APP_CONNECT_FOUR_DOCTRINE_DBAL_DATABASE=${value} bin/console doctrine:database:create \
+            --connection=connect_four \
+            --if-not-exists
+        APP_CONNECT_FOUR_DOCTRINE_DBAL_DATABASE=${value} bin/console doctrine:migrations:migrate \
+            --configuration=config/connect-four/migrations.yml \
+            --conn=connect_four \
+            --allow-no-migration \
+            --no-interaction
+    done
 fi

config/connect-four/config.yml

@@ -6,6 +6,7 @@ doctrine:
         connections:
             connect-four:
                 url: '%env(APP_CONNECT_FOUR_DOCTRINE_DBAL_URL)%'
+                dbname: '%env(APP_CONNECT_FOUR_DOCTRINE_DBAL_DATABASE)%'
                 server_version: 8.0
                 charset: utf8mb4
                 default_table_options:

config/connect-four/services/command_bus.yml

@@ -12,17 +12,10 @@ services:
         class: Gaming\Common\Bus\RetryBus
         public: false
         arguments:
-            - '@connect-four.doctrine-transactional-command-bus'
+            - '@connect-four.routing-command-bus'
             - 3
             - 'Gaming\Common\Domain\Exception\ConcurrencyException'
 
-    connect-four.doctrine-transactional-command-bus:
-        class: Gaming\Common\Port\Adapter\Bus\DoctrineTransactionalBus
-        public: false
-        arguments:
-            - '@connect-four.routing-command-bus'
-            - '@connect-four.doctrine-dbal'
-
     # This is pretty ugly. We can use tags, or create this via a factory in php.
     connect-four.routing-command-bus:
         class: Gaming\Common\Bus\RoutingBus

config/connect-four/services/game.yml

@@ -20,24 +20,20 @@ services:
             - '@connect-four.doctrine-dbal'
             - 'game'
             - '@connect-four.domain-event-publisher'
+            - '@connect-four.event-store'
             - '@connect-four.normalizer'
+            - !service
+                class: Gaming\Common\Sharding\Integration\Crc32ModShards
+                arguments:
+                    - '%env(csv:APP_CONNECT_FOUR_DOCTRINE_DBAL_SHARDS)%'
 
     connect-four.game-store:
         class: Gaming\ConnectFour\Port\Adapter\Persistence\Repository\PredisGameStore
         public: false
         arguments:
             - '@connect-four.predis'
             - 'game.'
-            - '@connect-four.event-store-game-finder'
-
-    connect-four.game-finder:
-        alias: 'connect-four.game-store'
-
-    connect-four.event-store-game-finder:
-        class: Gaming\ConnectFour\Port\Adapter\Persistence\Repository\EventStoreGameFinder
-        public: false
-        arguments:
-            - '@connect-four.event-store'
+            - '@connect-four.game-repository'
 
     connect-four.games-by-player-store:
         class: Gaming\ConnectFour\Port\Adapter\Persistence\Repository\PredisGamesByPlayerStore

config/connect-four/services/query_bus.yml

@@ -22,7 +22,7 @@ services:
         class: Gaming\ConnectFour\Application\Game\Query\GameHandler
         public: false
         arguments:
-            - '@connect-four.game-finder'
+            - '@connect-four.game-store'
 
     connect-four.query.games-by-player-handler:
         class: Gaming\ConnectFour\Application\Game\Query\GamesByPlayerHandler

docker-compose.production.yml

@@ -10,7 +10,9 @@ x-php-container:
         APP_RABBIT_MQ_DSN: "amqp://guest:guest@rabbit-mq:5672?receive_method=basic_consume&qos_prefetch_count=10&heartbeat=60"
         APP_CHAT_DOCTRINE_DBAL_URL: "mysqli://root:password@mysql:3306/chat?persistent=1"
         APP_CHAT_RUN_MIGRATIONS: "1"
-        APP_CONNECT_FOUR_DOCTRINE_DBAL_URL: "mysqli://root:password@mysql:3306/connect-four?persistent=1"
+        APP_CONNECT_FOUR_DOCTRINE_DBAL_URL: "mysqli://root:password@mysql:3306?persistent=1"
+        APP_CONNECT_FOUR_DOCTRINE_DBAL_DATABASE: "connect-four"
+        APP_CONNECT_FOUR_DOCTRINE_DBAL_SHARDS: "connect-four"
         APP_CONNECT_FOUR_RUN_MIGRATIONS: "1"
         APP_CONNECT_FOUR_PREDIS_CLIENT_URL: "redis://redis:6379?persistent=1"
         APP_IDENTITY_DOCTRINE_DBAL_URL: "mysqli://root:password@mysql:3306/identity?persistent=1"

src/Common/Sharding/Exception/ShardingException.php

@@ -0,0 +1,11 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Gaming\Common\Sharding\Exception;
+
+use Exception;
+
+final class ShardingException extends Exception
+{
+}

src/Common/Sharding/Integration/Crc32ModShards.php

@@ -0,0 +1,37 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Gaming\Common\Sharding\Integration;
+
+use Gaming\Common\Sharding\Exception\ShardingException;
+use Gaming\Common\Sharding\Shards;
+
+/**
+ * This implementation causes problems in the event of re-sharding,
+ * because it's very likely that values will be reassigned to another shard.
+ * This can lead to more network calls until everything is back in place.
+ * It's therefore not suitable for every use case and should be used with caution.
+ * A better alternative would be to use a consistent hashing algorithm,
+ * which would reduce the likelihood of values being reassigned to another shard.
+ */
+final class Crc32ModShards implements Shards
+{
+    /**
+     * @param string[] $shards
+     *
+     * @throws ShardingException
+     */
+    public function __construct(
+        private readonly array $shards
+    ) {
+        if (count($this->shards) === 0) {
+            throw new ShardingException('At least one shard must be specified.');
+        }
+    }
+
+    public function lookup(string $value): string
+    {
+        return $this->shards[crc32($value) % count($this->shards)];
+    }
+}

src/Common/Sharding/Shards.php

@@ -0,0 +1,15 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Gaming\Common\Sharding;
+
+use Gaming\Common\Sharding\Exception\ShardingException;
+
+interface Shards
+{
+    /**
+     * @throws ShardingException
+     */
+    public function lookup(string $value): string;
+}

src/ConnectFour/Application/Game/Command/AbortHandler.php

@@ -4,6 +4,7 @@
 
 namespace Gaming\ConnectFour\Application\Game\Command;
 
+use Gaming\ConnectFour\Domain\Game\Game;
 use Gaming\ConnectFour\Domain\Game\GameId;
 use Gaming\ConnectFour\Domain\Game\Games;
 
@@ -18,10 +19,9 @@ public function __construct(Games $games)
 
     public function __invoke(AbortCommand $command): void
     {
-        $game = $this->games->get(GameId::fromString($command->gameId()));
-
-        $game->abort($command->playerId());
-
-        $this->games->save($game);
+        $this->games->update(
+            GameId::fromString($command->gameId()),
+            static fn(Game $game) => $game->abort($command->playerId())
+        );
     }
 }

src/ConnectFour/Application/Game/Command/AssignChatHandler.php

@@ -4,6 +4,7 @@
 
 namespace Gaming\ConnectFour\Application\Game\Command;
 
+use Gaming\ConnectFour\Domain\Game\Game;
 use Gaming\ConnectFour\Domain\Game\GameId;
 use Gaming\ConnectFour\Domain\Game\Games;
 
@@ -18,10 +19,9 @@ public function __construct(Games $games)
 
     public function __invoke(AssignChatCommand $command): void
     {
-        $game = $this->games->get(GameId::fromString($command->gameId()));
-
-        $game->assignChat($command->chatId());
-
-        $this->games->save($game);
+        $this->games->update(
+            GameId::fromString($command->gameId()),
+            static fn(Game $game) => $game->assignChat($command->chatId())
+        );
     }
 }

src/ConnectFour/Application/Game/Command/JoinHandler.php

@@ -4,6 +4,7 @@
 
 namespace Gaming\ConnectFour\Application\Game\Command;
 
+use Gaming\ConnectFour\Domain\Game\Game;
 use Gaming\ConnectFour\Domain\Game\GameId;
 use Gaming\ConnectFour\Domain\Game\Games;
 
@@ -18,10 +19,9 @@ public function __construct(Games $games)
 
     public function __invoke(JoinCommand $command): void
     {
-        $game = $this->games->get(GameId::fromString($command->gameId()));
-
-        $game->join($command->playerId());
-
-        $this->games->save($game);
+        $this->games->update(
+            GameId::fromString($command->gameId()),
+            static fn(Game $game) => $game->join($command->playerId())
+        );
     }
 }

src/ConnectFour/Application/Game/Command/MoveHandler.php

@@ -4,6 +4,7 @@
 
 namespace Gaming\ConnectFour\Application\Game\Command;
 
+use Gaming\ConnectFour\Domain\Game\Game;
 use Gaming\ConnectFour\Domain\Game\GameId;
 use Gaming\ConnectFour\Domain\Game\Games;
 
@@ -18,10 +19,9 @@ public function __construct(Games $games)
 
     public function __invoke(MoveCommand $command): void
     {
-        $game = $this->games->get(GameId::fromString($command->gameId()));
-
-        $game->move($command->playerId(), $command->column());
-
-        $this->games->save($game);
+        $this->games->update(
+            GameId::fromString($command->gameId()),
+            static fn(Game $game) => $game->move($command->playerId(), $command->column())
+        );
     }
 }

src/ConnectFour/Application/Game/Command/OpenHandler.php

@@ -25,7 +25,7 @@ public function __invoke(OpenCommand $command): string
             $command->playerId()
         );
 
-        $this->games->save($game);
+        $this->games->add($game);
 
         return $game->id()->toString();
     }

src/ConnectFour/Application/Game/Command/ResignHandler.php

@@ -4,6 +4,7 @@
 
 namespace Gaming\ConnectFour\Application\Game\Command;
 
+use Gaming\ConnectFour\Domain\Game\Game;
 use Gaming\ConnectFour\Domain\Game\GameId;
 use Gaming\ConnectFour\Domain\Game\Games;
 
@@ -18,10 +19,9 @@ public function __construct(Games $games)
 
     public function __invoke(ResignCommand $command): void
     {
-        $game = $this->games->get(GameId::fromString($command->gameId()));
-
-        $game->resign($command->playerId());
-
-        $this->games->save($game);
+        $this->games->update(
+            GameId::fromString($command->gameId()),
+            static fn(Game $game) => $game->resign($command->playerId())
+        );
     }
 }

src/ConnectFour/Application/Game/Query/Exception/GameNotFoundException.php

@@ -4,9 +4,10 @@
 
 namespace Gaming\ConnectFour\Application\Game\Query;
 
-use Gaming\ConnectFour\Application\Game\Query\Exception\GameNotFoundException;
 use Gaming\ConnectFour\Application\Game\Query\Model\Game\Game;
 use Gaming\ConnectFour\Application\Game\Query\Model\Game\GameFinder;
+use Gaming\ConnectFour\Domain\Game\Exception\GameNotFoundException;
+use Gaming\ConnectFour\Domain\Game\GameId;
 
 final class GameHandler
 {
@@ -22,8 +23,6 @@ public function __construct(GameFinder $gameFinder)
      */
     public function __invoke(GameQuery $query): Game
     {
-        return $this->gameFinder->find(
-            $query->gameId()
-        );
+        return $this->gameFinder->find(GameId::fromString($query->gameId()));
     }
 }

src/ConnectFour/Application/Game/Query/GameHandler.php

@@ -4,12 +4,13 @@
 
 namespace Gaming\ConnectFour\Application\Game\Query\Model\Game;
 
-use Gaming\ConnectFour\Application\Game\Query\Exception\GameNotFoundException;
+use Gaming\ConnectFour\Domain\Game\Exception\GameNotFoundException;
+use Gaming\ConnectFour\Domain\Game\GameId;
 
 interface GameFinder
 {
     /**
      * @throws GameNotFoundException
      */
-    public function find(string $gameId): Game;
+    public function find(GameId $gameId): Game;
 }