feat: consensus appeals use wrong contract state and restore contract state after appeal successful (#799)

* feat: add rollup transaction db table

* fix: drop audit table, change id and nonce of rollup, only make one finalized rollup transaction

* create rollup transaction for every validator

* add function to mock in consensus test

* feat: add appeal window, accepted queue, basic test

* fix: old tests were stuck on accepted state, add usage of thread

* refactor consensus into states

* feat: added appeal flow when transaction is accepted including the loop when appeal failed and succeeded

* fix: adding tests for appeals and fixing minor bugs

* refactor: merge main and PR #573 into this branch

* feat: add appeal_failed in db, select new validators when appealed based on formula

* docs: cleanup consensus mechanism base file

* test: checking the number of validators for different appeals

* feat: adding appeal window to undetermined state

* feat: change timestamp_accepted and add appeal_undetermined in database

* feat: undetermined to pending, activate frontend button in undetermined state and add button also to modal

* feat: leader only has no appeal button and no appeal window

* feat: implement the state transitions to process the appeal
- Add N+2 validators, remove leader
- Use latest data of transaction when in pending state and not the old one from the crawler
- Write consensus data before setting status to have it updated in frontend when going to transaction info modal
- Do not deploy contract when transaction was in the undetermined state

* test: add test for leader appeals

* refactor: merge 593-appeals-add-validators-when-appealed into 604-appeals-implement-sequential-appeals-fail

* refactor: merging changed file permissions

* fix: appealing a write method gave a KeyError because of wrong conversation of transaction

* docs: update transaction_processor argument description

* fix: all appeals disagreed because of pending_transactions type

* refactor: undo change because of merge

* fix: do not reset finality window when leader appeal failed, leader_only check in appeal_window

* fix: set value in database migration file

* fix: add leader_only check for modal appeal button, comment out modal appeal button

* fix: do not show appeal button when finality window is finished, checking on finalized state gave a small delay in showing when appeal failed and finality window was exceeded

* fix: 2nd tx should not finish before 1st tx

* feat: add stop event async tasks, empty pending queue, rollback newer transactions to pending

* refactor: remove print statements wrt queue

* fix: keep the same validator set when tx is rollbacked

* fix: we do not emit messages when the transaction goes from undetermined to finalized

* fix: complete merge

* fix: redirect leader appeal is processed by appeal queue, not pending queue

* fix: test increase wait time to get to finalized

* fix: add rollback to pending of newer transactions for leader appeals

* fix: use appeal property of the transaction in frontend

* fix: comment out appeal buttons

* feat: contract contains accepted and finalized state

* fix: update old version of state of an existing contract, use accepted contract state in encoded_state for _SnapshotView reads and writes

* feat:restore contract state after validator appeal success

* fix: reusing validators after rollback clean up

* feat: add contractsnapshot to transaction, leader and validators use it inclusing for appeals, rollback restores the state with it as it is the state before applying the write method

* fix: remove print statement

* fix: leader appeal should keep the contract_snapshot

* refactor: clean up prints

* refactor: clean up prints

* fix: typo in test_helpers for consensus

* fix: resolve consensus base test fails

* fix: check status typo

* fix: update consensus test

* fix: update test_get_highest_timestamp

* fix: add activated to status match check in consensus test

* fix: add none check when transforming to dict for contract_snapshot

* fix: increase timings for failed test

* fix: put contract_snapshot to none in rollback function; set contract_snapshot in accepted state intead of proposing state

* Trigger GitHub Actions workflow

* fix: remove temporary code testing

---------

Co-authored-by: Cristiam Da Silva <cristiam86@gmail.com>

Author: kstroobants

backend/consensus/base.py

@@ -116,6 +116,7 @@ def contract_snapshot_factory(
         ret.balance = transaction.value or 0
         ret.states = {"accepted": {}, "finalized": {}}
         ret.encoded_state = ret.states["accepted"]
+        ret.ghost_contract_address = transaction.ghost_contract_address
         return ret
 
     # Return a ContractSnapshot instance for an existing contract
@@ -1146,6 +1147,23 @@ async def process_validator_appeal(
                         context.msg_handler,
                     )
 
+                    # Get the previous state of the contract
+                    previous_contact_state = (
+                        context.transaction.contract_snapshot.encoded_state
+                    )
+
+                    # Restore the contract state
+                    if previous_contact_state:
+                        # Get the contract snapshot for the transaction's target address
+                        leaders_contract_snapshot = context.contract_snapshot_factory(
+                            context.transaction.to_address
+                        )
+
+                        # Update the contract state with the previous state
+                        leaders_contract_snapshot.update_contract_state(
+                            accepted_state=previous_contact_state
+                        )
+
                     # Transaction will be picked up by _crawl_snapshot
                     break
                 state = next_state
@@ -1178,6 +1196,11 @@ def rollback_transactions(self, context: TransactionContext):
                 context.msg_handler,
             )
 
+            # Reset the contract snapshot for the transaction
+            context.transactions_processor.set_transaction_contract_snapshot(
+                future_transaction["hash"], None
+            )
+
         # Start the queue loop again
         self.start_pending_queue_task(address)
 
@@ -1655,6 +1678,7 @@ async def handle(self, context):
         )
 
         if context.transaction.appealed:
+
             # Update the consensus results with all new votes and validators
             context.consensus_data.votes = (
                 context.transaction.consensus_data.votes | context.votes
@@ -1779,7 +1803,6 @@ async def handle(self, context):
             context.transactions_processor.set_transaction_appeal(
                 context.transaction.hash, False
             )
-            context.transaction.appealed = False
 
         # Set the transaction result
         context.transactions_processor.set_transaction_result(
@@ -1824,51 +1847,59 @@ async def handle(self, context):
             )
         )
 
-        # Update contract state
         # Retrieve the leader's receipt from the consensus data
         leader_receipt = context.consensus_data.leader_receipt
 
-        # Get the contract snapshot for the transaction's target address
-        leaders_contract_snapshot = context.contract_snapshot_supplier()
-
-        # Do not deploy the contract if the execution failed
-        if leader_receipt.execution_result == ExecutionResultStatus.SUCCESS:
+        # Do not deploy or update the contract if validator appeal failed
+        if not context.transaction.appealed:
             # Get the contract snapshot for the transaction's target address
             leaders_contract_snapshot = context.contract_snapshot_supplier()
 
-            # Register contract if it is a new contract
-            if context.transaction.type == TransactionType.DEPLOY_CONTRACT:
-                new_contract = {
-                    "id": context.transaction.data["contract_address"],
-                    "data": {
-                        "state": {
-                            "accepted": leader_receipt.contract_state,
-                            "finalized": {},
-                        },
-                        "code": context.transaction.data["contract_code"],
-                        "ghost_contract_address": context.transaction.ghost_contract_address,
-                    },
-                }
-                leaders_contract_snapshot.register_contract(new_contract)
+            # Set the contract snapshot for the transaction for a future rollback
+            context.transactions_processor.set_transaction_contract_snapshot(
+                context.transaction.hash, leaders_contract_snapshot.to_dict()
+            )
 
-                # Send a message indicating successful contract deployment
-                context.msg_handler.send_message(
-                    LogEvent(
-                        "deployed_contract",
-                        EventType.SUCCESS,
-                        EventScope.GENVM,
-                        "Contract deployed",
-                        new_contract,
-                        transaction_hash=context.transaction.hash,
+            # Do not deploy or update the contract if the execution failed
+            if leader_receipt.execution_result == ExecutionResultStatus.SUCCESS:
+                # Register contract if it is a new contract
+                if context.transaction.type == TransactionType.DEPLOY_CONTRACT:
+                    new_contract = {
+                        "id": context.transaction.data["contract_address"],
+                        "data": {
+                            "state": {
+                                "accepted": leader_receipt.contract_state,
+                                "finalized": {},
+                            },
+                            "code": context.transaction.data["contract_code"],
+                            "ghost_contract_address": context.transaction.ghost_contract_address,
+                        },
+                    }
+                    leaders_contract_snapshot.register_contract(new_contract)
+
+                    # Send a message indicating successful contract deployment
+                    context.msg_handler.send_message(
+                        LogEvent(
+                            "deployed_contract",
+                            EventType.SUCCESS,
+                            EventScope.GENVM,
+                            "Contract deployed",
+                            new_contract,
+                            transaction_hash=context.transaction.hash,
+                        )
                     )
-                )
-            # Update contract state if it is an existing contract
-            else:
-                leaders_contract_snapshot.update_contract_state(
-                    accepted_state=leader_receipt.contract_state
+                # Update contract state if it is an existing contract
+                else:
+                    leaders_contract_snapshot.update_contract_state(
+                        accepted_state=leader_receipt.contract_state
+                    )
+
+                _emit_transactions(
+                    context, leader_receipt.pending_transactions, "accepted"
                 )
 
-            _emit_transactions(context, leader_receipt.pending_transactions, "accepted")
+        else:
+            context.transaction.appealed = False
 
         # Set the transaction appeal undetermined status to false and return appeal status
         if context.transaction.appeal_undetermined:
@@ -1991,15 +2022,14 @@ async def handle(self, context):
         # Retrieve the leader's receipt from the consensus data
         leader_receipt = context.transaction.consensus_data.leader_receipt
 
-        # Get the contract snapshot for the transaction's target address
-        leaders_contract_snapshot = context.contract_snapshot_factory(
-            context.transaction.to_address
-        )
-
         # Update contract state
         if (context.transaction.status == TransactionStatus.ACCEPTED) and (
             leader_receipt.execution_result == ExecutionResultStatus.SUCCESS
         ):
+            # Get the contract snapshot for the transaction's target address
+            leaders_contract_snapshot = context.contract_snapshot_factory(
+                context.transaction.to_address
+            )
             leaders_contract_snapshot.update_contract_state(
                 finalized_state=leader_receipt.contract_state
             )

backend/database_handler/contract_snapshot.py

@@ -1,6 +1,7 @@
 # database_handler/contract_snapshot.py
 from .models import CurrentState
 from sqlalchemy.orm import Session
+from typing import Optional
 
 
 # TODO: should ContractSnapshot be a dataclass with just the contract data? Snapshots shouldn't be allowed to be modified, so it doesn't make sense to modify the database
@@ -45,6 +46,33 @@ def __init__(self, contract_address: str | None, session: Session):
                 else None
             )
 
+    def to_dict(self):
+        return {
+            "contract_address": (
+                self.contract_address if self.contract_address else None
+            ),
+            "contract_code": self.contract_code if self.contract_code else None,
+            "encoded_state": self.encoded_state if self.encoded_state else {},
+            "states": self.states if self.states else {"accepted": {}, "finalized": {}},
+            "ghost_contract_address": (
+                self.ghost_contract_address if self.ghost_contract_address else None
+            ),
+        }
+
+    @classmethod
+    def from_dict(cls, input: dict | None) -> Optional["ContractSnapshot"]:
+        if input:
+            instance = cls.__new__(cls)
+            instance.session = None
+            instance.contract_address = input.get("contract_address", None)
+            instance.contract_code = input.get("contract_code", None)
+            instance.encoded_state = input.get("encoded_state", {})
+            instance.states = input.get("states", {"accepted": {}, "finalized": {}})
+            instance.ghost_contract_address = input.get("ghost_contract_address", None)
+            return instance
+        else:
+            return None
+
     def _load_contract_account(self) -> CurrentState:
         """Load and return the current state of the contract from the database."""
         result = (

backend/database_handler/migration/versions/15fde6faebaf_contract_snapshot.py

@@ -0,0 +1,36 @@
+"""contract_snapshot
+
+Revision ID: 15fde6faebaf
+Revises: d932a5fef8b1
+Create Date: 2025-01-06 10:42:19.972610
+
+"""
+
+from typing import Sequence, Union
+
+from alembic import op
+import sqlalchemy as sa
+from sqlalchemy.dialects import postgresql
+
+# revision identifiers, used by Alembic.
+revision: str = "15fde6faebaf"
+down_revision: Union[str, None] = "d932a5fef8b1"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    # ### commands auto generated by Alembic - please adjust! ###
+    op.add_column(
+        "transactions",
+        sa.Column(
+            "contract_snapshot", postgresql.JSONB(astext_type=sa.Text()), nullable=True
+        ),
+    )
+    # ### end Alembic commands ###
+
+
+def downgrade() -> None:
+    # ### commands auto generated by Alembic - please adjust! ###
+    op.drop_column("transactions", "contract_snapshot")
+    # ### end Alembic commands ###

backend/database_handler/models.py

@@ -100,6 +100,7 @@ class Transactions(Base):
     consensus_history: Mapped[Optional[dict]] = mapped_column(JSONB)
     timestamp_appeal: Mapped[Optional[int]] = mapped_column(BigInteger)
     appeal_processing_time: Mapped[Optional[int]] = mapped_column(Integer)
+    contract_snapshot: Mapped[Optional[dict]] = mapped_column(JSONB)
 
     # Relationship for triggered transactions
     triggered_by_hash: Mapped[Optional[str]] = mapped_column(

backend/database_handler/transactions_processor.py

@@ -70,6 +70,7 @@ def _parse_transaction_data(transaction_data: Transactions) -> dict:
             "consensus_history": transaction_data.consensus_history,
             "timestamp_appeal": transaction_data.timestamp_appeal,
             "appeal_processing_time": transaction_data.appeal_processing_time,
+            "contract_snapshot": transaction_data.contract_snapshot,
         }
 
     @staticmethod
@@ -182,6 +183,7 @@ def insert_transaction(
             consensus_history={},
             timestamp_appeal=None,
             appeal_processing_time=0,
+            contract_snapshot=None,
         )
 
         self.session.add(new_transaction)
@@ -483,3 +485,20 @@ def reset_transaction_appeal_processing_time(self, transaction_hash: str):
         )
         transaction.appeal_processing_time = 0
         self.session.commit()
+
+    def set_transaction_contract_snapshot(
+        self, transaction_hash: str, contract_snapshot: dict | None
+    ):
+        transaction = (
+            self.session.query(Transactions).filter_by(hash=transaction_hash).one()
+        )
+        transaction.contract_snapshot = contract_snapshot
+        self.session.commit()
+
+    def get_transaction_contract_snapshot(
+        self, transaction_hash: str
+    ) -> ContractSnapshot | None:
+        transaction = (
+            self.session.query(Transactions).filter_by(hash=transaction_hash).one()
+        )
+        return ContractSnapshot.from_dict(transaction.contract_snapshot)

backend/domain/types.py

@@ -8,6 +8,7 @@
 
 from backend.database_handler.models import TransactionStatus
 from backend.database_handler.types import ConsensusData
+from backend.database_handler.contract_snapshot import ContractSnapshot
 
 
 @dataclass()
@@ -89,6 +90,7 @@ class Transaction:
     consensus_history: dict = field(default_factory=dict)
     timestamp_appeal: int | None = None
     appeal_processing_time: int = 0
+    contract_snapshot: ContractSnapshot | None = None
 
     def to_dict(self):
         return {
@@ -118,6 +120,9 @@ def to_dict(self):
             "consensus_history": self.consensus_history,
             "timestamp_appeal": self.timestamp_appeal,
             "appeal_processing_time": self.appeal_processing_time,
+            "contract_snapshot": (
+                self.contract_snapshot.to_dict() if self.contract_snapshot else None
+            ),
         }
 
     @classmethod
@@ -149,4 +154,7 @@ def from_dict(cls, input: dict) -> "Transaction":
             consensus_history=input.get("consensus_history"),
             timestamp_appeal=input.get("timestamp_appeal"),
             appeal_processing_time=input.get("appeal_processing_time", 0),
+            contract_snapshot=ContractSnapshot.from_dict(
+                input.get("contract_snapshot")
+            ),
         )

tests/unit/consensus/test_base.py

@@ -793,6 +793,8 @@ async def test_exec_accepted_appeal_fail_three_times(consensus_algorithm):
     transactions_processor = TransactionsProcessorMock(
         [transaction_to_dict(transaction)]
     )
+    consensus_algorithm.consensus_sleep_time = 5
+    consensus_algorithm.finality_window_time = 15
 
     def get_vote():
         return Vote.AGREE