Merge branch 'PE-6474-initial-madr' into PE-6431

Author: atticusofsparta

.prettierrc

@@ -3,5 +3,6 @@
   "semi": true,
   "printWidth": 80,
   "singleQuote": true,
-  "trailingComma": "all"
+  "trailingComma": "all",
+  "proseWrap": "always"
 }

docs/madr/1-intial.md

@@ -1,4 +1,4 @@
-# Asset resolution by way of an ANT Registry
+# An event driven ANT registration system in AO to optimize ANT listings by access role
 
 - Status: accepted
 - Deciders: [Dylan], [Ariel], [Phil]
@@ -7,7 +7,32 @@
 
 ## Context and Problem Statement
 
-Performance of loading a user’s registered name is slow with the current method of loading all of the ArNS ecosystem (all ArNS names and ANTs registered to them). There is no way to quickly identify ownership of an ANT, let alone any AO process.
+Listing all ANTs registered to ArNS names and owned by a specific owner or
+owners becomes more computationally expensive as more ArNS names are registered.
+Additionally, its computationally infeasible to find ANTs not registered to an
+ArNS. GraphQL-based approaches cannot currently support either of these
+scenarios.
+
+We need to provide owner-based lookups that perform better than O(N).
+
+This is done both on the client side via [AoConnect], or as inter-process
+communication (like in the [ArNS Resolver](#arns-resolver)).
+
+### Additional Background
+
+The ArNS AO Process is responsible for recording which ANTs are registered to
+each ArNS name. It does not store or track additional information about the
+owner of the ANT. This reduces the amount of state managed, providing more
+concurrency in the operations in the overall name system's design, (e.g.
+individual ANT's activities do not impact performance of the ArNS Process).
+
+The downside of this separation is that listing all the ANTs for any owner is
+computationally expensive. You need to retrieve every ArNS record and perform a
+lookup of each associated ANT's owner, then filter the resulting owner map by
+the owner(s) in question - an O(N) operation where N is the total number of
+registered names. An additional downside is that unregistered (from an ArNS
+perspective) ANTs that one owns are not discoverable via this expensive
+algorithm.
 
 ## Decision Drivers
 
@@ -18,78 +43,111 @@ The main drivers for this decision revolve around:
 - **Scalability**
   - We cannot load the entire ArNS ecosystem client-side if it grows.
 - **Ecosystem Integrations**
-  - Having this utility process solely for ANT contract indexing provides integration points for other applications and reduces our reliance on other teams/products (e.g., Bazar profile processes).
+  - Having this utility process solely for ANT contract indexing provides
+    integration points for other applications and reduces our reliance on other
+    teams/products (e.g., Bazar profile processes).
 - **Maintainability**
   - Since we control the registry process, we can drive its maintenance.
 
 ## Considered Options
 
-### ANT Registry
+### Dedicated ANT Registry Process
+
+An AO Process that allows for permissionless reporting of the existence of ANTs
+by their Process ID that is capable of asynchronously, by way of AO messaging,
+retrieving, or being notified about, and then caching indexable information
+about ANTs, namely their owners and Controllers.
 
 - **Pros:**
   - Dedicated registry for ANTs.
   - Event-driven updates from ANTs.
-  - Separate state and compute from the resolver process.
+  - Separate state and compute from the resolver process, thus preserving system
+    concurrency while providing for better UX in the system.
+  - **Can support permissionless registration, thus allowing us to pre-register
+    every ANT we've previously created during our migration to AO AND allowing
+    3rd party Processes to create and register ANTs on behalf of users.**
 - **Cons:**
   - Additional maintenance required.
+  - Permissionlessness may invite spamming or DOSing
+  - Gas management requirements since the ANT Registry will have to send
+    messages to ANT processes
 
-### ArNS Resolver
+### Polling-based ArNS Resolver
+
+The ArNS Resolver is an AOS process that acts as an indexer for the entire ArNS
+ecosystem - it tracks owner and controller relationship by maintaining a state
+of all ANTs registered to ArNS names, thereby allowing efficient lookups of
+relational data from a central process.
 
 - **Pros:**
   - Capable of resolving records and ANTs.
 - **Cons:**
-  - Does not resolve ANTs that are not registered.
+  - **Does not resolve ANTs that are not registered.**
   - Complexity increases with integration of ANT updates.
+  - Performs unnecessary work for ANTs that have not changed
+  - **Potentially shoulders all the gas burden for fetching ANT information**
 
 ### Adding Ownership Info to the Network Process
 
+This would entail updates to the Network Process, either to make it so the ANT
+associated with an ArNS name can send it a state update (like the pattern the
+[ANT Registry](#dedicated-ant-registry-process) implements) on role changes (eg
+ANT is transferred or a controller is added to it), or removing said role
+changes from happening on the ANT and applying those handlers to the Network
+process to allow the owner of the domain to manage those fields directly on the
+Network process.
+
 - **Pros:**
   - Centralized control.
 - **Cons:**
   - Increases complexity of the Network process.
-  - Reduces flexibility of ANT processes.
+  - **Reduces flexibility of ANT processes.**
+  - **Massively increases computational load and process memory - i.e. reduction
+    in system concurrency**
 
 ### Bazar Profile Processes and Profile Registry
 
+[Bazar Profiles] are AOS processes that implement asset tracking and account abstraction,
+whereby the profile itself owns the assets it is tracking, and the profile is owned
+by the user. This allows for event driven portfolio management. The Profile Registry
+is a similar organism to the ANT Registry, therein tracking existing profiles that
+register themselves to at, allowing for profile discovery by user address.
+
 - **Pros:**
   - Promising technology.
   - Future integration potential.
 - **Cons:**
-  - Currently in rapid development.
+  - **Currently in rapid development.**
   - Managed by another team.
   - Complicates ownership resolution due to account abstraction.
 
 ## Decision Outcome
 
-It was decided to use an ANT registry due to the following downsides of other avenues:
-
-#### ArNS Resolver
-
-The ArNS Resolver interacts with the Network contract and is capable of resolving the records as well as the ANTs; however, it does not resolve ANTs that are not registered, only those that are registered. The ANT registry can implement event-driven updates from ANTs while separating the state and compute from the resolver process.
-
-#### Adding Ownership Info to the Network Process
-
-Part of the point of using the ANT processes for ownership control is to keep the state separated from the Network process. Adding this to the Network process would add complexity and reduce the flexibility of ANTs.
-
-#### Bazar Profile Processes and Profile Registry
-
-This tech is promising but still quite young and in rapid development at the time of writing, in the hands of another team making the architectural decisions. The profile process workflow is also an account abstraction, complicating the ownership resolution. With the ANT Registry, we should still be able to integrate with the profile processes in the future (since profile processes are, in fact, the owner of their listed assets).
+After weighing the pros and cons, it was decided to use an
+[ANT registry](#dedicated-ant-registry-process).
 
 ### Positive Consequences
 
-- Asset discovery will be optimized, allowing easy querying for users' assets, specifically ANT processes, which can then be used to retrieve owned domains from the network process.
+- Asset discovery will be optimized, allowing easy querying for users' assets,
+  specifically ANT processes, which can then be used to retrieve owned domains
+  from the network process.
 
 ### Negative Consequences
 
-- Maintenance: It is another process for us to maintain, though it seems necessary.
-- Future Redundancy: The problem this is solving may be solved in the future by more ecosystem-wide projects (e.g., Bazar profiles), making this redundant.
+- Maintenance: It is another process for us to maintain, though it seems
+  necessary.
+- Future Redundancy: The problem this is solving may be solved in the future by
+  more ecosystem-wide projects (e.g., Bazar profiles), making this redundant.
 
 ## Links
 
-- [Bazar Profiles](https://github.com/permaweb/ao-permaweb/tree/main/services/profiles)
+- [Bazar Profiles]
 
 ---
 
+[AoConnect]: (https://github.com/permaweb/ao/tree/main/connect)
+[Bazar Profiles]:
+  (https://github.com/permaweb/ao-permaweb/tree/main/services/profiles)
 [ADR Template]: (https://adr.github.io/)
 [Atticus]: (https://github.com/atticusofsparta)
 [Dylan]: (https://github.com/dtfiedler)

docs/madr/2-sqlite.md

@@ -1,4 +1,4 @@
-# Using aos-sqlite Module
+# Using aos-sqlite to manage data registered in the ANT Registry Process
 
 - Status: proposed
 - Deciders: [Dylan], [Ariel], [Phil]
@@ -7,18 +7,28 @@
 
 ## Context and Problem Statement
 
-The `aos-sqlite` module is a WebAssembly (Wasm) binary in the Arweave (ao) ecosystem that extends the functionality of the standard `aos` Wasm binary module. The `aos` module includes an `Eval` handler that consumes Lua strings and uses the `load` method of Lua to execute them. The `aos-sqlite` module is compiled with `lua-sqlite3` from C to Wasm, providing additional SQLite functionality. Given that directly using tables has demonstrated scalability issues, the SQLite implementation offers a potential solution. However, due to its novelty in the ecosystem, there are concerns about its stability in larger products. Therefore, using it here for the ANT Registry provides a suitable environment seperate from other products to test its stability.
+The `aos-sqlite` module, is a WebAssembly (Wasm) binary in the Arweave (ao)
+ecosystem, and extends the standard `aos` Wasm module. The `aos` module includes
+an `Eval` handler that executes Lua strings using the `load` method of the
+stdlib in Lua. The `aos-sqlite` module, compiled with `lua-sqlite3` from C to
+Wasm, adds SQLite functionality.
+
+Directly using Lua tables has shown a need for better tooling in manipulating
+and searching data, making the SQLite implementation a potential solution.
+However, its novelty raises concerns about reliability in larger products.
 
 ## Decision Drivers
 
 The main drivers for this decision are:
 
 - **Scalability**
-  - The current method of using tables has scalability issues that SQLite can potentially address.
+  - The current method of using tables has scalability issues that SQLite can
+    potentially address.
 - **Performance**
   - Improved performance in handling larger datasets with SQLite.
 - **Risk Management**
-  - Mitigating risks in larger products by testing the module in a controlled environment first.
+  - Mitigating risks in larger products by testing the module in a controlled
+    environment first.
 - **Ecosystem Integration**
   - Exploring new modules and their potential benefits within the ao ecosystem.
 
@@ -30,51 +40,50 @@ The main drivers for this decision are:
   - Known and stable.
   - No need for additional testing or integration.
 - **Cons:**
-  - Demonstrated scalability issues.
+  - Demonstrated scalability issues when attempting to retrieve 150k records in
+    ANT's.
   - Limited performance with larger datasets.
-  - Lack of strictness on data types
+  - Lack of strictness on data types.
 
 ### Option 2: Use aos-sqlite Module
 
 - **Pros:**
   - Addresses scalability issues with direct table usage.
   - Potential for improved performance with larger datasets.
   - Maintained by core team of AO
+  - Additional flexibility in the ways we can store and query for data
 - **Cons:**
   - New and relatively untested in the ecosystem.
   - Potential stability issues in larger products.
 
 ## Decision Outcome
 
-It was decided to use the `aos-sqlite` module to address scalability issues in a controlled environment. This approach allows us to test its effectiveness and performance before considering wider adoption in larger products.
-
-### Implementation Details
-
-#### aos-sqlite Module
-
-- **Functionality:** Extends the `aos` module with SQLite functionality, compiled with `lua-sqlite3` from C to Wasm.
-- **Handler:** Includes an `Eval` handler that consumes Lua strings and executes them using the `load` method of Lua.
-
-#### Testing and Scalability
-
-- **Controlled Environment:** The module will be tested in a controlled environment to assess its scalability and performance.
-- **Risk Mitigation:** By testing in a smaller scope, we aim to mitigate the risks associated with its novelty and potential instability in larger products.
+It was decided to use the `aos-sqlite` module to address scalability issues in
+the controlled environment of the ANT Registry, since it stands apart from other
+mission critical products. This approach allows us to test its effectiveness and
+performance before considering wider adoption in larger products.
 
 ### Positive Consequences
 
-- **Scalability Improvements:** Potential solution to scalability issues with direct table usage.
+- **Scalability Improvements:** Potential solution to scalability issues with
+  direct table usage.
 - **Performance Benefits:** Improved performance handling larger datasets.
-- **Future Integration:** Provides insights into the module's potential for wider adoption.
+- **Future Integration:** Provides insights into the module's potential for
+  wider adoption.
 
 ### Negative Consequences
 
-- **Stability Concerns:** New module with potential stability issues in larger products.
-- **Testing Effort:** Requires additional testing and monitoring to ensure reliability.
+- **Stability Concerns:** New module with potential stability issues in larger
+  products.
+- **Testing Effort:** Requires additional testing and monitoring to ensure
+  reliability.
 
 ## Implementation Recommendations
 
-- **Testing Framework:** Use a testing framework to evaluate the module's scalability and performance. ( eg with [AoLoader])
-- **Monitoring:** Implement monitoring tools like grafana to track the module's performance, stability, and resource usage.
+- **Testing Framework:** Use a testing framework to evaluate the module's
+  scalability and performance. ( eg with [AoLoader])
+- **Monitoring:** Implement monitoring tools like grafana to track the module's
+  performance, stability, and resource usage.
 
 ## Links