Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🪓 Split Output into Outputs -> Output[] #1671

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

agoose77
Copy link
Contributor

@agoose77 agoose77 commented Nov 22, 2024

This PR is an alternative to #1661 that attempts to solve #1026. It does the following:

  1. Introduces a distinction between internal AST and external AST.
  2. Upgrades external AST into internal AST (e.g. pulling in xref).
  3. Downgrades internal AST to external AST (e.g. on writing to disk).

Tip

Internal AST is the AST that our transforms (and plugins) operate upon.
External AST is the AST that we publish to e.g. the web.
The factoring of our AST into two "branches" means that we can defer breaking changes to an arbitrary point in the future.

New Internal AST Form

The internal AST has the following form:

type CodeBlock = {
  type: "block";
  kind: "notebook-code",
  children: [
    Code,
    Outputs
  ]
}

type Outputs = {
  type: "outputs";
  children: Output[];
  visibility: ...;
  identifier: ...;
  html_id: ...;
}

type Output = {
  type: "output";
  children: GenericNode[];
  jupyter_data: IOutput;
}

New External AST Form

type CodeBlock = {
  type: "block";
  kind: "notebook-code",
  children: [
    Code,
    Output
  ]
}
type Output = {
  type: "output";
  children: GenericNode[];
  data: IOutput[];
  _future_ast: Outputs
}

In this PR, I don't break our published AST at all by storing a "replacement" AST in a new node field.

Note

The pattern of _future_ast replacing the current node seems like it might be quite useful. A way to systematically define forwards-compatible AST.

Closes #1674

Copy link

changeset-bot bot commented Nov 22, 2024

⚠️ No Changeset found

Latest commit: 7ef0e1e

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@agoose77 agoose77 force-pushed the agoose77/feat-future-ast-outputs branch 4 times, most recently from e278262 to 1f3a8c2 Compare November 25, 2024 11:50
@agoose77 agoose77 changed the title ⬆️🤫 Lift and parse Markdown/text outputs from Jupyter outputs (option 2) ⬆️🤫 Split Output into Outputs -> Output[] Nov 25, 2024
@agoose77 agoose77 changed the title ⬆️🤫 Split Output into Outputs -> Output[] 🪓 Split Output into Outputs -> Output[] Nov 25, 2024
@agoose77 agoose77 force-pushed the agoose77/feat-future-ast-outputs branch from 1f3a8c2 to 48e8edd Compare January 14, 2025 14:06
@agoose77 agoose77 force-pushed the agoose77/feat-future-ast-outputs branch from 48e8edd to b257932 Compare January 14, 2025 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for representing per-output sub-ASTs in MyST AST
1 participant