feat(db-*): support atomic array $push db updates #13453

AlessioGr · 2025-08-13T06:55:31Z

This PR adds atomic $push support for array fields. It makes it possible to safely append new items to arrays, which is especially useful when running tasks in parallel (like job queues) where multiple processes might update the same record at the same time. By handling pushes atomically, we avoid race conditions and keep data consistent - especially on postgres, where the current implementation would nuke the entire array table before re-inserting every single array item.

The feature works for both localized and unlocalized arrays, and supports pushing either single or multiple items at once.

This PR is a requirement for reliably running parallel tasks in the job queue - see #13452.

Alongside documenting $push, this PR also adds documentation for $inc.

Changes to updatedAt behavior

#13335 allows us to override the updatedAt property instead of the db always setting it to the current date.

However, we are not able to skip updating the updatedAt property completely. This means, usage of $push results in 2 postgres db calls:

set updatedAt in main row
append array row in arrays table

This PR changes the behavior to only automatically set updatedAt if it's undefined. If you explicitly set it to null, this now allows you to skip the db adapter automatically setting updatedAt.

Note

I would like to note that this does differ from the behavior of other fields being set to null.

For other fields, setting them to null will cause the database to think we want to set it to null. In this PR, we created an exception for the updatedAt field, where setting it to null will simply skip it from the db update, neither setting it to the current date (this would be the behavior if it was undefined) nor setting it to null in the db.

The alternative to this would be adding a new property like disableTimestampUpdates, but that would be more code to maintain and less simple.

=> This allows us to use $push in just one single db call

Usage Examples

Pushing a single item to an array

const post = (await payload.db.updateOne({
  data: {
    array: {
      $push: {
        text: 'some text 2',
        id: new mongoose.Types.ObjectId().toHexString(),
      },
    },
  },
  collection: 'posts',
  id: post.id,
}))

Pushing a single item to a localized array

const post = (await payload.db.updateOne({
  data: {
    arrayLocalized: {
      $push: {
        en: {
          text: 'some text 2',
          id: new mongoose.Types.ObjectId().toHexString(),
        },
        es: {
          text: 'some text 2 es',
          id: new mongoose.Types.ObjectId().toHexString(),
        },
      },
    },
  },
  collection: 'posts',
  id: post.id,
}))

Pushing multiple items to an array

const post = (await payload.db.updateOne({
  data: {
    array: {
      $push: [
        {
          text: 'some text 2',
          id: new mongoose.Types.ObjectId().toHexString(),
        },
        {
          text: 'some text 3',
          id: new mongoose.Types.ObjectId().toHexString(),
        },
      ],
    },
  },
  collection: 'posts',
  id: post.id,
}))

Pushing multiple items to a localized array

const post = (await payload.db.updateOne({
  data: {
    arrayLocalized: {
      $push: {
        en: {
          text: 'some text 2',
          id: new mongoose.Types.ObjectId().toHexString(),
        },
        es: [
          {
            text: 'some text 2 es',
            id: new mongoose.Types.ObjectId().toHexString(),
          },
          {
            text: 'some text 3 es',
            id: new mongoose.Types.ObjectId().toHexString(),
          },
        ],
      },
    },
  },
  collection: 'posts',
  id: post.id,
}))

To see the specific tasks where the Asana app for GitHub is being used, see below:
- https://app.asana.com/0/0/1211110462564647

github-actions · 2025-08-13T07:00:41Z

📦 esbuild Bundle Analysis for payload

This analysis was generated by esbuild-bundle-analyzer. 🤖

Meta File	Out File	Size (raw)	Note
packages/next/meta_index.json	esbuild/index.js	753.94 KB	✅ No change
packages/payload/meta_index.json	esbuild/index.js	1.21 MB	✅ No change
packages/payload/meta_shared.json	esbuild/exports/shared.js	161.98 KB	✅ No change
packages/richtext-lexical/meta_client.json	esbuild/exports/client_optimized/index.js	259.50 KB	✅ No change
packages/ui/meta_client.json	esbuild/exports/client_optimized/index.js	1.15 MB	⚠️ +7 B (+0.0%)
packages/ui/meta_shared.json	esbuild/exports/shared_optimized/index.js	14.20 KB	✅ No change

Largest paths

These visualization shows top 20 largest paths in the bundle.

Meta file: packages/next/meta_index.json, Out file: esbuild/index.js

Path	Size
../../node_modules	${{\color{Goldenrod}{ ████████████████████ }}}$ 80.3%, 601.88 KB
dist/views/Version	${{\color{Goldenrod}{ █▋ }}}$ 6.7%, 50.09 KB
dist/views/Document	${{\color{Goldenrod}{ ▌ }}}$ 2.0%, 14.91 KB
dist/views/List	${{\color{Goldenrod}{ ▎ }}}$ 1.2%, 9.16 KB
dist/views/Root	${{\color{Goldenrod}{ ▏ }}}$ 0.9%, 7.08 KB
dist/views/Versions	${{\color{Goldenrod}{ ▏ }}}$ 0.8%, 6.30 KB
dist/views/API	${{\color{Goldenrod}{ ▏ }}}$ 0.8%, 5.98 KB
dist/views/Account	${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 5.32 KB
dist/elements/Nav	${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 4.84 KB
dist/elements/DocumentHeader	${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 4.81 KB
dist/views/Login	${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 4.39 KB
dist/views/Dashboard	${{\color{Goldenrod}{ ▏ }}}$ 0.5%, 3.68 KB
dist/views/ForgotPassword	${{\color{Goldenrod}{ }}}$ 0.4%, 3.09 KB
dist/layouts/Root	${{\color{Goldenrod}{ }}}$ 0.4%, 3.09 KB
dist/utilities/initPage	${{\color{Goldenrod}{ }}}$ 0.4%, 3.06 KB
dist/templates/Default	${{\color{Goldenrod}{ }}}$ 0.4%, 2.83 KB
dist/views/CreateFirstUser	${{\color{Goldenrod}{ }}}$ 0.3%, 2.61 KB
dist/views/BrowseByFolder	${{\color{Goldenrod}{ }}}$ 0.3%, 2.60 KB
dist/views/CollectionFolders	${{\color{Goldenrod}{ }}}$ 0.3%, 2.46 KB
dist/views/ResetPassword	${{\color{Goldenrod}{ }}}$ 0.3%, 2.41 KB
(other)	${{\color{Goldenrod}{ ████▉ }}}$ 19.7%, 147.38 KB

Meta file: packages/payload/meta_index.json, Out file: esbuild/index.js

Path	Size
../../node_modules	${{\color{Goldenrod}{ █████████████████▍ }}}$ 69.8%, 841.07 KB
dist/fields/hooks	${{\color{Goldenrod}{ ▊ }}}$ 3.3%, 40.21 KB
dist/collections/operations	${{\color{Goldenrod}{ ▊ }}}$ 3.0%, 35.98 KB
dist/auth/operations	${{\color{Goldenrod}{ ▎ }}}$ 1.3%, 15.22 KB
dist/queues/operations	${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 12.00 KB
dist/utilities/configToJSONSchema.js	${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 11.54 KB
dist/globals/operations	${{\color{Goldenrod}{ ▏ }}}$ 0.9%, 11.12 KB
dist/fields/config	${{\color{Goldenrod}{ ▏ }}}$ 0.9%, 10.82 KB
dist/fields/validations.js	${{\color{Goldenrod}{ ▏ }}}$ 0.8%, 9.38 KB
dist/bin/generateImportMap	${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 8.27 KB
dist/database/migrations	${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 7.79 KB
dist/collections/config	${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 7.79 KB
dist/uploads/fetchAPI-multipart	${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 7.74 KB
dist/collections/endpoints	${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 7.20 KB
dist/index.js	${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 7.11 KB
dist/config/orderable	${{\color{Goldenrod}{ ▏ }}}$ 0.5%, 6.25 KB
dist/auth/strategies	${{\color{Goldenrod}{ ▏ }}}$ 0.5%, 5.50 KB
dist/config/sanitize.js	${{\color{Goldenrod}{ ▏ }}}$ 0.5%, 5.44 KB
dist/auth/endpoints	${{\color{Goldenrod}{ }}}$ 0.4%, 5.41 KB
dist/utilities/telemetry	${{\color{Goldenrod}{ }}}$ 0.4%, 5.31 KB
(other)	${{\color{Goldenrod}{ ███████▌ }}}$ 30.2%, 363.70 KB

Meta file: packages/payload/meta_shared.json, Out file: esbuild/exports/shared.js

Path	Size
../../node_modules	${{\color{Goldenrod}{ ████████████████████ }}}$ 80.0%, 126.93 KB
dist/fields/validations.js	${{\color{Goldenrod}{ █▍ }}}$ 5.9%, 9.38 KB
dist/fields/baseFields	${{\color{Goldenrod}{ ▍ }}}$ 1.8%, 2.79 KB
dist/utilities/deepCopyObject.js	${{\color{Goldenrod}{ ▍ }}}$ 1.6%, 2.48 KB
dist/auth/cookies.js	${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 1.55 KB
dist/utilities/flattenTopLevelFields.js	${{\color{Goldenrod}{ ▏ }}}$ 0.9%, 1.41 KB
dist/fields/config	${{\color{Goldenrod}{ ▏ }}}$ 0.8%, 1.28 KB
dist/utilities/flattenAllFields.js	${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 943 B
dist/utilities/fieldSchemaToJSON.js	${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 917 B
dist/folders/utils	${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 916 B
dist/utilities/unflatten.js	${{\color{Goldenrod}{ ▏ }}}$ 0.5%, 779 B
dist/utilities/sanitizeUserDataForEmail.js	${{\color{Goldenrod}{ }}}$ 0.4%, 713 B
dist/collections/config	${{\color{Goldenrod}{ }}}$ 0.4%, 570 B
dist/bin/generateImportMap	${{\color{Goldenrod}{ }}}$ 0.4%, 559 B
dist/auth/sessions.js	${{\color{Goldenrod}{ }}}$ 0.3%, 545 B
dist/utilities/getSafeRedirect.js	${{\color{Goldenrod}{ }}}$ 0.3%, 423 B
dist/utilities/deepMerge.js	${{\color{Goldenrod}{ }}}$ 0.3%, 413 B
dist/utilities/getFieldPermissions.js	${{\color{Goldenrod}{ }}}$ 0.2%, 391 B
dist/utilities/formatLabels.js	${{\color{Goldenrod}{ }}}$ 0.2%, 380 B
dist/utilities/transformColumnPreferences.js	${{\color{Goldenrod}{ }}}$ 0.2%, 348 B
(other)	${{\color{Goldenrod}{ █████ }}}$ 20.0%, 31.82 KB

Meta file: packages/richtext-lexical/meta_client.json, Out file: esbuild/exports/client_optimized/index.js

Path	Size
dist/lexical/plugins	${{\color{Goldenrod}{ ███ }}}$ 12.1%, 30.91 KB
dist/features/experimental_table	${{\color{Goldenrod}{ ██▍ }}}$ 9.6%, 24.60 KB
dist/lexical/ui	${{\color{Goldenrod}{ ██▎ }}}$ 9.2%, 23.47 KB
dist/features/blocks	${{\color{Goldenrod}{ ██▏ }}}$ 8.7%, 22.43 KB
dist/packages/@lexical	${{\color{Goldenrod}{ █▊ }}}$ 7.4%, 18.99 KB
dist/features/link	${{\color{Goldenrod}{ █▊ }}}$ 7.0%, 17.96 KB
dist/features/toolbars	${{\color{Goldenrod}{ █▋ }}}$ 6.9%, 17.68 KB
dist/features/textState	${{\color{Goldenrod}{ █ }}}$ 4.3%, 10.98 KB
dist/features/upload	${{\color{Goldenrod}{ ▉ }}}$ 3.7%, 9.60 KB
dist/features/relationship	${{\color{Goldenrod}{ ▉ }}}$ 3.7%, 9.41 KB
dist/lexical/utils	${{\color{Goldenrod}{ ▊ }}}$ 3.1%, 8.08 KB
dist/features/debug	${{\color{Goldenrod}{ ▋ }}}$ 2.9%, 7.39 KB
dist/features/converters	${{\color{Goldenrod}{ ▋ }}}$ 2.7%, 7.04 KB
dist/utilities/fieldsDrawer	${{\color{Goldenrod}{ ▋ }}}$ 2.7%, 7.01 KB
dist/lexical/config	${{\color{Goldenrod}{ ▌ }}}$ 2.0%, 5.10 KB
dist/features/lists	${{\color{Goldenrod}{ ▍ }}}$ 1.9%, 4.95 KB
dist/lexical/theme	${{\color{Goldenrod}{ ▍ }}}$ 1.6%, 4.01 KB
dist/features/format	${{\color{Goldenrod}{ ▎ }}}$ 1.4%, 3.46 KB
dist/lexical/LexicalEditor.js	${{\color{Goldenrod}{ ▎ }}}$ 1.3%, 3.23 KB
dist/features/indent	${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 2.50 KB
(other)	${{\color{Goldenrod}{ █████████████████████▉ }}}$ 87.9%, 225.57 KB

Meta file: packages/ui/meta_client.json, Out file: esbuild/exports/client_optimized/index.js

Path	Size
../../node_modules	${{\color{Goldenrod}{ ████████████▌ }}}$ 50.3%, 572.93 KB
dist/elements/FolderView	${{\color{Goldenrod}{ ▋ }}}$ 2.7%, 30.39 KB
dist/elements/BulkUpload	${{\color{Goldenrod}{ ▌ }}}$ 2.2%, 24.57 KB
dist/elements/WhereBuilder	${{\color{Goldenrod}{ ▎ }}}$ 1.4%, 16.26 KB
dist/fields/Relationship	${{\color{Goldenrod}{ ▎ }}}$ 1.4%, 16.11 KB
dist/views/Edit	${{\color{Goldenrod}{ ▎ }}}$ 1.4%, 15.43 KB
dist/elements/Table	${{\color{Goldenrod}{ ▎ }}}$ 1.3%, 15.26 KB
dist/forms/Form	${{\color{Goldenrod}{ ▎ }}}$ 1.3%, 14.82 KB
dist/fields/Blocks	${{\color{Goldenrod}{ ▎ }}}$ 1.1%, 12.43 KB
dist/fields/Upload	${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 11.94 KB
dist/fields/Array	${{\color{Goldenrod}{ ▏ }}}$ 0.8%, 9.30 KB
dist/elements/QueryPresets	${{\color{Goldenrod}{ ▏ }}}$ 0.8%, 9.08 KB
dist/elements/PublishButton	${{\color{Goldenrod}{ ▏ }}}$ 0.8%, 8.57 KB
dist/providers/Folders	${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 8.38 KB
dist/elements/LivePreview	${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 8.20 KB
dist/elements/ListHeader	${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 7.98 KB
dist/views/CollectionFolder	${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 7.89 KB
dist/elements/HTMLDiff	${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 7.81 KB
dist/views/List	${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 7.18 KB
dist/elements/ReactSelect	${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 6.91 KB
(other)	${{\color{Goldenrod}{ ████████████▍ }}}$ 49.7%, 566.08 KB

Meta file: packages/ui/meta_shared.json, Out file: esbuild/exports/shared_optimized/index.js

Path	Size
dist/graphics/Logo	${{\color{Goldenrod}{ █████▋ }}}$ 22.9%, 3.12 KB
../../node_modules	${{\color{Goldenrod}{ ████▉ }}}$ 19.5%, 2.65 KB
dist/graphics/Icon	${{\color{Goldenrod}{ ██▊ }}}$ 11.2%, 1.52 KB
dist/utilities/formatDocTitle	${{\color{Goldenrod}{ ██▍ }}}$ 9.7%, 1.32 KB
dist/providers/TableColumns	${{\color{Goldenrod}{ █▌ }}}$ 6.3%, 862 B
dist/utilities/groupNavItems.js	${{\color{Goldenrod}{ █▌ }}}$ 6.0%, 814 B
dist/utilities/api.js	${{\color{Goldenrod}{ █▍ }}}$ 5.6%, 756 B
dist/elements/Translation	${{\color{Goldenrod}{ ▉ }}}$ 3.6%, 493 B
dist/elements/withMergedProps	${{\color{Goldenrod}{ ▋ }}}$ 2.5%, 339 B
dist/utilities/handleTakeOver.js	${{\color{Goldenrod}{ ▍ }}}$ 1.8%, 251 B
dist/elements/WithServerSideProps	${{\color{Goldenrod}{ ▍ }}}$ 1.7%, 232 B
dist/utilities/handleGoBack.js	${{\color{Goldenrod}{ ▎ }}}$ 1.2%, 168 B
dist/fields/mergeFieldStyles.js	${{\color{Goldenrod}{ ▎ }}}$ 1.2%, 159 B
dist/forms/Form	${{\color{Goldenrod}{ ▎ }}}$ 1.1%, 147 B
dist/utilities/abortAndIgnore.js	${{\color{Goldenrod}{ ▎ }}}$ 1.1%, 146 B
dist/utilities/hasSavePermission.js	${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 136 B
dist/utilities/handleBackToDashboard.js	${{\color{Goldenrod}{ ▏ }}}$ 0.9%, 129 B
dist/utilities/findLocaleFromCode.js	${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 84 B
dist/utilities/sanitizeID.js	${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 77 B
dist/utilities/isEditing.js	${{\color{Goldenrod}{ }}}$ 0.4%, 59 B
(other)	${{\color{Goldenrod}{ ███████████████████▎ }}}$ 77.1%, 10.49 KB

Details

Next to the size is how much the size has increased or decreased compared with the base branch of this PR.

‼️: Size increased by 20% or more. Special attention should be given to this.
⚠️: Size increased in acceptable range (lower than 20%).
✅: No change or even downsized.
🗑️: The out file is deleted: not found in base branch.
🆕: The out file is newly found: will be added to base branch.

…This allows usage of $push with just one single db call

jmikrut · 2025-08-22T19:56:46Z

we should chat about this one with @DanRibbens

AlessioGr added 3 commits August 11, 2025 16:25

mongo, int tests

e84f43f

postgres support

f61e7d0

Merge remote-tracking branch 'origin/main' into feat/$push

d2ca782

github-actions bot added the created-by: Payload team label Aug 13, 2025

AlessioGr added 8 commits August 13, 2025 00:21

fix tests

774ccee

Merge remote-tracking branch 'origin/main' into feat/$push

ef28b42

add tests for localized field within unlocalized array

2ab2b7b

mongodb localized push support

d78c047

improve test coverage

52c4adf

postgres localized arrays $push support

b3f23e8

import

1860f33

organize new tests

b9f6cb7

AlessioGr marked this pull request as ready for review August 21, 2025 20:38

AlessioGr requested a review from DanRibbens August 21, 2025 20:38

AlessioGr added 2 commits August 21, 2025 13:49

add docs

a3bbf38

docs

bb8e10b

AlessioGr enabled auto-merge (squash) August 21, 2025 22:33

AlessioGr added 6 commits August 22, 2025 10:20

Merge remote-tracking branch 'origin/main' into feat/$push

663c154

do not automatically set updatedAt if it's explicitly set to null

3a955f6

perf(drizzle): skip updating main row if there is no data to update. …

9c98fb3

…This allows usage of $push with just one single db call

add tests for new updatedAt behavior

d041687

Merge remote-tracking branch 'origin/main' into feat/$push

ace523b

document performance improvement from setting updatedAt to null

b1836f9

AlessioGr changed the title ~~feat(db-*): support atomic array push db updates~~ feat(db-*): support atomic array $push db updates Aug 22, 2025

AlessioGr disabled auto-merge August 22, 2025 19:52

AlessioGr enabled auto-merge (squash) August 22, 2025 19:52

AlessioGr disabled auto-merge August 22, 2025 20:05

AlessioGr enabled auto-merge (squash) August 22, 2025 20:06

AlessioGr added 2 commits August 22, 2025 14:51

fix

263029e

fix mongo test

73ffa10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(db-*): support atomic array $push db updates #13453

feat(db-*): support atomic array $push db updates #13453

AlessioGr commented Aug 13, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Aug 13, 2025 •

edited

Loading

Meta file: packages/next/meta_index.json, Out file: esbuild/index.js

Meta file: packages/payload/meta_index.json, Out file: esbuild/index.js

Meta file: packages/payload/meta_shared.json, Out file: esbuild/exports/shared.js

Meta file: packages/richtext-lexical/meta_client.json, Out file: esbuild/exports/client_optimized/index.js

Meta file: packages/ui/meta_client.json, Out file: esbuild/exports/client_optimized/index.js

Meta file: packages/ui/meta_shared.json, Out file: esbuild/exports/shared_optimized/index.js

Uh oh!

jmikrut commented Aug 22, 2025

Uh oh!

Uh oh!

feat(db-*): support atomic array $push db updates #13453

Are you sure you want to change the base?

feat(db-*): support atomic array $push db updates #13453

Conversation

AlessioGr commented Aug 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes to updatedAt behavior

Usage Examples

Pushing a single item to an array

Pushing a single item to a localized array

Pushing multiple items to an array

Pushing multiple items to a localized array

Uh oh!

github-actions bot commented Aug 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📦 esbuild Bundle Analysis for payload

Meta file: packages/next/meta_index.json, Out file: esbuild/index.js

Meta file: packages/payload/meta_index.json, Out file: esbuild/index.js

Meta file: packages/payload/meta_shared.json, Out file: esbuild/exports/shared.js

Meta file: packages/richtext-lexical/meta_client.json, Out file: esbuild/exports/client_optimized/index.js

Meta file: packages/ui/meta_client.json, Out file: esbuild/exports/client_optimized/index.js

Meta file: packages/ui/meta_shared.json, Out file: esbuild/exports/shared_optimized/index.js

Uh oh!

jmikrut commented Aug 22, 2025

Uh oh!

Uh oh!

AlessioGr commented Aug 13, 2025 •

edited

Loading

github-actions bot commented Aug 13, 2025 •

edited

Loading