Skip to content

Commit 9148fc9

Browse files
ebefarooquielylucasctflelylucas
authored
feat: add dedicated methods for cursor based pagination [CAPI-2357] (#2824)
* feat: add dedicated getManyWithCursor and getPublishedWithCursor methods for entry, asset, and content-type [CAPI-2357] Adds getManyWithCursor as a dedicated cursor based method for entry, asset, and content-type and getPublishedWithCursor for only the asset and entry entities * fix: removing getPublishedWithCursor methods, normalizing pagination response and params and building utilities to do so [CAPI-2357] * fix: getPublished does not currently support cursor based pagination, removing relevant methods for entities * fix: normalize pagination params to filter prevPage and prevNext if falsey * fix: normalize pagination response to parse next, prev tokens if present * test: add unit tests for create-environment-api and integration tests for getManyWithCursor for content-type, asset, and entry entities [CAPI-2357] * chore: updating README.md to reflect new cursor pagination methods [CAPI-2357] * chore: fix linting errors in new test suites and lib/ * chore: removing unused console.debug line [CAPI-2357] * fix: enforcing no skip param for query options, removing uneccessary console statements, adding cbp to toc in readme [CAPI-2357] * chore: update readme badge and change to master branch in workflow (#2819) Co-authored-by: Ely Lucas <ely.lucas@contentful.com> * chore: fixing linting errors in asset-integration-test [CAPI-2357] * fix: fixing typing to include CursorBasedParams not QueryParams for getManyWithCursor methods [CAPI-2357] * fix: infer typing for cursor pagination collection method per entity [CAPI-2357] * chore: updating param comment to reflect appropriate url, adding pagination example to readme [CAPI-2357] --------- Co-authored-by: Ely Lucas <ely.lucas+github@contentful.com> Co-authored-by: Ely Lucas <ely.lucas@contentful.com>
1 parent bfbd4a1 commit 9148fc9

15 files changed

+561
-20
lines changed

‎README.md‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,7 @@
5959
- [Configuration](#configuration)
6060
- [Reference Documentation](#reference-documentation)
6161
- [Contentful Javascript resources](#contentful-javascript-resources)
62+
- [Cursor Based Pagination](#cursor-based-pagination)
6263
- [REST API reference](#rest-api-reference)
6364
- [Versioning](#versioning)
6465
- [Reach out to us](#reach-out-to-us)
@@ -226,6 +227,25 @@ The benefits of using the "plain" version of the client, over the legacy version
226227
- The ability to scope CMA client instance to a specific `spaceId`, `environmentId`, and `organizationId` when initializing the client.
227228
- You can pass a concrete values to `defaults` and omit specifying these params in actual CMA methods calls.
228229

230+
## Cursor Based Pagination
231+
232+
Cursor-based pagination is supported on collection endpoints for content types, entries, and assets. To use cursor-based pagination, use the related entity methods `getAssetsWithCursor`, `getContentTypesWithCursor`, and `getEntriesWithCursor`
233+
234+
```js
235+
const response = await environment.getEntriesWithCursor({ limit: 10 });
236+
console.log(response.items); // Array of items
237+
console.log(response.pages?.next); // Cursor for next page
238+
```
239+
Use the value from `response.pages.next` to fetch the next page.
240+
241+
```js
242+
const secondPage = await environment.getEntriesWithCursor({
243+
limit: 2,
244+
pageNext: response.pages?.next,
245+
});
246+
console.log(secondPage.items); // Array of items
247+
```
248+
229249
## Legacy Client Interface
230250
231251
The following code snippet is an example of the legacy client interface, which reads and writes data as a sequence of nested requests:

‎lib/common-types.ts‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -359,6 +359,7 @@ export interface BasicQueryOptions {
359359
}
360360

361361
export interface BasicCursorPaginationOptions extends Omit<BasicQueryOptions, 'skip'> {
362+
skip?: never
362363
pageNext?: string
363364
pagePrev?: string
364365
}
@@ -487,6 +488,7 @@ type MRInternal<UA extends boolean> = {
487488
): MRReturn<'AppInstallation', 'getForOrganization'>
488489

489490
(opts: MROpts<'Asset', 'getMany', UA>): MRReturn<'Asset', 'getMany'>
491+
(opts: MROpts<'Asset', 'getManyWithCursor', UA>): MRReturn<'Asset', 'getManyWithCursor'>
490492
(opts: MROpts<'Asset', 'getPublished', UA>): MRReturn<'Asset', 'getPublished'>
491493
(opts: MROpts<'Asset', 'get', UA>): MRReturn<'Asset', 'get'>
492494
(opts: MROpts<'Asset', 'update', UA>): MRReturn<'Asset', 'update'>
@@ -567,6 +569,9 @@ type MRInternal<UA extends boolean> = {
567569

568570
(opts: MROpts<'ContentType', 'get', UA>): MRReturn<'ContentType', 'get'>
569571
(opts: MROpts<'ContentType', 'getMany', UA>): MRReturn<'ContentType', 'getMany'>
572+
(
573+
opts: MROpts<'ContentType', 'getManyWithCursor', UA>,
574+
): MRReturn<'ContentType', 'getManyWithCursor'>
570575
(opts: MROpts<'ContentType', 'update', UA>): MRReturn<'ContentType', 'update'>
571576
(opts: MROpts<'ContentType', 'create', UA>): MRReturn<'ContentType', 'create'>
572577
(opts: MROpts<'ContentType', 'createWithId', UA>): MRReturn<'ContentType', 'createWithId'>
@@ -616,6 +621,7 @@ type MRInternal<UA extends boolean> = {
616621
): MRReturn<'EnvironmentTemplateInstallation', 'getForEnvironment'>
617622

618623
(opts: MROpts<'Entry', 'getMany', UA>): MRReturn<'Entry', 'getMany'>
624+
(opts: MROpts<'Entry', 'getManyWithCursor', UA>): MRReturn<'Entry', 'getManyWithCursor'>
619625
(opts: MROpts<'Entry', 'getPublished', UA>): MRReturn<'Entry', 'getPublished'>
620626
(opts: MROpts<'Entry', 'get', UA>): MRReturn<'Entry', 'get'>
621627
(opts: MROpts<'Entry', 'patch', UA>): MRReturn<'Entry', 'patch'>
@@ -1234,6 +1240,11 @@ export type MRActions = {
12341240
headers?: RawAxiosRequestHeaders
12351241
return: CollectionProp<AssetProps>
12361242
}
1243+
getManyWithCursor: {
1244+
params: GetSpaceEnvironmentParams & CursorBasedParams & { releaseId?: string }
1245+
headers?: RawAxiosRequestHeaders
1246+
return: CursorPaginatedCollectionProp<AssetProps>
1247+
}
12371248
get: {
12381249
params: GetSpaceEnvironmentParams & { assetId: string; releaseId?: string } & QueryParams
12391250
headers?: RawAxiosRequestHeaders
@@ -1482,6 +1493,10 @@ export type MRActions = {
14821493
params: GetSpaceEnvironmentParams & QueryParams
14831494
return: CollectionProp<ContentTypeProps>
14841495
}
1496+
getManyWithCursor: {
1497+
params: GetSpaceEnvironmentParams & CursorBasedParams
1498+
return: CursorPaginatedCollectionProp<ContentTypeProps>
1499+
}
14851500
create: {
14861501
params: GetSpaceEnvironmentParams
14871502
payload: CreateContentTypeProps
@@ -1650,6 +1665,10 @@ export type MRActions = {
16501665
params: GetSpaceEnvironmentParams & QueryParams & { releaseId?: string }
16511666
return: CollectionProp<EntryProps<any>>
16521667
}
1668+
getManyWithCursor: {
1669+
params: GetSpaceEnvironmentParams & CursorBasedParams & { releaseId?: string }
1670+
return: CursorPaginatedCollectionProp<EntryProps<any>>
1671+
}
16531672
get: {
16541673
params: GetSpaceEnvironmentParams & { entryId: string; releaseId?: string } & QueryParams
16551674
return: EntryProps<any>

‎lib/common-utils.ts‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,10 @@
33
import { toPlainObject } from 'contentful-sdk-core'
44
import copy from 'fast-copy'
55
import type {
6+
BasicCursorPaginationOptions,
67
Collection,
78
CollectionProp,
9+
CursorBasedParams,
810
CursorPaginatedCollection,
911
CursorPaginatedCollectionProp,
1012
MakeRequest,
@@ -47,3 +49,50 @@ export function shouldRePoll(statusCode: number) {
4749
export async function waitFor(ms = 1000) {
4850
return new Promise((resolve) => setTimeout(resolve, ms))
4951
}
52+
53+
export function normalizeCursorPaginationParameters(
54+
query: BasicCursorPaginationOptions,
55+
): CursorBasedParams {
56+
const { pagePrev, pageNext, ...rest } = query
57+
58+
return {
59+
...rest,
60+
cursor: true,
61+
// omit pagePrev and pageNext if the value is falsy
62+
...(pagePrev ? { pagePrev } : null),
63+
...(pageNext ? { pageNext } : null),
64+
} as CursorBasedParams
65+
}
66+
67+
function extractQueryParam(key: string, url?: string): string | undefined {
68+
if (!url) return
69+
70+
const queryIndex = url.indexOf('?')
71+
if (queryIndex === -1) return
72+
73+
const queryString = url.slice(queryIndex + 1)
74+
return new URLSearchParams(queryString).get(key) ?? undefined
75+
}
76+
77+
const Pages = {
78+
prev: 'pagePrev',
79+
next: 'pageNext',
80+
} as const
81+
82+
const PAGE_KEYS = ['prev', 'next'] as const
83+
84+
export function normalizeCursorPaginationResponse<T>(
85+
data: CursorPaginatedCollectionProp<T>,
86+
): CursorPaginatedCollectionProp<T> {
87+
const pages: { prev?: string; next?: string } = {}
88+
89+
for (const key of PAGE_KEYS) {
90+
const token = extractQueryParam(Pages[key], data.pages?.[key])
91+
if (token) pages[key] = token
92+
}
93+
94+
return {
95+
...data,
96+
pages,
97+
}
98+
}

‎lib/create-environment-api.ts‎

Lines changed: 137 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,10 @@ import type {
77
CursorBasedParams,
88
QueryOptions,
99
} from './common-types'
10+
import {
11+
normalizeCursorPaginationParameters,
12+
normalizeCursorPaginationResponse,
13+
} from './common-utils'
1014
import type { BasicQueryOptions, MakeRequest } from './common-types'
1115
import entities from './entities'
1216
import type { CreateAppInstallationProps } from './entities/app-installation'
@@ -15,11 +19,11 @@ import type {
1519
CreateAppActionCallProps,
1620
AppActionCallRawResponseProps,
1721
} from './entities/app-action-call'
18-
import type {
19-
AssetFileProp,
20-
AssetProps,
21-
CreateAssetFromFilesOptions,
22-
CreateAssetProps,
22+
import {
23+
type AssetFileProp,
24+
type AssetProps,
25+
type CreateAssetFromFilesOptions,
26+
type CreateAssetProps,
2327
} from './entities/asset'
2428
import type { CreateAssetKeyProps } from './entities/asset-key'
2529
import type {
@@ -40,12 +44,12 @@ import type {
4044
} from './entities/release'
4145
import { wrapRelease, wrapReleaseCollection } from './entities/release'
4246

43-
import type { ContentTypeProps, CreateContentTypeProps } from './entities/content-type'
44-
import type {
45-
CreateEntryProps,
46-
EntryProps,
47-
EntryReferenceOptionsProps,
48-
EntryReferenceProps,
47+
import { type ContentTypeProps, type CreateContentTypeProps } from './entities/content-type'
48+
import {
49+
type CreateEntryProps,
50+
type EntryProps,
51+
type EntryReferenceOptionsProps,
52+
type EntryReferenceProps,
4953
} from './entities/entry'
5054
import type { EnvironmentProps } from './entities/environment'
5155
import type { CreateExtensionProps } from './entities/extension'
@@ -75,9 +79,10 @@ export type ContentfulEnvironmentAPI = ReturnType<typeof createEnvironmentApi>
7579
*/
7680
export default function createEnvironmentApi(makeRequest: MakeRequest) {
7781
const { wrapEnvironment } = entities.environment
78-
const { wrapContentType, wrapContentTypeCollection } = entities.contentType
79-
const { wrapEntry, wrapEntryCollection } = entities.entry
80-
const { wrapAsset, wrapAssetCollection } = entities.asset
82+
const { wrapContentType, wrapContentTypeCollection, wrapContentTypeCursorPaginatedCollection } =
83+
entities.contentType
84+
const { wrapEntry, wrapEntryCollection, wrapEntryTypeCursorPaginatedCollection } = entities.entry
85+
const { wrapAsset, wrapAssetCollection, wrapAssetTypeCursorPaginatedCollection } = entities.asset
8186
const { wrapAssetKey } = entities.assetKey
8287
const { wrapLocale, wrapLocaleCollection } = entities.locale
8388
const { wrapSnapshotCollection } = entities.snapshot
@@ -492,6 +497,44 @@ export default function createEnvironmentApi(makeRequest: MakeRequest) {
492497
},
493498
}).then((data) => wrapContentTypeCollection(makeRequest, data))
494499
},
500+
501+
/**
502+
* Gets a collection of Content Types with cursor based pagination
503+
* @param query - Object with cursor pagination parameters. Check the <a href="https://www.contentful.com/developers/docs/references/content-management-api/#/introduction/cursor-pagination">REST API reference</a> for more details.
504+
* @return Promise for a collection of Content Types
505+
* @example ```javascript
506+
* const contentful = require('contentful-management')
507+
*
508+
* const client = contentful.createClient({
509+
* accessToken: '<content_management_api_key>'
510+
* })
511+
*
512+
* client.getSpace('<space_id>')
513+
* .then((space) => space.getEnvironment('<environment-id>'))
514+
* .then((environment) => environment.getContentTypesWithCursor())
515+
* .then((response) => console.log(response.items))
516+
* .catch(console.error)
517+
* ```
518+
*/
519+
getContentTypesWithCursor(query: BasicCursorPaginationOptions = {}) {
520+
const raw = this.toPlainObject() as EnvironmentProps
521+
const normalizedQueryParams = normalizeCursorPaginationParameters(query)
522+
return makeRequest({
523+
entityType: 'ContentType',
524+
action: 'getMany',
525+
params: {
526+
spaceId: raw.sys.space.sys.id,
527+
environmentId: raw.sys.id,
528+
query: createRequestConfig({ query: normalizedQueryParams }).params,
529+
},
530+
}).then((data) =>
531+
wrapContentTypeCursorPaginatedCollection(
532+
makeRequest,
533+
normalizeCursorPaginationResponse(data),
534+
),
535+
)
536+
},
537+
495538
/**
496539
* Creates a Content Type
497540
* @param data - Object representation of the Content Type to be created
@@ -740,6 +783,45 @@ export default function createEnvironmentApi(makeRequest: MakeRequest) {
740783
}).then((data) => wrapEntryCollection(makeRequest, data))
741784
},
742785

786+
/**
787+
* Gets a collection of Entries with cursor based pagination
788+
* Warning: if you are using the select operator, when saving, any field that was not selected will be removed
789+
* from your entry in the backend
790+
* @param query - Object with cursor pagination parameters. Check the <a href="https://www.contentful.com/developers/docs/references/content-management-api/#/introduction/cursor-pagination">REST API reference</a> for more details.
791+
* @return Promise for a collection of Entries
792+
* @example ```javascript
793+
* const contentful = require('contentful-management')
794+
*
795+
* const client = contentful.createClient({
796+
* accessToken: '<content_management_api_key>'
797+
* })
798+
*
799+
* client.getSpace('<space_id>')
800+
* .then((space) => space.getEnvironment('<environment-id>'))
801+
* .then((environment) => environment.getEntriesWithCursor({'content_type': 'foo'})) // you can add more queries as 'key': 'value'
802+
* .then((response) => console.log(response.items))
803+
* .catch(console.error)
804+
* ```
805+
*/
806+
getEntriesWithCursor(query: BasicCursorPaginationOptions = {}) {
807+
const raw = this.toPlainObject() as EnvironmentProps
808+
const normalizedQueryParams = normalizeCursorPaginationParameters(query)
809+
return makeRequest({
810+
entityType: 'Entry',
811+
action: 'getMany',
812+
params: {
813+
spaceId: raw.sys.space.sys.id,
814+
environmentId: raw.sys.id,
815+
query: createRequestConfig({ query: normalizedQueryParams }).params,
816+
},
817+
}).then((data) =>
818+
wrapEntryTypeCursorPaginatedCollection(
819+
makeRequest,
820+
normalizeCursorPaginationResponse(data),
821+
),
822+
)
823+
},
824+
743825
/**
744826
* Gets a collection of published Entries
745827
* @param query - Object with search parameters. Check the <a href="https://www.contentful.com/developers/docs/javascript/tutorials/using-js-cda-sdk/#retrieving-entries-with-search-parameters">JS SDK tutorial</a> and the <a href="https://www.contentful.com/developers/docs/references/content-delivery-api/#/reference/search-parameters">REST API reference</a> for more details.
@@ -955,6 +1037,46 @@ export default function createEnvironmentApi(makeRequest: MakeRequest) {
9551037
},
9561038
}).then((data) => wrapAssetCollection(makeRequest, data))
9571039
},
1040+
1041+
/**
1042+
* Gets a collection of Assets with cursor based pagination
1043+
* Warning: if you are using the select operator, when saving, any field that was not selected will be removed
1044+
* from your entry in the backend
1045+
* @param query - Object with cursor pagination parameters. Check the <a href="https://www.contentful.com/developers/docs/references/content-management-api/#/introduction/cursor-pagination">REST API reference</a> for more details.
1046+
* @return Promise for a collection of Assets
1047+
* @example ```javascript
1048+
* const contentful = require('contentful-management')
1049+
*
1050+
* const client = contentful.createClient({
1051+
* accessToken: '<content_management_api_key>'
1052+
* })
1053+
*
1054+
* client.getSpace('<space_id>')
1055+
* .then((space) => space.getEnvironment('<environment-id>'))
1056+
* .then((environment) => environment.getAssetsWithCursor())
1057+
* .then((response) => console.log(response.items))
1058+
* .catch(console.error)
1059+
* ```
1060+
*/
1061+
getAssetsWithCursor(query: BasicCursorPaginationOptions = {}) {
1062+
const raw = this.toPlainObject() as EnvironmentProps
1063+
const normalizedQueryParams = normalizeCursorPaginationParameters(query)
1064+
return makeRequest({
1065+
entityType: 'Asset',
1066+
action: 'getMany',
1067+
params: {
1068+
spaceId: raw.sys.space.sys.id,
1069+
environmentId: raw.sys.id,
1070+
query: createRequestConfig({ query: normalizedQueryParams }).params,
1071+
},
1072+
}).then((data) =>
1073+
wrapAssetTypeCursorPaginatedCollection(
1074+
makeRequest,
1075+
normalizeCursorPaginationResponse(data),
1076+
),
1077+
)
1078+
},
1079+
9581080
/**
9591081
* Gets a collection of published Assets
9601082
* @param query - Object with search parameters. Check the <a href="https://www.contentful.com/developers/docs/javascript/tutorials/using-js-cda-sdk/#retrieving-entries-with-search-parameters">JS SDK tutorial</a> and the <a href="https://www.contentful.com/developers/docs/references/content-delivery-api/#/reference/search-parameters">REST API reference</a> for more details.
@@ -985,6 +1107,7 @@ export default function createEnvironmentApi(makeRequest: MakeRequest) {
9851107
},
9861108
}).then((data) => wrapAssetCollection(makeRequest, data))
9871109
},
1110+
9881111
/**
9891112
* Creates a Asset. After creation, call asset.processForLocale or asset.processForAllLocales to start asset processing.
9901113
* @param data - Object representation of the Asset to be created. Note that the field object should have an upload property on asset creation, which will be removed and replaced with an url property when processing is finished.

‎lib/entities/asset.ts‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,9 @@ import type {
88
EntityMetaSysProps,
99
MetadataProps,
1010
MakeRequest,
11+
CursorPaginatedCollectionProp,
1112
} from '../common-types'
12-
import { wrapCollection } from '../common-utils'
13+
import { wrapCollection, wrapCursorPaginatedCollection } from '../common-utils'
1314
import * as checks from '../plain/checks'
1415

1516
export type AssetProps<S = {}> = {
@@ -410,3 +411,8 @@ export function wrapAsset(makeRequest: MakeRequest, data: AssetProps): Asset {
410411
* @private
411412
*/
412413
export const wrapAssetCollection = wrapCollection(wrapAsset)
414+
415+
/**
416+
* @private
417+
*/
418+
export const wrapAssetTypeCursorPaginatedCollection = wrapCursorPaginatedCollection(wrapAsset)

0 commit comments

Comments
 (0)