-
Notifications
You must be signed in to change notification settings - Fork 200
feat: migrate sitemap to App Router ISR with automation #367
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
29 commits
Select commit
Hold shift + click to select a range
ad61c0a
feat: migrate to isr generation of sitemap
amaan-bhati d055f06
feat: fix the 502 + fix strictnull checks
amaan-bhati 600e65f
feat: create api server ts, server inly using node:https
amaan-bhati 2650275
feat: add post-deploy ISR sitemap prewarming workflow
amaan-bhati a37b8c2
feat: add missing files — GSC lib, vercel cron, CSP fixes, playwright…
amaan-bhati eef57e5
tests: add e2e tests for ISR sitemap and cron refresh endpoint
amaan-bhati 997fb87
fix: remove public/sitemap.xml — conflicts with app/sitemap.xml/route.ts
amaan-bhati 7a9a9a3
feat: fix and address copilot review#0
amaan-bhati 795b747
fix: harden tests and catch pre-push bugs
amaan-bhati c9db9f4
feat: resolve copilot review#1, notfound page tests
amaan-bhati 177e778
feat: address copilotreviews#2, parseint validate, add request timeout
amaan-bhati e4f1576
feat: fix copilot review#2, fix redirect(/404)
amaan-bhati e8308f9
feat: fix the not fond 404 page, fix nav/header navigation using router
amaan-bhati 296da97
feat: fix copilot review, status code check and navbar pathname regex
amaan-bhati 3774303
feat: address copilot review, get back userouter, update tests accord…
amaan-bhati 5d957a2
feat: address copilot review, fix endcursor guard, fix label for sear…
amaan-bhati 1deebda
feat: address copilot reviews, redirection conflict, update test
amaan-bhati a50e4ec
chore: address copilot review, minor comments
amaan-bhati d4fc5a8
feat: address copilot reviews: update error msg, fix slugs and typesa…
amaan-bhati 398aa4e
feat: address copilot comment, add warning when posts.edge is missing
amaan-bhati 75a3769
chore: address copilot review, add actionable error to avoid moving w…
amaan-bhati e09c1d3
feat: address copilot reviews, header matcher, gsc submission best ef…
amaan-bhati bb7fe32
feat: address copilot reviews, validate posts.pageinfo existence, add…
amaan-bhati 5bee196
feat: address copilot reviews, wrap wp fetches into a short timeout
amaan-bhati 0a9af32
feaT: remove pre warm to avoid extra setups and costs and webhooks
amaan-bhati de08f32
feat: address copilot review, unnecessary stetimeout handle lingering
amaan-bhati 56f271c
feat: address copilot comments: missing pageinfo guards, cursor safet…
amaan-bhati 47f97e1
feat: address copilot review, fix localhost base url
amaan-bhati f7d8700
feat: address copilot review, normalize loopback URLs
amaan-bhati File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| // Required by Next.js App Router. Every app/ directory MUST have a root layout | ||
| // and it MUST include <html> and <body> — Next.js 14 will not inject them | ||
| // automatically, and the App Router runtime crashes on root path without them | ||
| // (causing a blank screen on localhost:3000/). | ||
| // | ||
| // Global CSS is imported here so App Router pages (e.g. not-found.tsx) get | ||
| // Tailwind and site styles. Pages Router routes continue to load CSS via | ||
| // pages/_app.tsx as before — importing the same file twice is fine (deduped). | ||
| import "../styles/index.css"; | ||
|
|
||
| export default function RootLayout({ children }: { children: React.ReactNode }) { | ||
| return ( | ||
| <html lang="en"> | ||
| <body>{children}</body> | ||
| </html> | ||
| ); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,143 @@ | ||
| "use client"; | ||
|
|
||
| import React, { useState, useEffect, useMemo } from "react"; | ||
| import Image from "next/image"; | ||
| import Link from "next/link"; | ||
| import PostGrid from "../components/post-grid"; | ||
| import PostCard from "../components/post-card"; | ||
| import { Post } from "../types/post"; | ||
| import { getExcerpt } from "../utils/excerpt"; | ||
| import { FaSearch } from "react-icons/fa"; | ||
|
|
||
| interface Props { | ||
| latestPosts?: { edges: Array<{ node: Post }> }; | ||
| communityPosts?: { edges: Array<{ node: Post }> }; | ||
| technologyPosts?: { edges: Array<{ node: Post }> }; | ||
| } | ||
|
|
||
| export default function NotFoundClient({ latestPosts, communityPosts, technologyPosts }: Props) { | ||
| const [countdown, setCountdown] = useState(12); | ||
| const [searchTerm, setSearchTerm] = useState(""); | ||
|
|
||
| useEffect(() => { | ||
| const interval = setInterval(() => { | ||
| setCountdown((prev) => { | ||
| if (prev <= 1) { clearInterval(interval); window.location.href = "/blog"; return 0; } | ||
| return prev - 1; | ||
| }); | ||
| }, 1000); | ||
| return () => clearInterval(interval); | ||
| }, []); | ||
|
|
||
| const formatTime = (s: number) => s >= 60 ? `${Math.floor(s / 60)}m ${s % 60}s` : `${s}s`; | ||
|
|
||
| const allPosts = useMemo(() => | ||
| [...(latestPosts?.edges || []), ...(communityPosts?.edges || []), ...(technologyPosts?.edges || [])] | ||
| .filter((p, i, self) => i === self.findIndex((x) => x.node.slug === p.node.slug)), | ||
| [latestPosts, communityPosts, technologyPosts] | ||
| ); | ||
|
|
||
| const filteredPosts = useMemo(() => { | ||
| const t = searchTerm.toLowerCase(); | ||
| return allPosts.filter(({ node }) => | ||
| (node.title || "").toLowerCase().includes(t) || (node.excerpt || "").toLowerCase().includes(t) | ||
| ); | ||
| }, [allPosts, searchTerm]); | ||
|
|
||
| return ( | ||
| <div className="relative min-h-screen bg-gradient-to-br from-orange-50/15 via-orange-25/10 to-orange-100/12 overflow-hidden"> | ||
| <div className="absolute inset-0"> | ||
| <div className="absolute inset-0 bg-[linear-gradient(rgba(255,165,0,0.06)_1px,transparent_1px),linear-gradient(90deg,rgba(255,165,0,0.06)_1px,transparent_1px)] bg-[size:60px_60px]" /> | ||
| <div className="absolute top-20 left-10 w-40 h-40 bg-gradient-to-r from-orange-300/18 to-orange-400/12 rounded-full blur-2xl" /> | ||
| <div className="absolute top-40 right-20 w-32 h-32 bg-gradient-to-r from-orange-400/15 to-red-300/10 rounded-full blur-xl" /> | ||
| <div className="absolute bottom-40 left-1/4 w-48 h-48 bg-gradient-to-r from-orange-200/12 to-yellow-300/8 rounded-full blur-3xl" /> | ||
| <div className="absolute bottom-20 right-1/3 w-36 h-36 bg-gradient-to-r from-orange-300/15 to-orange-500/12 rounded-full blur-2xl" /> | ||
| <div className="absolute top-1/3 right-1/3 w-28 h-28 bg-gradient-to-r from-orange-200/18 to-orange-300/12 rounded-full blur-xl" /> | ||
| <div className="absolute top-32 right-1/4 w-20 h-20 border-2 border-orange-300/25 rotate-45 bg-orange-100/12" /> | ||
| <div className="absolute bottom-32 left-1/3 w-16 h-16 bg-orange-200/18 rounded-full" /> | ||
| <div className="absolute inset-0 bg-gradient-to-br from-orange-100/8 via-transparent to-orange-200/5" /> | ||
| </div> | ||
|
|
||
| <div className="relative z-10 max-w-7xl mx-auto px-4 md:px-8 pt-16"> | ||
| <div className="flex flex-col lg:flex-row items-center justify-between py-16"> | ||
| <div className="flex-1 lg:pr-12 mb-6 lg:mb-0"> | ||
| <h1 className="heading1 text-6xl lg:text-7xl font-bold mb-6 leading-tight max-w-lg"> | ||
| <span className="bg-gradient-to-r from-orange-300 via-orange-400 to-orange-500 bg-clip-text text-transparent"> | ||
| Oops! 404 Not Found... | ||
| </span> | ||
| </h1> | ||
| <p className="body text-lg lg:text-xl text-gray-600 mb-8 leading-relaxed max-w-lg"> | ||
| Looks like you have wandered off the beaten path. Our team is working to get you back on track and find what you are looking for. | ||
| </p> | ||
| <div className="flex flex-col items-start gap-4"> | ||
| <div className="flex flex-col sm:flex-row gap-3"> | ||
| <Link href="/" className="inline-flex items-center justify-center px-6 py-3 rounded-lg border border-gray-300 bg-white text-gray-700 font-semibold hover:bg-gray-50 transition-colors"> | ||
| Back To Home | ||
| </Link> | ||
| <button | ||
| onClick={() => { if (window.history.length > 1) window.history.back(); else window.location.href = "/blog"; }} | ||
| className="px-6 py-3 rounded-lg bg-orange-500 hover:bg-orange-600 text-white font-semibold transition-colors" | ||
| > | ||
| Back to Previous Page | ||
| </button> | ||
| </div> | ||
| <p className="text-sm text-gray-500 italic"> | ||
| Wait for <span className="text-orange-500 font-bold text-lg animate-pulse">{formatTime(countdown)}</span> for automatic redirect or click the buttons above or explore our latest blog posts below. | ||
| </p> | ||
| </div> | ||
| </div> | ||
| <div className="flex-1 lg:pl-12"> | ||
| <div className="w-full h-80 lg:h-[500px] relative"> | ||
| <Image src="/blog/images/error404.png" alt="404 Error Illustration" fill className="object-contain" priority /> | ||
| </div> | ||
| </div> | ||
| </div> | ||
|
|
||
| <div className="pt-4 pb-2"> | ||
| <div className="relative w-full mb-6"> | ||
| <input | ||
| type="text" | ||
| aria-label="Search posts" | ||
| placeholder="Search posts..." | ||
| value={searchTerm} | ||
| onChange={(e) => setSearchTerm(e.target.value)} | ||
| className="w-full p-4 pl-10 rounded-full border border-gray-300 focus:outline-none focus:ring-2 focus:ring-blue-500 focus:border-blue-500" | ||
| /> | ||
| <FaSearch className="absolute left-3 top-1/2 -translate-y-1/2 text-gray-500" /> | ||
| </div> | ||
| </div> | ||
|
|
||
| {searchTerm ? ( | ||
| <section className="py-4"> | ||
| <h2 className="bg-gradient-to-r from-orange-200 to-orange-100 bg-[length:100%_20px] bg-no-repeat bg-left-bottom w-max mb-8 text-4xl heading1 md:text-4xl font-bold tracking-tighter leading-tight">Search Results</h2> | ||
| {filteredPosts.length === 0 | ||
| ? <p className="text-center text-gray-500">No posts found matching that search</p> | ||
| : <PostGrid>{filteredPosts.map(({ node: post }) => <PostCard key={post.slug} title={post.title} coverImage={post.featuredImage} date={post.date} excerpt={getExcerpt(post.excerpt, 20)} author={post.ppmaAuthorName} slug={post.slug} isCommunity={post.categories?.edges?.some((e) => e.node.name === "community")} />)}</PostGrid> | ||
| } | ||
| </section> | ||
| ) : ( | ||
| <> | ||
| {(latestPosts?.edges?.length ?? 0) > 0 && ( | ||
| <section className="py-4"> | ||
| <h2 className="bg-gradient-to-r from-orange-200 to-orange-100 bg-[length:100%_20px] bg-no-repeat bg-left-bottom w-max mb-8 text-4xl heading1 md:text-4xl font-bold tracking-tighter leading-tight">Latest from Our Blog</h2> | ||
| <PostGrid>{(latestPosts?.edges ?? []).slice(0, 6).map(({ node: post }) => <PostCard key={post.slug} title={post.title} coverImage={post.featuredImage} date={post.date} excerpt={getExcerpt(post.excerpt, 20)} author={post.ppmaAuthorName} slug={post.slug} isCommunity={post.categories?.edges?.some((e) => e.node.name === "community")} />)}</PostGrid> | ||
| </section> | ||
| )} | ||
| {(communityPosts?.edges?.length ?? 0) > 0 && ( | ||
| <section className="py-4"> | ||
| <h2 className="bg-gradient-to-r from-orange-200 to-orange-100 bg-[length:100%_20px] bg-no-repeat bg-left-bottom w-max mb-8 text-4xl heading1 md:text-4xl font-bold tracking-tighter leading-tight">Latest Community Blogs</h2> | ||
| <PostGrid>{(communityPosts?.edges ?? []).slice(0, 6).map(({ node: post }) => <PostCard key={post.slug} title={post.title} coverImage={post.featuredImage} date={post.date} excerpt={getExcerpt(post.excerpt, 20)} author={post.ppmaAuthorName} slug={post.slug} isCommunity={true} />)}</PostGrid> | ||
| </section> | ||
| )} | ||
| {(technologyPosts?.edges?.length ?? 0) > 0 && ( | ||
| <section className="py-4"> | ||
| <h2 className="bg-gradient-to-r from-orange-200 to-orange-100 bg-[length:100%_20px] bg-no-repeat bg-left-bottom w-max mb-8 text-4xl heading1 md:text-4xl font-bold tracking-tighter leading-tight">Latest Technology Blogs</h2> | ||
| <PostGrid>{(technologyPosts?.edges ?? []).slice(0, 6).map(({ node: post }) => <PostCard key={post.slug} title={post.title} coverImage={post.featuredImage} date={post.date} excerpt={getExcerpt(post.excerpt, 20)} author={post.ppmaAuthorName} slug={post.slug} isCommunity={false} />)}</PostGrid> | ||
| </section> | ||
| )} | ||
| </> | ||
| )} | ||
| </div> | ||
| </div> | ||
| ); | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,58 @@ | ||
| // Required when app/ directory is present alongside pages/. | ||
| // Without this file, any unmatched App Router path shows a blank | ||
| // "__next_error__" shell instead of a usable page. | ||
| // | ||
| // This file renders a standalone 404 UI (NotFoundClient) — it does NOT redirect. | ||
| // redirect() cannot be used here: it is an RSC soft-navigation, not a hard HTTP | ||
| // redirect. Since /404 has no App Router page, calling redirect('/404') causes | ||
| // the router to invoke this boundary again → infinite loop. | ||
|
amaan-bhati marked this conversation as resolved.
|
||
| // | ||
| // Header/FloatingNavbar/NotFoundPage are intentionally not imported here because | ||
| // they use useRouter() from next/router, which throws "NextRouter was not mounted" | ||
| // in App Router context. Pages Router components are left completely unchanged. | ||
| import { getAllPostsForTechnology, getAllPostsForCommunity } from "../lib/api"; | ||
| import NotFoundClient from "./not-found-client"; | ||
|
|
||
| function withTimeout<T>(promise: Promise<T>, timeoutMs: number): Promise<T> { | ||
| let timeoutId: ReturnType<typeof setTimeout> | undefined; | ||
|
|
||
| const timeoutPromise = new Promise<T>((_, reject) => { | ||
| timeoutId = setTimeout(() => { | ||
| reject(new Error(`NotFound posts fetch timed out after ${timeoutMs}ms`)); | ||
| }, timeoutMs); | ||
| }); | ||
|
|
||
| return Promise.race([promise, timeoutPromise]).finally(() => { | ||
| if (timeoutId !== undefined) clearTimeout(timeoutId); | ||
| }); | ||
| } | ||
|
|
||
| export default async function NotFound() { | ||
| let latestPosts = { edges: [] as any[] }; | ||
| let communityPosts = { edges: [] as any[] }; | ||
|
amaan-bhati marked this conversation as resolved.
|
||
| let technologyPosts = { edges: [] as any[] }; | ||
|
|
||
| try { | ||
| // Posts are decorative; keep the 404 UI fast even if WordPress is slow/hung. | ||
| const [techPosts, commPosts] = await withTimeout(Promise.all([ | ||
| getAllPostsForTechnology(false, null), | ||
| getAllPostsForCommunity(false, null), | ||
| ]), 3000); | ||
| const allEdges = [...techPosts.edges, ...commPosts.edges].sort( | ||
| (a, b) => new Date(b.node.date).getTime() - new Date(a.node.date).getTime() | ||
| ); | ||
| latestPosts = { edges: allEdges.slice(0, 6) }; | ||
| technologyPosts = { edges: techPosts.edges.slice(0, 6) }; | ||
| communityPosts = { edges: commPosts.edges.slice(0, 6) }; | ||
| } catch { | ||
| // Posts are decorative — render without them rather than blank screen. | ||
| } | ||
|
amaan-bhati marked this conversation as resolved.
|
||
|
|
||
| return ( | ||
| <NotFoundClient | ||
| latestPosts={latestPosts} | ||
| communityPosts={communityPosts} | ||
| technologyPosts={technologyPosts} | ||
| /> | ||
| ); | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,78 @@ | ||
| import { getAllPostsForSitemap } from "../../lib/api-server"; | ||
| import { | ||
| adaptPostsForSitemap, | ||
| assertFullSitemap, | ||
| buildAuthorEntries, | ||
| buildPostEntries, | ||
| buildTagEntries, | ||
| dedupeEntries, | ||
| getLatestModified, | ||
| getStaticFallbackXml, | ||
| serializeSitemap, | ||
| STATIC_ROUTES, | ||
| } from "../../lib/sitemap"; | ||
|
|
||
| // ISR: Vercel caches this response in its CDN edge cache for 1 hour. | ||
| // | ||
| // After first generation: every request is served from CDN (<10ms, no Lambda invoked). | ||
| // After TTL expires: stale version served immediately, regeneration happens in background. | ||
| // If WordPress is down during regen: Vercel keeps serving the previous good version automatically. | ||
| export const revalidate = 3600; | ||
|
|
||
| export async function GET(): Promise<Response> { | ||
| try { | ||
| // getAllPostsForSitemap() uses node:https directly (see lib/api-server.ts) to | ||
| // bypass Next.js App Router's RSC fetch instrumentation, which causes Cloudflare | ||
| // to return 502 HTML when global fetch() is used in this context. | ||
| const allPostsResult = await getAllPostsForSitemap(); | ||
| const posts = adaptPostsForSitemap(allPostsResult); | ||
|
|
||
| // Reject partial WordPress responses — ISR will not cache a thrown error, | ||
| // so Vercel keeps serving the previous good cached version automatically. | ||
| assertFullSitemap(posts); | ||
|
|
||
| // Static routes get lastmod = newest post modification time, | ||
| // so listing pages reflect when the freshest underlying content changed. | ||
| const latestModified = getLatestModified(posts) ?? new Date().toISOString(); | ||
| const staticEntries = STATIC_ROUTES.map((r) => ({ | ||
| ...r, | ||
| lastModified: latestModified, | ||
| })); | ||
|
|
||
| const entries = dedupeEntries([ | ||
| ...staticEntries, | ||
| ...buildPostEntries(posts), | ||
| ...buildAuthorEntries(posts), | ||
| ...buildTagEntries(posts), | ||
| ]); | ||
|
|
||
| const xml = serializeSitemap(entries); | ||
|
|
||
| return new Response(xml, { | ||
| status: 200, | ||
| headers: { | ||
| "Content-Type": "application/xml", | ||
| // s-maxage: CDN caches for 1h. stale-while-revalidate: serve stale while | ||
| // regenerating in background. max-age=0: browsers always revalidate with CDN. | ||
| "Cache-Control": "public, max-age=0, s-maxage=3600, stale-while-revalidate=3600", | ||
| }, | ||
| }); | ||
| } catch (error) { | ||
| console.error( | ||
| "Sitemap generation failed — serving static-routes-only fallback. " + | ||
| "Verify WORDPRESS_API_URL is reachable and WPGraphQL is responding.", | ||
| error | ||
| ); | ||
|
|
||
| // ISR does NOT cache non-2xx responses. | ||
| // no-store prevents any downstream proxy from caching this degraded response, | ||
| // so crawlers will retry on the next request once WordPress is back. | ||
| return new Response(getStaticFallbackXml(), { | ||
| status: 503, | ||
| headers: { | ||
| "Content-Type": "application/xml", | ||
| "Cache-Control": "no-store", | ||
| }, | ||
| }); | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.