Skip to content

fix: handle --youtube apify when HTML fetch fails #64

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
entropyy0 wants to merge 1 commit into
base: main
Choose a base branch
from
Open

fix: handle --youtube apify when HTML fetch fails #64

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 31 additions & 8 deletions packages/core/src/content/transcript/providers/youtube.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,36 @@ export const fetchTranscript = async (
)
}

// Handle explicit apify mode before HTML check — Apify doesn't need HTML.
// Fixes: when HTML fetch fails, explicit --youtube apify was skipped entirely.
if (mode === 'apify') {
if (!options.apifyApiToken) {
throw new Error('Missing APIFY_API_TOKEN for --youtube apify')
}
pushHint('YouTube: fetching transcript (Apify)')
attemptedProviders.push('apify')
const apifyTranscript = await fetchTranscriptWithApify(
options.fetch,
options.apifyApiToken,
url
)
if (apifyTranscript) {
return {
text: normalizeTranscriptText(apifyTranscript),
source: 'apify',
metadata: { provider: 'apify' },
attemptedProviders,
}
}
attemptedProviders.push('unavailable')
return {
text: null,
source: 'unavailable',
metadata: { provider: 'youtube', reason: 'no_transcript_available' },
attemptedProviders,
}
}

if (!html) {
return { text: null, source: null, attemptedProviders }
}
Expand Down Expand Up @@ -249,14 +279,7 @@ export const fetchTranscript = async (
}
}

// Explicit apify mode: allow forcing it, but require a token.
if (mode === 'apify') {
if (!options.apifyApiToken) {
throw new Error('Missing APIFY_API_TOKEN for --youtube apify')
}
const apifyResult = await tryApify('YouTube: fetching transcript (Apify)')
if (apifyResult) return apifyResult
}
// Note: explicit apify mode is handled before the HTML check (above).

// Auto mode: if yt-dlp cannot run (no binary/credentials), fall back to Apify last-last.
if (mode === 'auto' && !canRunYtDlp) {
Expand Down
62 changes: 62 additions & 0 deletions tests/transcript.youtube-provider.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,68 @@ describe('YouTube transcript provider module', () => {
).toEqual({ text: null, source: null, attemptedProviders: [] })
})

it('uses apify mode even when HTML is null (fixes #51)', async () => {
apify.fetchTranscriptWithApify.mockResolvedValue('Hello from apify')

const result = await fetchTranscript(
{
url: 'https://www.youtube.com/watch?v=abcdefghijk',
html: null,
resourceKey: null,
},
{
...baseOptions,
apifyApiToken: 'TOKEN',
youtubeTranscriptMode: 'apify',
}
)

expect(result.text).toBe('Hello from apify')
expect(result.source).toBe('apify')
expect(result.attemptedProviders).toEqual(['apify'])
expect(api.extractYoutubeiTranscriptConfig).not.toHaveBeenCalled()
expect(captions.fetchTranscriptFromCaptionTracks).not.toHaveBeenCalled()
expect(ytdlp.fetchTranscriptWithYtDlp).not.toHaveBeenCalled()
})

it('returns unavailable when apify mode fails with null HTML', async () => {
apify.fetchTranscriptWithApify.mockResolvedValue(null)

const result = await fetchTranscript(
{
url: 'https://www.youtube.com/watch?v=abcdefghijk',
html: null,
resourceKey: null,
},
{
...baseOptions,
apifyApiToken: 'TOKEN',
youtubeTranscriptMode: 'apify',
}
)

expect(result.text).toBeNull()
expect(result.source).toBe('unavailable')
expect(result.attemptedProviders).toEqual(['apify', 'unavailable'])
})

it('throws when apify mode used without token and HTML is null', async () => {
await expect(
fetchTranscript(
{
url: 'https://www.youtube.com/watch?v=abcdefghijk',
html: null,
resourceKey: null,
},
{
...baseOptions,
apifyApiToken: null,
youtubeTranscriptMode: 'apify',
}
)
).rejects.toThrow(/Missing APIFY_API_TOKEN/i)
})

it('uses apify-only mode and skips web + yt-dlp', async () => {
apify.fetchTranscriptWithApify.mockResolvedValue('Hello from apify')

Expand Down
Loading