| 123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296 |
- /**
- * 音频生成扫描器
- *
- * 作为 fire-and-forget 触发方式的兜底保障:
- * - 定时扫描 content_completed 但无音频的章节,自动入队
- * - 发现 audio_generating 卡死(有 audioUrl 但 genStage 没推进)→ 自动修复
- * - 统计遗漏修复数量,方便监控告警
- *
- * 设计原则:
- * 1. 纯兜底,不取代 fire-and-forget(保持实时性)
- * 2. 幂等安全:已处理的章节不会重复入队
- * 3. 可观测:每次扫描输出统计摘要
- */
- import { prisma } from '../../models';
- import { bookStore } from './book-generator.store';
- import { tryAutoMerge } from './book-generator.store';
- // ============ 配置 ============
- const SCAN_INTERVAL_MS = 30_000; // 扫描间隔 30 秒
- const LOG_STATS_INTERVAL = 10; // 每 10 轮输出一次统计摘要
- // ============ 统计 ============
- interface ScanStats {
- totalScans: number;
- foundMissing: number; // 发现 content_completed 但无音频的章节
- enqueuedSuccess: number; // 成功入队
- enqueuedFailed: number; // 入队失败
- fixedStuck: number; // 修复了卡死状态(有 audioUrl 但 genStage 不对)
- lastScanTime: number;
- }
- const stats: ScanStats = {
- totalScans: 0,
- foundMissing: 0,
- enqueuedSuccess: 0,
- enqueuedFailed: 0,
- fixedStuck: 0,
- lastScanTime: 0,
- };
- // ============ 核心扫描 ============
- /**
- * 额外扫描:检查叶节点音频完成但1级章节缺少合并音频的情况
- * 修复 advanceChapter 失败导致 tryAutoMerge 被跳过的问题
- */
- async function scanAndTryAutoMerge(): Promise<void> {
- try {
- // 找到所有 audio_completed 的叶节点(level >= 2),其所属1级章节没有合并音频
- const leafNodesWithAudio = await prisma.bookChapter.findMany({
- where: {
- genStage: 'audio_completed',
- audioUrl: { not: '' },
- level: { gte: 2 },
- },
- select: {
- id: true,
- bookId: true,
- level: true,
- parentId: true,
- },
- take: 50,
- });
- // 按 parentId (level=2) 或 parentId→parentId (level=3) 分组找到1级章节
- // 为避免复杂查询,直接对每个叶节点调用 tryAutoMerge(内部会判断是否所有兄弟节点都已就绪)
- for (const leaf of leafNodesWithAudio) {
- tryAutoMerge(leaf.id, leaf.bookId, leaf.level, leaf.parentId).catch(() => {});
- }
- } catch (err: any) {
- // 不影响主扫描流程
- }
- }
- /**
- * 扫描并修复:找到 content_completed 但没有音频的叶节点,自动创建 TTS 任务
- */
- async function scanAndEnqueue(): Promise<void> {
- const startTime = Date.now();
- stats.totalScans++;
- stats.lastScanTime = startTime;
- try {
- // ---- 步骤 1:查找遗漏的章节(content_completed 但无音频) ----
- const missingChapters = await prisma.bookChapter.findMany({
- where: {
- genStage: 'content_completed',
- content: { not: null },
- OR: [
- { audioUrl: '' },
- { audioUrl: null },
- ],
- // 确保没有进行中的 TTS 任务(避免重复入队)
- ttsTasks: {
- none: { status: { in: ['pending', 'processing'] } },
- },
- },
- include: { book: true },
- take: 50, // 每次最多处理 50 条,避免一次性拉取过多
- orderBy: { id: 'asc' },
- });
- // ---- 步骤 2a:查找卡死章节(audio_generating 且有 audioUrl) ----
- const stuckChapters = await prisma.bookChapter.findMany({
- where: {
- genStage: 'audio_generating',
- audioUrl: { not: '' },
- // 确认没有进行中的任务
- ttsTasks: {
- none: { status: { in: ['pending', 'processing'] } },
- },
- },
- take: 50,
- orderBy: { id: 'asc' },
- });
- // ---- 步骤 2b:查找 TTS 失败但 genStage 未回退的章节(audio_generating + 无 audioUrl + 无进行中任务) ----
- const orphanAudioChapters = await prisma.bookChapter.findMany({
- where: {
- genStage: 'audio_generating',
- content: { not: null },
- OR: [
- { audioUrl: '' },
- { audioUrl: null },
- ],
- // 确认没有进行中的任务
- ttsTasks: {
- none: { status: { in: ['pending', 'processing'] } },
- },
- },
- take: 50,
- orderBy: { id: 'asc' },
- });
- // ---- 步骤 3a:修复卡死章节(有音频文件,直接推进 + 触发自动合并检查) ----
- for (const chapter of stuckChapters) {
- try {
- await bookStore.updateChapterById(chapter.id, {
- genStage: 'audio_completed',
- });
- stats.fixedStuck++;
- console.log(`[AudioScanner] 🔧 修复卡死: chapterId=${chapter.id} (audio_generating → audio_completed)`);
- // 修复后检查是否需要触发自动合并(advanceChapter 失败时 tryAutoMerge 会被跳过)
- tryAutoMerge(chapter.id, chapter.bookId, chapter.level, chapter.parentId).catch((mergeErr: any) => {
- console.warn(`[AudioScanner] 修复卡死后自动合并失败 chapterId=${chapter.id}:`, mergeErr.message);
- });
- } catch (err: any) {
- console.error(`[AudioScanner] 修复卡死失败 chapterId=${chapter.id}:`, err.message);
- }
- }
- // ---- 步骤 3b:回退孤立的 audio_generating 章节(无音频,TTS 任务已失败但 genStage 未回退) ----
- let orphanFixed = 0;
- for (const chapter of orphanAudioChapters) {
- try {
- // 回退到 content_completed,后续扫描会重新入队
- await bookStore.updateChapterById(chapter.id, {
- genStage: 'content_completed',
- });
- orphanFixed++;
- console.log(`[AudioScanner] 🔄 回退孤立场: chapterId=${chapter.id} (audio_generating → content_completed,将重新入队)`);
- } catch (err: any) {
- console.error(`[AudioScanner] 回退孤立场失败 chapterId=${chapter.id}:`, err.message);
- }
- }
- // ---- 步骤 4:为遗漏章节创建 TTS 任务(带防死循环保护) ----
- // 过滤掉 24h 内已失败超过 MAX_RETRY_PER_DAY 次的章节,避免 Provider 全部不可用时无限重试
- const MAX_RETRY_PER_DAY = 5;
- const twentyFourHoursAgo = new Date(Date.now() - 24 * 60 * 60 * 1000);
- let safeChapters = missingChapters;
- if (missingChapters.length > 0) {
- const failedCounts = await prisma.ttsTask.groupBy({
- by: ['chapterId'],
- where: {
- chapterId: { in: missingChapters.map(c => c.id) },
- status: 'failed',
- createdAt: { gte: twentyFourHoursAgo },
- },
- _count: { id: true },
- });
- const failedCountMap = new Map(failedCounts.map(f => [f.chapterId, f._count.id]));
- safeChapters = missingChapters.filter(ch => {
- const count = failedCountMap.get(ch.id) || 0;
- if (count >= MAX_RETRY_PER_DAY) {
- // 超过重试上限 → 标记 failed,用户可以手工重新生成
- bookStore.updateChapterById(ch.id, {
- genStage: 'failed',
- contentError: `TTS重试${count}次全部失败,请手工生成音频`,
- }).catch(() => {});
- console.log(`[AudioScanner] 🔒 章节${ch.id} 24h内已失败${count}次,标记为failed(等候用户手工处理)`);
- return false;
- }
- return true;
- });
- if (safeChapters.length > 0) {
- console.log(`[AudioScanner] 发现 ${safeChapters.length} 个缺少音频的章节(跳过${missingChapters.length - safeChapters.length}个已达重试上限),开始入队...`);
- }
- stats.foundMissing += safeChapters.length;
- } else {
- stats.foundMissing += missingChapters.length;
- }
- for (const chapter of safeChapters) {
- try {
- const result = await bookStore.generateChapterAudioById(chapter.id);
- if (result === null) {
- // 入队失败(可能内容为空等),记录但不计入统计
- console.warn(`[AudioScanner] ⚠️ 入队失败(return null): chapterId=${chapter.id}`);
- } else if (result.audioUrl) {
- // 已有音频(复用/已完成),也算成功
- stats.enqueuedSuccess++;
- } else {
- // 成功创建 pending 任务
- stats.enqueuedSuccess++;
- }
- } catch (err: any) {
- stats.enqueuedFailed++;
- console.error(`[AudioScanner] ❌ 入队异常 chapterId=${chapter.id}:`, err.message);
- }
- }
- // ---- 步骤 5:额外扫描 - 检查叶节点音频完成但1级章节缺少合并音频 ----
- await scanAndTryAutoMerge();
- // ---- 步骤 6:输出本轮摘要 ----
- const elapsed = Date.now() - startTime;
- const roundTotal = missingChapters.length + stuckChapters.length + orphanAudioChapters.length;
- if (roundTotal > 0) {
- console.log(
- `[AudioScanner] 本轮扫描完成 (${elapsed}ms): ` +
- `遗漏=${missingChapters.length}, 卡死修复=${stuckChapters.length}, 孤章回退=${orphanFixed}, ` +
- `成功=${stats.enqueuedSuccess}, 失败=${stats.enqueuedFailed}`
- );
- }
- // 定期输出累计统计
- if (stats.totalScans % LOG_STATS_INTERVAL === 0) {
- console.log(
- `[AudioScanner] 📊 累计统计: 扫描${stats.totalScans}轮, ` +
- `累计发现遗漏=${stats.foundMissing}, 累计修复卡死=${stats.fixedStuck}, ` +
- `入队成功=${stats.enqueuedSuccess}, 入队失败=${stats.enqueuedFailed}`
- );
- }
- } catch (err: any) {
- console.error(`[AudioScanner] 扫描异常:`, err.message);
- }
- }
- // ============ 生命周期 ============
- let timer: ReturnType<typeof setInterval> | null = null;
- let isRunning = false;
- export function startAudioScanner(): void {
- if (isRunning) {
- console.warn('[AudioScanner] 已在运行,跳过重复启动');
- return;
- }
- isRunning = true;
- console.log(`[AudioScanner] 音频扫描器已启动 (间隔=${SCAN_INTERVAL_MS / 1000}s, 兜底模式)`);
- // 启动后立即执行一次扫描(处理上一次运行期间遗留的)
- scanAndEnqueue().catch(err => {
- console.error('[AudioScanner] 首次扫描失败:', err);
- });
- // 定时轮询
- timer = setInterval(() => {
- scanAndEnqueue().catch(err => {
- console.error('[AudioScanner] 定时扫描失败:', err);
- });
- }, SCAN_INTERVAL_MS);
- }
- export function stopAudioScanner(): void {
- if (!isRunning) return;
- isRunning = false;
- if (timer) {
- clearInterval(timer);
- timer = null;
- }
- console.log(`[AudioScanner] 扫描器已停止 (累计: ${stats.totalScans}轮, 修复${stats.fixedStuck}个卡死, 入队${stats.enqueuedSuccess}个遗漏)`);
- }
- export function getAudioScannerStats(): ScanStats {
- return { ...stats };
- }
|