feat(embedder): use summary for file embedding in semantic pipeline

[yangxinxin-7]

Summary

• When files in a code repository are processed through the semantic pipeline, use the pre-generated summary (AST skeleton or LLM summary) for embedding instead of raw file content
• Add is_code_repo flag to SemanticMsg and propagate it through the pipeline: ResourceProcessor → Summarizer → SemanticMsg → SemanticDagExecutor
• Detect code repositories via source_format == "repository" (set by CodeRepositoryParser) and pass is_code_repo=True when enqueuing semantic processing
• use_summary in _file_summary_task is now gated on is_code_repo, so plain text / markdown / other non-repo resources continue to embed raw file content
• Truncate AST skeleton to max_skeleton_chars (12000 chars, ~3000 tokens) before embedding to prevent oversized input
• Add max_skeleton_chars config field to SemanticConfig

Why

Raw file content was being sent directly to the embedding API even when a semantic summary had already been generated. For large files this caused the embedding API to reject the request with a token limit error (e.g. OpenAI 8192 token limit). Using the bounded summary instead of raw content fixes this.

However, using summary for all file types (including markdown, plain text) was incorrect — for those files the raw content is the meaningful representation. Summary-based embedding is only appropriate for code files where AST skeletons provide a better semantic signal.

Paths unaffected:

• index_resource direct indexing path (use_summary defaults to False)
• Memory files (handled separately in memory_extractor.py)
• Non-repo resources (markdown, plain text, etc.) — always use raw content

Closes

Closes #616

[qin-ctx]

只有代码会走这个路径吗

[yangxinxin-7]

好像有点问题，我再check下

[yangxinxin-7]

只有代码会走这个路径吗

这个已经改了下，现在是只有code repo的情况才会这样了