llm multi turn conversation

[redyoshi]

if i'm not satisfied with the result when using LLMExtractionStrategy, is there any way to start a multi turn conversion with LLM?

[Alexrosulek]

there is not

[jverneuer]

Yes, absolutely --- you're free to implement whatever approach you want
in your own code. The only catch is that you'll need to build it yourself.

What @Alexrosulek probably meant is, that it might not be the most practical approach and is not build into crawl4ai.

You can define your own strategy, for example:

class ReasoningExtractionStrategy(ExtractionStrategy):
    ...

and use just like any other extraction strategy.

Inside, you can reuse the existing strategies to get up and running quickly,
then branch out into your own custom logic. A simple example would be reusing the existing LLM strategy
but tweaking the prompt you send on each iteration --- nothing too complicated.

If you want to get more advanced, you could even have another LLM agent act as a QA reviewer in the loop,
going back and forth until the QA agent is satisfied. Probably using another instance of the extraction LLM as QA.

That said, running such a process during crawling doesn't really make much sense.
Even without knowing your problem space it sounds a lot like post processing.

If you want to go down that route look how i extended a filter, basically the same pattern as extending
the extraction strategie stat data aware pruning filter

[redyoshi]

Thank you for your detailed answer, I will carefully study the examples.