Hello, thank you for your impressive work on MovieAgent! We are the authors of ViStoryBench. We have recently developed ViStoryBench, a comprehensive benchmark suite designed to evaluate story visualization models across diverse narrative structures, visual styles, and character settings. Our benchmark features 80 curated stories, 344 characters, and introduces a suite of 12 automated metrics to systematically assess key aspects like character consistency, style similarity, prompt adherence, and aesthetic quality.
ViStoryBench Evaluation
In our extensive evaluation of over 18 methods, we are delighted to report that MovieAgent demonstrated excellent performance, ranking among the top contenders in several critical metrics.
Notably, MovieAgent (SD3) achieved:
An Alignment Score of 76.1, showcasing its strong ability to adhere to complex textual prompts.
An Inception Score of 15.02, indicating high diversity in generation.
An Aesthetics Score of 5.33, confirming the high visual quality of the output.
| Method |
CSD (Style) Cross |
CSD (Style) Self |
CIDS (Character) Cross |
CIDS (Character) Self |
Alignment Score |
OCCM Score |
Inception Score |
Aesthetics Score |
Copy-Paste Degree |
| MovieAgent (ROICtrl) |
20.0 |
49.3 |
30.5 |
51.4 |
36.5 |
86.7 |
11.63 |
4.65 |
0.33 |
| MovieAgent (SD3) |
30.9 |
48.3 |
34.9 |
52.2 |
76.1 |
87.5 |
15.02 |
5.33 |
-0.40 |
These results, which you can find in Table 2 of our paper, strongly validate MovieAgent's advanced capabilities. We believe our benchmark effectively highlights the technical strengths of your model.
If you find our benchmark helpful, we would be honored if ViStoryBench could be considered for citation in your work. For any questions or further information, please contact us at vistorybench@126.com.
The fine-grained results are shown below.
Hello, thank you for your impressive work on MovieAgent! We are the authors of ViStoryBench. We have recently developed ViStoryBench, a comprehensive benchmark suite designed to evaluate story visualization models across diverse narrative structures, visual styles, and character settings. Our benchmark features 80 curated stories, 344 characters, and introduces a suite of 12 automated metrics to systematically assess key aspects like character consistency, style similarity, prompt adherence, and aesthetic quality.
ViStoryBench Evaluation
In our extensive evaluation of over 18 methods, we are delighted to report that MovieAgent demonstrated excellent performance, ranking among the top contenders in several critical metrics.
Notably, MovieAgent (SD3) achieved:
An Alignment Score of 76.1, showcasing its strong ability to adhere to complex textual prompts.
An Inception Score of 15.02, indicating high diversity in generation.
An Aesthetics Score of 5.33, confirming the high visual quality of the output.
These results, which you can find in Table 2 of our paper, strongly validate MovieAgent's advanced capabilities. We believe our benchmark effectively highlights the technical strengths of your model.
If you find our benchmark helpful, we would be honored if ViStoryBench could be considered for citation in your work. For any questions or further information, please contact us at vistorybench@126.com.
The fine-grained results are shown below.