Fuctional code with docker based dependencies. Proposed solution in readme.md#1
Fuctional code with docker based dependencies. Proposed solution in readme.md#1Srishti-1806 wants to merge 4 commits intomasterfrom
Conversation
This proposal enhances the original idea in several key ways: 1. Accessibility and Scalability Unlike traditional methods that require multispectral or X-ray data, this system works with standard RGB images, making it deployable in low-resource environments and scalable across large datasets. 2. Software-first Approach By shifting from hardware-dependent imaging to AI-driven reconstruction, the project reduces cost and increases usability. This allows broader adoption across institutions, researchers, and independent analysts. 3. End-to-End Automated Pipeline The proposed system integrates multiple stages—denoising, segmentation, inpainting, enhancement, and upscaling—into a single cohesive pipeline, reducing manual intervention. 4. Generative Reconstruction instead of Detection Rather than only identifying whether a hidden image exists, this approach attempts to reconstruct plausible underlying visuals, providing richer insights. 5. Deployment-ready Architecture The system is designed with FastAPI and Docker-based deployment, ensuring reproducibility, scalability, and ease of integration into real-world applications.
|
Your trial period has expired. To continue using this feature, please upgrade to a paid plan here or book a time to chat here. |
|
I’ve submitted a proposal for the ArtExtract project, focusing on a software-driven pipeline for hidden image reconstruction. [https://github.com/Srishti-1806/OS_1_PAINTING_IN_A_PAINTING] While traditional methods are heavily dependent on expensive multispectral and X-ray hardware, my approach pivots to an AI-first reconstruction strategy using standard RGB images. By leveraging Diffusion Models, Mask2Former for semantic segmentation, and ControlNet for structural integrity, this pipeline moves beyond simple detection toward probabilistic, high-fidelity reconstruction. I’ve designed this as a deployment-ready system (FastAPI/Docker) to ensure it’s not just a research concept but a scalable tool for digital archivists and researchers who lack access to high-end imaging equipment. Looking forward to your thoughts on this end-to-end automated approach! |
Srishti-1806
left a comment
Srishti-1806
left a comment
There was a problem hiding this comment.
I'm aiming to target Painting_in_painting problem.
This proposal stands out due to its practicality, innovation, and real-world applicability:
It transforms a research-heavy concept into a deployable engineering solution
It removes reliance on costly imaging techniques, making the solution democratized and scalable
It combines multiple state-of-the-art AI models into a cohesive, production-ready system
It focuses not just on detection but on meaningful reconstruction and visualization
It is built with deployment and usability in mind, ensuring impact beyond experimentation.
Though I have attached a small working prototype with it https://github.com/Srishti-1806/humanai-foundation.github.io/pull/1/, a large sacle implementation of the idea requires a detailed roadmap.
Execution plan is as follows:
Phase 1: Research & Environment Setup (Weeks 1-2)
Infrastructure: Set up the development environment using Docker to ensure reproducibility.
Data Acquisition: Curate a synthetic dataset by layering "hidden" images under primary paintings using digital blending (alpha-compositing) to simulate overpaints.
Baseline Selection: Implement a basic GAN-based inpainting baseline for performance comparison.
Phase 2: Semantic Segmentation & Masking (Weeks 3-4)
Region Detection: Integrate Mask2Former to identify regions of interest, such as damaged areas or suspected overpainted sections.
Edge-based Masking: Develop custom OpenCV-based scripts to generate precise masks that guide the diffusion model, ensuring it only "reconstructs" where necessary.
Phase 3: Generative Reconstruction Pipeline (Weeks 5-7)
Stable Diffusion Integration: Implement the Diffusers library for high-fidelity inpainting.
Structure Preservation: Integrate ControlNet (Canny/Depth edge control). This is crucial to ensure the reconstructed hidden image maintains a logical geometric composition relative to the brushstroke patterns of the original painting.
Fine-tuning (Optional): Explore LoRA (Low-Rank Adaptation) on art history datasets to improve the "painterly" quality of reconstructions.
Phase 4: Post-Processing & Enhancement (Weeks 8-9)
Super-Resolution: Integrate Real-ESRGAN to upscale the low-resolution latent outputs of the diffusion model to high-quality archival standards.
Color Correction: Implement histogram matching algorithms to ensure the reconstructed layers blend naturally with the historical palette of the era.
Phase 5: API Development & Optimization (Weeks 10-11)
Backend Engineering: Develop a robust FastAPI wrapper to serve the model as an end-to-end service.
Performance Tuning: Optimize the pipeline for both CPU and GPU execution (quantization or Half-Precision FP16) to ensure accessibility for researchers with varying hardware.
Phase 6: Evaluation & Documentation (Week 12)
Metrics: Evaluate performance using PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity Index) on synthetic data.
Final Delivery: Complete the documentation, including a comprehensive README.md, API usage guides, and a final report on the reconstruction accuracy.
1 participant
This proposal enhances the original idea in several key ways:
https://github.com/Srishti-1806/OS_1_PAINTING_IN_A_PAINTING
USP OF THE SOLUTION (replaced database dependency with Generative AI pipene integrated with ControlNet model (Canny Edge Detection) for refined image generation after segmentation ).
Instead of a static retrieval-based system that is limited by a fixed database, I am proposing a dynamic Generative Reconstruction Pipeline. This not only reduces server-side latency and storage overhead but also ensures the system can generalize to artworks that have never been digitally cataloged before.
Stable Diffusion XL (SDXL) or ControlNet Tile can be integrated to produce high resolution images as part of future developements in the proposed idea.
Key addons:
combined = cv2.bitwise_or(edges, seg_mask) -- in pipeline.py ensures that imprinting is done only where distortion exists or missing pixels exist).
Features:
Unlike traditional methods that require multispectral or X-ray data, this system works with standard RGB images, making it deployable in low-resource environments and scalable across large datasets.
By shifting from hardware-dependent imaging to AI-driven reconstruction, the project reduces cost and increases usability. This allows broader adoption across institutions, researchers, and independent analysts.
The proposed system integrates multiple stages—denoising, segmentation, inpainting, enhancement, and upscaling—into a single cohesive pipeline, reducing manual intervention.
Rather than only identifying whether a hidden image exists, this approach attempts to reconstruct plausible underlying visuals, providing richer insights.
The system is designed with FastAPI and Docker-based deployment, ensuring reproducibility, scalability, and ease of integration into real-world applications.