HuggingFaceM4/Idefics3-8B-Llama3 crash fix

[sywangyi]

server:
text-generation-launcher --model-id=HuggingFaceM4/Idefics3-8B-Llama3 -p 8080

client
curl -N 0.0.0.0:8080/generate_stream
-X POST
-d '{"inputs":"What is in the picture?\n\n","parameters":{"max_new_tokens":100, "seed": 42, "do_sample":true}}'
-H 'Content-Type: application/json'

crash happens because slot allocated in rust is not large enough.
gaudi and XPU has the same issue, assume cuda has same issue too.
the image process logic is copied from https://github.com/huggingface/transformers/blob/main/src/transformers/models/idefics3/image_processing_idefics3.py#L118

[sywangyi]

@Narsil @regisss please help review, thanks

[sywangyi]

following is the crash in XPU

2025-06-16T01:25:16.438091Z INFO text_generation_launcher: image_id 0 start_idx 0 end_idx 2314, length 2314
2025-06-16T01:25:17.269690Z ERROR batch{batch_size=1}:prefill:prefill{id=0 size=1}:prefill{id=0 size=1}: text_generation_router_v3::client: backends/v3/src/client/mo
d.rs:45: Server error: transport error
2025-06-16T01:25:17.269815Z ERROR batch{batch_size=1}:prefill:clear_cache{batch_id=Some(0)}:clear_cache{batch_id=Some(0)}: text_generation_router_v3::client: backend
s/v3/src/client/mod.rs:45: Server error: error trying to connect: Connection refused (os error 111)
2025-06-16T01:25:17.269831Z ERROR generate_stream{parameters=GenerateParameters { best_of: None, temperature: None, repetition_penalty: None, frequency_penalty: None
, top_k: None, top_p: None, typical_p: None, do_sample: true, max_new_tokens: Some(100), return_full_text: None, stop: [], truncate: None, watermark: false, details:
false, decoder_input_details: false, seed: Some(42), top_n_tokens: None, grammar: None, adapter_id: None }}:async_stream:generate_stream:schedule:infer:send_error:
text_generation_router_v3::backend: backends/v3/src/backend.rs:546: Request failed during generation: Server error: transport error
2025-06-16T01:25:17.333341Z ERROR shard-manager: text_generation_launcher: Shard complete standard error output:

[W616 01:23:57.772114900 OperatorEntry.cpp:154] Warning: Warning only once for all operators, other operators may also be overridden.
Overriding a previously registered kernel for the same operator and the same dispatch key
operator: aten::geometric_(Tensor(a!) self, float p, *, Generator? generator=None) -> Tensor(a!)
registered at /pytorch/build/aten/src/ATen/RegisterSchema.cpp:6
dispatch key: XPU
previous kernel: registered at /pytorch/aten/src/ATen/VmapModeRegistrations.cpp:37
new kernel: registered at /build/intel-pytorch-extension/build/Release/csrc/gpu/csrc/gpu/xpu/ATen/RegisterXPU_0.cpp:186 (function operator())
2025-06-16 01:23:59.160 | INFO | text_generation_server.utils.import_utils::76 - Detected system ipex
AssertHandler::printMessage
/pytorch/third_party/torch-xpu-ops/src/ATen/native/xpu/sycl/Indexing.h:620: operator(): global id: [2080,0,0], local id: [0,0,0] Assertion index >= -sizes_[i] && in dex < sizes_[i] && "index out of bounds" failed.
/pytorch/third_party/torch-xpu-ops/src/ATen/native/xpu/sycl/Indexing.h:620: operator(): global id: [2081,0,0], local id: [1,0,0] Assertion index >= -sizes_[i] && in dex < sizes_[i] && "index out of bounds" failed.
/pytorch/third_party/torch-xpu-ops/src/ATen/native/xpu/sycl/Indexing.h:620: operator(): global id: [2082,0,0], local id: [2,0,0] Assertion index >= -sizes_[i] && in dex < sizes_[i] && "index out of bounds" failed.

[drbh]

LGTM in general, but maybe the scaling logic can be reduced a bit?

possibly something with less branching like:

    let max_longest_edge_for_image_resize = config.get_max_longest_edge_for_image_resize();
    let max_image_size = config.get_max_image_size();

    let (height, width) = {
        let h = height as f32;
        let w = width as f32;
        
        // First resize to max_longest_edge (always scale to this size)
        let scale1 = max_longest_edge / h.max(w);
        let (h, w) = (h * scale1, w * scale1);
        
        // Ensure we dont exceed max_size (only scale down)
        let scale2 = (max_size / h.max(w)).min(1.0);
        
        ((h * scale2) as usize, (w * scale2) as usize)
    };