Skip to content

Commit 28f937f

Browse files
Zehui-LinZehui-Lin
committed
Add Project Pages PerceptGuide
1 parent 98f1726 commit 28f937f

File tree

6 files changed

+521
-0
lines changed

6 files changed

+521
-0
lines changed

‎PerceptGuide/index.html‎

Lines changed: 381 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,381 @@
1+
<html>
2+
3+
<head>
4+
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
5+
6+
<meta name="viewport" content="width=device-width, initial-scale=1.0">
7+
<meta name="keywords"
8+
content="An orchestration learning framework for ultrasound imaging: Prompt-Guided Hyper-Perception and Attention-Matching Downstream Synchronization, PerceptGuide">
9+
<meta name="description"
10+
content="An orchestration learning framework for ultrasound imaging: Prompt-Guided Hyper-Perception and Attention-Matching Downstream Synchronization, MedIA 2025">
11+
<title>An orchestration learning framework for ultrasound imaging: Prompt-Guided Hyper-Perception and Attention-Matching Downstream Synchronization</title>
12+
13+
<link rel="stylesheet" type="text/css" href="./resources/text.css">
14+
15+
<meta property="og:image" content='./resources/perceptguide.png'>
16+
<meta property="og:title"
17+
content="An orchestration learning framework for ultrasound imaging: Prompt-Guided Hyper-Perception and Attention-Matching Downstream Synchronization, MedIA 2025">
18+
</head>
19+
20+
21+
22+
23+
<body>
24+
25+
<br>
26+
<center>
27+
<span style="font-size:36px">
28+
An orchestration learning framework for ultrasound imaging: Prompt-Guided Hyper-Perception and Attention-Matching Downstream Synchronization </span>
29+
</center>
30+
<br>
31+
32+
<!-- 第一行作者 -->
33+
<table align="center" width="800px">
34+
<tbody>
35+
<tr>
36+
37+
<td align="center" width="160px">
38+
<center>
39+
<span style="font-size:18px"><a href="https://zehui-lin.github.io/">Zehui
40+
Lin</a><sup>1</sup></span>
41+
</center>
42+
</td>
43+
44+
<td align="center" width="160px">
45+
<center>
46+
<span style="font-size:18px"><a href="https://scholar.google.com/citations?user=6WNtJa0AAAAJ&hl=en&oi=ao">Shuo
47+
Li</a><sup>2</sup></span>
48+
</center>
49+
</td>
50+
51+
<td align="center" width="160px">
52+
<center>
53+
<span style="font-size:18px"><a href="https://scholar.google.com/citations?user=8pnz5L4AAAAJ&hl=en&oi=ao">Shanshan
54+
Wang</a><sup>3</sup></span>
55+
</center>
56+
</td>
57+
58+
<td align="center" width="160px">
59+
<center>
60+
<span style="font-size:18px"><a href="https://zhifan-gao.github.io/">Zhifan
61+
Gao</a><sup>4</sup></span>
62+
</center>
63+
</td>
64+
65+
<td align="center" width="160px">
66+
<center>
67+
<span style="font-size:18px"><a
68+
href="https://scholar.google.com/citations?hl=en&user=lxxn3CcAAAAJ">Yue
69+
Sun</a><sup>1</sup></span>
70+
</center>
71+
</td>
72+
73+
</tr>
74+
</tbody>
75+
</table><br>
76+
77+
78+
<!-- 第二行作者 -->
79+
<table align="center" width="750px">
80+
<tbody>
81+
<tr>
82+
83+
84+
<td align="center" width="160px">
85+
<center>
86+
<span style="font-size:18px"><a href="https://scholar.google.com/citations?user=GXD7ppMAAAAJ&hl=en&oi=ao">Chan-Tong Lam</a><sup>1</sup></span>
87+
</center>
88+
</td>
89+
90+
91+
<td align="center" width="160px">
92+
<center>
93+
<span style="font-size:18px">Xindi Hu<sup>5</sup></span>
94+
</center>
95+
</td>
96+
97+
98+
<td align="center" width="160px">
99+
<center>
100+
<span style="font-size:18px"><a href="https://xy0806.github.io/">Xin Yang</a><sup>6</sup></span>
101+
</center>
102+
</td>
103+
104+
105+
<td align="center" width="160px">
106+
<center>
107+
<span style="font-size:18px"><a
108+
href="https://scholar.google.com/citations?user=J27J2VUAAAAJ&hl=en&oi=ao">Dong
109+
Ni</a><sup>6</sup></span>
110+
</center>
111+
</td>
112+
113+
<td align="center" width="160px">
114+
<center>
115+
<span style="font-size:18px"><a
116+
href="https://scholar.google.com/citations?user=lLg3WRkAAAAJ&hl=en&oi=ao">Tao
117+
Tan</a><sup>1,<img class="round" style="width:20px"
118+
src="./resources/corresponding_fig.png"></sup></span>
119+
</center>
120+
</td>
121+
122+
</tr>
123+
</tbody>
124+
</table><br>
125+
126+
127+
128+
129+
<!-- 单位 -->
130+
<table align="center" width="700px">
131+
<tbody>
132+
<tr>
133+
<td align="center" width="300px">
134+
<center>
135+
<span style="font-size:18px"><sup>1</sup>Faculty of Applied Sciences, Macao Polytechnic
136+
University</span>
137+
</center>
138+
</td>
139+
</tr>
140+
</tbody>
141+
</table>
142+
143+
<table align="center" width="700px">
144+
<tbody>
145+
<tr>
146+
<td align="center" width="600px">
147+
<center>
148+
<span style="font-size:18px"><sup>2</sup>Department of Biomedical Engineering and the Department of Computer and Data Science, Case Western Reserve University</span>
149+
</center>
150+
</td>
151+
</tr>
152+
</tbody>
153+
</table>
154+
155+
<table align="center" width="700px">
156+
<tbody>
157+
<tr>
158+
<td align="center" width="600px">
159+
<center>
160+
<span style="font-size:18px"><sup>3</sup>Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences</span>
161+
</center>
162+
</td>
163+
</tr>
164+
</tbody>
165+
</table>
166+
167+
<table align="center" width="700px">
168+
<tbody>
169+
<tr>
170+
<td align="center" width="600px">
171+
<center>
172+
<span style="font-size:18px"><sup>4</sup>School of Biomedical Engineering,
173+
Sun Yat-sen University</span>
174+
</center>
175+
</td>
176+
177+
</tr>
178+
</tbody>
179+
</table>
180+
181+
182+
183+
<table align="center" width="700px">
184+
<tbody>
185+
<tr>
186+
<td align="center" width="600px">
187+
<center>
188+
<span style="font-size:18px"><sup>5</sup>Shenzhen RayShape Medical Technology Co. Ltd.</span>
189+
</center>
190+
</td>
191+
</tr>
192+
</tbody>
193+
</table>
194+
195+
196+
<table align="center" width="700px">
197+
<tbody>
198+
<tr>
199+
<td align="center" width="600px">
200+
<center>
201+
<span style="font-size:18px"><sup>6</sup>School of Biomedical Engineering, Shenzhen
202+
University</span>
203+
</center>
204+
</td>
205+
206+
</tr>
207+
</tbody>
208+
</table>
209+
210+
<br>
211+
<hr>
212+
<center>
213+
<h1>Abstract</h1>
214+
</center>
215+
<div align="justify">
216+
Ultrasound imaging is pivotal in clinical diagnostics due to its affordability, portability, safety, real-time capability, and non-invasive nature. It is widely utilized for examining various organs, such as the breast, thyroid, ovary, cardiac, and more. However, the manual interpretation and annotation of ultrasound images are time-consuming and prone to variability among physicians. While single-task artificial intelligence (AI) solutions have been explored, they are not ideal for scaling AI applications in medical imaging. Foundation models, although a trending solution, often struggle with real-world medical datasets due to factors such as noise, variability, and the incapability of flexibly aligning prior knowledge with task adaptation. To address these limitations, we propose an orchestration learning framework named PerceptGuide for general-purpose ultrasound classification and segmentation. Our framework incorporates a novel orchestration mechanism based on prompted hyper-perception, which adapts to the diverse inductive biases required by different ultrasound datasets. Unlike self-supervised pre-trained models, which require extensive fine-tuning, our approach leverages supervised pre-training to directly capture task-relevant features, providing a stronger foundation for multi-task and multi-organ ultrasound imaging. To support this research, we compiled a large-scale Multi-task, Multi-organ public ultrasound dataset (M
217+
-US), featuring images from 9 organs and 16 datasets, encompassing both classification and segmentation tasks. Our approach employs four specific prompts—Object, Task, Input, and Position—to guide the model, ensuring task-specific adaptability. Additionally, a downstream synchronization training stage is introduced to fine-tune the model for new data, significantly improving generalization capabilities and enabling real-world applications. Experimental results demonstrate the robustness and versatility of our framework in handling multi-task and multi-organ ultrasound image processing, outperforming both specialist models and existing general AI solutions. Compared to specialist models, our method improves segmentation from 82.26% to 86.45%, classification from 71.30% to 79.08%, while also significantly reducing model parameters.
218+
</div>
219+
<br>
220+
221+
222+
223+
<hr>
224+
<center>
225+
<h1>Architecture</h1>
226+
</center>
227+
<p>
228+
<figure>
229+
<img style="width:700px" src='./resources/perceptguide.png'>
230+
<br><br>
231+
<figcaption>This figure illustrates the framework of our proposed network, named PerceptGuide. We utilize Swin-Unet (Cao et al., 2022) as the backbone. In the classification decoder,
232+
there are skip connections similar to those in the segmentation decoder, but no upsampling layers. The hyper-perception operations are present in each block of the transformer
233+
in every layer of the decoders. During the downstream synchronization stage, we fine-tune only the MLP layers involved in generating prompt embeddings
234+
</figcaption>
235+
</figure>
236+
</p>
237+
238+
239+
<hr>
240+
<center>
241+
<h1>Highlights</h1>
242+
</center>
243+
244+
<p style="font-weight: bold; font-size: large;">
245+
A novel orchestration learning framework (PerceptGuide) for multi-task, multi-organ ultrasound analysis.
246+
</p>
247+
248+
<p style="font-weight: bold; font-size: large;">
249+
The design of a hyper-perception module that leverages four semantic prompts to guide the learning process.
250+
</p>
251+
252+
<p style="font-weight: bold; font-size: large;">
253+
The creation of M2-US, a comprehensive, large-scale publicly available dataset for ultrasound imaging.
254+
</p>
255+
256+
<p style="font-weight: bold; font-size: large;">
257+
The introduction of a downstream synchronization stage for efficient model fine-tuning and improved generalization.
258+
</p>
259+
260+
<p style="font-weight: bold; font-size: large;">
261+
Significant improvements in segmentation and classification performance compared to existing approaches.
262+
</p>
263+
264+
265+
<hr>
266+
<center>
267+
<h1>Graphical abstract</h1>
268+
</center>
269+
<p>
270+
<center><img style="width:700px" src='./resources/ga.jpg'></center>
271+
</p>
272+
<br>
273+
274+
275+
<hr>
276+
<center>
277+
<h1>Publications</h1>
278+
</center>
279+
280+
<script type="text/javascript">
281+
function show_hide(eid) {
282+
var x = document.getElementById(eid);
283+
if (x.style.display === "none") {
284+
x.style.display = "block";
285+
} else {
286+
x.style.display = "none";
287+
}
288+
}
289+
</script>
290+
291+
<table align="center" width="700px">
292+
<tbody>
293+
<tr>
294+
<td>
295+
<a href="https://zehui-lin.github.io/PerceptGuide/"><img class="layered-paper-big" style="height:175px"
296+
src="./resources/perceptguide_page1.png"></a>
297+
</td>
298+
299+
<td width="500px">
300+
<span style="font-size:18px">
301+
302+
<div class="title">
303+
<b>An orchestration learning framework for ultrasound imaging: Prompt-Guided Hyper-Perception and Attention-Matching Downstream Synchronization.</b>
304+
</div>
305+
306+
<div class="authors">
307+
Zehui Lin,
308+
Shuo Li,
309+
Shanshan Wang,
310+
Zhifan Gao,
311+
Yue Sun,
312+
Chan-Tong Lam,
313+
Xindi Hu,
314+
Xin Yang,
315+
Dong Ni,
316+
Tao Tan
317+
</div>
318+
319+
<div class="journal">
320+
<b>MedIA, 2025</b>
321+
</div>
322+
323+
<div class="links">
324+
<a href="javascript:;" onclick="show_hide('Biblin2025orchestration')"> Bibtex </a>
325+
| <a href="https://doi.org/10.1016/j.media.2025.103639"> PDF </a>
326+
| <a href="https://arxiv.org/abs/2409.12345"> Temporary Accessible PDF [expired: 2025-09-30] </a>
327+
| <a href="https://github.com/Zehui-Lin/PerceptGuide"> Code </a>
328+
</div>
329+
330+
<div style="display: none;" class="BibtexExpand" id="Biblin2025orchestration">
331+
<div style="width:500px;overflow:visible;">
332+
<pre class="bibtex" style="font-size:12px">@article{lin2025orchestration,
333+
title={An orchestration learning framework for ultrasound imaging: Prompt-guided hyper-perception and attention-matching Downstream Synchronization},
334+
author={Lin, Zehui and Li, Shuo and Wang, Shanshan and Gao, Zhifan and Sun, Yue and Lam, Chan-Tong and Hu, Xindi and Yang, Xin and Ni, Dong and Tan, Tao},
335+
journal={Medical Image Analysis},
336+
pages={103639},
337+
year={2025},
338+
publisher={Elsevier}
339+
}
340+
</pre>
341+
</div>
342+
</div>
343+
</span>
344+
</td>
345+
</tr>
346+
</tbody>
347+
</table>
348+
349+
350+
351+
<br><br>
352+
<br><br>
353+
<hr>
354+
355+
<center>
356+
<h1>Support</h1>
357+
</center>
358+
359+
<p style="font-size:18px;">
360+
All resources, including data, code, and pretrained models, can be found here (<a href="https://github.com/Zehui-Lin/PerceptGuide/releases/tag/v1.0.0">link</a>).
361+
</p>
362+
363+
364+
<br><br>
365+
<hr>
366+
367+
<center>
368+
<h1>Acknowledgements</h1>
369+
</center>
370+
This work was supported by Science and Technology Development Fund of Macao (0021/2022/AGJ) and Science and Technology Development Fund of Macao (0041/2023/RIB2).
371+
372+
<br><br>
373+
<br><br>
374+
375+
<p style="text-align:center;font-size:14px;">
376+
Webpage template modified from <a href="https://richzhang.github.io/splitbrainauto/">Richard Zhang</a>.
377+
</p>
378+
379+
380+
</body>
381+
</html>

‎PerceptGuide/resources/corresponding_fig.png‎

66.7 KB
Loading

‎PerceptGuide/resources/ga.jpg‎

330 KB
Loading

‎PerceptGuide/resources/perceptguide.png‎

307 KB
Loading

‎PerceptGuide/resources/perceptguide_page1.png‎

101 KB
Loading

0 commit comments

Comments
 (0)