-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathImageAutofocus.html
More file actions
297 lines (228 loc) · 22.2 KB
/
ImageAutofocus.html
File metadata and controls
297 lines (228 loc) · 22.2 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
<!DOCTYPE html>
<html>
<head>
<title>LifeHack Microscope: ImLock Autofocus/Stabilisation</title>
<meta name="description" content="An infra-red brighfield image based autofocus and lateral drift correction device. Capable of maintaining lock upto 50 microns away from the cover-slip">
<meta name="author" content="Josh Edwards">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="stylesheet" type="text/css" href="main.css">
<link rel="stylesheet" type="text/css" href="menu.css">
<!-- Global site tag (gtag.js) - Google Analytics -->
<script async src="https://www.googletagmanager.com/gtag/js?id=G-B9XZF5GEV3"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'G-B9XZF5GEV3');
</script>
</head>
<body>
<div id="myNav" class="overlay">
<a href="javascript:void(0)" class="closebtn" onclick="closeNav()">×</a>
<div class="grid-container">
<a href="home.html"><div class="hero-image-home">
<div class="menu-text"><h1>Home</h1></div>
</div></a>
<a href="MainBody.html"><div class="hero-image-MainBody">
<div class="menu-text"><h1>Main Body</h1></div>
</div></a>
<a href="BeamExpansion.html"><div class="hero-image-BeamExpansion">
<div class="menu-text"><h1>Beam Expansion Module</h1></div>
</div></a>
<a href="Excitation.html"><div class="hero-image-Excitation">
<div class="menu-text"><h1>Excitation Module</h1></div>
</div></a>
<a href="FocusShifter.html"><div class="hero-image-FocusShifter">
<div class="menu-text"><h1>Focus Shifter</h1></div>
</div></a>
<a href="IncubationBox.html"><div class="hero-image-IncubationBox">
<div class="menu-text"><h1>Incubation Box</h1></div>
</div></a>
<a href="https://github.com/HoldenLab/LifeHack"><div class="hero-image-github">
<div class="menu-text"><h1>Git Hub</h1></div>
</div></a>
<a href="ImageAutofocus.html"><div class="hero-image-ImageAutofocus">
<div class="menu-text"><h1>ImLock</h1></div>
</div></a>
<a href="Operation.html"><div class="hero-image-Operation">
<div class="menu-text"><h1>Operation</h1></div>
</div></a>
</div>
</div>
<script>
function openNav() {
document.getElementById("myNav").style.height = "100%";
}
function closeNav() {
document.getElementById("myNav").style.height = "0px";
}
</script>
<div class="hero-image">
<div class="hero-text">
<h1>ImLock</h1>
<p>Infra-Red Image Based Deep Autofocus and Live Drift Correction</p>
</div>
</div>
<div class="nav-wrapper" style="height:80px;">
<div id="navbar">
<a href="javascript:void(0)" onclick="openNav()">☰ Menu</a>
</div>
</div>
<div class="sidenav">
<a href="#Description">Description</a>
<a href="#Parts">Parts</a>
<a href="#Construction">Construction</a>
<a href="#Software">Software</a>
<a href="https://github.com/HoldenLab/DeepAutoFocus">Go to GitHub</a>
<a href="https://github.com/HoldenLab/LifeHack/issues">Report Bug/Feedback</a>
<hr>
<p>Project Contributors: Josh Edwards, Kevin Whitley, Sudeer Peneti, Yann Cesbron, Seamus Holden</p>
<p>Website by Josh Edwards</p>
<p>Updated: April 2023</p>
</div>
<div class="page">
<video src="Video/ir-image-autofocus-animation.mp4" width="90%" controls autoplay muted></video>
<h1 id="Description">Description</h1><hr>
<p>This module provides the capability for infra-red brightfield imaging concurrently with visible band fluorescence. This hardware is combined with custom software to conduct image based, deep autofocus at up to 50 microns in depth. We have found this especially useful in imaging samples set in PDMS-microholes (used in Whitley et al Nat Commun 2021) where the cells themselves are held away from the cover surface.</p>
<img src="Images/Image-Autofocus.jpg" width="60%">
<h1 id="Parts">Parts</h1><hr>
<iframe src="Parts-List/ImageAutofocus-Parts.htm" width="100%" height="500px" title="Parts"></iframe>
<div class="minipage">
<a class="button-design" href="Parts-List/ImageAutofocus-Parts.xlsx">⇩ Download ⇩</a>
<a class="button-design" href="https://github.com/HoldenLab/LifeHack/tree/master/Image%20Autofocus/Parts">⇩ Get Custom Part Designs ⇩</a>
</div>
<p>If you are adding this to an existing system it can be simpler to add hardware after the camera port. If this is the case we recomend using a <a href="https://www.cairn-research.co.uk/product/twincam">TwinCam</a> to give a second pathway for the IR camera.</p>
<h1 id="Construction">Construction</h1><hr>
<a class="button-design" href="https://a360.co/3p8be5k" target="_blank">👁 View CAD Design Online 👁</a>
<p>As this module has only one pathway and few degrees of freedom for adjustment it can be constructed and attached to the system simply following the designs. The combiner cube is the only part that needs alignment and this is detailed in the <a href="MainBody.html#Construction">Main Body</a> module.</p>
<p>To add the ImLock to an existing microscope system we recomend adding the second camera after the camera port for simplicity. This can be done easily using products such as CAIRN's <a href="https://www.cairn-research.co.uk/product/twincam/">TwinCam</a>.</p>
<p>In principle this sytem can operate at a range of wavelengths. However, the suggested camera suffers from reduced sensitivity into the IR range. As such, poor performance has been observed when operating this system at 1050nm due to low SNR. To achieve a sufficient SNR whilst avoiding clashes with the CRISP system operating at 940nm we recomend operating with 850nm illumination.</p>
<img src="Images/IMX250_sensitivity.jpg" width="40%">
<h1 id="Software">Software</h1><hr>
<p>This is a MicroManager hardware autofocus plugin developed between the Henriques and Holden laboratories. This plugin will perform microscopy autofocus and lateral drift correction using the cross correlated signal from infrared brightfield microscopy. It is a development on the principle described in McGorty et al, Optical Nanoscopy 2013. As such, this software is not coupled to any particular hardware and may be used with cameras and stages other than those described here.</p>
<h2>Instructions: Please select a version</h2>
<p>The original "Main" version of the plugin works within a single instance of micromanager. However, when using this with cameras other than the Blackfly camera in the parts-list we frequently see issues. A second "Single-Core" version is available that will run in a separate instance of micromanager to the data aquisition which avoids these problems. We recomend trying the "Main" version first and using the "Single-Core" version if this does not work with your system.</p>
<div class="tab">
<button class="tablinks" onclick="openVersion(event, 'Main')">Main</button>
<button class="tablinks" onclick="openVersion(event, 'Single-Core')">Single Core</button>
</div>
<div id="Main" class="tabcontent">
<p>This is the primary version of ImLock. It is designed to work within a single instance of micromanager alongside data collection. Testing with a few cameras has yielded mixed results so if this fails with your camera please try the other version.</p>
<h3>Installation</h3>
<a class="button-design" href="https://github.com/HoldenLab/DeepAutoFocus/releases/tag/v2.0.0">👁 Get Software from GitHub 👁</a>
<p>From the folder "dist" in the GitHub repository, copy "ImLock.jar" and the content of the "lib" folder into the Micromanager "plugins/micro-manager" directory. In the case of these files already existing in the target directory, the target directory files should be retained.</p>
<p>The camera used for this module must be added to a MicroManager config file separate from the main system config file. Instructions are available <a href="https://micro-manager.org/wiki/Spinnaker">here</a> to install the correct drivers for the Blackfly camera.</p>
<p>The plugin may now be launched in MicroManager from "plugins/Beta/ImLock". On opening the <a href="https://github.com/HoldenLab/LifeHack/blob/master/Operation/BlackflyCamera.cfg">Config file</a> containing the camera's device adapter should be loaded using the "select configuration file" and "load hardware" buttons under the configuration tab (This will happen automatically on subsequent openings).</p>
<h3>Use</h3>
<p>The plugin does not have the ability to trigger the illumination LED so this must be controlled manually. If using this as part of the LifeHack system this can be done through MicroManager. Our testing has been conducted with the Tiger Controller's MAINTAIN(MA) setting set to "3 - Motors on during wait time".</p>
<img src="Images/ImLock_v2_GUI.png" width="100%">
<p>To use the plugin, First "get background image" to perform background subtraction. The stage will scan around the sample to record multiple fields of view for averaging.</p>
<p>Next "calibrate pixel size" to calculate the pixels-to-nm conversion factor and axis orientations. The stage will move laterally in user-defined steps and calculate the pixel displacements by cross-correlation.</p>
<p>"Start" takes a reference z-stack (if using XYZ or Z correction) or a single reference image (if using XY correction) and begins the drift correction protocol selected using the pulldown menu.</p>
<p>The tick-box options on the control panel are mostly self explanitory. "Show drift plots" produces plots of the live position reading to aid with setting gains(described in "Variables").</p>
<p>If "Save drift data to file" is ticked, positions will be saved in a columnwise ".csv" file with columns "time, x, y, z".</p>
<h3>Complex Experiments</h3>
<p>The ImLock is capable of working with Z stacking and XY scanning timelapse experiments as long as the interval between time points is >5 seconds. Drift corrections will be made between the acquisition time points and the acquisition's position list updated to apply these corrections to every FOV.</p>
<p>If you wish to conduct long timelapse experiments on live samples that will change substantially over time the initial calibration stack will become obsolete. The ImLock can be set to re-record the calibration stack at set intervals through the experiment to keep it relevant. If the interval between the experiment's time points is >5 seconds this will be done between time points so as not to disrupt the experiment.</p>
<h3>Settings</h3>
<h4>Configuration Tab</h4>
<ul>
<li><b>Background subtraction step size:</b> Lateral move between background subtraction images. We find that a step size of 50 um works well.</li>
<li><b>Calibration step size:</b> Lateral move between images during the pixels-to-nm calibration routine. We find that a step size of 1-5 um works well.</li>
<li><b>Exposure time:</b> Exposure time of the camera, should be balanced to minimise LED power and motion blur.</li>
<li><b>Maximum ROI to analyse:</b> Sets the size of the area to perform cross-correlation on. The limits of this are camera-dependent. Larger fields of view will require more processing time but use more of the sample to lock on to.</li>
<li><b>Pixels to trim from edges:</b> Removes edge pixels from cross-correlation calculations. Can be used to speed up processing if the camera has a minumum ROI limit.</li>
<li><b>Step size for Z correction:</b> Sets the depth between the images of the reference z-stack taken at the beginning of a run.</li>
<li><b>Zp (Axial proportional gain):</b> Proportional gain sets the strength of correction relative to the position error. higher values will cause faster responses but can lead to overcorrections. A value of 1 is a good starting point</li>
<li><b>Zi (Axial integral gain):</b> Integral gain used to correct for a stable offset from the zero position. Can induce oscillations if too strong. Often this value can be left at 0</li>
<li><b>Lp (Lateral proportional gain):</b> Proportional gain sets the strength of correction relative to the position error. higher values will cause faster responses but can lead to overcorrections. A value of 1 is a good starting point</li>
<li><b>Li (Lateral integral gain):</b> Integral gain used to correct for a stable offset from the zero position. Can induce oscillations if too strong. Often this value can be left at 0</li>
<li><b>Time between corrections:</b> Sets the dwell time between stage corrections to avoid high frequency corrections to noise. Must be set greater than the set exposure time.</li>
<li><b>Time between reference updates:</b> Sets the time between updates to the reference stack. Useful if your sample changes significantly over time. Setting to 0 turns this feature off.</li>
<li><b>Maximum translation:</b> Limit on correction size to prevent stage crashing.</li>
</ul>
<h4>Advanced Tab</h4>
<ul>
<li><b>Lateral Gain Bias:</b> Bias of gain between X and Y axies on a scale from -1 to 1. Positive values bias towards X and negative towards Y. Should not be needed for most systems.</li>
<li><b>Axial Minimum Move (nm):</b> Minimum move command that will be sent to the Z stage. Any smaller corrections will be ignored. Useful for stages with poor encoder resolution (usually not needed for piezo stages). Set to the same as the encoder resolution if you see odd behaviour when set to 0.</li>
<li><b>Lateral Minimum Move (nm):</b> Minimum move command that will be sent to the XY stage. Any smaller corrections will be ignored. Useful for stages with poor encoder resolution. Set to the same as the encoder resolution if you see odd behaviour when set to 0.</li>
</ul>
</div>
<div id="Single-Core" class="tabcontent">
<p>This is a secondary version designed to work robustly with any camera normally supported by Micro-Manager. It is designed to work within a second instance of Micro-Manager seperate to data collection. Working in separate Micro-Manager instances means some features require additional hardware so try the other version first.</p>
<h3>Installation</h3>
<a class="button-design" href="https://github.com/HoldenLab/DeepAutoFocus/tree/JE-SingleCore">👁 Get Software from GitHub 👁</a>
<p>From the folder "dist" in the GitHub repository, copy "ImLock.jar" and the content of the "lib" folder into the Micromanager "plugins/micro-manager" directory. In the case of these files already existing in the target directory, the target directory files should be retained.</p>
<p>The camera and stages used must be added to a MicroManager config file. Use this config file when launching Micro-Manager.</p>
<p>The plugin may now be launched in MicroManager from "plugins/Beta/ImLock". On opening the plugin will use the configuration already loaded by micromanager.</p>
<h3>Use</h3>
<p>The plugin does not have the ability to trigger the illumination LED so this must be controlled manually. If using this as part of the LifeHack system this can be done through MicroManager. Our testing has been conducted with the Tiger Controller's MAINTAIN(MA) setting set to "3 - Motors on during wait time".</p>
<img src="Images/ImLock_SingleCore.png" width="100%">
<p>To use the plugin, First "get background image" to perform background subtraction. The stage will scan around the sample to record multiple fields of view for averaging.</p>
<p>Next "calibrate pixel size" to calculate the pixels-to-nm conversion factor and axis orientations. The stage will move laterally in user-defined steps and calculate the pixel displacements by cross-correlation.</p>
<p>"Start" takes a reference z-stack (if using XYZ or Z correction) or a single reference image (if using XY correction) and begins the drift correction protocol selected using the pulldown menu.</p>
<p>The tick-box options on the control panel are mostly self explanitory. "Show drift plots" produces plots of the live position reading to aid with setting gains(described in "Variables").</p>
<p>If "Save drift data to file" is ticked, positions will be saved in a columnwise ".csv" file with columns "time, x, y, z".</p>
<h3>Settings</h3>
<h4>Configuration Tab</h4>
<ul>
<li><b>Background subtraction step size:</b> Lateral move between background subtraction images. We find that a step size of 50 um works well.</li>
<li><b>Calibration step size:</b> Lateral move between images during the pixels-to-nm calibration routine. We find that a step size of 1-5 um works well.</li>
<li><b>Exposure time:</b> Exposure time of the camera, should be balanced to minimise LED power and motion blur.</li>
<li><b>Maximum ROI to analyse:</b> Sets the size of the area to perform cross-correlation on. The limits of this are camera-dependent. Larger fields of view will require more processing time but use more of the sample to lock on to.</li>
<li><b>Pixels to trim from edges:</b> Removes edge pixels from cross-correlation calculations. Can be used to speed up processing if the camera has a minumum ROI limit.</li>
<li><b>Step size for Z correction:</b> Sets the depth between the images of the reference z-stack taken at the beginning of a run.</li>
<li><b>Zp (Axial proportional gain):</b> Proportional gain sets the strength of correction relative to the position error. higher values will cause faster responses but can lead to overcorrections. A value of 1 is a good starting point</li>
<li><b>Zi (Axial integral gain):</b> Integral gain used to correct for a stable offset from the zero position. Can induce oscillations if too strong. Often this value can be left at 0</li>
<li><b>Lp (Lateral proportional gain):</b> Proportional gain sets the strength of correction relative to the position error. higher values will cause faster responses but can lead to overcorrections. A value of 1 is a good starting point</li>
<li><b>Li (Lateral integral gain):</b> Integral gain used to correct for a stable offset from the zero position. Can induce oscillations if too strong. Often this value can be left at 0</li>
<li><b>Time between corrections:</b> Sets the dwell time between stage corrections to avoid high frequency corrections to noise. Must be set greater than the set exposure time.</li>
<li><b>Time between reference updates:</b> Sets the time between updates to the reference stack. Useful if your sample changes significantly over time. Setting to 0 turns this feature off.</li>
<li><b>Maximum translation:</b> Limit on shift to prevent stage crashing.</li>
</ul>
<h4>Advanced Tab</h4>
<ul>
<li><b>Lateral Gain Bias:</b> Bias of gain between X and Y axies on a scale from -1 to 1. Positive values bias towards X and negative towards Y. Should not be needed for most systems.</li>
<li><b>Axial Minimum Move (nm):</b> Minimum move command that will be sent to the Z stage. Any smaller corrections will be ignored. Useful for stages with poor encoder resolution (usually not needed for piezo stages). Set to the same as the encoder resolution if you see odd behaviour when set to 0.</li>
<li><b>Lateral Minimum Move (nm):</b> Minimum move command that will be sent to the XY stage. Any smaller corrections will be ignored. Useful for stages with poor encoder resolution. Set to the same as the encoder resolution if you see odd behaviour when set to 0.</li>
</ul>
<h3>Multi-Dimensional Acquisitions (Optional)</h3>
<p>To perform imaging across multiple fields of view and or Z-stacks while using this version of ImLock, add an Arduino uno to the Micro-Manager config file. This will need to be set up as an "Arduino-Input" device.</p>
<p>Upon opening the ImLock plugin go to the configuration panel and select the Arduino using the "Trigger" pulldown. The plugin is now set up to recieve external interupt commands via the Arduino.</p>
<p>If using Micro-Manager for data collection you will need to add a second arduino to the config file for the data collection instance of Micro-Manager. This will need to be set up as an "Arduino-Shutter" device with the output wired as below. Within Micromanager set this up as a Multi-shutter device with your acquisition shutter so that the two are triggered together. Connect this Arduino to the first Arduino as its input and the two instances of micromanager will be able to talk to eachother.</p>
<img src="Images/ImLock_Arduino.png" width="50%">
<p>If you are using other software you will need your own way of providing the interupt signals to the input Arduino, <a href=https://www.cairn-research.co.uk>cairn-research</a> may be able to help with this.</p>
</div>
</div>
<div class="bottomnav">
<a href="http://creativecommons.org/licenses/by-nc-sa/4.0/">License: Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.
<a href="https://blogs.ncl.ac.uk/holdenlab/">Holden Lab Website</a>
</div>
<script>
window.onscroll = function() {myFunction()};
var navbar = document.getElementById("navbar");
var sticky = navbar.offsetTop;
function myFunction() {
if (window.pageYOffset >= sticky) {
navbar.classList.add("sticky")
} else {
navbar.classList.remove("sticky");
}
}
function openVersion(evt, cityName) {
var i, tabcontent, tablinks;
tabcontent = document.getElementsByClassName("tabcontent");
for (i = 0; i < tabcontent.length; i++) {
tabcontent[i].style.display = "none";
}
tablinks = document.getElementsByClassName("tablinks");
for (i = 0; i < tablinks.length; i++) {
tablinks[i].className = tablinks[i].className.replace(" active", "");
}
document.getElementById(cityName).style.display = "block";
evt.currentTarget.className += " active";
}
document.getElementById("defaultOpen").click();
</script>
</body>
</html>