|
1 | | -# dbXmatch |
2 | | -In-database cross-match of astronomical objects. |
| 1 | +# [**SQLxMatch: In-Database Spatial Cross-Match of Astronomical Catalogs**](https://github.com/sciserver/SQLxMatch) |
3 | 2 |
|
| 3 | +##### Manuchehr Taghizadeh-Popp <sup>1*</sup> and Laszlo Dobos<sup>1,2</sup> |
| 4 | +<sup>1</sup> Department of Physics and Astronomy, Johns Hopkins University, Baltimore, MD, USA.<br> |
| 5 | +<sup>2</sup> Department of Physics of Complex Systems, Eötvös Loránd University, Budapest, Hungary.<br> |
| 6 | +<sup>*</sup> Leading contributor email: mtaghiza [at] jhu.edu | Help Desk: sciserver-helpdesk [at] jhu.edu |
| 7 | +<br><br> |
4 | 8 |
|
5 | | -<h3> Cross-matching objects in astronomical databases.</h3> |
| 9 | +`SQLxMatch` (or *sequel cross match*) is a SQL stored procedure that allows to perform 2-dimensional spatial cross-matches and cone searches across multiple astronomical catalogs stored in relational databases. |
| 10 | +This procedure implements the `Zones Algorithm` ([[1]](https://arxiv.org/abs/cs/0701171), [[2]](https://arxiv.org/abs/cs/0408031)), which leverages relational database algebra and B-Trees to cross-match the database tables or views containing the catalogs. To run a cross-match, these tables must simply contain at least the Right Ascension (RA) or Longitude, Declination (Dec) or Latitude, and unique object identifier (ID) columns. |
6 | 11 |
|
7 | | -[SciServer](https://apps.sciserver.org) offers the capability of cross-matching objects across multiple astronomical catalogs. These catalogs are available as tables in remote databases `in the cloud` through the [CasJobs](https://skyserver.sdss.org/CasJobs) web interface. |
| 12 | +We have integrated `SQLxMatch` with more than 50 astronomical catalogs, and made those publicly available as tables in remote SQL Server databases `in the cloud` through the [CasJobs](https://skyserver.sdss.org/CasJobs) website, as part of the [SciServer](https://www.sciserver.org) science platform ([[3]](https://www.sciencedirect.com/science/article/abs/pii/S2213133720300664)). <br> |
| 13 | +To improve the execution speed, we install the cross-match code in a SQL Server database supported by fast NVMe storage with a RAID 6 configuration. We also place the catalog tables in several databases in the same physical server, thus avoiding having to move data across servers with a potentially slower network conection. |
8 | 14 |
|
9 | | -The cross-match is run by executing a simple SQL stored procedure called `sp_xmatch`. This procedure implements the Zones Algorithm ([[1]](https://arxiv.org/abs/cs/0701171), [[2]](https://arxiv.org/abs/cs/0408031), which involves relational database algebra and B-Trees to run a 2-dimensional spatial cross-match between 2 catalogs of objects stored as database tables. These input tables must contain at least RA, Dec and ID columns. |
10 | 15 |
|
11 | | -The advantage of this <i>`in-database`</i> remote cross-match, compared to other <i>`in-memory`</i> local cross-match software libraries, is that it uses the remote database server's own computing/storage resources to filter and cross-match the full catalogs right away, having only a relatively small-sized cross-match output table returned to the user. |
12 | | -This can be faster and more efficient than having users to download the catalogs into their own computers (if they have big enough storage), and then load them in python for filtering and running the cross-match, for example. |
| 16 | +The advantage of this <i>`in-database`</i> remote cross-match, compared to other <i>`in-memory`</i> local cross-match software libraries, is that the users leverage the remote database server's own (and potentially bigger) computing/memory/storage resources to filter and cross-match the full catalogs right away, having only a relatively small-sized cross-match output table returned to them. |
| 17 | +This can be faster and more efficient than having users to download the full catalogs into their own computers (if they have enough storage), and then load them in python for filtering and running the cross-match, for instance. |
13 | 18 |
|
14 | 19 |
|
| 20 | + |
| 21 | +## **Documentation** |
| 22 | + |
| 23 | + |
| 24 | +Instructions on how to install and operate `SQLxMatch` can found under the [docs](https://github.com/sciserver/sqlxmatch/docs) folder. |
| 25 | + |
| 26 | + |
| 27 | +## **Examples** |
| 28 | + |
| 29 | +Example Jupyter Notebooks and demos can be found under [demo](https://github.com/sciserver/sqlxmatch/demo) folder. |
| 30 | + |
| 31 | + |
| 32 | +## **License** |
| 33 | + |
| 34 | +All code and contents of this repository are licensed under the Apache 2.0 license. For more details see the [LICENSE.txt](https://github.com/sciserver/sqlxmatch/LICENSE.txt) file. |
0 commit comments