A secure, browser-based application for visualizing high-dimensional 'omics and other data matrices with heatmaps, PCA, and t-SNE. V. 0.1
+
+
Video tutorials
+
+ Watch a 2-min video on
+ YouTube
+ to get a quick start. Here is another
+
+ video
+ on countries with negative population growth.
+
+
+
+
Overview
+
DataMap is a Shiny application designed to provide researchers and data analysts with an easy-to-use tool for visualizing high-dimensional data matrices. What makes DataMap unique is that it runs entirely in your browser through Shinylive technology, ensuring your data never leaves your device, providing both security and scalability.
+
+
+ Why DataMap?
+
+
Runs completely in your browser - no server required
+
Your data stays on your device, providing maximum security
+
Interactive visualizations including heatmaps, PCA, and t-SNE
+
Support for data transformations (scaling, normalization, etc.)
+
Exportable code to reproduce your analysis in R
+
Supports multiple data formats (CSV, TSV, TXT, Excel)
+
Customizable visualization parameters
+
+
+
+
+
+
Getting Started
+
+
Launching the Application
+
DataMap is accessed through your web browser. Simply navigate to the DataMap URL to launch the application. No installation is required.
+
+
Interface Overview
+
The DataMap interface consists of:
+
+
Left Sidebar: Contains controls for data upload, transformations, and visualization settings
+
Main Panel: Displays the visualizations in tabbed panels (Heatmap, PCA, t-SNE, Code)
+
+
+
+
+
Data Upload
+
+
Supported File Formats
+
DataMap supports the following file formats:
+
+
CSV files (.csv)
+
Tab-separated files (.tsv, .txt)
+
Excel files (.xls, .xlsx)
+
+
+
Data Format Requirements
+
Your data should be organized in a matrix format where:
+
+
Rows typically represent features (genes, metabolites, proteins, etc.)
+
Columns typically represent samples or observations
+
The first row may contain column headers (sample names)
+
The first column may contain row identifiers (feature names)
+
+
+
+
+
+
+
+
+
+
+
Name1
+
Name2
+
Name3
+
Name4
+
Name5
+
Name6
+
+
+
5.6
+
2.9
+
4.3
+
9.1
+
3.1
+
3.4
+
+
+
3.2
+
2.1
+
NA
+
2.8
+
1.5
+
2.9
+
+
+
8.9
+
8.7
+
9.0
+
8.5
+
3.5
+
2.4
+
+
+
1.9
+
8.0
+
1.0
+
1.5
+
2.5
+
1.8
+
+
+
1.0
+
2.1
+
1.5
+
3.1
+
6.6
+
7.8
+
+
+
The column names should not have duplicates. Missing values in the data matrix are tolerated.
+
+
+
+
+
+
GeneID
+
Name
+
Sample1
+
Sample2
+
Sample3
+
Sample4
+
Protein_coding
+
+
+
Gene1
+
DNA Repair 2
+
5.6
+
7.8
+
4.3
+
9.1
+
Yes
+
+
+
Gene2
+
Energy 3B
+
3.2
+
2.1
+
3.5
+
2.8
+
Yes
+
+
+
Gene3
+
Stress related 1
+
8.9
+
8.7
+
9.0
+
8.5
+
No
+
+
+
Gene4
+
Transporter 2
+
1.2
+
1.0
+
1.4
+
1.3
+
Yes
+
+
+
Gene5
+
+
6.7
+
6.9
+
6.2
+
7.1
+
Yes
+
+
+
The first column contains row names (here is gene ids).
+ The row must not contain duplicates. Otherwise, it will be ignored, just like the 2nd column.
+ You can have one or more columns with categorical data, such as the last one here.
+ Such information will be used to label rows. You can also save such columns in a separate
+ file to be uploaded as row annotations (See below).
+
+
+
+
Optional Annotation Files
+
DataMap allows you to upload additional annotation files to provide context for your data:
+
+
Column Annotation File
+
This file provides metadata about your samples (columns). The file should have:
+
+
Column names matching the column names in your main data file
+
Rows containing different annotations (e.g., treatment, time point, etc.)
+
+
+
+
SampleID
+
Sample1
+
Sample2
+
Sample3
+
Sample4
+
+
+
Treatment
+
Control
+
Treatment
+
Control
+
Treatment
+
+
+
TimePoint
+
0h
+
0h
+
24h
+
24h
+
+
+
+
Row Annotation File
+
This file provides metadata about your features (rows). The file should have:
+
+
Row names matching the row names in your main data file
+
Columns containing different annotations (e.g., pathways, functions, etc.)
+
+
+
+
GeneID
+
Pathway
+
Function
+
+
+
Gene1
+
Glycolysis
+
Enzyme
+
+
+
Gene2
+
TCA Cycle
+
Transporter
+
+
+
Gene3
+
Glycolysis
+
Transcription Factor
+
+
+
Gene4
+
TCA Cycle
+
Enzyme
+
+
+
Gene5
+
Lipid Metabolism
+
Signaling
+
+
+
+
Uploading Your Data
+
+
Click the Data Files button in the sidebar
+
Use the file upload controls to select and upload your files
+
Configure import settings (delimiter, headers, row names) in the dialog
+
Preview your data to ensure it's imported correctly
+
Click Import Data to load your data into the application
+
+
+
+ Tip: The application will attempt to auto-detect the optimal import settings, but you can adjust these manually if needed.
+
+
+
+
+
Data Transformation
+
+
DataMap provides various options for transforming your data to improve visualization and analysis:
+
+
Available Transformations
+
+
Log10 Transformation: Useful for skewed data with large ranges
+
Missing Value Handling: Replace missing values with zeros, means, or medians
+
Scaling Options:
+
+
Center by row: Subtract row means from each row
+
Scale by row (Z-score): Standardize each row to have mean 0 and standard deviation 1
+
Center by column: Subtract column means from each column
+
Scale by column (Z-score): Standardize each column to have mean 0 and standard deviation 1
+
+
+
Outlier Capping: Limit extreme values based on a Z-score threshold
+
Row Filtering: Keep only the most variable rows (features) for clearer visualization
+
+
+
Applying Transformations
+
+
Click the Data Prep. button in the sidebar
+
Select the desired transformations in the dialog
+
Preview the effect of your transformations in the histogram
+
Click Apply Changes to implement the transformations
+
+
+
+ Note: The application will recommend appropriate transformations based on your data characteristics, such as skewness and the presence of missing values.
+
+
+
+
+
Visualization Options
+
+
Heatmap
+
The heatmap visualization allows you to see patterns in your data matrix through color intensity.
+
+
Customization Options
+
+
Clustering: Enable/disable clustering for rows and columns
+
Clustering Method: Choose from various linkage methods (complete, average, single, ward.D, etc.)
t-SNE is a nonlinear dimensionality reduction technique that is particularly good at visualizing high-dimensional data clusters.
+
+
t-SNE Options
+
+
Analysis Mode: Choose whether to analyze row vectors or column vectors
+
Perplexity: Adjust the perplexity parameter to balance local and global structure (typically between 5-50)
+
Annotation Coloring: Automatically colors points by row or column annotations
+
+
+
+
+
Generating Reproducible Code
+
One of the most powerful features of DataMap is its ability to generate reproducible R code for your analysis.
+
+
Code Generation
+
The application automatically generates R code that replicates all steps of your analysis:
+
+
Data import with the correct parameters
+
Data transformations applied
+
Heatmap generation with all customized settings
+
PCA and t-SNE configurations
+
+
+
Accessing the Code
+
+
Navigate to the Code tab in the main panel
+
Review the generated R code
+
Click Download Code as R Script to save the code for later use
+
+
+
+ Tip: This feature is especially useful for documentation, publications, and ensuring reproducibility of your analysis.
+
+
+
+
+
Exporting Results
+
+
Saving Visualizations
+
You can save your visualizations in various formats:
+
+
PDF: Click the PDF button to download the current heatmap as a PDF file
+
PNG: Click the PNG button to download the current heatmap as a PNG image
+
+
+
+
+
+
Tips and Best Practices
+
+
Data Preparation
+
+
Data Size: While the application can handle large matrices, performance may decrease with extremely large datasets (e.g., >50,000 rows). Consider filtering your data to the most relevant features before uploading.
+
Missing Values: Consider how to handle missing values based on your data type. For gene expression data, replacing with zeros often makes sense, while for other data types, means or medians might be more appropriate.
+
Log Transformation: For gene expression data and many other biological datasets, log transformation often improves visualization by reducing the effect of extreme values.
+
+
+
Visualization
+
+
Row Names: Displaying row names on large matrices can make the heatmap unreadable. Use the toggle to show or hide row names as needed.
+
Clustering: Different distance metrics and linkage methods can reveal different patterns. Explore multiple combinations to find the most informative visualization for your data.
+
Color Palettes: For divergent data (with both positive and negative values), use divergent color schemes like RdBu. For sequential data, gradients like YlOrRd or Blues are more appropriate.
+
+
+
Troubleshooting
+
+
Import Issues: If your data isn't importing correctly, check for special characters in headers, mixed data types, or unusual delimiters.
+
Missing Row/Column Annotations: Ensure that the identifiers in your annotation files exactly match those in your main data file.
+
t-SNE Performance: t-SNE can be computationally intensive for large datasets. If you experience slow performance, try reducing the number of rows with the row filtering option.
+
+
+
+
+
FAQ
+
+
General Questions
+
+
Is my data secure?
+
Yes. DataMap runs entirely in your browser using WebAssembly technology. Your data never leaves your device and is not sent to any server, providing maximum security and privacy.
+
+
What is the maximum file size I can upload?
+
Since the application runs in your browser, the maximum file size depends on your device's memory. However, this browser app gets slow when there are 5000 rows or columns. Users can install DataMap as an R package to run it natively.
+
+
Can I analyze RNA-seq (bulk or single cell) data?
+
Yes, DataMap is particularly well-suited for analyzing gene expression data from RNA-seq or microarray experiments. The scaling and transformation options are designed with these data types in mind.
+
+
How do I cite DataMap in my publication?
+
If you use DataMap in your research, please cite it as: "Ge, X. (2025). DataMap: A secure browser-based application for visualizing high-dimensional data matrices. [Software]. Available from: [URL]"
+
+
What browsers are supported?
+
DataMap works best with modern browsers like Chrome, Firefox, Edge, and Safari. We recommend using the latest version of your preferred browser for optimal performance.
+
+
My visualization is taking a long time to generate. What can I do?
+
For large datasets, consider these strategies to improve performance:
+
+
Try install DataMap as an R package. See instruction on the GitHub page.
+
Filter your data to fewer rows (use the "Keep top most variable rows" option)
+
Use simpler clustering methods (like "single" or "average" instead of "complete" or "ward.D")
+
Reduce the perplexity value for t-SNE
+
+
+
Can I use DataMap offline?
+
Yes, once loaded in your browser, DataMap can operate without an internet connection. You can bookmark the page for offline access in some browsers.
+
+
\ No newline at end of file
diff --git a/inst/www/pca.png b/inst/www/pca.png
new file mode 100644
index 0000000..eaa4f4a
Binary files /dev/null and b/inst/www/pca.png differ
diff --git a/inst/www/tsne.png b/inst/www/tsne.png
new file mode 100644
index 0000000..a49e317
Binary files /dev/null and b/inst/www/tsne.png differ
diff --git a/man/create_dr_plot.Rd b/man/create_dr_plot.Rd
new file mode 100644
index 0000000..0e9e0c5
--- /dev/null
+++ b/man/create_dr_plot.Rd
@@ -0,0 +1,54 @@
+% Generated by roxygen2: do not edit by hand
+% Please edit documentation in R/utilities.R
+\name{create_dr_plot}
+\alias{create_dr_plot}
+\title{Create Dimensionality Reduction Plot}
+\usage{
+create_dr_plot(
+ coords_data,
+ x_label,
+ y_label,
+ point_annot = NULL,
+ fontsize = 12,
+ show_labels = FALSE,
+ point_labels = NULL
+)
+}
+\arguments{
+\item{coords_data}{A numeric matrix or data frame containing at least two columns for the x and y coordinates.}
+
+\item{x_label}{A character string specifying the label for the x-axis.}
+
+\item{y_label}{A character string specifying the label for the y-axis.}
+
+\item{point_annot}{Optional data frame containing annotation data. The first column is used for assigning colors,
+and if a second column is provided, its values are used for assigning point shapes.}
+
+\item{fontsize}{A numeric value specifying the base font size used for labels, axes, and legends. Default is 12.}
+
+\item{show_labels}{Logical flag indicating whether to display text labels next to points. Default is FALSE.}
+
+\item{point_labels}{Optional character vector containing labels for each point. Must have at least as many elements as rows in coords_data when show_labels is TRUE.}
+}
+\value{
+A recorded plot object that can be replayed using \code{replayPlot()}.
+}
+\description{
+Generates a scatter plot for dimensionality reduction with support for annotations that alter
+point colors and shapes, and optionally includes point labels.
+}
+\details{
+The function sets up the plotting environment, adjusts plot limits based on whether labels are shown,
+and manages legends for both color and shape annotations. If annotation data is provided, the first column
+determines the color palette (using a rainbow palette) and the second column (if available) assigns point shapes.
+}
+\examples{
+# Example with simulated data:
+coords <- matrix(rnorm(200), ncol = 2)
+annot <- data.frame(Group = sample(c("A", "B", "C"), 100, replace = TRUE),
+ Type = sample(c("X", "Y"), 100, replace = TRUE))
+plot_obj <- create_dr_plot(coords, "X Axis", "Y Axis", point_annot = annot, show_labels = TRUE,
+ point_labels = paste("P", 1:100, sep=""))
+replayPlot(plot_obj)
+
+}
diff --git a/man/datamap-package.Rd b/man/datamap-package.Rd
new file mode 100644
index 0000000..3203633
--- /dev/null
+++ b/man/datamap-package.Rd
@@ -0,0 +1,15 @@
+% Generated by roxygen2: do not edit by hand
+% Please edit documentation in R/datamap-package.R
+\docType{package}
+\name{datamap-package}
+\alias{datamap}
+\alias{datamap-package}
+\title{datamap: Interactive Visualization of Data Matrices}
+\description{
+A Shiny application for visualizing data matrices with heatmaps, PCA, and t-SNE. Allows users to upload, transform, and visualize their data interactively.
+}
+\author{
+\strong{Maintainer}: Steven Ge \email{Xijin.Ge@sdstate.edu}
+
+}
+\keyword{internal}
diff --git a/man/datamap_resource.Rd b/man/datamap_resource.Rd
new file mode 100644
index 0000000..2512312
--- /dev/null
+++ b/man/datamap_resource.Rd
@@ -0,0 +1,18 @@
+% Generated by roxygen2: do not edit by hand
+% Please edit documentation in R/utilities.R
+\name{datamap_resource}
+\alias{datamap_resource}
+\title{Find resource files for datamap}
+\usage{
+datamap_resource(path)
+}
+\arguments{
+\item{path}{Path to the resource within the package}
+}
+\value{
+The full path to the resource file
+}
+\description{
+Find resource files for datamap
+}
+\keyword{internal}
diff --git a/man/process_column_annotations.Rd b/man/process_column_annotations.Rd
new file mode 100644
index 0000000..ddda69b
--- /dev/null
+++ b/man/process_column_annotations.Rd
@@ -0,0 +1,24 @@
+% Generated by roxygen2: do not edit by hand
+% Please edit documentation in R/utilities.R
+\name{process_column_annotations}
+\alias{process_column_annotations}
+\title{Process Column Annotations for Heatmap}
+\usage{
+process_column_annotations(main_data_cols, annotation_df, selected_annotations)
+}
+\arguments{
+\item{main_data_cols}{A character vector of column names corresponding to the main dataset.}
+
+\item{annotation_df}{A data frame containing annotation information where rows correspond to different annotation types
+and columns represent samples.}
+
+\item{selected_annotations}{A vector indicating the rows (by their names or indices) in annotation_df to be selected.}
+}
+\value{
+A data frame with the main dataset's column names as its row names and the selected annotation types as its column names.
+If there are no annotations selected, no common samples between datasets, or if all annotations fail to process, the function returns NULL.
+}
+\description{
+This function processes column annotations by aligning selected annotation rows with the main dataset's columns.
+It attempts to safely subset and merge annotation data based on common sample names and handles errors during processing.
+}
diff --git a/man/process_row_annotations.Rd b/man/process_row_annotations.Rd
new file mode 100644
index 0000000..fc07725
--- /dev/null
+++ b/man/process_row_annotations.Rd
@@ -0,0 +1,34 @@
+% Generated by roxygen2: do not edit by hand
+% Please edit documentation in R/utilities.R
+\name{process_row_annotations}
+\alias{process_row_annotations}
+\title{Process Row Annotations}
+\usage{
+process_row_annotations(
+ main_data_rows,
+ file_annotation_df = NULL,
+ factor_annotation_df = NULL,
+ selected_annotations
+)
+}
+\arguments{
+\item{main_data_rows}{A vector of row identifiers from the main data set.}
+
+\item{file_annotation_df}{An optional data frame containing row annotations from an uploaded file.
+Default is NULL.}
+
+\item{factor_annotation_df}{An optional data frame containing row annotations based on factors.
+Default is NULL.}
+
+\item{selected_annotations}{A character vector specifying the names of annotations to extract.}
+}
+\value{
+A data frame with combined annotations that correspond to the main data rows. Returns
+NULL if no annotations are selected or if no matching annotations are found.
+}
+\description{
+This function processes and combines annotation data for rows from two potential sources:
+a file-uploaded annotation data frame and an auto-detected factor annotation data frame. It selects
+desired annotation columns based on the provided list, aligns these annotations with the main data rows,
+and merges them into a single data frame.
+}
diff --git a/man/run_app.Rd b/man/run_app.Rd
new file mode 100644
index 0000000..345567b
--- /dev/null
+++ b/man/run_app.Rd
@@ -0,0 +1,19 @@
+% Generated by roxygen2: do not edit by hand
+% Please edit documentation in R/main_app.R
+\name{run_app}
+\alias{run_app}
+\title{Run the DataMap Shiny application}
+\usage{
+run_app()
+}
+\value{
+A Shiny application object
+}
+\description{
+Run the DataMap Shiny application
+}
+\examples{
+if (interactive()) {
+ run_app()
+}
+}
diff --git a/man/transform_server.Rd b/man/transform_server.Rd
new file mode 100644
index 0000000..3130bb0
--- /dev/null
+++ b/man/transform_server.Rd
@@ -0,0 +1,19 @@
+% Generated by roxygen2: do not edit by hand
+% Please edit documentation in R/mod_transform.R
+\name{transform_server}
+\alias{transform_server}
+\title{Server function for preprocessing module}
+\usage{
+transform_server(id, data)
+}
+\arguments{
+\item{id}{The module namespace id}
+
+\item{data}{Reactive data frame to process}
+}
+\value{
+A list with the processed data reactive and reproducible code
+}
+\description{
+Server function for preprocessing module
+}
diff --git a/man/transform_ui.Rd b/man/transform_ui.Rd
new file mode 100644
index 0000000..c3f5121
--- /dev/null
+++ b/man/transform_ui.Rd
@@ -0,0 +1,17 @@
+% Generated by roxygen2: do not edit by hand
+% Please edit documentation in R/mod_transform.R
+\name{transform_ui}
+\alias{transform_ui}
+\title{UI function for preprocessing module button}
+\usage{
+transform_ui(id)
+}
+\arguments{
+\item{id}{The module namespace id}
+}
+\value{
+A button that triggers the preprocessing modal
+}
+\description{
+UI function for preprocessing module button
+}