DeepResearchLayers

A Python library for novel and experimental deep learning layers.

deep_layers bridges the gap between mathematical theory and usable code, providing "plug-and-play" implementations for PyTorch (and TensorFlow/Keras) of complex layers from research papers.

Features

Dual-Backend Compatibility

Supports both PyTorch and TensorFlow (via deep_layers/torch and deep_layers/tf).

Implemented Layers

All layers are available for both PyTorch and TensorFlow.

deep_layers.vision

• CoordConv (coord_conv): Coordinate Convolution.
• DropBlock (dropblock): Structured dropout regularization.
• GLU (glu): Gated Linear Unit.
• Involution (involution): Inverted convolution.

deep_layers.sequence

• Hyena (hyena): Hyena Hierarchy operator.
• Linear Attention (linear_attention): Transformers are RNNs.
• Mamba (mamba): Selective State Space Model.
• Retention (retention): RetNet layer.

deep_layers.graph

• GCN (gcn): Graph Convolutional Network.
• NTN (ntn): Neural Tensor Network.
• PointNet (pointnet): PointNet Set Abstraction.
• Set Transformer (set_transformer): Permutation-invariant attention.
• SGR (sgr): Symbolic Graph Reasoning.

deep_layers.scientific

• CORAL (coral): Coordinate-based Neural Field Operator.
• DeepONet (deeponet): Deep Operator Network.
• DEQ (deq): Deep Equilibrium Models.
• HyperLayer (hyperlayer): HyperNetwork-based dynamic layer.
• KAN (kan): Kolmogorov-Arnold Network.
• Neural ODE (neural_ode): Ordinary Differential Equation solver layer.
• PhyCRNet (phycrnet): Physics-Informed Convolutional-Recurrent.
• PirateNet (piratenet): Physics-Informed Residual Adaptive Network.
• Sparse Memory (sparse_memory): Differentiable memory with sparse reads/writes.
• Steerable Conv (steerable_conv): E(2)-Equivariant Steerable CNN.
• VQ (vq): Vector Quantization layer.

Installation

Install with PyTorch support:

pip install "deep_layers[torch]"

Install with TensorFlow support:

pip install "deep_layers[tf]"

Install with both:

pip install "deep_layers[all]"

For development/editable install:

git clone https://github.com/yourusername/deep_layers.git
cd deep_layers
pip install -e .

Usage

PyTorch Example

import torch
from deep_layers.torch.vision import CoordConv

# Initialize layer
layer = CoordConv(in_channels=3, out_channels=64, kernel_size=3)

# Forward pass
x = torch.randn(1, 3, 224, 224)
output = layer(x)
print(output.shape)

TensorFlow Example

import tensorflow as tf
from deep_layers.tf.vision import CoordConv

# Initialize layer
layer = CoordConv(filters=64, kernel_size=3)

# Forward pass
x = tf.random.normal((1, 224, 224, 3))
output = layer(x)
print(output.shape)