Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
193 changes: 193 additions & 0 deletions CONDA_PUBLISHING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,193 @@
# Publishing protegrity-developer-python to Conda

This guide provides instructions for building and publishing the `protegrity-developer-python` package to Anaconda.org (conda) under the Protegrity organization.

## Prerequisites

Before you begin, ensure you have the following installed:

1. **Anaconda or Miniconda**
```bash
# Download and install from: https://docs.conda.io/en/latest/miniconda.html
```

2. **conda-build**
```bash
conda install conda-build
```

3. **anaconda-client** (for uploading to Anaconda.org)
```bash
conda install anaconda-client
```

4. **Account Setup**
- Create an account at https://anaconda.org
- Create or join the "Protegrity" organization on Anaconda.org
- Get appropriate permissions to publish packages under the organization

## Building the Conda Package

### Step 1: Navigate to the Project Root

```bash
cd /home/yigalr/projects/pty-dev
Copy link

Copilot AI Nov 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This path appears to be a personal/local directory path (/home/yigalr/projects/pty-dev) that should not be included in documentation. It should be replaced with a generic placeholder like cd /path/to/protegrity-developer-python or simply removed since the build command can be run from any location within the project root.

Suggested change
cd /home/yigalr/projects/pty-dev
cd /path/to/protegrity-developer-python

Copilot uses AI. Check for mistakes.
```

### Step 2: Build the Package

Build the conda package using the recipe in the `conda-recipe/` directory:

```bash
conda build conda-recipe/
```

This will:
- Download the source from PyPI (or use local source if configured)
Copy link

Copilot AI Nov 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The description states "Download the source from PyPI (or use local source if configured)", but the current meta.yaml is configured to use local source (path: ..) and there is no PyPI source URL configured. This is misleading and should be updated to accurately reflect that the recipe builds from local source by default.

Suggested change
- Download the source from PyPI (or use local source if configured)
- Use the local source directory to build the package (as configured in meta.yaml)

Copilot uses AI. Check for mistakes.
- Create the conda package
- Run tests to verify the package works correctly
- Output the package location (typically in your conda-bld directory)

### Step 3: Verify the Build

After a successful build, conda-build will display the path to the created package, something like:
```
anaconda upload /path/to/conda-bld/noarch/protegrity-developer-python-1.0.0-py_0.tar.bz2
```

You can test the package locally before uploading:

```bash
# Create a test environment
conda create -n test-protegrity python=3.12

# Activate the environment
conda activate test-protegrity

# Install the locally built package
conda install --use-local protegrity-developer-python

# Test the import
python -c "import protegrity_developer_python; import appython; print('Success!')"

# Clean up
conda deactivate
conda env remove -n test-protegrity
```

## Publishing to Anaconda.org

### Step 1: Login to Anaconda.org

```bash
anaconda login
```

Enter your Anaconda.org credentials when prompted.

### Step 2: Upload the Package

Upload the package to the Protegrity organization channel:

```bash
anaconda upload -u Protegrity /path/to/conda-bld/noarch/protegrity-developer-python-1.0.0-py_0.tar.bz2
```

Replace `/path/to/conda-bld/noarch/protegrity-developer-python-1.0.0-py_0.tar.bz2` with the actual path from your build output.

**Options:**
- Add `--force` to overwrite an existing package version
- Add `--label main` to publish to the main channel (default)
- Add `--label dev` to publish to a development channel

### Step 3: Verify the Upload

Visit your package page:
```
https://anaconda.org/Protegrity/protegrity-developer-python
```

## Installing the Published Package

Once published, users can install the package using:

```bash
# Install from the Protegrity channel
conda install -c Protegrity protegrity-developer-python

# Or add the channel permanently
conda config --add channels Protegrity
conda install protegrity-developer-python
```

## Building from Local Source (Alternative)

If you want to build from local source instead of PyPI:

1. Edit `conda-recipe/meta.yaml` and uncomment the `path: ..` line in the `source` section
2. Comment out or remove the `url` line
3. Run the build command from the project root:
Comment on lines +128 to +129
Copy link

Copilot AI Nov 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation refers to a url line in the source section that needs to be commented out or removed when building from local source. However, the current meta.yaml file only contains path: .. and no url line exists. This instruction is misleading and should be removed or updated to reflect the actual file content.

Suggested change
2. Comment out or remove the `url` line
3. Run the build command from the project root:
2. Run the build command from the project root:

Copilot uses AI. Check for mistakes.
```bash
conda build conda-recipe/
```

## Updating the Package

To publish a new version:

1. Update the version in `pyproject.toml`
2. Update the version in `conda-recipe/meta.yaml`
3. Increment the build number in `meta.yaml` if needed (for same version but different build)
4. Rebuild and upload following the steps above

## Troubleshooting

### Build Failures

- **Missing dependencies**: Ensure all dependencies are available in conda channels
- **Test failures**: Check the test section in `meta.yaml`
- **Python version issues**: Verify the Python version constraints

### Upload Issues

- **Authentication failed**: Run `anaconda login` again
- **Permission denied**: Ensure you have upload permissions for the Protegrity organization
- **Package exists**: Use `--force` flag to overwrite, or increment the build number

### Getting Help

- Conda build documentation: https://docs.conda.io/projects/conda-build/
- Anaconda.org documentation: https://docs.anaconda.com/anacondaorg/

## CI/CD Integration (Optional)

For automated publishing, you can integrate conda builds into your CI/CD pipeline:

```yaml
# Example GitHub Actions workflow snippet
- name: Build conda package
run: |
conda install conda-build anaconda-client
conda build conda-recipe/

- name: Upload to Anaconda
env:
ANACONDA_TOKEN: ${{ secrets.ANACONDA_TOKEN }}
run: |
anaconda -t $ANACONDA_TOKEN upload -u Protegrity /path/to/package/*.tar.bz2
```

Store your Anaconda token as a secret in your CI/CD system.

## Package Maintenance

- Regularly update dependencies
- Test against new Python versions
- Monitor user feedback and issues
- Keep documentation up to date
- Follow semantic versioning for releases

## Support

For issues specific to the conda package, please file an issue at:
https://github.com/Protegrity-Developer-Edition/protegrity-developer-python/issues
3 changes: 3 additions & 0 deletions conda-recipe/bld.bat
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
REM Install the package using pip
%PYTHON% -m pip install . -vv
if errorlevel 1 exit 1
4 changes: 4 additions & 0 deletions conda-recipe/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/bin/bash
Copy link

Copilot AI Nov 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Unix build script lacks error handling. If the pip install command fails, the script will continue silently. Consider adding set -e at the beginning of the script (after the shebang) to ensure the script exits immediately if any command fails, or add explicit error checking like || exit 1 after the pip install command, similar to the Windows build script.

Suggested change
#!/bin/bash
#!/bin/bash
set -e

Copilot uses AI. Check for mistakes.

# Install the package using pip
$PYTHON -m pip install . -vv
53 changes: 53 additions & 0 deletions conda-recipe/meta.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
{% set name = "protegrity-developer-python" %}
{% set version = "1.0.0" %}
Copy link

Copilot AI Nov 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The version is hardcoded in the conda recipe, which means it needs to be manually updated whenever the version in pyproject.toml changes. Consider using a dynamic approach to extract the version from pyproject.toml to ensure version consistency and reduce maintenance overhead. You can use Jinja2 to load the version from a file or use {{ load_file_regex(pattern='version = "(.*)"', file='pyproject.toml')[1] }}.

Suggested change
{% set version = "1.0.0" %}
{% set version = load_file_regex(pattern='version = "(.*)"', file='../pyproject.toml')[1] %}

Copilot uses AI. Check for mistakes.

package:
name: {{ name|lower }}
version: {{ version }}

source:
# Building from local source
path: ..

build:
number: 0
noarch: python

requirements:
host:
- python >=3.12.11
- pip
- setuptools >=61.0
run:
- python >=3.12.11
Comment on lines +18 to +22
Copy link

Copilot AI Nov 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Python version constraint >=3.12.11 is overly restrictive. Python patch versions (the third number) are typically used for bug fixes within a minor release, and requiring a specific patch version is unusual unless there's a critical security fix or bug that affects the package. Consider using >=3.12 or >=3.12.0 instead to allow any Python 3.12.x version, which is the more common pattern in conda packages and aligns with the README badge that shows "Python 3.12+".

Suggested change
- python >=3.12.11
- pip
- setuptools >=61.0
run:
- python >=3.12.11
- python >=3.12
- pip
- setuptools >=61.0
run:
- python >=3.12

Copilot uses AI. Check for mistakes.
- requests

test:
imports:
- protegrity_developer_python
- appython
commands:
- pip check
requires:
- pip

about:
home: https://www.protegrity.com/developers
license: MIT
license_family: MIT
license_file: LICENSE
summary: Python module for integrating Protegrity's Data Discovery and Protection APIs into GenAI and traditional applications
description: |
This repository contains two powerful modules designed to handle different aspects of data protection:

- protegrity_developer_python - Focuses on data discovery, classification, and redaction of
Personally Identifiable Information (PII) in unstructured text
- appython - Provides comprehensive data protection and unprotection capabilities for structured data

Part of the Protegrity Developer Edition suite for integrating Protegrity's APIs into applications.
doc_url: http://developer.docs.protegrity.com
dev_url: https://github.com/Protegrity-Developer-Edition/protegrity-developer-python

extra:
recipe-maintainers:
- Protegrity