forked from Shifty/pyserveX
537b783726
- Create a GitHub Actions workflow for testing with Python 3.12 and 3.13. - Update Makefile to include release management commands and pipeline checks. - Document the CI/CD pipeline structure and usage in PIPELINE.md. - Add structlog for structured logging and enhance logging utilities. - Implement release management script for automated versioning and tagging. - Modify logging configuration to support structured logging and improved formatting. - Update dependencies in pyproject.toml and poetry.lock to include structlog. - Enhance access logging in server and middleware to include structured data.
179 lines
4.0 KiB
Markdown
179 lines
4.0 KiB
Markdown
# CI/CD Pipeline for PyServeX
|
|
|
|
This document describes the complete CI/CD pipeline for the PyServeX project, including linting, testing, building, and automated release creation.
|
|
|
|
## Pipeline Structure
|
|
|
|
### 1. Linting Stage (`lint.yaml`)
|
|
**Triggers:**
|
|
- Push to any branch
|
|
- Pull request to any branch
|
|
|
|
**Checks:**
|
|
- Black (code formatting)
|
|
- isort (import sorting)
|
|
- flake8 (code analysis)
|
|
- mypy (type checking)
|
|
|
|
### 2. Testing Stage (`test.yaml`)
|
|
**Triggers:**
|
|
- Push to branches: `dev`, `master`, `main`
|
|
- Pull request to branches: `dev`, `master`, `main`
|
|
|
|
**Checks:**
|
|
- pytest on Python 3.12 and 3.13
|
|
- Coverage report generation
|
|
- Artifact storage with reports
|
|
|
|
### 3. Build and Release Stage (`release.yaml`)
|
|
**Triggers:**
|
|
- Push tag matching `v*.*.*`
|
|
- Manual trigger through Gitea interface
|
|
|
|
**Actions:**
|
|
- Package build via Poetry
|
|
- Changelog generation
|
|
- Draft release creation
|
|
- Artifact upload (.whl and .tar.gz)
|
|
|
|
### 4. Main Pipeline (`pipeline.yaml`)
|
|
Coordinates execution of all stages and provides results summary.
|
|
|
|
## How to Use
|
|
|
|
### Local Development
|
|
|
|
```bash
|
|
# Environment initialization
|
|
make init
|
|
|
|
# Check all stages locally
|
|
make pipeline-check
|
|
|
|
# Code formatting
|
|
make format
|
|
|
|
# Run tests
|
|
make test-cov
|
|
```
|
|
|
|
### Creating a Release
|
|
|
|
#### Automatic method (recommended):
|
|
```bash
|
|
# Patch release (x.x.X)
|
|
make release-patch
|
|
|
|
# Minor release (x.X.0)
|
|
make release-minor
|
|
|
|
# Major release (X.0.0)
|
|
make release-major
|
|
```
|
|
|
|
#### Manual method:
|
|
```bash
|
|
# 1. Update version
|
|
poetry version patch # or minor/major
|
|
|
|
# 2. Commit changes
|
|
git add pyproject.toml
|
|
git commit -m "bump version to $(poetry version -s)"
|
|
|
|
# 3. Create tag
|
|
git tag v$(poetry version -s)
|
|
|
|
# 4. Push to server
|
|
git push origin main
|
|
git push origin v$(poetry version -s)
|
|
```
|
|
|
|
## Working with Releases
|
|
|
|
### What happens automatically:
|
|
1. **When tag** `v*.*.*` is created, pipeline starts
|
|
2. **Linting executes** - code quality check
|
|
3. **Tests run** - functionality verification
|
|
4. **Package builds** - wheel and tarball creation
|
|
5. **Draft release created** - automatic creation in Gitea
|
|
|
|
### What needs manual action:
|
|
1. **Go to Gitea interface** in Releases section
|
|
2. **Find created draft** release
|
|
3. **Edit description** according to template in `RELEASE_TEMPLATE.md`
|
|
4. **Publish release** (remove "Draft" status)
|
|
|
|
## Configuration
|
|
|
|
### Pipeline files:
|
|
- `.gitea/workflows/lint.yaml` - Linting
|
|
- `.gitea/workflows/test.yaml` - Testing
|
|
- `.gitea/workflows/release.yaml` - Build and release
|
|
- `.gitea/workflows/pipeline.yaml` - Main pipeline
|
|
- `.gitea/RELEASE_TEMPLATE.md` - Release template
|
|
|
|
### Scripts:
|
|
- `scripts/release.sh` - Automated release creation
|
|
- `Makefile` - Development and release commands
|
|
|
|
## Environment Setup
|
|
|
|
### Gitea Actions variables:
|
|
- `GITHUB_TOKEN` - for release creation (usually configured automatically)
|
|
|
|
### Access permissions:
|
|
- Repository release creation permissions
|
|
- Tag push permissions
|
|
|
|
## Monitoring
|
|
|
|
### Stage statuses:
|
|
- **Success** - stage completed successfully
|
|
- **Failure** - stage failed with error
|
|
- **Skipped** - stage skipped (e.g., tests for non-listed branches)
|
|
|
|
### Artifacts:
|
|
- **Coverage reports** - test coverage reports
|
|
- **Build artifacts** - built packages (.whl, .tar.gz)
|
|
- **Changelog** - automatically generated changelog
|
|
|
|
## Troubleshooting
|
|
|
|
### Common issues:
|
|
|
|
1. **Linting fails:**
|
|
```bash
|
|
make format # Auto-formatting
|
|
make lint # Check issues
|
|
```
|
|
|
|
2. **Tests fail:**
|
|
```bash
|
|
make test # Local test run
|
|
```
|
|
|
|
3. **Build error:**
|
|
```bash
|
|
make clean # Clean temporary files
|
|
make build # Rebuild
|
|
```
|
|
|
|
4. **Tag already exists:**
|
|
```bash
|
|
git tag -d v1.0.0 # Delete locally
|
|
git push origin :refs/tags/v1.0.0 # Delete on server
|
|
```
|
|
|
|
## Additional Resources
|
|
|
|
- [Poetry documentation](https://python-poetry.org/docs/)
|
|
- [Gitea Actions documentation](https://docs.gitea.io/en-us/actions/)
|
|
- [Pytest documentation](https://docs.pytest.org/)
|
|
- [Black documentation](https://black.readthedocs.io/)
|
|
|
|
---
|
|
|
|
**Author:** Ilya Glazunov
|
|
**Project:** PyServeX
|
|
**Documentation version:** 1.0
|