GitHub Actions: Python CI/CD Pipeline

TL;DR

• Bottom line: GitHub Actions provides native CI/CD for Python projects with matrix testing across versions, built-in pip/poetry caching, pytest integration, and PyPI publishing via trusted publishing (OIDC).
• Key tool/command: actions/setup-python@v5 with cache: 'pip' and pip install -r requirements.txt.
• Watch out for: Not pinning dependency versions in requirements.txt -- floating versions cause non-deterministic builds.
• Works with: Python 3.9, 3.10, 3.11, 3.12, 3.13 on ubuntu-latest, windows-latest, macos-latest.

Constraints

• Always pin Python version explicitly (e.g., '3.12') -- never use just python3 which varies across runner images.
• Use pip install -r requirements.txt with pinned versions -- avoid floating version specifiers in CI.
• Python 3.7 and 3.8 are EOL -- minimum version for new projects should be 3.9+.
• Pin action versions to major tags (e.g., actions/setup-python@v5) -- never use @main in production.
• Use OIDC trusted publishing for PyPI uploads -- never store PyPI passwords as secrets.

Quick Reference

Decision Tree

START
├── Library published to PyPI?
│   ├── YES → Matrix [3.10, 3.11, 3.12, 3.13] + tox + build + publish via OIDC
│   └── NO ↓
├── Django/Flask/FastAPI application?
│   ├── YES → Single Python version + pytest + deploy job with environment protection
│   └── NO ↓
├── Data science / ML project?
│   ├── YES → Single version + conda/mamba setup + large runner for GPU tests
│   └── NO ↓
├── CLI tool?
│   ├── YES → Matrix versions + pytest + build with PyInstaller or shiv
│   └── NO ↓
└── DEFAULT → Basic CI: checkout → setup-python → pip install → pytest

Step-by-Step Guide

1. Create the workflow file

Create .github/workflows/ci.yml in your repository. [src1]

name: Python CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

permissions:
  contents: read

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ['3.10', '3.11', '3.12', '3.13']
    steps:
      - uses: actions/checkout@v4
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
          cache: 'pip'
      - run: pip install -r requirements.txt
      - run: pytest

Verify: Push to GitHub → Actions tab → see 4 parallel matrix jobs.

2. Add linting and type checking

Run linters before tests to fail fast. [src4]

  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
          cache: 'pip'
      - run: pip install ruff mypy
      - run: ruff check .
      - run: ruff format --check .

Verify: Introduce a lint error → push → workflow fails at ruff check step.

3. Configure pip caching

The cache: 'pip' option caches the pip download directory. [src2]

      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
          cache: 'pip'
          cache-dependency-path: |
            requirements.txt
            requirements-dev.txt

Verify: Check workflow logs for "Cache restored" on second run.

4. Add test coverage with pytest

Generate coverage reports and upload as artifacts. [src7]

      - run: pip install pytest pytest-cov
      - run: pytest --cov=src --cov-report=xml --cov-report=html --junitxml=report.xml
      - name: Upload coverage
        if: matrix.python-version == '3.12'
        uses: actions/upload-artifact@v4
        with:
          name: coverage-report
          path: htmlcov/

Verify: After workflow completes → Artifacts section shows coverage-report.

5. Publish to PyPI with trusted publishing

Use OIDC trusted publishing -- no secrets needed. [src5]

  publish:
    needs: test
    runs-on: ubuntu-latest
    if: github.event_name == 'release'
    environment: pypi
    permissions:
      id-token: write
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - run: pip install build
      - run: python -m build
      - uses: pypa/gh-action-pypi-publish@release/v1

Verify: Create a GitHub Release → check pypi.org for new version.

Code Examples

Complete CI/CD workflow with lint, test matrix, and deploy

# .github/workflows/ci.yml
name: Python CI/CD

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

permissions:
  contents: read

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
          cache: 'pip'
      - run: pip install ruff mypy
      - run: ruff check .
      - run: ruff format --check .

  test:
    runs-on: ubuntu-latest
    needs: lint
    strategy:
      fail-fast: false
      matrix:
        python-version: ['3.10', '3.11', '3.12', '3.13']
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
          cache: 'pip'
      - run: pip install -r requirements.txt
      - run: pytest --tb=short -q --junitxml=report.xml --cov=src --cov-report=xml

  deploy:
    runs-on: ubuntu-latest
    needs: test
    if: github.ref == 'refs/heads/main' && github.event_name == 'push'
    environment: production
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
          cache: 'pip'
      - run: pip install -r requirements.txt
      - name: Deploy
        run: echo "Deploy your application here"
        env:
          DEPLOY_KEY: ${{ secrets.DEPLOY_KEY }}

Django CI with PostgreSQL service container

# .github/workflows/django.yml
name: Django CI

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: test_db
        ports:
          - 5432:5432
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
    env:
      DATABASE_URL: postgres://postgres:postgres@localhost:5432/test_db
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
          cache: 'pip'
      - run: pip install -r requirements.txt
      - run: python manage.py test --verbosity=2

Anti-Patterns

Wrong: Using floating Python version

# ❌ BAD — '3.x' resolves to different versions over time
- uses: actions/setup-python@v5
  with:
    python-version: '3.x'

Correct: Pinning exact Python minor version

# ✅ GOOD — deterministic, same version every run
- uses: actions/setup-python@v5
  with:
    python-version: '3.12'

Wrong: Using PyPI password in secrets

# ❌ BAD — password-based auth is deprecated
- run: twine upload dist/*
  env:
    TWINE_USERNAME: __token__
    TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }}

Correct: Using OIDC trusted publishing

# ✅ GOOD — no secrets needed; OIDC proves workflow identity to PyPI
permissions:
  id-token: write
steps:
  - uses: pypa/gh-action-pypi-publish@release/v1

Wrong: Installing unpinned dependencies

# ❌ BAD — unpinned dependencies change between runs
- run: pip install django pytest requests

Correct: Using requirements file with pinned versions

# ✅ GOOD — deterministic install from lockfile
- run: pip install -r requirements.txt
# requirements.txt: django==5.1.2, pytest==8.3.3

Common Pitfalls

• Python version as number: YAML interprets 3.10 as 3.1 (float). Fix: always quote: '3.10'. [src1]
• Cache not restoring: cache: 'pip' needs a dependency file. Fix: ensure requirements.txt exists and specify cache-dependency-path. [src2]
• Different behavior on Windows: Backslash paths, line endings, case-insensitive imports. Fix: add runs-on: windows-latest to matrix. [src4]
• tox-gh-actions version mismatch: GitHub Actions Python version not mapped to tox envs. Fix: ensure [gh-actions] section in tox.ini matches matrix. [src4]
• pytest not finding tests: Default discovery looks for test_*.py. Fix: configure testpaths in pyproject.toml. [src7]
• Import errors in CI: Package not installed in editable mode. Fix: use pip install -e ".[dev]". [src1]

Diagnostic Commands

# Check installed Python version
python --version

# List installed packages with versions
pip list

# Check pip cache location
pip cache dir

# Verify all dependencies are satisfied
pip check

# Run pytest with verbose output
pytest -v --tb=long

# List all tox environments
tox -l

# Validate workflow YAML locally
actionlint .github/workflows/ci.yml

# Check workflow run status via CLI
gh run list --workflow=ci.yml --limit=5

Version History & Compatibility

When to Use / When Not to Use

Important Caveats

• Python version 3.10 without quotes is parsed as 3.1 by YAML. Always quote: '3.10'.
• setup-python caches the pip download cache, not the virtual environment. Re-installation still occurs.
• Free tier provides 2,000 minutes/month for private repos. Public repos get unlimited minutes.
• Service containers (PostgreSQL, Redis) only work on Linux runners, not Windows or macOS.
• OIDC trusted publishing requires configuring the trusted publisher on PyPI before first publish.