OIA Data Collection

Purpose

This recipe produces an anonymized, multi-platform communication metadata dataset ready for organizational network analysis. It extracts interaction patterns from Slack, email, Jira, Git, calendar, and HRIS — capturing who communicates with whom, how often, and through which channels — without accessing message content. [src3, src4]

Prerequisites

• Signed scope document from OIA Engagement Playbook
• Data processing agreement — signed by client legal, metadata-only scope
• Slack Bot OAuth token — from Slack API (requires Business+ plan)
• Microsoft Graph OAuth token — from Azure Portal (requires admin consent)
• Jira API token — from Atlassian
• Python 3.10+ with pip installed

Constraints

• GDPR: extract metadata only — timestamps, sender/receiver IDs, channel names. Never message body content. [src3]
• All individual identifiers must be pseudonymized (hash employee IDs) before analysis. [src4]
• Minimum 90 days of historical data required to avoid seasonal bias.
• Data retention: delete all raw exports within 30 days of engagement completion.
• Rate limits: Slack API (20 req/min), Microsoft Graph (10,000 req/10min). [src1, src2]

Tool Selection Decision

Which platform combination?
├── Slack + Google Workspace
│   └── PATH A: Slack API + Google Admin SDK
├── Slack + Microsoft 365
│   └── PATH B: Slack API + Microsoft Graph API
├── Microsoft Teams + Microsoft 365
│   └── PATH C: Microsoft Graph API (unified)
└── Mixed or restricted access
    └── PATH D: Manual exports + survey supplement

Execution Flow

Step 1: Inventory Available Systems

Duration: 2-4 hours · Tool: Client IT coordination

Map all communication and collaboration platforms in use. Document API availability, admin access status, data retention period, and user count per system.

Verify: Complete system list with API availability confirmed. · If failed: Document gaps, proceed with available systems.

Step 2: Legal & Privacy Framework

Duration: 1-2 days · Tool: Legal coordination

Draft data processing agreement, employee transparency notice, anonymization protocol (SHA-256 hash of employee ID + engagement salt), and data retention schedule. [src3]

Verify: Legal sign-off received, transparency notice sent. · If failed: Escalate to executive sponsor.

Step 3: Slack Data Export

Duration: 4-8 hours · Tool: Python slack_sdk or Slack Admin Panel

Extract communication metadata: message timestamps, channel names, reaction counts, thread depths, sender IDs (anonymized). NOT message content. [src1]

# Extract metadata per channel (no message content)
from slack_sdk import WebClient
client = WebClient(token="xoxb-YOUR-BOT-TOKEN")
# conversations.history → timestamps, thread_ts, reply_count, reactions

Verify: Export covers all target departments, row count > 1000 for 90-day window. · If failed: Reduce batch size, add sleep intervals.

Step 4: Email Metadata Collection

Duration: 4-8 hours · Tool: Microsoft Graph API or Google Workspace Admin SDK

Extract email metadata: sender/receiver pairs, timestamps, volume per relationship. NOT body content. [src2]

Verify: Metadata covers 90+ days, cross-reference user count with org chart. · If failed: Implement exponential backoff for throttling.

Step 5: Project Tool Data

Duration: 2-4 hours · Tool: Jira REST API, GitHub API, calendar API

Extract ticket velocity, assignment patterns, PR review turnaround, meeting density.

Verify: Data covers 90+ days, ticket count > 100. · If failed: Use CSV export from admin panel.

Step 6: HRIS Data Integration

Duration: 2-4 hours · Tool: HRIS export (BambooHR, Workday, or manual)

Extract org structure: reporting lines, department assignments, tenure, role titles. Cross-reference with communication data.

Verify: > 90% of communication senders matched to HRIS records. · If failed: Build org chart manually from client document.

Output Schema

{
  "output_type": "oia_communication_dataset",
  "format": "CSV + JSON",
  "files": [
    {"name": "slack_metadata.csv", "description": "Slack channel activity metadata"},
    {"name": "email_metadata.csv", "description": "Email sender/receiver pair metadata"},
    {"name": "project_metadata.csv", "description": "Jira/GitHub project management metadata"},
    {"name": "org_structure.json", "description": "Anonymized org chart"},
    {"name": "combined_oia_dataset.csv", "description": "Merged dataset for network analysis"}
  ]
}

Quality Benchmarks

If below minimum: Extend data collection period, add missing platform integrations, or supplement with survey data.

Error Handling

Cost Breakdown

Anti-Patterns

Wrong: Accessing message content without consent

Reading Slack messages or email bodies for richer data. Result: GDPR violation, client trust destroyed. [src3]

Correct: Metadata only, always

Extract timestamps, sender/receiver pairs, channel names, reaction counts. Metadata under legitimate interest is legally defensible.

Wrong: Skipping anonymization during collection

Processing data with real names and anonymizing later. Result: PII on consultant's machine creates breach risk. [src3]

Correct: Anonymize at point of extraction

Apply hash immediately when extracting from each API. Raw employee IDs should never persist in any file.

Wrong: Using only one communication channel

Extracting Slack alone and declaring complete. Result: misses email-heavy relationships common in senior leadership. [src4]

Correct: Multi-channel triangulation

Extract from all available platforms and merge for a complete picture.

When This Matters

Use when an agent needs to collect organizational communication metadata for network analysis. This is Step 2-3 of the OIA engagement lifecycle. Requires signed data processing agreement and admin-level API access. Output feeds directly into the network analysis execution recipe.