--- # === IDENTITY === id: consulting/recipes/signal-source-audit/2026 canonical_question: "How do you audit industry signal sources across regulatory, behavioral, visual, and unstructured?" aliases: - "Industry signal audit methodology" - "Data source inventory for intent signals" - "How to score and rank signal sources for vertical intelligence" entity_type: execution_recipe domain: consulting > recipes > Signal Source Audit region: global jurisdiction: global temporal_scope: 2026-2027 # === VERIFICATION === last_verified: 2026-03-29 confidence: 0.85 version: 1.0 first_published: 2026-03-29 # === TEMPORAL VALIDITY === temporal_validity: status: evolving last_breaking_change: "Initial release — signal audit methodology v1.0" next_review: 2026-09-25 change_sensitivity: high # === CONSTRAINTS === constraints: - "Only evaluate publicly accessible or commercially licensable data sources — no grey-area scraping" - "Minimum 15 sources must be identified for a vertical to be considered viable" - "Every source must be scored on all 4 dimensions — partial scoring invalidates viability calculation" - "Refresh rate assessment requires verifying at least 3 consecutive update cycles" - "Cost estimates must include both access fees and processing/storage costs" # === SKIP CONDITIONS === skip_this_unit_if: - condition: "User needs full Signal Stack lifecycle, not just the audit" use_instead: "consulting/recipes/signal-stack-engagement-playbook/2026" - condition: "User already has signal sources and needs taxonomy design" use_instead: "consulting/recipes/signal-taxonomy-workshop/2026" # === AGENT HINTS === inputs_needed: - key: target_vertical question: "Which industry vertical is being audited?" type: text - key: geography question: "What geographic scope?" type: choice options: ["US only", "US + EU", "global", "specific country"] - key: budget_range question: "What is the monthly data source budget?" type: choice options: ["$0 (public only)", "$500-$2K/month", "$2K-$5K/month", "$5K+/month"] # === EXECUTION METADATA === execution: required_inputs: - name: "Target vertical definition" source: "client/strategy" format: "document" - name: "Existing data source list (if any)" source: "client/data-team" format: "spreadsheet" outputs: - name: "Signal Source Audit Report" format: "spreadsheet + document" description: "Scored inventory of all identified signal sources with viability assessment" - name: "Source Priority Matrix" format: "visualization" description: "2x2 matrix plotting signal quality vs accessibility for each source" tools_required: - name: "Web browser + search" purpose: "Identify and evaluate data sources" tier: "free" cost: "$0" alternatives: [] - name: "Spreadsheet" purpose: "Structured scoring and ranking" tier: "free" cost: "$0" alternatives: ["Google Sheets", "Excel", "Airtable"] credentials_needed: [] estimated_duration: "5-10 days" estimated_cost: "$2K-$5K (consulting time)" # === DISTRIBUTION === canonical_source: "https://knowledgelib.io/consulting/recipes/signal-source-audit/2026" suggested_citation: "Source: knowledgelib.io — AI Knowledge Library (verified 2026-03-29)" # === RELATED UNITS === related_kos: depends_on: [] feeds_into: - id: "consulting/recipes/signal-taxonomy-workshop/2026" label: "Taxonomy workshop uses audit outputs as input" - id: "consulting/recipes/mvp-pipeline-build/2026" label: "MVP build connects to top-ranked sources" related_to: - id: "consulting/recipes/signal-stack-engagement-playbook/2026" label: "Master playbook — this is Phase 1" # === SOURCES === sources: - id: src1 title: "Competitive Intelligence Advantage" author: McGonagle, J.J. & Vella, C.M. url: https://www.springer.com/gp/book/9783030098247 type: industry_report published: 2012-01-15 reliability: high - id: src2 title: "Data-Driven Marketing" author: Jeffery, M. url: https://www.wiley.com/en-us/Data+Driven+Marketing-p-9780470504543 type: academic_paper published: 2010-04-06 reliability: authoritative - id: src3 title: "Web Scraping with Python — Legal Considerations" author: Mitchell, R. url: https://www.oreilly.com/library/view/web-scraping-with/9781491985564/ type: technical_blog published: 2018-04-01 reliability: high - id: src4 title: "Alternative Data in Investment Management" author: CFA Institute url: https://www.cfainstitute.org/en/research/foundation/2020/alternative-data type: industry_report published: 2020-09-01 reliability: authoritative - id: src5 title: "Signal Detection Theory and ROC Analysis" author: Macmillan, N.A. & Creelman, C.D. url: https://www.cambridge.org/core/books/detection-theory/3B8C9980B13E2391B3F31715E06F5AB4 type: academic_paper published: 2004-11-01 reliability: authoritative --- # Signal Source Audit ## Purpose This recipe executes a systematic audit of all available data sources that could provide intent signals for a target industry vertical. It produces a scored inventory covering regulatory databases, behavioral data sources, visual signals, and unstructured media — enabling a go/no-go decision on vertical viability and prioritized source selection for pipeline construction. [src1, src4] ## Prerequisites - [ ] **Target vertical** defined with clear industry boundaries (SIC/NAICS codes or equivalent) - [ ] **Geographic scope** determined — US, EU, global, or specific markets - [ ] **Budget parameters** established for data source licensing - [ ] **Existing data sources** inventoried (if any) to avoid duplicate evaluation ## Constraints - Only evaluate publicly accessible or commercially licensable sources. Grey-area scraping creates legal liability that undermines the entire engagement. [src3] - Every source must be scored on all 4 dimensions (accessibility, cost, refresh rate, signal-to-noise). Partial scoring invalidates the viability calculation. - Refresh rate assessment requires verifying at least 3 consecutive update cycles — a single snapshot is insufficient. - Cost estimates must include both direct access fees and downstream processing/storage costs. [src4] - Minimum 15 sources must be identified for a vertical to pass viability threshold. ## Tool Selection Decision ``` Which audit depth? ├── Quick assessment (3-5 days) │ └── PATH A: Desktop research — web search, database directories, known source inventories ├── Standard audit (5-10 days) │ └── PATH B: Desktop + API testing — verify data quality with sample pulls from top sources ├── Deep audit (10-15 days) │ └── PATH C: Desktop + API testing + vendor interviews — negotiate pricing, assess reliability └── Competitive audit └── PATH D: Add competitor signal analysis — what data are competitors likely using ``` | Path | Scope | Cost | Speed | Confidence | |------|-------|------|-------|-----------| | A: Quick | Surface-level identification | $2K-$3K | 3-5 days | Moderate | | B: Standard | Identification + quality verification | $3K-$5K | 5-10 days | High | | C: Deep | Full evaluation + vendor negotiation | $5K-$8K | 10-15 days | Very high | | D: Competitive | Standard + competitor analysis | $4K-$7K | 7-12 days | High | ## Execution Flow ### Step 1: Inventory Regulatory Databases **Duration**: 1-2 days **Tool**: Web research + government database directories Identify all regulatory and government databases relevant to the target vertical. For each database, document: agency name, database URL, data format (API/bulk download/search only), update frequency, geographic coverage, and content scope. Common regulatory sources by sector: - **Environmental**: EPA ECHO, state DEQ databases, OSHA inspection records - **Financial**: SEC EDGAR, FDIC, state licensing boards - **Healthcare**: FDA databases (FAERS, 510(k)), CMS, state health department records - **Construction/Real estate**: building permits, zoning changes, property records - **Food/Agriculture**: FDA inspections, USDA databases, state agriculture boards Score each source: accessibility (1-5), cost (1-5), refresh rate (1-5), signal-to-noise ratio (1-5). [src1] **Verify**: Minimum 5 regulatory sources identified and scored. Each score justified with evidence. **If failed**: Vertical may be lightly regulated — this is not disqualifying but shifts weight to behavioral sources. ### Step 2: Map Behavioral Data Sources **Duration**: 1-2 days **Tool**: Web research + API documentation review Identify behavioral data sources that reveal company activity and intent: DNS/WHOIS changes (technology adoption), job board postings (hiring signals), review site activity (customer sentiment shifts), app store data (product launches), patent filings (R&D direction), press releases, conference speaker lists. For each source, assess: - **Accessibility**: Is there an API? Bulk download? Requires scraping? - **Cost**: Free, freemium, or paid? What are volume pricing tiers? - **Refresh rate**: Real-time, daily, weekly, monthly? - **Signal-to-noise**: What percentage of data points are actually actionable? [src2, src5] **Verify**: Minimum 5 behavioral sources identified and scored. At least 2 with API access confirmed. **If failed**: If fewer than 3 behavioral sources available, vertical may lack sufficient digital footprint for automated intelligence. ### Step 3: Assess Visual Signal Availability **Duration**: 0.5-1 day **Tool**: Satellite/street imagery platform evaluation Evaluate whether visual signals are relevant and available for the vertical: satellite imagery (construction activity, fleet size, parking lot occupancy), street-level imagery (store conditions, signage changes), aerial photography. This step is highly vertical-dependent — skip for purely digital verticals. Score applicable visual sources on same 4 dimensions. Note: visual signal processing typically requires specialized ML models, adding $2K-$10K in development cost. [src4] **Verify**: Visual source relevance determined — either scored sources identified or explicit "not applicable" documentation. **If failed**: Visual signals are optional for most verticals. Document as "not applicable" and continue. ### Step 4: Identify Unstructured Media Sources **Duration**: 1-2 days **Tool**: Media monitoring platform evaluation + web research Identify unstructured text and media sources: industry publications, trade journals, conference proceedings, podcast transcripts, social media (LinkedIn company pages, Twitter/X industry hashtags), forum discussions (Reddit, industry-specific forums), news feeds. Assess each for: volume, relevance density, extraction difficulty (structured text vs audio/video), and timeliness. Unstructured sources require NLP/LLM processing — factor this into cost estimates. [src1, src2] **Verify**: Minimum 5 unstructured sources identified. At least 2 with manageable extraction difficulty (text-based, not audio/video). **If failed**: If unstructured sources are predominantly audio/video, budget additional $1K-$3K/month for transcription services. ### Step 5: Score and Rank All Sources **Duration**: 1 day **Tool**: Spreadsheet + scoring framework Compile all identified sources into a single scored inventory. Calculate composite viability score per source: Composite Score = (Accessibility x 0.30) + (Cost x 0.20) + (Refresh Rate x 0.25) + (Signal-to-Noise x 0.25) Rank sources by composite score. Plot on 2x2 priority matrix: X-axis = signal quality (refresh + SNR), Y-axis = accessibility (access + cost). Sources in top-right quadrant are immediate priorities. [src5] **Verify**: All sources scored on all 4 dimensions. Priority matrix generated. Overall vertical viability score calculated (sum of top 10 source composite scores / 50). **If failed**: If overall viability score < 0.60, recommend alternative vertical or supplemental paid data sources to close gaps. ### Step 6: Deliver Audit Report **Duration**: 0.5-1 day **Tool**: Document generation Produce the Signal Source Audit Report containing: - Executive summary with go/no-go recommendation - Complete source inventory with scores - Priority matrix visualization - Cost projection for top 10 sources (monthly access + processing) - Risk assessment (source reliability, legal considerations, single-source dependencies) - Recommended next step (proceed to taxonomy workshop or pivot vertical) **Verify**: Report reviewed by engagement lead. Go/no-go recommendation clearly stated with supporting data. **If failed**: If recommendation is ambiguous, request additional domain expert input before finalizing. ## Output Schema ```json { "output_type": "signal_source_audit", "format": "spreadsheet + document", "sections": [ {"name": "source_inventory", "type": "array", "description": "All identified sources with 4-dimension scoring", "required": true}, {"name": "priority_matrix", "type": "object", "description": "2x2 quality vs accessibility visualization", "required": true}, {"name": "viability_score", "type": "number", "description": "Overall vertical viability 0.0-1.0", "required": true}, {"name": "cost_projection", "type": "object", "description": "Monthly cost for top 10 sources", "required": true}, {"name": "risk_assessment", "type": "array", "description": "Legal, reliability, and dependency risks", "required": true}, {"name": "recommendation", "type": "string", "description": "Go/no-go with rationale", "required": true} ], "expected_sections": "6", "sort_order": "composite score descending" } ``` ## Quality Benchmarks | Quality Metric | Minimum Acceptable | Good | Excellent | |---------------|-------------------|------|-----------| | Total sources identified | > 15 | > 25 | > 40 | | Sources with API access | > 3 | > 8 | > 15 | | Signal categories covered | 3 of 4 | 4 of 4 | 4 of 4 + niche | | Cost accuracy (vs actual) | Within 50% | Within 25% | Within 10% | | Refresh rate verified (3 cycles) | > 50% of sources | > 75% | > 90% | **If below minimum**: Extend audit by 2-3 days or consider that the vertical may lack sufficient signal density for automated intelligence. ## Error Handling | Error | Likely Cause | Recovery Action | |-------|-------------|----------------| | Cannot find regulatory databases | Vertical is lightly regulated | Shift weight to behavioral/media sources; document regulatory gap | | API access denied during testing | Rate limits or authentication required | Contact vendor for evaluation access; estimate based on documentation | | Source refresh rate inconsistent | Irregular data publication schedule | Use minimum observed frequency for scoring; add reliability risk flag | | Cost information unavailable | Enterprise pricing, requires sales call | Use comparable source pricing as estimate; flag for vendor negotiation | | Fewer than 15 sources total | Vertical has limited digital footprint | Recommend supplemental paid sources or consider vertical pivot | ## Cost Breakdown | Component | Quick ($2K-$3K) | Standard ($3K-$5K) | Deep ($5K-$8K) | |-----------|-----------------|---------------------|----------------| | Regulatory source inventory | $500-$800 | $800-$1.2K | $1.2K-$2K | | Behavioral source mapping | $500-$800 | $800-$1.2K | $1.2K-$2K | | Visual + unstructured sources | $300-$500 | $500-$800 | $800-$1.2K | | Scoring + ranking | $300-$500 | $500-$800 | $800-$1.2K | | Report + recommendation | $400-$400 | $400-$800 | $800-$1.5K | | **Total** | **$2K-$3K** | **$3K-$5K** | **$5K-$8K** | ## Anti-Patterns ### Wrong: Counting sources without scoring them Listing 30 data sources without assessing quality, accessibility, or cost. Result: pipeline built on unreliable or inaccessible sources fails within first month. [src1] ### Correct: Score every source on all 4 dimensions Each source gets a 1-5 rating on accessibility, cost, refresh rate, and signal-to-noise. Composite scores drive prioritization. ### Wrong: Ignoring legal accessibility constraints Identifying valuable data sources that require scraping behind login walls or violate terms of service. Result: cease-and-desist letters, data source cut off mid-engagement. [src3] ### Correct: Verify legal access for every source Confirm each source is either publicly accessible, has an official API, or offers commercial licensing. Document the access method and legal basis. ### Wrong: Single-snapshot refresh rate assessment Checking a source once and assuming the observed update frequency is reliable. Result: pipeline depends on source that updates irregularly. [src4] ### Correct: Verify 3 consecutive update cycles Monitor each top-priority source across at least 3 update cycles before committing pipeline dependency. ## When This Matters Use when an agent needs to evaluate whether a target industry vertical has sufficient signal density for an automated intelligence pipeline. This is Phase 1 of the Signal Stack engagement — its output directly determines whether to proceed with taxonomy design and pipeline construction, or pivot to an alternative vertical. ## Related Units - [Signal Stack Engagement Playbook](/consulting/recipes/signal-stack-engagement-playbook/2026) - [Signal Taxonomy Workshop](/consulting/recipes/signal-taxonomy-workshop/2026) - [MVP Pipeline Build](/consulting/recipes/mvp-pipeline-build/2026)