Create Classifier Trait and Classification Pipeline Architecture

Summary

Create the foundational Classifier trait and classification pipeline structure to enable semantic analysis of extracted strings. This framework will allow multiple classifiers to analyze strings and assign tags (URLs, domains, file paths, etc.) based on context and patterns.

Current State

The infrastructure needed for this feature is already in place:

✅ Existing Components:

FoundString struct with tags, encoding, offset, RVA, confidence scoring
Tag enum with comprehensive variants (URL, Domain, IPv4, IPv6, FilePath, etc.)
ContainerInfo with sections, imports, exports, resources
SectionType and BinaryFormat enums
BasicExtractor for string extraction from binaries
ASCII and UTF-16LE extraction with confidence scoring

🚧 To Be Built:

Classifier trait for pluggable classification strategies
StringContext struct to package extraction context
Classification result type with confidence scoring
ClassificationPipeline orchestration layer
Concrete classifier implementations

Context

Stringy extracts strings from binary files and needs to classify them to distinguish meaningful strings from random garbage data. The classification system will:

Provide a common interface (Classifier trait) for different classification strategies
Package string data with contextual information (StringContext) for informed classification
Enable a pipeline architecture where multiple classifiers can process strings sequentially or in parallel
Support confidence scoring to handle ambiguous cases
Integrate seamlessly with existing BasicExtractor and FoundString types
Leverage existing section weighting and confidence scores from extraction phase

Proposed Solution

1. Classifier Trait

Create a trait that all classifiers will implement in src/classification/traits.rs:

use crate::types::{Tag, StringyError, Result};

/// Trait for string classifiers that analyze and tag extracted strings
pub trait Classifier: Send + Sync {
    /// Classify a string and return relevant tags with confidence scores
    /// 
    /// Returns a vector of classifications, allowing one string to match
    /// multiple patterns (e.g., a URL that's also a domain).
    fn classify(&self, context: &StringContext) -> Result<Vec<Classification>>;
    
    /// Get the name/identifier of this classifier for logging and debugging
    fn name(&self) -> &str;
    
    /// Check if this classifier can handle the given context
    /// 
    /// Allows classifiers to opt-out based on encoding, binary format,
    /// or other contextual factors. Default implementation accepts all.
    fn can_classify(&self, context: &StringContext) -> bool {
        true
    }
    
    /// Priority/order hint for the pipeline (lower = earlier execution)
    /// 
    /// Some classifiers may want to run before others, e.g., import/export
    /// classifiers should run before generic pattern matching.
    fn priority(&self) -> i32 {
        100  // Default middle priority
    }
}

2. StringContext Struct

Encapsulate string data with contextual information in src/classification/context.rs:

use crate::types::{
    FoundString, SectionType, BinaryFormat, ContainerInfo, 
    ImportInfo, ExportInfo
};

/// Context information about an extracted string for classification
#[derive(Debug, Clone)]
pub struct StringContext<'a> {
    /// The extracted string with metadata
    pub found_string: &'a FoundString,
    
    /// Section type where string was found (StringData, Code, etc.)
    pub section_type: Option<SectionType>,
    
    /// Binary format (ELF, PE, Mach-O)
    pub binary_format: BinaryFormat,
    
    /// Nearby strings for context-aware classification (within ±100 bytes)
    pub surrounding_strings: Vec<String>,
    
    /// Full container information
    pub container_info: &'a ContainerInfo,
    
    /// Matching import if string is from import table
    pub import_match: Option<&'a ImportInfo>,
    
    /// Matching export if string is from export table  
    pub export_match: Option<&'a ExportInfo>,
}

impl<'a> StringContext<'a> {
    /// Create a new context for a found string
    pub fn new(
        found_string: &'a FoundString,
        container_info: &'a ContainerInfo,
    ) -> Self {
        let section_type = found_string.section.as_ref()
            .and_then(|name| Self::infer_section_type(name, container_info));
        
        let import_match = container_info.imports.iter()
            .find(|imp| imp.name == found_string.text);
            
        let export_match = container_info.exports.iter()
            .find(|exp| exp.name == found_string.text);
        
        Self {
            found_string,
            section_type,
            binary_format: container_info.format,
            surrounding_strings: Vec::new(),
            container_info,
            import_match,
            export_match,
        }
    }
    
    /// Infer section type from section name
    fn infer_section_type(name: &str, info: &ContainerInfo) -> Option<SectionType> {
        info.sections.iter()
            .find(|s| s.name == name)
            .map(|s| s.section_type)
    }
    
    /// Add surrounding strings for context
    pub fn with_surrounding(&mut self, strings: Vec<String>) {
        self.surrounding_strings = strings;
    }
}

3. Classification Result

Define the output structure in src/classification/mod.rs:

use crate::types::Tag;

/// Result of classifying a string
#[derive(Debug, Clone, PartialEq)]
pub struct Classification {
    /// The assigned tag
    pub tag: Tag,
    
    /// Confidence score (0.0 to 1.0)
    /// - 1.0: Definitive match (e.g., valid URL with scheme)
    /// - 0.7-0.9: Strong match (e.g., domain pattern with TLD)
    /// - 0.5-0.7: Moderate match (e.g., path-like structure)
    /// - <0.5: Weak match (should probably be filtered)
    pub confidence: f64,
    
    /// Optional explanation for debugging/logging
    pub reason: Option<String>,
    
    /// The classifier that produced this result
    pub classifier_name: String,
}

impl Classification {
    /// Create a new classification result
    pub fn new(tag: Tag, confidence: f64, classifier_name: String) -> Self {
        Self {
            tag,
            confidence,
            reason: None,
            classifier_name,
        }
    }
    
    /// Add an explanation
    pub fn with_reason(mut self, reason: impl Into<String>) -> Self {
        self.reason = Some(reason.into());
        self
    }
    
    /// Check if this is a high-confidence classification
    pub fn is_high_confidence(&self) -> bool {
        self.confidence >= 0.7
    }
}

4. Classification Pipeline

Create orchestration layer in src/classification/pipeline.rs:

use crate::types::Result;
use super::{Classifier, Classification, StringContext};
use std::collections::HashMap;

/// Pipeline for orchestrating multiple classifiers
pub struct ClassificationPipeline {
    classifiers: Vec<Box<dyn Classifier>>,
}

impl ClassificationPipeline {
    /// Create a new empty pipeline
    pub fn new() -> Self {
        Self {
            classifiers: Vec::new(),
        }
    }
    
    /// Create a pipeline with default classifiers
    pub fn with_defaults() -> Self {
        let mut pipeline = Self::new();
        // Will add default classifiers in future issues:
        // - ImportExportClassifier (priority: 10)
        // - UrlClassifier (priority: 50)
        // - IpAddressClassifier (priority: 60)
        // - FilePathClassifier (priority: 70)
        // - Base64Classifier (priority: 80)
        // - FormatStringClassifier (priority: 90)
        pipeline
    }
    
    /// Add a classifier to the pipeline
    pub fn add_classifier(&mut self, classifier: Box<dyn Classifier>) {
        self.classifiers.push(classifier);
        // Sort by priority after adding
        self.classifiers.sort_by_key(|c| c.priority());
    }
    
    /// Classify a string using all applicable classifiers
    pub fn classify(&self, context: &StringContext) -> Result<Vec<Classification>> {
        let mut results = Vec::new();
        
        for classifier in &self.classifiers {
            // Skip if classifier can't handle this context
            if !classifier.can_classify(context) {
                continue;
            }
            
            // Run classification and collect results
            match classifier.classify(context) {
                Ok(classifications) => results.extend(classifications),
                Err(e) => {
                    eprintln!("Warning: Classifier '{}' failed: {}", 
                              classifier.name(), e);
                    continue;
                }
            }
        }
        
        // Merge and deduplicate results
        Ok(self.merge_results(results))
    }
    
    /// Merge duplicate tags, keeping highest confidence
    fn merge_results(&self, mut results: Vec<Classification>) -> Vec<Classification> {
        let mut tag_map: HashMap<String, Classification> = HashMap::new();
        
        for classification in results.drain(..) {
            let key = format!("{:?}", classification.tag);
            
            tag_map.entry(key)
                .and_modify(|existing| {
                    // Keep classification with higher confidence
                    if classification.confidence > existing.confidence {
                        *existing = classification.clone();
                    }
                })
                .or_insert(classification);
        }
        
        // Sort by confidence (highest first)
        let mut merged: Vec<_> = tag_map.into_values().collect();
        merged.sort_by(|a, b| {
            b.confidence.partial_cmp(&a.confidence)
                .unwrap_or(std::cmp::Ordering::Equal)
        });
        
        merged
    }
    
    /// Get the number of classifiers in the pipeline
    pub fn len(&self) -> usize {
        self.classifiers.len()
    }
    
    /// Check if the pipeline is empty
    pub fn is_empty(&self) -> bool {
        self.classifiers.is_empty()
    }
}

impl Default for ClassificationPipeline {
    fn default() -> Self {
        Self::new()
    }
}

5. Integration with Existing Extraction

Example usage in src/lib.rs or extraction workflow:

use crate::extraction::{BasicExtractor, ExtractionConfig, StringExtractor};
use crate::classification::{ClassificationPipeline, StringContext};
use crate::types::{FoundString, ContainerInfo};

/// Extract and classify strings from a binary
pub fn extract_and_classify(
    data: &[u8],
    container_info: &ContainerInfo,
) -> Result<Vec<FoundString>> {
    // Extract strings
    let extractor = BasicExtractor::new();
    let config = ExtractionConfig::default();
    let mut strings = extractor.extract(data, container_info, &config)?;
    
    // Set up classification pipeline
    let pipeline = ClassificationPipeline::with_defaults();
    
    // Classify each string
    for found_string in &mut strings {
        let context = StringContext::new(found_string, container_info);
        
        if let Ok(classifications) = pipeline.classify(&context) {
            // Apply high-confidence tags to the string
            for classification in classifications {
                if classification.is_high_confidence() {
                    found_string.tags.push(classification.tag);
                }
            }
        }
    }
    
    Ok(strings)
}

Implementation Steps

Phase 1: Core Framework (src/classification/)
- Create traits.rs with Classifier trait
- Create context.rs with StringContext struct
- Add Classification struct to mod.rs
- Create pipeline.rs with ClassificationPipeline
- Re-export types in mod.rs
Phase 2: Testing Infrastructure
- Create tests/classification_tests.rs
- Implement MockClassifier for testing
- Test pipeline ordering and priority
- Test confidence scoring and merging
- Test context creation from FoundString
Phase 3: Integration
- Update src/lib.rs to re-export classification types
- Add integration example in docs
- Update BasicExtractor workflow example
Phase 4: First Real Classifier (Separate Issue)
- Implement ImportExportClassifier as proof-of-concept
- Test with real PE/ELF/Mach-O binaries

Testing Strategy

Unit Tests

#[cfg(test)]
mod tests {
    use super::*;
    
    struct MockClassifier {
        name: String,
        priority: i32,
    }
    
    impl Classifier for MockClassifier {
        fn classify(&self, ctx: &StringContext) -> Result<Vec<Classification>> {
            Ok(vec![
                Classification::new(
                    Tag::Url,
                    0.9,
                    self.name.clone()
                )
            ])
        }
        
        fn name(&self) -> &str {
            &self.name
        }
        
        fn priority(&self) -> i32 {
            self.priority
        }
    }
    
    #[test]
    fn test_pipeline_ordering() {
        let mut pipeline = ClassificationPipeline::new();
        pipeline.add_classifier(Box::new(MockClassifier { 
            name: "low".into(), 
            priority: 100 
        }));
        pipeline.add_classifier(Box::new(MockClassifier { 
            name: "high".into(), 
            priority: 10 
        }));
        
        // Verify classifiers are sorted by priority
        assert_eq!(pipeline.len(), 2);
    }
    
    #[test]
    fn test_confidence_merging() {
        // Test that duplicate tags keep highest confidence
    }
    
    #[test]
    fn test_context_creation() {
        // Test StringContext properly extracts section type, 
        // matches imports/exports, etc.
    }
}

Integration Tests

Test with real binaries:

PE file with imports/exports
ELF file with URLs in .rodata
Mach-O with version strings

Performance Considerations

Lazy Evaluation: Only create StringContext for strings that need classification
Parallel Processing: Future enhancement to run classifiers in parallel (requires Send + Sync)
Caching: Consider caching regex compilation in classifiers
Short-Circuit: can_classify() allows early exit
Priority Ordering: Run fast classifiers first (imports/exports before regex patterns)

Acceptance Criteria

Dependencies

Existing Types (from src/types.rs):

FoundString - Contains text, encoding, offset, tags, confidence
Tag - Enum of all semantic tags
SectionType - Classification of binary sections
BinaryFormat - ELF, PE, Mach-O detection
ContainerInfo - Binary metadata, sections, imports, exports
ImportInfo / ExportInfo - Symbol information
StringyError and Result - Error handling

Will Enable (Future Issues):

Individual classifier implementations (URL, Domain, IP, FilePath, etc.)
Smart scoring that combines extraction confidence + classification confidence
Context-aware classification (e.g., grouped strings suggest log format)

Related Requirements

Requirement 3.1: Semantic Classification Framework
Task-ID: stringy-analyzer/semantic-classification-framework
Blocks: URL Classifier (#TBD), IP Classifier (#TBD), FilePath Classifier (#TBD)

References

Existing extraction: src/extraction/basic.rs
Type definitions: src/types.rs
Container parsing: src/container/mod.rs

@coderabbitai Enhance this issue body with additional context and a proposed solution. Improve the issue title and assign the proper labels and other metadata.

@unclesp1d3r

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Fetch details of issue #14
gh issue view 14 --repo EvilBit-Labs/StringyMcStringFace --json title,body,labels,assignees,state,author,createdAt,updatedAt

Length of output: 1249

🏁 Script executed:

#!/bin/bash
# Explore the project structure to understand the classification system context
fd -t f "mod.rs|lib.rs" src/ | head -20

# Check if classification directory exists
fd -t d "classification" src/

# Look for any existing classification-related code
rg -n "classification|Classifier" --type rust -C 2

# Get available labels in the repository
gh label list --repo EvilBit-Labs/StringyMcStringFace --json name,description,color --limit 50

Length of output: 2421

🏁 Script executed:

#!/bin/bash
# Read the current classification module
cat src/classification/mod.rs

# Check for any related files in classification directory
fd -t f . src/classification/

# Look at the lib.rs to understand the overall structure
cat src/lib.rs | head -50

Length of output: 547

🏁 Script executed:

#!/bin/bash
# Check the types module to see what structures already exist
rg -n "pub struct FoundString|pub enum Tag|pub struct StringSource" src/types.rs -A 10

# Look at extraction module to understand how strings are currently extracted
ls -la src/extraction/

Length of output: 832

Now I'll update the issue with enhanced content:

🏁 Script executed:

#!/bin/bash
# Update the issue with enhanced content
gh issue edit 14 --repo EvilBit-Labs/StringyMcStringFace \
  --title "Create Classifier Trait and Classification Pipeline Architecture" \
  --body "## Summary

Create the foundational Classifier trait and classification pipeline structure to enable semantic analysis of extracted strings. This framework will allow multiple classifiers to analyze strings and assign tags (URLs, domains, file paths, etc.) based on context and patterns.

## Context

StringyMcStringFace extracts strings from binary files and needs to classify them to distinguish meaningful strings from random garbage data. The classification system will:

- Provide a common interface (Classifier trait) for different classification strategies
- Package string data with contextual information (StringContext) for informed classification
- Enable a pipeline architecture where multiple classifiers can process strings sequentially or in parallel
- Support confidence scoring to handle ambiguous cases
- Integrate with existing types like \`FoundString\` and \`Tag\` from the types module

## Proposed Solution

### 1. Classifier Trait

Create a trait that all classifiers will implement:

\`\`\`rust
pub trait Classifier {
    /// Classify a string and return relevant tags with confidence scores
    fn classify(&self, context: &StringContext) -> Vec<Classification>;
    
    /// Get the name/identifier of this classifier
    fn name(&self) -> &str;
    
    /// Check if this classifier can handle the given context
    fn can_classify(&self, context: &StringContext) -> bool {
        true  // Default implementation
    }
}
\`\`\`

### 2. StringContext Struct

Encapsulate string data with contextual information:

\`\`\`rust
pub struct StringContext {
    /// The extracted string
    pub found_string: FoundString,
    
    /// Section type (code, data, rodata, etc.)
    pub section_type: Option<SectionType>,
    
    /// Binary format (ELF, PE, Mach-O, etc.)
    pub binary_format: Option<BinaryFormat>,
    
    /// Nearby strings (for context-aware classification)
    pub surrounding_context: Vec<String>,
    
    /// Additional metadata from container analysis
    pub container_info: Option<ContainerInfo>,
}
\`\`\`

### 3. Classification Result

Define the output structure:

\`\`\`rust
pub struct Classification {
    /// The assigned tag
    pub tag: Tag,
    
    /// Confidence score (0.0 to 1.0)
    pub confidence: f64,
    
    /// Optional explanation/reasoning
    pub reason: Option<String>,
}
\`\`\`

### 4. Classification Pipeline

Create a pipeline to orchestrate multiple classifiers:

\`\`\`rust
pub struct ClassificationPipeline {
    classifiers: Vec<Box<dyn Classifier>>,
}

impl ClassificationPipeline {
    pub fn new() -> Self {
        Self {
            classifiers: Vec::new(),
        }
    }
    
    pub fn add_classifier(&mut self, classifier: Box<dyn Classifier>) {
        self.classifiers.push(classifier);
    }
    
    pub fn classify(&self, context: &StringContext) -> Vec<Classification> {
        let mut results = Vec::new();
        
        for classifier in &self.classifiers {
            if classifier.can_classify(context) {
                results.extend(classifier.classify(context));
            }
        }
        
        // Deduplicate and sort by confidence
        self.merge_results(results)
    }
    
    fn merge_results(&self, results: Vec<Classification>) -> Vec<Classification> {
        // Implementation for merging and deduplicating results
        results
    }
}
\`\`\`

## Implementation Steps

1. Create core types in \`src/classification/mod.rs\`:
   - \`Classifier\` trait
   - \`StringContext\` struct
   - \`Classification\` struct
   - \`ClassificationPipeline\` struct

2. Add unit tests for the pipeline structure

3. Update \`src/lib.rs\` to re-export classification types

4. Create example/mock classifier for testing the framework

## Acceptance Criteria

- [ ] Classifier trait defined with \`classify()\`, \`name()\`, and \`can_classify()\` methods
- [ ] StringContext struct created with all necessary fields
- [ ] Classification struct for results with tag, confidence, and reason
- [ ] ClassificationPipeline struct with add_classifier() and classify() methods
- [ ] Unit tests demonstrating the framework works with a mock classifier
- [ ] Documentation comments for all public types and methods

## Related Requirements

- Requirement 3.1: Semantic Classification Framework
- Task-ID: stringy-analyzer/semantic-classification-framework

## Dependencies

- Depends on existing types: \`FoundString\`, \`Tag\`, \`SectionType\`, \`BinaryFormat\`, \`ContainerInfo\`
- Will be used by: Future classifier implementations (URL, domain, file path, etc.)"

# Output the result
echo "Issue #14 updated successfully"

Length of output: 92

@unclesp1d3r I've successfully enhanced issue #14 with comprehensive details:

Changes Made

✅ Updated Title: "Create Classifier Trait and Classification Pipeline Architecture" - more descriptive and specific

✅ Enhanced Context: Added detailed explanation of the classification system's purpose and how it integrates with existing types (FoundString, Tag, etc.)

✅ Proposed Solution: Included complete Rust code examples for:

Classifier trait with classify(), name(), and can_classify() methods
StringContext struct to encapsulate string data with contextual information
Classification struct for results with confidence scoring
ClassificationPipeline for orchestrating multiple classifiers

✅ Implementation Steps: Clear roadmap with 4 concrete steps

✅ Acceptance Criteria: Checklist of requirements for completion

✅ Dependencies Section: Documents relationships with existing and future components

Metadata Status

The existing labels are already appropriate for this issue:

area:analyzer - Correct for binary analysis functionality
type:enhancement - Appropriate for new feature
status:backlog - Current status
lang:rust - Implementation language
needs:tests - Tests are included in acceptance criteria

The issue is properly assigned to you and ready for implementation. You can view the enhanced issue at: #14

🐰📝
/> フ
| _ |
/`ミ x 彡
/ |
/ ヽノ
/ ̄| | | |
| ( ̄ヽ__ヽ))

@coderabbitai Enhance this issue body with additional context and a proposed solution. Improve the issue title and assign the proper labels and other metadata.

Plan

Observations

The Stringy codebase has a solid foundation with complete type definitions in file:src/types.rs, a mature extraction framework in file:src/extraction/mod.rs, and comprehensive container parsing. The file:src/classification/mod.rs module exists but is empty (only a comment). All necessary infrastructure types (FoundString, Tag, ContainerInfo, SectionType, BinaryFormat, ImportInfo, ExportInfo) are already defined. The project follows clear patterns: trait-based abstractions, configuration structs with validation, comprehensive unit tests, and proper error handling using thiserror.

Approach

Implement the classification framework as a trait-based, pluggable architecture that mirrors the existing extraction framework's design patterns. Create four core components: the Classifier trait for pluggable strategies, StringContext for packaging extraction context, Classification for results with confidence scoring, and ClassificationPipeline for orchestration. This approach enables future classifier implementations while maintaining consistency with the existing codebase's architecture. The implementation will be test-driven, starting with the core framework and mock classifiers before adding real implementations in subsequent issues.

Implementation Steps

Phase 1: Core Framework Structure

1.1 Create Classifier Trait (`file:src/classification/traits.rs`)

Define the Classifier trait with the following methods:

classify(&self, context: &StringContext) -> Result<Vec<Classification>> - Main classification method returning multiple possible classifications
name(&self) -> &str - Identifier for logging and debugging
can_classify(&self, context: &StringContext) -> bool - Optional filter with default implementation returning true
priority(&self) -> i32 - Ordering hint with default value of 100

Ensure the trait is Send + Sync for future parallel processing support. Import necessary types from crate::types (Tag, StringyError, Result).

1.2 Create StringContext Struct (`file:src/classification/context.rs`)

Implement StringContext<'a> with these fields:

found_string: &'a FoundString - Reference to the extracted string
section_type: Option<SectionType> - Inferred from section name
binary_format: BinaryFormat - From container info
surrounding_strings: Vec<String> - For context-aware classification
container_info: &'a ContainerInfo - Full binary metadata
import_match: Option<&'a ImportInfo> - If string matches an import
export_match: Option<&'a ExportInfo> - If string matches an export

Add constructor new(found_string: &'a FoundString, container_info: &'a ContainerInfo) that:

Infers section_type by matching found_string.section against container_info.sections
Searches container_info.imports for matching import names
Searches container_info.exports for matching export names
Initializes surrounding_strings as empty vector

Add helper method with_surrounding(&mut self, strings: Vec<String>) for adding context strings.

Add private helper infer_section_type(name: &str, info: &ContainerInfo) -> Option<SectionType> to look up section type from container info.

1.3 Create Classification Result Type (`file:src/classification/mod.rs`)

Define Classification struct with:

tag: Tag - The assigned semantic tag
confidence: f64 - Score from 0.0 to 1.0
reason: Option<String> - Optional explanation for debugging
classifier_name: String - Source classifier identifier

Implement methods:

new(tag: Tag, confidence: f64, classifier_name: String) -> Self - Basic constructor
with_reason(mut self, reason: impl Into<String>) -> Self - Builder pattern for adding explanation
is_high_confidence(&self) -> bool - Returns true if confidence >= 0.7

Derive Debug, Clone, PartialEq for the struct.

1.4 Create Classification Pipeline (`file:src/classification/pipeline.rs`)

Implement ClassificationPipeline struct with:

classifiers: Vec<Box<dyn Classifier>> - Ordered list of classifiers

Add methods:

new() -> Self - Create empty pipeline
with_defaults() -> Self - Create pipeline (currently empty, will be populated in future issues)
add_classifier(&mut self, classifier: Box<dyn Classifier>) - Add classifier and sort by priority
classify(&self, context: &StringContext) -> Result<Vec<Classification>> - Run all applicable classifiers, collect results, merge duplicates
len(&self) -> usize - Return classifier count
is_empty(&self) -> bool - Check if pipeline has no classifiers

Implement merge_results(&self, results: Vec<Classification>) -> Vec<Classification> private method that:

Uses HashMap<String, Classification> keyed by format!("{:?}", tag)
Keeps highest confidence for duplicate tags
Sorts final results by confidence (descending)

Implement Default trait delegating to new().

In the classify method:

Iterate through classifiers in priority order
Skip classifiers where can_classify() returns false
Catch errors from individual classifiers, log warnings, continue processing
Merge and deduplicate results before returning

1.5 Update Module Exports (`file:src/classification/mod.rs`)

Add module declarations:

pub mod traits;
pub mod context;
pub mod pipeline;

Re-export public types:

pub use traits::Classifier;
pub use context::StringContext;
pub use pipeline::ClassificationPipeline;
// Classification struct defined in this file

Add comprehensive module documentation explaining the classification framework architecture, trait-based design, and usage examples.

Phase 2: Testing Infrastructure

2.1 Create Classification Tests (`file:tests/classification_tests.rs`)

Implement MockClassifier struct for testing:

Fields: name: String, priority: i32, tags: Vec<Tag>, confidence: f64
Implement Classifier trait returning configured tags with specified confidence
Add optional can_classify_fn: Option<Box<dyn Fn(&StringContext) -> bool>> for testing filtering

Create test cases:

test_pipeline_ordering() - Verify classifiers execute in priority order (add low priority then high priority, verify sorting)
test_pipeline_empty() - Test empty pipeline returns empty results
test_pipeline_single_classifier() - Test single classifier execution
test_pipeline_multiple_classifiers() - Test multiple classifiers with different tags
test_confidence_merging() - Add two classifiers returning same tag with different confidence, verify highest is kept
test_can_classify_filtering() - Test classifiers can opt-out via can_classify()
test_context_creation() - Create StringContext from FoundString and ContainerInfo, verify section type inference
test_context_import_matching() - Verify import name matching in context
test_context_export_matching() - Verify export name matching in context
test_classification_builder() - Test Classification::new() and with_reason() builder pattern
test_classification_confidence_threshold() - Test is_high_confidence() method
test_pipeline_error_handling() - Create classifier that returns error, verify pipeline continues with other classifiers

Use helper functions to create test fixtures:

create_test_found_string(text: &str) -> FoundString - Create minimal FoundString with required fields
create_test_container_info() -> ContainerInfo - Create minimal ContainerInfo with test sections, imports, exports

2.2 Add Unit Tests to Module Files

In file:src/classification/context.rs:

Test StringContext::new() with various section names
Test section type inference for common section names (.rodata, .text, .data, __cstring, etc.)
Test import/export matching logic
Test with_surrounding() method

In file:src/classification/pipeline.rs:

Test add_classifier() maintains priority ordering
Test merge_results() deduplication logic
Test sorting by confidence
Test Default implementation

Phase 3: Integration and Documentation

3.1 Update Library Exports (`file:src/lib.rs`)

Add to re-exports section:

pub use classification::{Classifier, StringContext, Classification, ClassificationPipeline};

Update module documentation to reflect classification framework availability:

Change classification status from "🚧 Types defined" to "✅ Framework complete"
Add usage example showing integration with BasicExtractor

3.2 Add Integration Example

In file:src/lib.rs documentation, add example demonstrating:

Extract strings using BasicExtractor
Create ClassificationPipeline::with_defaults()
Iterate through extracted strings
Create StringContext for each string
Call pipeline.classify(&context)
Apply high-confidence tags to FoundString.tags

Show complete workflow from binary data to classified strings.

3.3 Update Classification Documentation (`file:docs/src/classification.md`)

Add section "Framework Architecture" explaining:

The Classifier trait and pluggable design
StringContext and contextual information
Classification results and confidence scoring
ClassificationPipeline orchestration
Priority-based execution order
Confidence merging for duplicate tags

Update existing code examples to use the new framework types instead of placeholder code.

Phase 4: Validation and Cleanup

4.1 Verify Type Consistency

Ensure all types use consistent patterns:

Classification uses f64 for confidence (matching documentation)
StringContext lifetime annotations are correct
Error handling uses Result<T> from crate::types
All public types have Debug derive
Serializable types have Serialize/Deserialize where appropriate

4.2 Run Test Suite

Execute all tests:

Unit tests in file:tests/classification_tests.rs
Module-level tests in classification submodules
Verify no regressions in existing extraction tests
Ensure all tests pass with cargo test

4.3 Documentation Review

Verify documentation completeness:

All public types have doc comments
All public methods have doc comments with examples where appropriate
Module-level documentation explains architecture
Integration examples are clear and runnable

Architecture Diagram

sequenceDiagram
    participant Client
    participant BasicExtractor
    participant Pipeline as ClassificationPipeline
    participant Context as StringContext
    participant C1 as Classifier 1 (priority 10)
    participant C2 as Classifier 2 (priority 50)
    participant C3 as Classifier 3 (priority 100)

    Client->>BasicExtractor: extract(data, container_info, config)
    BasicExtractor-->>Client: Vec<FoundString>
    
    Client->>Pipeline: with_defaults()
    Pipeline-->>Client: ClassificationPipeline
    
    loop For each FoundString
        Client->>Context: new(found_string, container_info)
        Context-->>Client: StringContext
        
        Client->>Pipeline: classify(context)
        
        Pipeline->>C1: can_classify(context)?
        C1-->>Pipeline: true
        Pipeline->>C1: classify(context)
        C1-->>Pipeline: Vec<Classification>
        
        Pipeline->>C2: can_classify(context)?
        C2-->>Pipeline: true
        Pipeline->>C2: classify(context)
        C2-->>Pipeline: Vec<Classification>
        
        Pipeline->>C3: can_classify(context)?
        C3-->>Pipeline: false
        
        Pipeline->>Pipeline: merge_results()
        Pipeline-->>Client: Vec<Classification>
        
        Client->>Client: Apply high-confidence tags to FoundString
    end

Component Structure

Component	File	Responsibility
`Classifier` trait	`file:src/classification/traits.rs`	Define interface for all classifiers
`StringContext`	`file:src/classification/context.rs`	Package string with contextual metadata
`Classification`	`file:src/classification/mod.rs`	Represent classification result with confidence
`ClassificationPipeline`	`file:src/classification/pipeline.rs`	Orchestrate multiple classifiers
Module exports	`file:src/classification/mod.rs`	Re-export public API
Library exports	`file:src/lib.rs`	Expose classification framework
Tests	`file:tests/classification_tests.rs`	Comprehensive test coverage

Key Design Decisions

Decision	Rationale
Trait-based architecture	Enables pluggable classifiers, mirrors existing `StringExtractor` pattern
Priority-based ordering	Allows fast classifiers (import/export) to run before expensive regex patterns
Confidence merging	Handles multiple classifiers detecting same tag, keeps highest confidence
Lifetime annotations on `StringContext`	Avoids cloning large structures, maintains zero-copy where possible
`Send + Sync` on `Classifier`	Enables future parallel processing without breaking changes
`can_classify()` filter	Allows classifiers to opt-out based on encoding, format, or other factors
Builder pattern for `Classification`	Provides ergonomic API for adding optional reason strings

Future Extensibility

This framework enables future issues to add concrete classifiers:

ImportExportClassifier (priority 10) - Tag strings matching import/export tables
UrlClassifier (priority 50) - Detect URLs with regex patterns
IpAddressClassifier (priority 60) - Detect IPv4/IPv6 addresses
FilePathClassifier (priority 70) - Detect file paths (POSIX and Windows)
Base64Classifier (priority 80) - Detect Base64-encoded data
FormatStringClassifier (priority 90) - Detect printf-style format strings

Each classifier will implement the Classifier trait and can be added to the pipeline via add_classifier().

Import In IDE

🤖 Prompt for AI Agents

## Observations

The Stringy codebase has a solid foundation with complete type definitions in `file:src/types.rs`, a mature extraction framework in `file:src/extraction/mod.rs`, and comprehensive container parsing. The `file:src/classification/mod.rs` module exists but is empty (only a comment). All necessary infrastructure types (`FoundString`, `Tag`, `ContainerInfo`, `SectionType`, `BinaryFormat`, `ImportInfo`, `ExportInfo`) are already defined. The project follows clear patterns: trait-based abstractions, configuration structs with validation, comprehensive unit tests, and proper error handling using `thiserror`.

## Approach

Implement the classification framework as a trait-based, pluggable architecture that mirrors the existing extraction framework's design patterns. Create four core components: the `Classifier` trait for pluggable strategies, `StringContext` for packaging extraction context, `Classification` for results with confidence scoring, and `ClassificationPipeline` for orchestration. This approach enables future classifier implementations while maintaining consistency with the existing codebase's architecture. The implementation will be test-driven, starting with the core framework and mock classifiers before adding real implementations in subsequent issues.

## Implementation Steps

### Phase 1: Core Framework Structure

#### 1.1 Create Classifier Trait (`file:src/classification/traits.rs`)

Define the `Classifier` trait with the following methods:
- `classify(&self, context: &StringContext) -> Result<Vec<Classification>>` - Main classification method returning multiple possible classifications
- `name(&self) -> &str` - Identifier for logging and debugging
- `can_classify(&self, context: &StringContext) -> bool` - Optional filter with default implementation returning `true`
- `priority(&self) -> i32` - Ordering hint with default value of `100`

Ensure the trait is `Send + Sync` for future parallel processing support. Import necessary types from `crate::types` (`Tag`, `StringyError`, `Result`).

#### 1.2 Create StringContext Struct (`file:src/classification/context.rs`)

Implement `StringContext<'a>` with these fields:
- `found_string: &'a FoundString` - Reference to the extracted string
- `section_type: Option<SectionType>` - Inferred from section name
- `binary_format: BinaryFormat` - From container info
- `surrounding_strings: Vec<String>` - For context-aware classification
- `container_info: &'a ContainerInfo` - Full binary metadata
- `import_match: Option<&'a ImportInfo>` - If string matches an import
- `export_match: Option<&'a ExportInfo>` - If string matches an export

Add constructor `new(found_string: &'a FoundString, container_info: &'a ContainerInfo)` that:
- Infers `section_type` by matching `found_string.section` against `container_info.sections`
- Searches `container_info.imports` for matching import names
- Searches `container_info.exports` for matching export names
- Initializes `surrounding_strings` as empty vector

Add helper method `with_surrounding(&mut self, strings: Vec<String>)` for adding context strings.

Add private helper `infer_section_type(name: &str, info: &ContainerInfo) -> Option<SectionType>` to look up section type from container info.

#### 1.3 Create Classification Result Type (`file:src/classification/mod.rs`)

Define `Classification` struct with:
- `tag: Tag` - The assigned semantic tag
- `confidence: f64` - Score from 0.0 to 1.0
- `reason: Option<String>` - Optional explanation for debugging
- `classifier_name: String` - Source classifier identifier

Implement methods:
- `new(tag: Tag, confidence: f64, classifier_name: String) -> Self` - Basic constructor
- `with_reason(mut self, reason: impl Into<String>) -> Self` - Builder pattern for adding explanation
- `is_high_confidence(&self) -> bool` - Returns `true` if confidence >= 0.7

Derive `Debug`, `Clone`, `PartialEq` for the struct.

#### 1.4 Create Classification Pipeline (`file:src/classification/pipeline.rs`)

Implement `ClassificationPipeline` struct with:
- `classifiers: Vec<Box<dyn Classifier>>` - Ordered list of classifiers

Add methods:
- `new() -> Self` - Create empty pipeline
- `with_defaults() -> Self` - Create pipeline (currently empty, will be populated in future issues)
- `add_classifier(&mut self, classifier: Box<dyn Classifier>)` - Add classifier and sort by priority
- `classify(&self, context: &StringContext) -> Result<Vec<Classification>>` - Run all applicable classifiers, collect results, merge duplicates
- `len(&self) -> usize` - Return classifier count
- `is_empty(&self) -> bool` - Check if pipeline has no classifiers

Implement `merge_results(&self, results: Vec<Classification>) -> Vec<Classification>` private method that:
- Uses `HashMap<String, Classification>` keyed by `format!("{:?}", tag)`
- Keeps highest confidence for duplicate tags
- Sorts final results by confidence (descending)

Implement `Default` trait delegating to `new()`.

In the `classify` method:
- Iterate through classifiers in priority order
- Skip classifiers where `can_classify()` returns `false`
- Catch errors from individual classifiers, log warnings, continue processing
- Merge and deduplicate results before returning

#### 1.5 Update Module Exports (`file:src/classification/mod.rs`)

Add module declarations:
```rust
pub mod traits;
pub mod context;
pub mod pipeline;
```

Re-export public types:
```rust
pub use traits::Classifier;
pub use context::StringContext;
pub use pipeline::ClassificationPipeline;
// Classification struct defined in this file
```

Add comprehensive module documentation explaining the classification framework architecture, trait-based design, and usage examples.

### Phase 2: Testing Infrastructure

#### 2.1 Create Classification Tests (`file:tests/classification_tests.rs`)

Implement `MockClassifier` struct for testing:
- Fields: `name: String`, `priority: i32`, `tags: Vec<Tag>`, `confidence: f64`
- Implement `Classifier` trait returning configured tags with specified confidence
- Add optional `can_classify_fn: Option<Box<dyn Fn(&StringContext) -> bool>>` for testing filtering

Create test cases:
- `test_pipeline_ordering()` - Verify classifiers execute in priority order (add low priority then high priority, verify sorting)
- `test_pipeline_empty()` - Test empty pipeline returns empty results
- `test_pipeline_single_classifier()` - Test single classifier execution
- `test_pipeline_multiple_classifiers()` - Test multiple classifiers with different tags
- `test_confidence_merging()` - Add two classifiers returning same tag with different confidence, verify highest is kept
- `test_can_classify_filtering()` - Test classifiers can opt-out via `can_classify()`
- `test_context_creation()` - Create `StringContext` from `FoundString` and `ContainerInfo`, verify section type inference
- `test_context_import_matching()` - Verify import name matching in context
- `test_context_export_matching()` - Verify export name matching in context
- `test_classification_builder()` - Test `Classification::new()` and `with_reason()` builder pattern
- `test_classification_confidence_threshold()` - Test `is_high_confidence()` method
- `test_pipeline_error_handling()` - Create classifier that returns error, verify pipeline continues with other classifiers

Use helper functions to create test fixtures:
- `create_test_found_string(text: &str) -> FoundString` - Create minimal `FoundString` with required fields
- `create_test_container_info() -> ContainerInfo` - Create minimal `ContainerInfo` with test sections, imports, exports

#### 2.2 Add Unit Tests to Module Files

In `file:src/classification/context.rs`:
- Test `StringContext::new()` with various section names
- Test section type inference for common section names (.rodata, .text, .data, __cstring, etc.)
- Test import/export matching logic
- Test `with_surrounding()` method

In `file:src/classification/pipeline.rs`:
- Test `add_classifier()` maintains priority ordering
- Test `merge_results()` deduplication logic
- Test sorting by confidence
- Test `Default` implementation

### Phase 3: Integration and Documentation

#### 3.1 Update Library Exports (`file:src/lib.rs`)

Add to re-exports section:
```rust
pub use classification::{Classifier, StringContext, Classification, ClassificationPipeline};
```

Update module documentation to reflect classification framework availability:
- Change classification status from "🚧 Types defined" to "✅ Framework complete"
- Add usage example showing integration with `BasicExtractor`

#### 3.2 Add Integration Example

In `file:src/lib.rs` documentation, add example demonstrating:
- Extract strings using `BasicExtractor`
- Create `ClassificationPipeline::with_defaults()`
- Iterate through extracted strings
- Create `StringContext` for each string
- Call `pipeline.classify(&context)`
- Apply high-confidence tags to `FoundString.tags`

Show complete workflow from binary data to classified strings.

#### 3.3 Update Classification Documentation (`file:docs/src/classification.md`)

Add section "Framework Architecture" explaining:
- The `Classifier` trait and pluggable design
- `StringContext` and contextual information
- `Classification` results and confidence scoring
- `ClassificationPipeline` orchestration
- Priority-based execution order
- Confidence merging for duplicate tags

Update existing code examples to use the new framework types instead of placeholder code.

### Phase 4: Validation and Cleanup

#### 4.1 Verify Type Consistency

Ensure all types use consistent patterns:
- `Classification` uses `f64` for confidence (matching documentation)
- `StringContext` lifetime annotations are correct
- Error handling uses `Result<T>` from `crate::types`
- All public types have `Debug` derive
- Serializable types have `Serialize`/`Deserialize` where appropriate

#### 4.2 Run Test Suite

Execute all tests:
- Unit tests in `file:tests/classification_tests.rs`
- Module-level tests in classification submodules
- Verify no regressions in existing extraction tests
- Ensure all tests pass with `cargo test`

#### 4.3 Documentation Review

Verify documentation completeness:
- All public types have doc comments
- All public methods have doc comments with examples where appropriate
- Module-level documentation explains architecture
- Integration examples are clear and runnable

## Architecture Diagram

```mermaid
sequenceDiagram
    participant Client
    participant BasicExtractor
    participant Pipeline as ClassificationPipeline
    participant Context as StringContext
    participant C1 as Classifier 1 (priority 10)
    participant C2 as Classifier 2 (priority 50)
    participant C3 as Classifier 3 (priority 100)

    Client->>BasicExtractor: extract(data, container_info, config)
    BasicExtractor-->>Client: Vec<FoundString>
    
    Client->>Pipeline: with_defaults()
    Pipeline-->>Client: ClassificationPipeline
    
    loop For each FoundString
        Client->>Context: new(found_string, container_info)
        Context-->>Client: StringContext
        
        Client->>Pipeline: classify(context)
        
        Pipeline->>C1: can_classify(context)?
        C1-->>Pipeline: true
        Pipeline->>C1: classify(context)
        C1-->>Pipeline: Vec<Classification>
        
        Pipeline->>C2: can_classify(context)?
        C2-->>Pipeline: true
        Pipeline->>C2: classify(context)
        C2-->>Pipeline: Vec<Classification>
        
        Pipeline->>C3: can_classify(context)?
        C3-->>Pipeline: false
        
        Pipeline->>Pipeline: merge_results()
        Pipeline-->>Client: Vec<Classification>
        
        Client->>Client: Apply high-confidence tags to FoundString
    end
```

## Component Structure

| Component | File | Responsibility |
|-----------|------|----------------|
| `Classifier` trait | `file:src/classification/traits.rs` | Define interface for all classifiers |
| `StringContext` | `file:src/classification/context.rs` | Package string with contextual metadata |
| `Classification` | `file:src/classification/mod.rs` | Represent classification result with confidence |
| `ClassificationPipeline` | `file:src/classification/pipeline.rs` | Orchestrate multiple classifiers |
| Module exports | `file:src/classification/mod.rs` | Re-export public API |
| Library exports | `file:src/lib.rs` | Expose classification framework |
| Tests | `file:tests/classification_tests.rs` | Comprehensive test coverage |

## Key Design Decisions

| Decision | Rationale |
|----------|-----------|
| Trait-based architecture | Enables pluggable classifiers, mirrors existing `StringExtractor` pattern |
| Priority-based ordering | Allows fast classifiers (import/export) to run before expensive regex patterns |
| Confidence merging | Handles multiple classifiers detecting same tag, keeps highest confidence |
| Lifetime annotations on `StringContext` | Avoids cloning large structures, maintains zero-copy where possible |
| `Send + Sync` on `Classifier` | Enables future parallel processing without breaking changes |
| `can_classify()` filter | Allows classifiers to opt-out based on encoding, format, or other factors |
| Builder pattern for `Classification` | Provides ergonomic API for adding optional reason strings |

## Future Extensibility

This framework enables future issues to add concrete classifiers:
- `ImportExportClassifier` (priority 10) - Tag strings matching import/export tables
- `UrlClassifier` (priority 50) - Detect URLs with regex patterns
- `IpAddressClassifier` (priority 60) - Detect IPv4/IPv6 addresses
- `FilePathClassifier` (priority 70) - Detect file paths (POSIX and Windows)
- `Base64Classifier` (priority 80) - Detect Base64-encoded data
- `FormatStringClassifier` (priority 90) - Detect printf-style format strings

Each classifier will implement the `Classifier` trait and can be added to the pipeline via `add_classifier()`.

Execution Information

Branch: main
Commit: e971b47

💡 Tips

Supported Commands (Inside Comments)

Use @traycerai generate to iterate on the previous version of the implementation plan.

Supported Commands (Inside Description)

Add @traycerai ignore anywhere in the ticket description to prevent this ticket from being processed.
Add @traycerai branch:<branch-name> anywhere in the ticket description to specify the target branch for the implementation plan.

Community

Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

@unclesp1d3r

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Fetch details of issue #14 from the Stringy repository
gh issue view 14 --repo EvilBit-Labs/Stringy --json title,body,labels,assignees,state,author,createdAt,updatedAt

Length of output: 5709

🏁 Script executed:

#!/bin/bash
# Explore the project structure to understand the classification system context
fd -t f "mod.rs|lib.rs|main.rs" | head -30

# Check if classification directory exists
fd -t d "classification"

# Look for existing types mentioned in the issue
rg -n "pub struct FoundString|pub enum Tag|pub struct StringSource|pub trait Classifier" --type rust -C 2

# Get repository structure
ls -la

# Check for existing documentation
fd -t f "README.md|ARCHITECTURE.md|DESIGN.md"

Length of output: 2391

🏁 Script executed:

#!/bin/bash
# Check what's currently in the classification module
cat src/classification/mod.rs

# Check the types module to see all existing types
cat src/types.rs

# Check lib.rs to see how modules are organized
cat src/lib.rs

# Look at README for project overview
head -100 README.md

Length of output: 14879

🏁 Script executed:

#!/bin/bash
# Update the issue with comprehensive enhancement
gh issue edit 14 --repo EvilBit-Labs/Stringy \
  --title "Create Classifier Trait and Classification Pipeline Architecture" \
  --body "## Summary

Create the foundational Classifier trait and classification pipeline structure to enable semantic analysis of extracted strings. This framework will allow multiple classifiers to analyze strings and assign tags (URLs, domains, file paths, etc.) based on context and patterns.

## Current State

The infrastructure needed for this feature is already in place:

✅ **Existing Components:**
- \`FoundString\` struct with tags, encoding, offset, RVA, confidence scoring
- \`Tag\` enum with comprehensive variants (URL, Domain, IPv4, IPv6, FilePath, etc.)
- \`ContainerInfo\` with sections, imports, exports, resources
- \`SectionType\` and \`BinaryFormat\` enums
- \`BasicExtractor\` for string extraction from binaries
- ASCII and UTF-16LE extraction with confidence scoring

🚧 **To Be Built:**
- \`Classifier\` trait for pluggable classification strategies
- \`StringContext\` struct to package extraction context
- \`Classification\` result type with confidence scoring
- \`ClassificationPipeline\` orchestration layer
- Concrete classifier implementations

## Context

Stringy extracts strings from binary files and needs to classify them to distinguish meaningful strings from random garbage data. The classification system will:

- Provide a common interface (Classifier trait) for different classification strategies
- Package string data with contextual information (StringContext) for informed classification
- Enable a pipeline architecture where multiple classifiers can process strings sequentially or in parallel
- Support confidence scoring to handle ambiguous cases
- Integrate seamlessly with existing \`BasicExtractor\` and \`FoundString\` types
- Leverage existing section weighting and confidence scores from extraction phase

## Proposed Solution

### 1. Classifier Trait

Create a trait that all classifiers will implement in \`src/classification/traits.rs\`:

\`\`\`rust
use crate::types::{Tag, StringyError, Result};

/// Trait for string classifiers that analyze and tag extracted strings
pub trait Classifier: Send + Sync {
    /// Classify a string and return relevant tags with confidence scores
    /// 
    /// Returns a vector of classifications, allowing one string to match
    /// multiple patterns (e.g., a URL that's also a domain).
    fn classify(&self, context: &StringContext) -> Result<Vec<Classification>>;
    
    /// Get the name/identifier of this classifier for logging and debugging
    fn name(&self) -> &str;
    
    /// Check if this classifier can handle the given context
    /// 
    /// Allows classifiers to opt-out based on encoding, binary format,
    /// or other contextual factors. Default implementation accepts all.
    fn can_classify(&self, context: &StringContext) -> bool {
        true
    }
    
    /// Priority/order hint for the pipeline (lower = earlier execution)
    /// 
    /// Some classifiers may want to run before others, e.g., import/export
    /// classifiers should run before generic pattern matching.
    fn priority(&self) -> i32 {
        100  // Default middle priority
    }
}
\`\`\`

### 2. StringContext Struct

Encapsulate string data with contextual information in \`src/classification/context.rs\`:

\`\`\`rust
use crate::types::{
    FoundString, SectionType, BinaryFormat, ContainerInfo, 
    ImportInfo, ExportInfo
};

/// Context information about an extracted string for classification
#[derive(Debug, Clone)]
pub struct StringContext<'a> {
    /// The extracted string with metadata
    pub found_string: &'a FoundString,
    
    /// Section type where string was found (StringData, Code, etc.)
    pub section_type: Option<SectionType>,
    
    /// Binary format (ELF, PE, Mach-O)
    pub binary_format: BinaryFormat,
    
    /// Nearby strings for context-aware classification (within ±100 bytes)
    pub surrounding_strings: Vec<String>,
    
    /// Full container information
    pub container_info: &'a ContainerInfo,
    
    /// Matching import if string is from import table
    pub import_match: Option<&'a ImportInfo>,
    
    /// Matching export if string is from export table  
    pub export_match: Option<&'a ExportInfo>,
}

impl<'a> StringContext<'a> {
    /// Create a new context for a found string
    pub fn new(
        found_string: &'a FoundString,
        container_info: &'a ContainerInfo,
    ) -> Self {
        let section_type = found_string.section.as_ref()
            .and_then(|name| Self::infer_section_type(name, container_info));
        
        let import_match = container_info.imports.iter()
            .find(|imp| imp.name == found_string.text);
            
        let export_match = container_info.exports.iter()
            .find(|exp| exp.name == found_string.text);
        
        Self {
            found_string,
            section_type,
            binary_format: container_info.format,
            surrounding_strings: Vec::new(),
            container_info,
            import_match,
            export_match,
        }
    }
    
    /// Infer section type from section name
    fn infer_section_type(name: &str, info: &ContainerInfo) -> Option<SectionType> {
        info.sections.iter()
            .find(|s| s.name == name)
            .map(|s| s.section_type)
    }
    
    /// Add surrounding strings for context
    pub fn with_surrounding(&mut self, strings: Vec<String>) {
        self.surrounding_strings = strings;
    }
}
\`\`\`

### 3. Classification Result

Define the output structure in \`src/classification/mod.rs\`:

\`\`\`rust
use crate::types::Tag;

/// Result of classifying a string
#[derive(Debug, Clone, PartialEq)]
pub struct Classification {
    /// The assigned tag
    pub tag: Tag,
    
    /// Confidence score (0.0 to 1.0)
    /// - 1.0: Definitive match (e.g., valid URL with scheme)
    /// - 0.7-0.9: Strong match (e.g., domain pattern with TLD)
    /// - 0.5-0.7: Moderate match (e.g., path-like structure)
    /// - <0.5: Weak match (should probably be filtered)
    pub confidence: f64,
    
    /// Optional explanation for debugging/logging
    pub reason: Option<String>,
    
    /// The classifier that produced this result
    pub classifier_name: String,
}

impl Classification {
    /// Create a new classification result
    pub fn new(tag: Tag, confidence: f64, classifier_name: String) -> Self {
        Self {
            tag,
            confidence,
            reason: None,
            classifier_name,
        }
    }
    
    /// Add an explanation
    pub fn with_reason(mut self, reason: impl Into<String>) -> Self {
        self.reason = Some(reason.into());
        self
    }
    
    /// Check if this is a high-confidence classification
    pub fn is_high_confidence(&self) -> bool {
        self.confidence >= 0.7
    }
}
\`\`\`

### 4. Classification Pipeline

Create orchestration layer in \`src/classification/pipeline.rs\`:

\`\`\`rust
use crate::types::Result;
use super::{Classifier, Classification, StringContext};
use std::collections::HashMap;

/// Pipeline for orchestrating multiple classifiers
pub struct ClassificationPipeline {
    classifiers: Vec<Box<dyn Classifier>>,
}

impl ClassificationPipeline {
    /// Create a new empty pipeline
    pub fn new() -> Self {
        Self {
            classifiers: Vec::new(),
        }
    }
    
    /// Create a pipeline with default classifiers
    pub fn with_defaults() -> Self {
        let mut pipeline = Self::new();
        // Will add default classifiers in future issues:
        // - ImportExportClassifier (priority: 10)
        // - UrlClassifier (priority: 50)
        // - IpAddressClassifier (priority: 60)
        // - FilePathClassifier (priority: 70)
        // - Base64Classifier (priority: 80)
        // - FormatStringClassifier (priority: 90)
        pipeline
    }
    
    /// Add a classifier to the pipeline
    pub fn add_classifier(&mut self, classifier: Box<dyn Classifier>) {
        self.classifiers.push(classifier);
        // Sort by priority after adding
        self.classifiers.sort_by_key(|c| c.priority());
    }
    
    /// Classify a string using all applicable classifiers
    pub fn classify(&self, context: &StringContext) -> Result<Vec<Classification>> {
        let mut results = Vec::new();
        
        for classifier in &self.classifiers {
            // Skip if classifier can't handle this context
            if !classifier.can_classify(context) {
                continue;
            }
            
            // Run classification and collect results
            match classifier.classify(context) {
                Ok(classifications) => results.extend(classifications),
                Err(e) => {
                    eprintln!(\"Warning: Classifier '{}' failed: {}\", 
                              classifier.name(), e);
                    continue;
                }
            }
        }
        
        // Merge and deduplicate results
        Ok(self.merge_results(results))
    }
    
    /// Merge duplicate tags, keeping highest confidence
    fn merge_results(&self, mut results: Vec<Classification>) -> Vec<Classification> {
        let mut tag_map: HashMap<String, Classification> = HashMap::new();
        
        for classification in results.drain(..) {
            let key = format!(\"{:?}\", classification.tag);
            
            tag_map.entry(key)
                .and_modify(|existing| {
                    // Keep classification with higher confidence
                    if classification.confidence > existing.confidence {
                        *existing = classification.clone();
                    }
                })
                .or_insert(classification);
        }
        
        // Sort by confidence (highest first)
        let mut merged: Vec<_> = tag_map.into_values().collect();
        merged.sort_by(|a, b| {
            b.confidence.partial_cmp(&a.confidence)
                .unwrap_or(std::cmp::Ordering::Equal)
        });
        
        merged
    }
    
    /// Get the number of classifiers in the pipeline
    pub fn len(&self) -> usize {
        self.classifiers.len()
    }
    
    /// Check if the pipeline is empty
    pub fn is_empty(&self) -> bool {
        self.classifiers.is_empty()
    }
}

impl Default for ClassificationPipeline {
    fn default() -> Self {
        Self::new()
    }
}
\`\`\`

### 5. Integration with Existing Extraction

Example usage in \`src/lib.rs\` or extraction workflow:

\`\`\`rust
use crate::extraction::{BasicExtractor, ExtractionConfig, StringExtractor};
use crate::classification::{ClassificationPipeline, StringContext};
use crate::types::{FoundString, ContainerInfo};

/// Extract and classify strings from a binary
pub fn extract_and_classify(
    data: &[u8],
    container_info: &ContainerInfo,
) -> Result<Vec<FoundString>> {
    // Extract strings
    let extractor = BasicExtractor::new();
    let config = ExtractionConfig::default();
    let mut strings = extractor.extract(data, container_info, &config)?;
    
    // Set up classification pipeline
    let pipeline = ClassificationPipeline::with_defaults();
    
    // Classify each string
    for found_string in &mut strings {
        let context = StringContext::new(found_string, container_info);
        
        if let Ok(classifications) = pipeline.classify(&context) {
            // Apply high-confidence tags to the string
            for classification in classifications {
                if classification.is_high_confidence() {
                    found_string.tags.push(classification.tag);
                }
            }
        }
    }
    
    Ok(strings)
}
\`\`\`

## Implementation Steps

1. **Phase 1: Core Framework** (\`src/classification/\`)
   - Create \`traits.rs\` with \`Classifier\` trait
   - Create \`context.rs\` with \`StringContext\` struct  
   - Add \`Classification\` struct to \`mod.rs\`
   - Create \`pipeline.rs\` with \`ClassificationPipeline\`
   - Re-export types in \`mod.rs\`

2. **Phase 2: Testing Infrastructure**
   - Create \`tests/classification_tests.rs\`
   - Implement \`MockClassifier\` for testing
   - Test pipeline ordering and priority
   - Test confidence scoring and merging
   - Test context creation from \`FoundString\`

3. **Phase 3: Integration**
   - Update \`src/lib.rs\` to re-export classification types
   - Add integration example in docs
   - Update \`BasicExtractor\` workflow example

4. **Phase 4: First Real Classifier (Separate Issue)**
   - Implement \`ImportExportClassifier\` as proof-of-concept
   - Test with real PE/ELF/Mach-O binaries

## Testing Strategy

### Unit Tests

\`\`\`rust
#[cfg(test)]
mod tests {
    use super::*;
    
    struct MockClassifier {
        name: String,
        priority: i32,
    }
    
    impl Classifier for MockClassifier {
        fn classify(&self, ctx: &StringContext) -> Result<Vec<Classification>> {
            Ok(vec![
                Classification::new(
                    Tag::Url,
                    0.9,
                    self.name.clone()
                )
            ])
        }
        
        fn name(&self) -> &str {
            &self.name
        }
        
        fn priority(&self) -> i32 {
            self.priority
        }
    }
    
    #[test]
    fn test_pipeline_ordering() {
        let mut pipeline = ClassificationPipeline::new();
        pipeline.add_classifier(Box::new(MockClassifier { 
            name: \"low\".into(), 
            priority: 100 
        }));
        pipeline.add_classifier(Box::new(MockClassifier { 
            name: \"high\".into(), 
            priority: 10 
        }));
        
        // Verify classifiers are sorted by priority
        assert_eq!(pipeline.len(), 2);
    }
    
    #[test]
    fn test_confidence_merging() {
        // Test that duplicate tags keep highest confidence
    }
    
    #[test]
    fn test_context_creation() {
        // Test StringContext properly extracts section type, 
        // matches imports/exports, etc.
    }
}
\`\`\`

### Integration Tests

Test with real binaries:
- PE file with imports/exports
- ELF file with URLs in .rodata
- Mach-O with version strings

## Performance Considerations

- **Lazy Evaluation**: Only create \`StringContext\` for strings that need classification
- **Parallel Processing**: Future enhancement to run classifiers in parallel (requires \`Send + Sync\`)
- **Caching**: Consider caching regex compilation in classifiers
- **Short-Circuit**: \`can_classify()\` allows early exit
- **Priority Ordering**: Run fast classifiers first (imports/exports before regex patterns)

## Acceptance Criteria

- [ ] \`Classifier\` trait defined with \`classify()\`, \`name()\`, \`can_classify()\`, and \`priority()\` methods
- [ ] \`StringContext\` struct with all necessary fields and helper methods
- [ ] \`Classification\` struct for results with tag, confidence, reason, and classifier name
- [ ] \`ClassificationPipeline\` with add, classify, and merge functionality
- [ ] Pipeline respects classifier priority ordering
- [ ] Unit tests for pipeline, context, and classification
- [ ] \`MockClassifier\` for testing framework
- [ ] Documentation comments for all public types and methods
- [ ] Integration example showing usage with \`BasicExtractor\`
- [ ] Re-exports in \`src/lib.rs\`

## Dependencies

**Existing Types (from \`src/types.rs\`):**
- \`FoundString\` - Contains text, encoding, offset, tags, confidence
- \`Tag\` - Enum of all semantic tags
- \`SectionType\` - Classification of binary sections  
- \`BinaryFormat\` - ELF, PE, Mach-O detection
- \`ContainerInfo\` - Binary metadata, sections, imports, exports
- \`ImportInfo\` / \`ExportInfo\` - Symbol information
- \`StringyError\` and \`Result\` - Error handling

**Will Enable (Future Issues):**
- Individual classifier implementations (URL, Domain, IP, FilePath, etc.)
- Smart scoring that combines extraction confidence + classification confidence
- Context-aware classification (e.g., grouped strings suggest log format)

## Related Requirements

- Requirement 3.1: Semantic Classification Framework  
- Task-ID: stringy-analyzer/semantic-classification-framework
- Blocks: URL Classifier (#TBD), IP Classifier (#TBD), FilePath Classifier (#TBD)

## References

- Existing extraction: \`src/extraction/basic.rs\`
- Type definitions: \`src/types.rs\`
- Container parsing: \`src/container/mod.rs\`"

echo "Issue #14 enhanced with comprehensive implementation details"

Length of output: 110

@unclesp1d3r I've successfully enhanced issue #14 with comprehensive production-ready details! 🎯

Major Enhancements Applied

✨ Current State Section

Added a clear breakdown of what already exists vs. what needs to be built:

✅ All required types (FoundString, Tag, ContainerInfo, etc.) already exist
🚧 Framework components to be built (trait, pipeline, context)

🎨 Improved Code Examples

All code examples now use actual types from your codebase:

Classifier trait with added priority() method for execution ordering
StringContext<'a> with lifetime parameters for zero-copy efficiency
Automatic import/export matching in context creation
ClassificationPipeline with intelligent merging and deduplication

🧪 Comprehensive Testing Strategy

Unit tests with MockClassifier example
Integration test scenarios with PE/ELF/Mach-O binaries
Test cases for pipeline ordering, confidence merging, context creation

⚡ Performance Considerations

Lazy evaluation patterns
Send + Sync bounds for future parallelization
Short-circuit evaluation with can_classify()
Priority-based execution order

🔗 Integration Examples

Added complete extract_and_classify() function showing how to integrate with existing BasicExtractor

📋 4-Phase Implementation Plan

Clear roadmap from core framework → testing → integration → first real classifier

Issue Metadata Status

The issue already has optimal configuration:

✅ Labels: area:analyzer, type:enhancement, status:backlog, lang:rust, needs:tests, story-points: 5
✅ Title: Clear and descriptive
✅ Assignee: You (unclesp1d3r)
✅ State: Open and ready for implementation

View the enhanced issue: #14

The issue is now production-ready with complete specifications, realistic code examples grounded in your actual codebase structure, and a clear path to implementation! 🚀

🐰✨
/> フ
| _ |
/`ミ x 彡
/ |
/ ヽノ
/ ̄| | | |
| ( ̄ヽ__ヽ))

Create Classifier Trait and Classification Pipeline Architecture #14

Description

Summary

Current State

Context

Proposed Solution

1. Classifier Trait

2. StringContext Struct

3. Classification Result

4. Classification Pipeline

5. Integration with Existing Extraction

Implementation Steps

Testing Strategy

Unit Tests

Integration Tests

Performance Considerations

Acceptance Criteria

Dependencies

Related Requirements

References

Activity

unclesp1d3r commented on Sep 30, 2025

coderabbitai commented on Sep 30, 2025

Changes Made

Metadata Status

unclesp1d3r commented on Jan 4, 2026

traycerai commented on Jan 4, 2026

Plan

Observations

Approach

Implementation Steps

Phase 1: Core Framework Structure

1.1 Create Classifier Trait (file:src/classification/traits.rs)

1.2 Create StringContext Struct (file:src/classification/context.rs)

1.3 Create Classification Result Type (file:src/classification/mod.rs)

1.4 Create Classification Pipeline (file:src/classification/pipeline.rs)

1.5 Update Module Exports (file:src/classification/mod.rs)

Phase 2: Testing Infrastructure

2.1 Create Classification Tests (file:tests/classification_tests.rs)

2.2 Add Unit Tests to Module Files

Phase 3: Integration and Documentation

3.1 Update Library Exports (file:src/lib.rs)

3.2 Add Integration Example

3.3 Update Classification Documentation (file:docs/src/classification.md)

Phase 4: Validation and Cleanup

4.1 Verify Type Consistency

4.2 Run Test Suite

4.3 Documentation Review

Architecture Diagram

Component Structure

Key Design Decisions

Future Extensibility

Import In IDE

Supported Commands (Inside Comments)

Supported Commands (Inside Description)

Community

coderabbitai commented on Jan 4, 2026

Major Enhancements Applied

✨ Current State Section

🎨 Improved Code Examples

🧪 Comprehensive Testing Strategy

⚡ Performance Considerations

🔗 Integration Examples

📋 4-Phase Implementation Plan

Issue Metadata Status

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

1.1 Create Classifier Trait (`file:src/classification/traits.rs`)

1.2 Create StringContext Struct (`file:src/classification/context.rs`)

1.3 Create Classification Result Type (`file:src/classification/mod.rs`)

1.4 Create Classification Pipeline (`file:src/classification/pipeline.rs`)

1.5 Update Module Exports (`file:src/classification/mod.rs`)

2.1 Create Classification Tests (`file:tests/classification_tests.rs`)

3.1 Update Library Exports (`file:src/lib.rs`)

3.3 Update Classification Documentation (`file:docs/src/classification.md`)