Repository navigation
Commit bf9920f
authored
Add String Extraction Framework with Noise Filtering and Benchmarks (#108)
* feat(extraction): Introduce BasicExtractor and ExtractionConfig for string extraction
- Added the BasicExtractor struct to implement a sequential ASCII/UTF-8 string extraction algorithm from binary data.
- Introduced the ExtractionConfig struct to allow customization of extraction parameters, including minimum and maximum string lengths, encoding selection, and section filtering.
- Updated documentation to include usage examples for both BasicExtractor and ExtractionConfig.
- Added integration tests to validate the functionality of string extraction from various binary formats, including ELF and PE.
This enhancement improves the library's ability to extract meaningful strings from binary files, facilitating better analysis and understanding of their content.
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
* feat(extraction): Add ASCII string extraction module and update documentation
- Introduced a new module for ASCII string extraction, providing foundational functionality for extracting contiguous printable ASCII sequences from binary data.
- Implemented `extract_ascii_strings` and `extract_from_section` functions, along with an `ExtractionConfig` struct for customizable extraction parameters.
- Updated the main extraction module documentation to include details about the new ASCII extraction capabilities and usage examples.
- Enhanced the existing documentation to reflect the addition of ASCII extraction in the library's structure and functionality.
This addition improves the library's ability to handle ASCII string extraction, paving the way for future enhancements in UTF-8 and other encoding types.
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
* refactor(extraction): Rename ExtractionConfig to AsciiExtractionConfig and update related documentation
- Renamed `ExtractionConfig` to `AsciiExtractionConfig` to better reflect its purpose in ASCII string extraction.
- Updated all references in the codebase and documentation to use the new name, ensuring consistency.
- Enhanced documentation to clarify usage examples and configuration details for ASCII extraction.
- Improved function signatures and comments to align with the new naming convention.
This refactor improves code clarity and maintains a consistent naming scheme across the extraction module, facilitating better understanding and usage of the ASCII extraction functionality.
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
* feat(extraction): Enhance ASCII string extraction with noise filtering and new benchmarks
- Added a new `entropy` dependency to support advanced noise filtering capabilities in ASCII string extraction.
- Introduced a new benchmark for ASCII extraction, `ascii_extraction`, to evaluate performance under various conditions.
- Updated the `FoundString` struct to include a `confidence` field, representing the likelihood of a string being legitimate versus noise.
- Enhanced the ASCII extraction functions to compute confidence scores using a composite noise filter, allowing for better filtering of false positives.
- Updated documentation to reflect the new noise filtering features and provided examples for configuring extraction parameters.
- Added unit and integration tests to validate the new noise filtering functionality and ensure robust performance.
This enhancement significantly improves the accuracy and reliability of ASCII string extraction, making it more effective for analyzing binary data.
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
* chore(deps): Update dependencies in Cargo.toml
- Bumped `entropy` to version 0.4.2 for improved functionality.
- Updated `pelite` to version 0.10.0 for better compatibility.
- Upgraded `serde_json` to version 1.0.145 to incorporate the latest features and fixes.
- Updated `insta` to version 1.43.2 and `tempfile` to version 3.23.0 for enhanced testing capabilities.
These updates ensure the project utilizes the latest versions of dependencies, improving overall stability and performance.
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
* chore(ci): Update GitHub Actions dependencies and configuration
- Updated `actions/upload-artifact` from version 4 to 5 across multiple workflows for improved artifact management.
- Updated `actions/download-artifact` from version 5 to 6 to leverage new features and enhancements.
- Updated `actions/attest-build-provenance` from version 2 to 3 for better build provenance tracking.
- Adjusted the `targets` list formatting in `dist-workspace.toml` for improved readability.
These updates ensure the workflows are utilizing the latest versions of the actions, enhancing overall CI/CD performance and reliability.
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
---------
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>1 parent bd0b1b5 commit bf9920f
17 files changed
Lines changed: 4942 additions & 43 deletions
File tree
- .github/workflows
- benches
- docs/src
- src
- extraction
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
66 | 66 | | |
67 | 67 | | |
68 | 68 | | |
69 | | - | |
| 69 | + | |
70 | 70 | | |
71 | 71 | | |
72 | 72 | | |
| |||
82 | 82 | | |
83 | 83 | | |
84 | 84 | | |
85 | | - | |
| 85 | + | |
86 | 86 | | |
87 | 87 | | |
88 | 88 | | |
| |||
135 | 135 | | |
136 | 136 | | |
137 | 137 | | |
138 | | - | |
| 138 | + | |
139 | 139 | | |
140 | 140 | | |
141 | 141 | | |
| |||
151 | 151 | | |
152 | 152 | | |
153 | 153 | | |
154 | | - | |
| 154 | + | |
155 | 155 | | |
156 | 156 | | |
157 | 157 | | |
| |||
168 | 168 | | |
169 | 169 | | |
170 | 170 | | |
171 | | - | |
| 171 | + | |
172 | 172 | | |
173 | 173 | | |
174 | 174 | | |
| |||
190 | 190 | | |
191 | 191 | | |
192 | 192 | | |
193 | | - | |
| 193 | + | |
194 | 194 | | |
195 | 195 | | |
196 | 196 | | |
| |||
202 | 202 | | |
203 | 203 | | |
204 | 204 | | |
205 | | - | |
| 205 | + | |
206 | 206 | | |
207 | 207 | | |
208 | 208 | | |
| |||
233 | 233 | | |
234 | 234 | | |
235 | 235 | | |
236 | | - | |
| 236 | + | |
237 | 237 | | |
238 | 238 | | |
239 | 239 | | |
| |||
259 | 259 | | |
260 | 260 | | |
261 | 261 | | |
262 | | - | |
| 262 | + | |
263 | 263 | | |
264 | 264 | | |
265 | 265 | | |
266 | 266 | | |
267 | 267 | | |
268 | 268 | | |
269 | | - | |
| 269 | + | |
270 | 270 | | |
271 | 271 | | |
272 | 272 | | |
| |||
279 | 279 | | |
280 | 280 | | |
281 | 281 | | |
282 | | - | |
| 282 | + | |
283 | 283 | | |
284 | 284 | | |
285 | 285 | | |
286 | 286 | | |
287 | 287 | | |
288 | 288 | | |
289 | | - | |
| 289 | + | |
290 | 290 | | |
291 | 291 | | |
292 | 292 | | |
| |||
326 | 326 | | |
327 | 327 | | |
328 | 328 | | |
329 | | - | |
| 329 | + | |
330 | 330 | | |
331 | 331 | | |
332 | 332 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
| 23 | + | |
23 | 24 | | |
24 | | - | |
| 25 | + | |
25 | 26 | | |
26 | | - | |
| 27 | + | |
27 | 28 | | |
28 | 29 | | |
29 | 30 | | |
30 | 31 | | |
31 | | - | |
32 | | - | |
| 32 | + | |
| 33 | + | |
33 | 34 | | |
34 | 35 | | |
35 | 36 | | |
| |||
43 | 44 | | |
44 | 45 | | |
45 | 46 | | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
13 | | - | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
14 | 20 | | |
15 | 21 | | |
16 | 22 | | |
| |||
49 | 55 | | |
50 | 56 | | |
51 | 57 | | |
52 | | - | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
0 commit comments