Repository navigation
[Feat][SDK-601] Add built in scrubbing #377
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 9 commits
Commits
Show all changes
24 commits
Select commit
Hold shift + click to select a range
2ba9494
feat(scrubbing): add StringUrlSanitizer and DefaultUrlSanitizer
buongarzoni 9049736
feat(scrubbing): add ScrubDataTransformer
buongarzoni 7a9ca73
feat(config): expose redactedKeys and urlSanitizer
buongarzoni 00dc4cb
feat(notifier): apply built-in scrubbing to every payload
buongarzoni bf3629d
refactor(okhttp): reuse the shared DefaultUrlSanitizer
buongarzoni 9fee327
refactor(api): update imports
buongarzoni 347ad48
refactor(scrubbing): update imports
buongarzoni 306b105
refactor(config): update imports
buongarzoni 52d5d40
refactor(okhttp): reuse the shared DefaultUrlSanitizer
buongarzoni 82fb70b
fix(scrubbing): scrub Frame.locals in Body.rollbarThreads
buongarzoni 8c06fd1
fix(scrubbing): match percent-encoded query parameter names
buongarzoni a09f485
fix(scrubbing): scrub Request.params and Request.metadata
buongarzoni 57399ec
fix(scrubbing): traverse collections and arrays when scrubbing nested…
buongarzoni 5482e72
fix(telemetry): sanitize URLs recorded as network telemetry events
buongarzoni f7bc6f3
test(scrubbing): cover ordering, reconfiguration and the okhttp sanit…
buongarzoni 4c21b07
docs: add scrubbing documentation
buongarzoni 1fc3ed6
feat(scrubbing): seed field scrubbing with a built-in key list
buongarzoni f016ac2
feat(config): expose useDefaultRedactedKeys
buongarzoni 9ee5bc7
test(scrubbing): cover the built-in key list and the opt-out
buongarzoni f357b6c
test(scrubbing): prove secrets are redacted with no configuration
buongarzoni 2ede510
docs: document the built-in redacted key list
buongarzoni 0aa3993
perf(scrubbing): match literal keys without the regex engine
buongarzoni 0c5a376
fix(scrubbing): redact Message.metadata in the body content + ignore …
buongarzoni 01cd471
fix(scrubbing): redact Message.metadata in the body content + ignore …
buongarzoni File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
51 changes: 51 additions & 0 deletions
51
rollbar-api/src/main/java/com/rollbar/api/scrubbing/DefaultUrlSanitizer.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| package com.rollbar.api.scrubbing; | ||
|
|
||
| /** | ||
| * Default {@link StringUrlSanitizer} that strips userinfo, query string, and fragment from URLs. | ||
| * Uses string scanning rather than {@code java.net.URI} to avoid allocation on clean URLs | ||
| * and to preserve the original percent-encoding without normalization. | ||
| */ | ||
| public final class DefaultUrlSanitizer implements StringUrlSanitizer { | ||
|
|
||
| public static final DefaultUrlSanitizer INSTANCE = new DefaultUrlSanitizer(); | ||
|
|
||
| private DefaultUrlSanitizer() { | ||
| } | ||
|
|
||
| @Override | ||
| public String sanitize(String url) { | ||
| if (url == null) { | ||
| return null; | ||
| } | ||
| // Fast path: no characters that can introduce query string, fragment, or userinfo. | ||
| if (url.indexOf('?') < 0 && url.indexOf('#') < 0 && url.indexOf('@') < 0) { | ||
| return url; | ||
| } | ||
| return strip(url); | ||
| } | ||
|
|
||
| private static String strip(String url) { | ||
| int end = url.length(); | ||
| int q = url.indexOf('?'); | ||
| int f = url.indexOf('#'); | ||
| if (q >= 0 && q < end) { | ||
| end = q; | ||
| } | ||
| if (f >= 0 && f < end) { | ||
| end = f; | ||
| } | ||
| // Strip userinfo: find "://" then the last "@" before the first "/" after the authority start. | ||
| String result = url.substring(0, end); | ||
| int schemeEnd = result.indexOf("://"); | ||
| if (schemeEnd >= 0) { | ||
| int hostStart = schemeEnd + 3; | ||
| int slashAfterHost = result.indexOf('/', hostStart); | ||
| int searchEnd = slashAfterHost < 0 ? result.length() : slashAfterHost; | ||
| int at = result.lastIndexOf('@', searchEnd); | ||
| if (at >= hostStart) { | ||
| result = result.substring(0, hostStart) + result.substring(at + 1); | ||
| } | ||
| } | ||
| return result; | ||
| } | ||
| } |
18 changes: 18 additions & 0 deletions
18
rollbar-api/src/main/java/com/rollbar/api/scrubbing/StringUrlSanitizer.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| package com.rollbar.api.scrubbing; | ||
|
|
||
| /** | ||
| * Sanitizes a URL string before it is included in a Rollbar payload. | ||
| * Implementations should strip sensitive components such as userinfo, | ||
| * query parameters, and fragments. | ||
| */ | ||
| @FunctionalInterface | ||
| public interface StringUrlSanitizer { | ||
| /** | ||
| * Returns a sanitized version of the given URL string, or {@code null} if | ||
| * the input is {@code null}. | ||
| * | ||
| * @param url the raw URL string, may be {@code null}. | ||
| * @return the sanitized URL, or {@code null}. | ||
| */ | ||
| String sanitize(String url); | ||
| } |
96 changes: 96 additions & 0 deletions
96
rollbar-api/src/test/java/com/rollbar/api/scrubbing/DefaultUrlSanitizerTest.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,96 @@ | ||
| package com.rollbar.api.scrubbing; | ||
|
|
||
| import org.junit.Test; | ||
|
|
||
| import static org.junit.Assert.*; | ||
|
|
||
| public class DefaultUrlSanitizerTest { | ||
|
|
||
| private final DefaultUrlSanitizer sanitizer = DefaultUrlSanitizer.INSTANCE; | ||
|
|
||
| @Test | ||
| public void nullInputReturnsNull() { | ||
| assertNull(sanitizer.sanitize(null)); | ||
| } | ||
|
|
||
| @Test | ||
| public void cleanUrlUnchanged() { | ||
| String url = "https://example.com/api/v1/things"; | ||
| assertEquals(url, sanitizer.sanitize(url)); | ||
| } | ||
|
|
||
| @Test | ||
| public void queryStringStripped() { | ||
| assertEquals( | ||
| "https://example.com/search", | ||
| sanitizer.sanitize("https://example.com/search?token=abc&page=1") | ||
| ); | ||
| } | ||
|
|
||
| @Test | ||
| public void fragmentStripped() { | ||
| assertEquals( | ||
| "https://example.com/page", | ||
| sanitizer.sanitize("https://example.com/page#section") | ||
| ); | ||
| } | ||
|
|
||
| @Test | ||
| public void userinfoStripped() { | ||
| assertEquals( | ||
| "https://example.com/path", | ||
| sanitizer.sanitize("https://user:pass@example.com/path") | ||
| ); | ||
| } | ||
|
|
||
| @Test | ||
| public void allThreeScrubbed() { | ||
| assertEquals( | ||
| "https://example.com/path", | ||
| sanitizer.sanitize("https://admin:secret@example.com/path?token=xyz#top") | ||
| ); | ||
| } | ||
|
|
||
| @Test | ||
| public void malformedUrlNoException() { | ||
| // Should not throw; best-effort strip | ||
| String result = sanitizer.sanitize("not-a-url?query=sensitive"); | ||
| assertNotNull(result); | ||
| assertFalse(result.contains("sensitive")); | ||
| } | ||
|
|
||
| @Test | ||
| public void malformedUrlWithUserinfo() { | ||
| String result = sanitizer.sanitize("http://user:secret@host/path?q=1"); | ||
| assertNotNull(result); | ||
| assertFalse(result.contains("secret")); | ||
| assertFalse(result.contains("q=1")); | ||
| } | ||
|
|
||
| @Test | ||
| public void emptyStringUnchanged() { | ||
| assertEquals("", sanitizer.sanitize("")); | ||
| } | ||
|
|
||
| @Test | ||
| public void cleanUrlReturnedAsSameInstance() { | ||
| String url = "https://example.com/api/v1/things"; | ||
| assertSame(url, sanitizer.sanitize(url)); | ||
| } | ||
|
|
||
| @Test | ||
| public void percentEncodedPathPreserved() { | ||
| // No ?, #, or @ — fast path must return the same instance without normalizing encoding. | ||
| String url = "https://example.com/path%20with%20spaces"; | ||
| assertSame(url, sanitizer.sanitize(url)); | ||
| } | ||
|
|
||
| @Test | ||
| public void atSignInPathNotTreatedAsUserinfo() { | ||
| // The @ is after the first path slash, so it is not userinfo. | ||
| String url = "https://example.com/users/@alice?token=x"; | ||
| String result = sanitizer.sanitize(url); | ||
| assertTrue(result.contains("@alice")); | ||
| assertFalse(result.contains("token")); | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[P1] Seed field scrubbing with a meaningful default list
When
redactedKeysis unset, this stores an empty list, soScrubDataTransformerleaves GET/POST parameters,request.query_string, custom data, metadata, andFrame.localsuntouched. Stripping the query fromrequest.urldoes not close that gap: the web request provider serializes parsed GET parameters and the raw query string separately, so/login?password=hunter2still sendshunter2under bothrequest.get.passwordandrequest.query_stringwith the default configuration.That means the default behavior only protects the built-in header deny-list and URL strings; it does not satisfy the stated goal that sensitive data is redacted by default. Please seed this with an additive built-in field list (for example password/passwd, secret, token/access_token, auth/authentication/authorization), ideally with an explicit override mechanism, and add a no-configuration end-to-end test proving the secret is absent from every relevant request representation.