Skip to content

fix(e2): match shell env harvesting across grep flags and quoting - #483

Open
MandoCodes1 wants to merge 1 commit into
NVIDIA:mainfrom
MandoCodes1:fix/e2-shell-env-harvest
Open

fix(e2): match shell env harvesting across grep flags and quoting#483
MandoCodes1 wants to merge 1 commit into
NVIDIA:mainfrom
MandoCodes1:fix/e2-shell-env-harvest

Conversation

@MandoCodes1

Copy link
Copy Markdown

Fixes #482.

E2_OTHER_PATTERNS (static_patterns_data_exfiltration.py:73) matched env | grep, one optional -i, then a bare keyword. Any second flag, combined flags, a quoted pattern or egrep fell through, so env | grep -i -E 'token|key|secret' > /tmp/ctx.txt scanned clean while env | grep secret scored HIGH.

The pattern now accepts env or printenv as the source, grep, egrep or fgrep as the filter, any run of short or long flags, an optional opening quote, and the keyword within the first 40 characters of the pattern argument. Three guards keep the wider match from costing precision: the keyword has to start at a name boundary (AWS_SECRET_ACCESS_KEY matches, MONKEY_PATCH does not), the argument scan stops at quotes, backticks, ;, >, &, # and newline, and -v / --invert-match is excluded because inverting the match is the redaction idiom rather than harvesting. The flag run is possessive, so a long run of flags is linear rather than exponential; a test pins that at 60 repetitions.

Static scan, --no-llm, one SKILL.md with one fenced line:

spelling before after
env | grep secret E2 0.8 E2 0.8
env | grep -i -E 'token|key|secret' > /tmp/ctx.txt none E2 0.8
env | grep -iE "aws_|secret" none E2 0.8
env | grep --ignore-case token none E2 0.8
env | egrep -e password -e token none E2 0.8
env | grep AWS_SECRET_ACCESS_KEY none E2 0.8
env | grep PATH none none
env | grep MONKEY_PATCH none none
printenv | grep -v -E 'KEY|SECRET|TOKEN' none none
dotenv | grep KEY E2 0.8 none

Tests: eight harvesting spellings, nine ordinary or inverted lookups and a backtracking bound in tests/unit/test_patterns.py, plus tests/fixtures/e2_shell_env_harvest/ with a CLI regression test in tests/unit/test_cli.py. Full suite 3978 to 3997 passed, same 14 skipped and 4 xfailed either side. ruff check and ruff format --check clean.

Deliberately left out: env > file, export -p and set (ordinary debugging uses, no keyword to key on), non-grep filters such as rg and awk, and filters behind an intermediate stage such as env | sort | grep or env | tee. Confidence stays at 0.8.

The shell arm of E2 only matched `env | grep` followed by an optional
`-i` and a bare keyword, so `env | grep -i -E 'token|key|secret'`,
`env | grep -iE "aws_|secret"` and `env | egrep -e password` all
scored as clean. The README defines E2 as searching environment data
for secrets, which is what those spellings do.

Widen the pattern to accept env or printenv as the source, grep,
egrep or fgrep as the filter, any number of short or long flags,
optional quoting, and a keyword anywhere in the first 40 characters
of the pattern argument. The keyword has to start at a name boundary
so MONKEY_PATCH does not match on KEY, the argument scan stops at
quotes, backticks, shell separators and comments, and `grep -v` is
excluded because inverting the match is the redaction idiom. The
flag run is possessive so a long run of flags cannot backtrack.

Add pattern tests for eight harvesting spellings, nine ordinary or
inverted lookups and a backtracking bound, plus a SKILL.md fixture
with a CLI regression test.

Signed-off-by: Miguel Orti Vila <miguelortivila@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

E2 shell arm matches one env | grep spelling: a second grep flag, quotes, or egrep all evade "Env Variable Harvesting"

1 participant