Skip to content

Handle errors in lookup script #5096

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lukaszgryglicki merged 7 commits into from
Jun 24, 2026
Merged

Handle errors in lookup script #5096

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
d8a4645
Handle errors in lookup script
lukaszgryglicki Jun 24, 2026
d2b0cc8
Fix reporting errors
lukaszgryglicki Jun 24, 2026
e148d0b
Update lookup script
lukaszgryglicki Jun 24, 2026
568acbe
Address AI feedback
lukaszgryglicki Jun 24, 2026
02f196a
Address AI feedback
lukaszgryglicki Jun 24, 2026
d79ad39
Address AI feedback
lukaszgryglicki Jun 24, 2026
ac25cca
Address AI feedback
lukaszgryglicki Jun 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions utils/lookup_all_logs.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@ if [ -z "$STAGE" ]
then
export STAGE=dev
fi

# Fail fast: if any lookup helper reports a failure (aws/jq error, exit 3/4), abort here
# instead of running on and ending with a successful `ls` that would hide the failure.
set -e
Comment thread
lukaszgryglicki marked this conversation as resolved.
Outdated
REGION=us-east-1 DEBUG=1 DTFROM="${1}" DTTO="${2}" ./utils/search_aws_log_group.sh 'githubactivity' "${3}" > githubactivity.log
REGION=us-east-1 DEBUG=1 DTFROM="${1}" DTTO="${2}" ./utils/search_aws_log_group.sh 'apiv1' "${3}" > v1.log
REGION=us-east-1 DEBUG=1 DTFROM="${1}" DTTO="${2}" ./utils/search_aws_log_group.sh 'apiv2' "${3}" > v2.log
Expand Down
27 changes: 24 additions & 3 deletions utils/search_aws_log_group.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,18 +77,39 @@ DTF=$(date -u -d @$(echo "${DTFROM}/1000" | bc) "+%F %T.%6N")
DTT=$(date -u -d @$(echo "${DTTO}/1000" | bc) "+%F %T.%6N")
echo "Date range: ${DTF} .. ${DTT} (from ${DTFROM} to ${DTTO})"

# Capture aws output to a temp file first (no pipe), so an aws failure and a jq failure
# are reported accurately and independently. In a pipe, a jq failure can SIGPIPE aws and
# surface as 141, misclassifying it as an aws failure.
raw_log="$(mktemp)" || { echo "ERROR: mktemp failed — cannot capture aws output" >&2; exit 2; }
Comment thread
lukaszgryglicki marked this conversation as resolved.
Outdated
trap 'rm -f "${raw_log}"' EXIT

if [ -z "${search}" ]
then
if [ ! -z "${DEBUG}" ]
then
echo "aws --region \"${REGION}\" --profile \"lfproduct-${STAGE}\" logs filter-log-events --log-group-name \"/aws/lambda/${log_group}\" --start-time \"${DTFROM}\" --end-time \"${DTTO}\""
fi
aws --region "${REGION}" --profile "lfproduct-${STAGE}" logs filter-log-events --log-group-name "/aws/lambda/${log_group}" --start-time "${DTFROM}" --end-time "${DTTO}" | jq -r '.events | sort_by(.timestamp)'
aws --region "${REGION}" --profile "lfproduct-${STAGE}" logs filter-log-events --log-group-name "/aws/lambda/${log_group}" --start-time "${DTFROM}" --end-time "${DTTO}" > "${raw_log}"
Comment thread
lukaszgryglicki marked this conversation as resolved.
Outdated
else
if [ ! -z "${DEBUG}" ]
then
echo "aws --region \"${REGION}\" --profile \"lfproduct-${STAGE}\" logs filter-log-events --log-group-name \"/aws/lambda/${log_group}\" --start-time \"${DTFROM}\" --end-time \"${DTTO}\" --filter-pattern \"${search}\""
echo "aws --region \"${REGION}\" --profile \"lfproduct-${STAGE}\" logs filter-log-events --log-group-name \"/aws/lambda/${log_group}\" --start-time \"${DTFROM}\" --end-time \"${DTTO}\" --filter-pattern '\"${search}\"'"
Comment thread
lukaszgryglicki marked this conversation as resolved.
Outdated
fi
aws --region "${REGION}" --profile "lfproduct-${STAGE}" logs filter-log-events --log-group-name "/aws/lambda/${log_group}" --start-time "${DTFROM}" --end-time "${DTTO}" --filter-pattern "\"${search}\"" | jq -r '.events | sort_by(.timestamp)'
aws --region "${REGION}" --profile "lfproduct-${STAGE}" logs filter-log-events --log-group-name "/aws/lambda/${log_group}" --start-time "${DTFROM}" --end-time "${DTTO}" --filter-pattern "\"${search}\"" > "${raw_log}"
Comment thread
lukaszgryglicki marked this conversation as resolved.
Outdated
fi
aws_rc=$?

# An aws failure (expired SSO, no access, crashed CLI) is NOT "no events": report it and
# exit non-zero instead of leaving an empty/[] result that looks like "no hits". aws's own
# error is shown above on stderr.
if [ "${aws_rc}" -ne 0 ]
then
echo "ERROR: aws failed (rc=${aws_rc}) — output above is NOT 'no events'; logs were not retrieved. Try: aws sso login --profile \"lfproduct-${STAGE}\"" >&2
exit 3
fi
if ! jq -r '.events | sort_by(.timestamp)' < "${raw_log}"
then
echo "ERROR: jq failed — logs were retrieved but could not be parsed (is jq installed?)." >&2
exit 4
fi

Loading