Review BenchmarksMethods, controls & recorded results

Keep redirect authentication checks intact through an agent edit

Removing Requests’ early hostname check lets two cross-host redirect cases retain credentials in this local control. Eight URL-only tests expose the change. A broad Perch behavior rule missed it; a revised rule naming the early guard flagged it and accepted both clean controls.

Source controlRuntime cases passingFirst behavior ruleRevised guard rule
Upstream method8/8No rule findingNo rule finding
Injected hostname-guard removal6/8No rule findingRule finding
Equivalent comparison refactor8/8No rule findingNo rule finding

Captured October 19, 2026 with jev-latest. These are three versions of one extracted method, with one fresh check per version for each rule. The unchanged upstream method is a passing control. The removed guard is an illustrative local mutation, not a reported Requests vulnerability.

Inspect the pinned method and exact changes

Requests source at commit 611c6162cbc4 supplies SessionRedirectMixin.should_strip_auth. The executable harness extracts no additional Requests behavior: it supplies Python’s urlparse and the HTTP/HTTPS default ports, then calls that method with URL strings. No network requests or credentials are used.

Exact upstream method · Injected regression · Equivalent refactor · Source identity and SHA-256 hashes · Apache-2.0 license · Requests notices

Reproduce the eight URL controls

Download the controls and run the harness with Python 3.10 or newer:

mkdir requests-auth-check && cd requests-auth-check
for file in run.py source.json upstream.py regression.py refactor.py perch.yaml explicit-guard.yaml LICENSE NOTICE; do
  curl -fLO "https://reviewbenchmarks.dev/evaluations/requests-redirect-auth/$file"
done
python3 run.py

The cases cover two cross-host redirects, same-host path changes, a standard HTTP-to-HTTPS upgrade, a downgrade, default-port equivalence, a changed port and a custom-port upgrade. Recorded inputs, expected outcomes and runtime results retains each outcome.

Keep the first miss when revising the rule

The first rule describes the required behavior:

- name: redirect-host-auth-boundary
  where: "*.py"
  ensure: >-
    Read should_strip_auth only. Whenever the redirect hostname differs from
    the original hostname it returns true to strip Authorization, including
    an HTTP-to-HTTPS redirect to a different host. A same-host HTTP-to-HTTPS
    upgrade on standard ports may keep Authorization.

It returned no finding for all three versions. The next rule states the implementation requirement more explicitly: reject a hostname mismatch before the compatibility branches can preserve Authorization.

- name: redirect-host-auth-boundary
  where: "*.py"
  ensure: >-
    Before either HTTP-to-HTTPS compatibility logic or default-port logic can
    return false, should_strip_auth compares old_parsed.hostname with
    new_parsed.hostname and returns true when those hostnames differ. This
    early hostname rejection must remain present after a refactor.

The revised rule flagged the removed guard and accepted the upstream method and equivalent refactor. Both runs are retained: First rule results · Revised rule results. The revision was made after observing the first miss, so this is a rule-authoring demonstration, not an independent accuracy estimate.

Check an agent edit and repair the finding

Keep your existing perch.yaml. Add the revised rule to a separate file under .perch/rules/ and adapt its selector to your Requests checkout. Configure the agent you use with perch setup claude-code, perch setup codex, perch setup cursor or perch setup pi.

perch check src/requests/sessions.py::SessionRedirectMixin.should_strip_auth --rules redirect-host-auth-boundary --json

Run the saved check after the edit. Inspect any finding, repair the hostname guard, and run the identical check again. Run the URL regression tests separately. A passing model check does not establish that the complete client is secure.

Read the time, cost and verification limits

The six fresh checks took 0.358 s, 0.329 s, 0.349 s, 0.536 s, 0.376 s, 0.371 s in recorded command wall time. At 10¢ per 1,000 fresh rule checks, six checks have a rate-derived estimate of 0.06¢; actual billed cost was unavailable. Versions, commands, timestamps and raw outputs are in the result files. Installation and authentication time are excluded.

Only one method and one deliberately introduced regression were examined. The revised rule had seen the first failure during authoring. Fresh model judgments can vary; unchanged requests reuse cached answers. CodeRabbit and Greptile were not run on these inputs, so this page establishes no comparative quality ranking.