Skip to content

feat(runner): align top-level input schema with inner (initPreset enum + drop preAuthenticate section caption) #15

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
DaveHanns merged 1 commit into from
Jun 17, 2026
Merged

feat(runner): align top-level input schema with inner (initPreset enum + drop preAuthenticate section caption) #15

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 11 additions & 8 deletions .actor/input_schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,19 +70,22 @@
"title": "Pre-authenticate Apify CLI",
"type": "boolean",
"description": "If true (default), the runner populates ~/.apify/auth.json with APIFY_TOKEN via 'apify login --token' BEFORE the agent starts. This mirrors what every real Apify developer has after running 'apify login' once on their machine — 'apify push' from the agent will work without the agent having to discover auth flow. Set to false for raw-unauthed measurement (the agent has to figure out auth on its own — surfaces F11 in the eval pack's FINDINGS.md).",
"default": true,
"sectionCaption": "Authentication"
"default": true
},
"initPreset": {
"title": "Tool Setup Preset",
"type": "string",
"description": "Configures what tools the agent has access to. Use different presets to compare MCP vs CLI performance on the same scenario.",
"enum": ["none", "mcp_native", "cli_native", "mcpc"],
"description": "Configures what tools the agent has access to. 'None' is a safe default that logs which CLI tools are available and wires up mcpConfigJson if provided. The *_native presets MAKE a surface available without preventing the others; the *_only presets ENFORCE exclusive use of a single surface (PATH-shimming + MCP-config gating + trajectory-based hard reject of cross-surface leaks). Compare _native variants to measure 'does the agent prefer X when X is available'; compare _only variants to measure 'is surface X sufficient on its own'.",
"enum": ["none", "mcp_native", "cli_native", "mcpc", "api_native", "mcp_only", "cli_only", "api_only"],
"enumTitles": [
"None — agent uses only built-in capabilities",
"MCP Native — agent gets MCP servers from config JSON",
"CLI Native — agent uses command-line tools (gh, apify-cli)",
"mcpc — MCP servers via mcpc CLI bridge"
"None — logs availability of apify/gh/curl/jq; wires up mcpConfigJson if provided",
"MCP Native — agent gets MCP servers from config JSON (CLI / REST still available)",
"CLI Native — agent uses command-line tools (MCP / REST still available)",
"mcpc — MCP servers via mcpc CLI bridge",
"API Native — agent uses raw HTTPS against api.apify.com (CLI / MCP still available)",
"MCP Only — exclusive MCP surface: CLI binaries shimmed off PATH, REST tooling blocked, in-agent WebFetch/WebSearch hard-rejected",
"CLI Only — exclusive CLI surface: curl/wget shimmed off PATH, MCP not loaded, in-agent WebFetch/WebSearch hard-rejected",
"API Only — exclusive REST surface: apify shimmed off PATH, MCP not loaded, any apify-cli or MCP tool use hard-rejected"
],
"default": "none"
},
Expand Down
Loading