Skip to content

Commit b1bb38b

Browse files
ksfikasclaude
andcommitted
Source form: a new source is registered step by step; the button says which tab is next until every required field is filled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
1 parent 368fc76 commit b1bb38b

3 files changed

Lines changed: 84 additions & 2 deletions

File tree

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -189,7 +189,7 @@ provider's uploads until they switch), and for an HTTPS push source **Rotate API
189189
key** (`/provider/sources/{id}/rotate-key`: the old key stops working immediately;
190190
before this a lost key meant deleting and re-registering the source).
191191

192-
**Input and Output are steps** (Transform Box): a transform's list reads **IN** -> the processing steps (the first is *Read XML/JSON/CSV/Excel*) -> **OUT**. How callers ask an API source's published address (the GET->POST turn-around and its template) is set in the Load Box, on the source's row (Request -> Edit). A source can have several pipelines (each with its own steps, output and on or off switch; a new one is added from the Transform Box page, empty or as a copy of one of the source's pipelines): the Transform page has a switcher, and a pipeline's pages live under /transform/sources/<id>/pipelines/<name>/. The first is named default. Every enabled pipeline runs on an arriving file and stores its own output. Index tables belong to one source (Extract Box, the source's menu: Index tables): uploaded, replaced and deleted there, and only that source's pipelines can look things up in them; a table name is unique within its source, never shared. Deleting a source deletes everything that belongs to it: its pipelines, index tables, publish setting, stored runs, files and outputs. The source form (Extract Box, register and edit) is split into tabs -- General (name, title, e-mail), Connection (pull or push, the connector, endpoint, schedule and credentials), Request and Limits (every rate limit, timeout, retry and depth setting together) -- and only the tabs a source needs are shown: a push source and SFTP / FTP have General and Connection (a push source's how-they-send and allowed senders are its Connection), an HTTP API all four.
192+
**Input and Output are steps** (Transform Box): a transform's list reads **IN** -> the processing steps (the first is *Read XML/JSON/CSV/Excel*) -> **OUT**. How callers ask an API source's published address (the GET->POST turn-around and its template) is set in the Load Box, on the source's row (Request -> Edit). A source can have several pipelines (each with its own steps, output and on or off switch; a new one is added from the Transform Box page, empty or as a copy of one of the source's pipelines): the Transform page has a switcher, and a pipeline's pages live under /transform/sources/<id>/pipelines/<name>/. The first is named default. Every enabled pipeline runs on an arriving file and stores its own output. Index tables belong to one source (Extract Box, the source's menu: Index tables): uploaded, replaced and deleted there, and only that source's pipelines can look things up in them; a table name is unique within its source, never shared. Deleting a source deletes everything that belongs to it: its pipelines, index tables, publish setting, stored runs, files and outputs. The source form (Extract Box, register and edit) is split into tabs -- General (name, title, e-mail), Connection (pull or push, the connector, endpoint, schedule and credentials), Request and Limits (every rate limit, timeout, retry and depth setting together) -- and only the tabs a source needs are shown: a push source and SFTP / FTP have General and Connection (a push source's how-they-send and allowed senders are its Connection), an HTTP API all four. On a new source the button leads through the tabs ("Next: Connection", ...) and becomes Register source once every required field is filled.
193193
The two ends are fixed (not removable or movable, not offered under "Add a step") and each has its own page, opened by its Edit button:
194194
*Input* (`/transform/sources/{id}/steps/input`) says where the data comes from -- for a source that stores data, the newest file the steps are built
195195
on; for an API source, its sample answer (paste one, or fetch one with the source's test request); *Output* (`.../steps/output`) holds the format and

‎providerharvest_service/templates/provider/source_form.html‎

Lines changed: 68 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -424,7 +424,8 @@ <h1>{{ "Edit source" if editing else "Register a new source" }}</h1>
424424
</div>
425425

426426
<div class="form-actions">
427-
<button type="submit">{{ macros.icon(("check" if editing else "plus"), 16) }} {{ "Save changes" if editing else "Register source" }}</button>
427+
<button type="submit" id="submit-btn">{{ macros.icon(("check" if editing else "plus"), 16) }} <span id="submit-label">{{ "Save changes" if editing else "Register source" }}</span></button>
428+
<span id="submit-hint" class="muted" style="margin-left:0.8rem; font-size:0.85rem;"></span>
428429
</div>
429430
</form>
430431
</div>
@@ -493,6 +494,7 @@ <h1>{{ "Edit source" if editing else "Register a new source" }}</h1>
493494
if (available.indexOf(name) < 0) name = 'general';
494495
activeTab = name;
495496
document.querySelectorAll('[data-tab]').forEach(function (el) { el.classList.toggle('tab-off', el.dataset.tab !== name); });
497+
if (typeof updateWizard === 'function') setTimeout(updateWizard, 0);
496498
document.querySelectorAll('[data-tab-btn]').forEach(function (button) {
497499
var on = button.dataset.tabBtn === name;
498500
button.classList.toggle('active', on);
@@ -508,5 +510,70 @@ <h1>{{ "Edit source" if editing else "Register a new source" }}</h1>
508510
var first = document.querySelector('#source-form .error'), holder = first && first.closest('[data-tab]');
509511
setTab(holder ? holder.dataset.tab : 'general'); // opens on the tab of the first problem, if there is one
510512
})();
513+
514+
// A new source is registered step by step: the button says which tab comes next until every required field has been filled, and
515+
// only then becomes "Register source". (The server still checks everything; this only guides.) Editing keeps a plain Save button.
516+
var WIZARD = {{ 'false' if editing else 'true' }};
517+
var TAB_LABELS = {general: 'General', connection: 'Connection', request: 'Request', limits: 'Limits'};
518+
var FIELD_NAMES = {
519+
name: 'name', title: 'title', endpoint_url: 'endpoint URL / host', api_key: 'API key', basic_username: 'username', basic_password: 'password',
520+
oauth2_client_id: 'client ID', oauth2_client_secret: 'client secret', oauth2_token_url: 'token URL', mtls_client_cert: 'client certificate',
521+
mtls_client_key: 'client key', sftp_username: 'username', sftp_password: 'password or private key', host_key_fingerprint: 'host-key fingerprint',
522+
ftp_username: 'username', ftp_password: 'password', ftp_plain_ftp_acknowledged: 'FTPS, or the plain-FTP acknowledgment'
523+
};
524+
function requiredFields() {
525+
var fields = [['name', 'general'], ['title', 'general']]; // [field id, tab]
526+
var kind = document.querySelector('input[name="kind"]:checked');
527+
if ((kind ? kind.value : 'pull') === 'push') return fields;
528+
fields.push(['endpoint_url', 'connection']);
529+
var transport = document.getElementById('transport-type-pull').value;
530+
if (transport === 'http') {
531+
var auth = document.querySelector('input[name="auth_type"]:checked');
532+
var needs = {api_key: ['api_key'], basic_auth: ['basic_username', 'basic_password'],
533+
oauth2_client_credentials: ['oauth2_client_id', 'oauth2_client_secret', 'oauth2_token_url'], mtls: ['mtls_client_cert', 'mtls_client_key']};
534+
(needs[auth ? auth.value : 'api_key'] || []).forEach(function (id) { fields.push([id, 'connection']); });
535+
} else if (transport === 'sftp' || transport === 'scp') {
536+
fields.push(['sftp_username', 'connection']);
537+
if (!document.getElementById('sftp_private_key').value.trim()) fields.push(['sftp_password', 'connection']);
538+
fields.push(['host_key_fingerprint', 'connection']);
539+
} else if (transport === 'ftp') {
540+
fields.push(['ftp_username', 'connection'], ['ftp_password', 'connection']);
541+
if (!document.getElementById('ftp_use_tls').checked) fields.push(['ftp_plain_ftp_acknowledged', 'connection']);
542+
}
543+
return fields;
544+
}
545+
function isFilled(id) {
546+
var el = document.getElementById(id);
547+
if (!el) return true;
548+
return el.type === 'checkbox' ? el.checked : el.value.trim() !== '';
549+
}
550+
function missingFields() { return requiredFields().filter(function (f) { return !isFilled(f[0]); }); }
551+
function updateWizard() {
552+
if (!WIZARD) return;
553+
var missing = missingFields(), button = document.getElementById('submit-btn');
554+
var here = missing.filter(function (f) { return f[1] === activeTab; });
555+
var elsewhere = missing.filter(function (f) { return f[1] !== activeTab; });
556+
var target = '';
557+
if (elsewhere.length) {
558+
var later = elsewhere.filter(function (f) { return TAB_ORDER.indexOf(f[1]) > TAB_ORDER.indexOf(activeTab); });
559+
target = (later.length ? later[0] : elsewhere[0])[1];
560+
}
561+
button.dataset.next = target;
562+
document.getElementById('submit-label').textContent = target ? 'Next: ' + TAB_LABELS[target] : 'Register source';
563+
button.disabled = here.length > 0;
564+
document.getElementById('submit-hint').textContent = here.length ? 'Still needed here: ' + here.map(function (f) { return FIELD_NAMES[f[0]] || f[0]; }).join(', ') : '';
565+
}
566+
document.getElementById('submit-btn').addEventListener('click', function (event) {
567+
if (!WIZARD || !this.dataset.next) return; // nothing missing elsewhere: this click registers the source
568+
event.preventDefault();
569+
var target = this.dataset.next;
570+
setTab(target);
571+
var first = missingFields().filter(function (f) { return f[1] === target; })[0];
572+
var field = first && document.getElementById(first[0]);
573+
if (field && field.focus) field.focus();
574+
});
575+
document.getElementById('source-form').addEventListener('input', updateWizard);
576+
document.getElementById('source-form').addEventListener('change', updateWizard);
577+
updateWizard();
511578
</script>
512579
{% endblock %}

‎tests/test_source_form_tabs.py‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,3 +71,18 @@ def test_delivery_connector_endpoint_and_schedule_come_in_that_order_on_the_conn
7171
page = client.get("/provider/sources/new").text
7272
positions = [page.index(marker) for marker in ('id="kind-pull"', 'class="connector-grid"', 'id="endpoint_url"', 'id="frequency"', "<label>Authentication</label>")]
7373
assert positions == sorted(positions)
74+
75+
76+
def test_a_new_source_is_registered_step_by_step_and_an_edit_is_just_saved(client, _isolated_db):
77+
new_page = client.get("/provider/sources/new").text
78+
assert "var WIZARD = true;" in new_page and 'id="submit-btn"' in new_page and 'id="submit-label">Register source<' in new_page
79+
assert "'Next: ' + TAB_LABELS[target]" in new_page and "Still needed here: " in new_page
80+
for needed in ("endpoint_url", "api_key", "basic_username", "oauth2_token_url", "mtls_client_key", "sftp_username", "host_key_fingerprint", "ftp_password", "ftp_plain_ftp_acknowledged"):
81+
assert "%s:" % needed in new_page.split("var FIELD_NAMES")[1].split("function requiredFields")[0] or needed == "endpoint_url" # the hint names what is missing
82+
client.post("/provider/sources/new", data={
83+
"name": "wiz", "title": "Wiz", "kind": "pull", "transport_type": "http", "endpoint_url": "https://registry.example.gr/api/", "frequency": "DAILY",
84+
"auth_type": "api_key", "api_key": "k",
85+
})
86+
source_id = re.search(r"/provider/sources/([^/]+)/edit", client.get("/provider/sources").text).group(1)
87+
edit_page = client.get("/provider/sources/%s/edit" % source_id).text
88+
assert "var WIZARD = false;" in edit_page and 'id="submit-label">Save changes<' in edit_page

0 commit comments

Comments
 (0)