Rework Code Base for Single Instance by JVickery-TBS · Pull Request #1583 · open-data/ckanext-canada

JVickery-TBS · 2025-06-18T20:14:52Z

No description provided.

- Tearing out all unnecessary code.

# Conflicts: # ckanext/canada/tests/test_trim_package.py ### RESOLVED.

codecov · 2025-10-07T16:53:50Z

❌ 1 Tests Failed:

Tests completed	Failed	Passed	Skipped
317	1	316	0

View the full list of 1 ❄️ flaky test(s)

ckanext/canada/tests/test_webforms.py::TestPackageWebForms::test_new_dataset_required_fields

Flake rate in main: 100.00% (Passed 0 times, Failed 1 times)

Stack Traces | 11.7s run time

self = <ckanext.canada.tests.test_webforms.TestPackageWebForms object at 0x7f7a34168400>
app = <ckan.tests.helpers.CKANTestApp object at 0x7f7a3381fe50>

    @mock.patch.object(h, 'flash_success', flashes.mock_flash)
    @mock.patch.object(h, 'get_flashed_messages', flashes.mock_get_flashed_messages)
    @mock.patch.object(h, 'is_registry_domain', mock_is_registry_domain)
    # @pytest.mark.skip(reason='TODO: finalize shceming pages form layouts')
    def test_new_dataset_required_fields(self, app):
        dataset_id = 'f3e4adb9-6e32-4cb4-bf68-1eab9d1288f5'
    
        offset = h.url_for('dataset.new')
        response = app.get(offset, extra_environ=self.extra_environ_tester,
                           environ_overrides=self.environ_overrides_tester,
                           follow_redirects=False)  # no need for redirects
    
        assert 'Create Dataset' in response.body
        assert 'Before you can create a dataset you need to create an organization' not in response.body
    
        response = app.post(offset,
                            data=self._filled_dataset_form(dataset_id),
                            extra_environ=self.extra_environ_tester,
                            environ_overrides=self.environ_overrides_tester,
                            follow_redirects=False)  # no need for redirects
    
        offset, _host = get_relative_offset_from_response(response)
        response = app.get(offset, extra_environ=self.extra_environ_tester,
                           environ_overrides=self.environ_overrides_tester,
                           follow_redirects=False)  # no need for redirects
    
        assert 'Add data to the dataset' in response.body
    
        response = app.post(offset,
                            data=self._filled_resource_form(dataset_id),
                            extra_environ=self.extra_environ_tester,
                            environ_overrides=self.environ_overrides_tester,
                            follow_redirects=False)  # no need for redirects
    
        offset, _host = get_relative_offset_from_response(response)
        response = app.get(offset, extra_environ=self.extra_environ_tester,
                           environ_overrides=self.environ_overrides_tester,
                           follow_redirects=False)  # no need for redirects
    
        assert 'Resource added' in response.body
    
        # test editor user
        dataset_id = make_uuid()
    
        offset = h.url_for('dataset.new')
        response = app.get(offset, extra_environ=self.extra_environ_editor,
                           environ_overrides=self.environ_overrides_editor)
    
>       assert 'Create Dataset' in response.body
E       assert 'Create Dataset' in '<!DOCTYPE html><!--[if lt IE 9]>\n  <html class="no-js lt-ie9" lang="en" dir="ltr"><![endif]--> <!--[if gt IE 8]><!-->\n  <html class="no-js" lang="en" dir="ltr"> <!--<![endif]-->\n  <head>\n    \n  <meta charset="utf-8" />\n      <meta name="csrf_field_name" content="_csrf_token" />\n      <meta name="_csrf_token" content="IjcyMWM2M2U4MTFjMmJiYjQyMzBmNzRkZjU3ZDhhNGJkZDFjZjgxYjAi.ano9Dg.uVczY4ifror1p8lJmXrwYl7zppo" />\n\n      <meta name="generator" content="ckan 2.10.8" />\n      <meta name="viewport" content="width=device-width, initial-scale=1.0">\n  \n    <meta property="og:title" content="Error 500 - CKAN">\n  \n  \n    \n  \n\n    <title>Error 500 - CKAN</title>\n\n    \n    \n  <link rel="shortcut icon" href="https://www.canada..../wet-boew/assets/favicon.ico" />\n    \n  \n    \n    \n    \n  \n  \n  \n\n  \n  \n      \n      \n      \n    \n  \n\n  \n\n\n    \n      \n      <link rel="stylesheet" href="TEST_TEMPLATE_HEAD_END.css" type="text/css">\n    \n\n    \n    \n  <script src="https://www.canada..../cdts/compiled/soyutils.js" integrity="sha384-hfwnpowMIP7hDqCMoNULlqSq7k2nu8R7zl+zHfYpNc5iokyd+Gb...s.forEach(function(_link){\n    _link.setAttribute(\'nonce\', \'VvuFHC58rNEYDaWkjSToSZ\');\n    newWetFooterString += _link.outerHTML;\n  });\n  document.write(newWetFooterString);\n  \n</script>\n  \n  \n    \n      \n    \n  \n  \n  \n  \n    \n  \n  \n    \n\n  \n\n  \n    \n        <script nonce="VvuFHC58rNEYDaWkjSToSZ">\n          setTimeout(function(){ $(\'#timeout\').modal(\'show\'); }, -1 * 1000);\n        </script>\n      \n      <script nonce="VvuFHC58rNEYDaWkjSToSZ">\n        window.addEventListener(\'load\', function(){\n          $(document).ready(function(){\n            if (navigator.appName == \'Microsoft Internet Explorer\' ||\n            !!(navigator.userAgent.match(/Trident/) || navigator.userAgent.match(/rv:11/)) ||\n            (typeof $.browser !== "undefined" && $.browser.msie == 1))\n            {\n              let ieWarning = document.getElementById("ie_warning");\n              if( ieWarning.length > 0 ){\n                $(ieWarning).show();\n              }\n            }\n          });\n        });\n      </script>\n    \n  \n\n      <div role="region"><strong>TEST TEMPLATE_FOOTER_END TEST</strong></div>\n\n    \n    \n    \n    \n  </body>\n</html>'
E        +  where '<!DOCTYPE html><!--[if lt IE 9]>\n  <html class="no-js lt-ie9" lang="en" dir="ltr"><![endif]--> <!--[if gt IE 8]><!-->\n  <html class="no-js" lang="en" dir="ltr"> <!--<![endif]-->\n  <head>\n    \n  <meta charset="utf-8" />\n      <meta name="csrf_field_name" content="_csrf_token" />\n      <meta name="_csrf_token" content="IjcyMWM2M2U4MTFjMmJiYjQyMzBmNzRkZjU3ZDhhNGJkZDFjZjgxYjAi.ano9Dg.uVczY4ifror1p8lJmXrwYl7zppo" />\n\n      <meta name="generator" content="ckan 2.10.8" />\n      <meta name="viewport" content="width=device-width, initial-scale=1.0">\n  \n    <meta property="og:title" content="Error 500 - CKAN">\n  \n  \n    \n  \n\n    <title>Error 500 - CKAN</title>\n\n    \n    \n  <link rel="shortcut icon" href="https://www.canada..../wet-boew/assets/favicon.ico" />\n    \n  \n    \n    \n    \n  \n  \n  \n\n  \n  \n      \n      \n      \n    \n  \n\n  \n\n\n    \n      \n      <link rel="stylesheet" href="TEST_TEMPLATE_HEAD_END.css" type="text/css">\n    \n\n    \n    \n  <script src="https://www.canada..../cdts/compiled/soyutils.js" integrity="sha384-hfwnpowMIP7hDqCMoNULlqSq7k2nu8R7zl+zHfYpNc5iokyd+Gb...s.forEach(function(_link){\n    _link.setAttribute(\'nonce\', \'VvuFHC58rNEYDaWkjSToSZ\');\n    newWetFooterString += _link.outerHTML;\n  });\n  document.write(newWetFooterString);\n  \n</script>\n  \n  \n    \n      \n    \n  \n  \n  \n  \n    \n  \n  \n    \n\n  \n\n  \n    \n        <script nonce="VvuFHC58rNEYDaWkjSToSZ">\n          setTimeout(function(){ $(\'#timeout\').modal(\'show\'); }, -1 * 1000);\n        </script>\n      \n      <script nonce="VvuFHC58rNEYDaWkjSToSZ">\n        window.addEventListener(\'load\', function(){\n          $(document).ready(function(){\n            if (navigator.appName == \'Microsoft Internet Explorer\' ||\n            !!(navigator.userAgent.match(/Trident/) || navigator.userAgent.match(/rv:11/)) ||\n            (typeof $.browser !== "undefined" && $.browser.msie == 1))\n            {\n              let ieWarning = document.getElementById("ie_warning");\n              if( ieWarning.length > 0 ){\n                $(ieWarning).show();\n              }\n            }\n          });\n        });\n      </script>\n    \n  \n\n      <div role="region"><strong>TEST TEMPLATE_FOOTER_END TEST</strong></div>\n\n    \n    \n    \n    \n  </body>\n</html>' = <WrapperTestResponse 19098 bytes [500 INTERNAL SERVER ERROR]>.body

.../canada/tests/test_webforms.py:115: AssertionError

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

- Syntax fixes. - Typing fixes. - Started working on the dataset visibility validator. - Added more validation to portal release date field. - Changed the published search facet to capacity. - Set capacity=private for non standard dataset types.

- After validator for dataset visibility. - Added more sysadmin UI for user publishing perms.

- Side effect free.

- Removed geozoom.

- Fix different logic for single instance.

- Force True for non portal package types. - Override activity stuff for logged out users to show a dummy user. - Minor template UI fixes. - Added UI for private/public packages. - Removed old logic methods. - Removed anon actions. - Use fq_list instead of complicated fq for package search. - Handled remaining search issues between registry and portal. - Added helper cli subcommand to set non portal package types to private. - Added more bootstrap4+ classes.

- Improved some code.

- Language switcher for language domains. - Minor remplate fixes. - Pinned required dependency versions.

- Try/catch for dataset search.

- Flake8 fix.

wardi

looks good, just a few minor suggestions

- Added change log files.

- Pyright fixes. - Flake8 fixes.

# Conflicts: # ckanext/canada/assets/internal/canada_internal.css # ckanext/canada/assets/public/canada_public.css # ckanext/canada/templates/admin/portal_sync.html ### RESOLVED.

- Various changes from feedback. - Change contextual datastore flags to just be one text[] db field.

- Improved suffix choices. - Minor logic changes for service inventory suffixes. - Fix newlines not displaying in pd datatables.

- Pytests don't like f string.

- Flake8 fixes. - Test fix for db func change. - Pyright fixes.

# Conflicts: # ckanext/canada/assets/public/canada_public.css # ckanext/canada/plugin/dataset_plugin.py # ckanext/canada/plugin/form_plugin.py # ckanext/canada/schemas/doc.yaml # ckanext/canada/validators.py ### RESOLVED.

# Conflicts: # .github/workflows/pytest.yml # ckanext/canada/assets/internal/canada_internal.css # ckanext/canada/assets/public/canada_public.css # ckanext/canada/plugin/dataset_plugin.py # ckanext/canada/plugin/internal_plugin.py # ckanext/canada/plugin/public_plugin.py ### RESOLVED.

# Conflicts: # ckanext/canada/assets/public/canada_public.css # ckanext/canada/plugin/public_plugin.py ### RESOLVED.

- Add correct domain to recombinant redirect tests.

# Conflicts: # ckanext/canada/tests/test_webforms.py ### RESOLVED.

# Conflicts: # .github/workflows/pytest.yml # ckanext/canada/assets/public/canada_public.css ### RESOLVED.

# Conflicts: # ckanext/canada/assets/public/canada_public.css ### RESOLVED.

- Bypass validation during migrations.

# Conflicts: # ckanext/canada/assets/datatables/pd_datatable.css # ckanext/canada/assets/public/canada_public.css # ckanext/canada/helpers.py # ckanext/canada/logic.py # ckanext/canada/migration/canada_logic/versions/004_28a9be077875_create_ref_data_hash_tables.py # ckanext/canada/model.py # ckanext/canada/pd.py # ckanext/canada/plugin/internal_plugin.py ### RESOLVED.

- Flake8 fixes.

- Added script to warn of leaving canada.ca domainspace.

- Added change log file.

# Conflicts: # ckanext/canada/plugin/form_plugin.py # ckanext/canada/validators.py ### RESOLVED.

JVickery-TBS added 2 commits June 18, 2025 18:33

removal(dev): tear it out;

9dc480b

- Tearing out all unnecessary code.

removal(dev): tear it out;

41c979d

- Tearing out all unnecessary code.

JVickery-TBS requested a review from wardi June 18, 2025 20:14

JVickery-TBS assigned wardi and JVickery-TBS Jun 18, 2025

JVickery-TBS added enhancement Business Requirement labels Jun 18, 2025

JVickery-TBS added 3 commits October 7, 2025 16:27

Merge branch 'refactor/is-registry' into feature/single-instance

380699a

# Conflicts: # ckanext/canada/tests/test_trim_package.py ### RESOLVED.

Merge branch 'refactor/is-registry' into feature/single-instance

3c052ea

# Conflicts: # ckanext/canada/tests/test_trim_package.py ### RESOLVED.

Merge branch 'refactor/is-registry' into feature/single-instance

db9fabb

JVickery-TBS added 17 commits October 7, 2025 20:29

feat(dev): validators, facets, syntax;

cdc4f75

- Syntax fixes. - Typing fixes. - Started working on the dataset visibility validator. - Added more validation to portal release date field. - Changed the published search facet to capacity. - Set capacity=private for non standard dataset types.

feat(dev): validators;

6bf2a1b

- After validator for dataset visibility. - Added more sysadmin UI for user publishing perms.

Merge branch 'master' into feature/single-instance

978bb86

fix(logic): user show;

b996589

- Side effect free.

removal(js): webassets;

3c210e9

- Removed geozoom.

fix(dev): misc logic;

1e815eb

- Fix different logic for single instance.

fix(dev): misc logic;

2d3fd89

- Fix different logic for single instance.

feat(dev): some TODOs;

0cc412e

- Improved some code.

Merge branch 'master' into feature/single-instance

5243dd8

Merge branch 'master' into feature/single-instance

9502402

Merge branch 'master' into feature/single-instance

11187cd

Merge branch 'master' into feature/single-instance

8334f52

Merge branch 'master' into feature/single-instance

91d6ef4

fix(dev): language switcher;

4033abe

- Language switcher for language domains. - Minor remplate fixes. - Pinned required dependency versions.

fix(logic): dataset search;

44e80c4

- Try/catch for dataset search.

fix(syntax): lint;

b4dd6d1

- Flake8 fix.

JVickery-TBS marked this pull request as ready for review October 17, 2025 14:54

wardi reviewed Oct 17, 2025

View reviewed changes

JVickery-TBS added 30 commits July 30, 2026 13:24

feat(misc): changelog;

e4cf9f1

- Added change log files.

fix(lint): syntax;

cb720c5

- Pyright fixes. - Flake8 fixes.

fix(lint): syntax;

62e3cd3

- Pyright fixes. - Flake8 fixes.

Merge branch 'master' into feature/single-instance

9f48e74

# Conflicts: # ckanext/canada/assets/internal/canada_internal.css # ckanext/canada/assets/public/canada_public.css # ckanext/canada/templates/admin/portal_sync.html ### RESOLVED.

refactor(dev): feedback;

611d7cd

- Various changes from feedback. - Change contextual datastore flags to just be one text[] db field.

feat(pd): suffix choices;

bdc4ef4

- Improved suffix choices. - Minor logic changes for service inventory suffixes. - Fix newlines not displaying in pd datatables.

fix(tests): service filter;

19757bd

- Pytests don't like f string.

fix(syntax): lint;

31c160d

- Flake8 fixes. - Test fix for db func change. - Pyright fixes.

Merge branch 'master' into feature/single-instance

f0977ba

# Conflicts: # ckanext/canada/assets/public/canada_public.css # ckanext/canada/plugin/dataset_plugin.py # ckanext/canada/plugin/form_plugin.py # ckanext/canada/schemas/doc.yaml # ckanext/canada/validators.py ### RESOLVED.

Merge branch 'master' into feature/single-instance

f960c4b

Merge branch 'master' into feature/single-instance

5c66c0b

# Conflicts: # ckanext/canada/assets/public/canada_public.css # ckanext/canada/plugin/public_plugin.py ### RESOLVED.

fix(tests): domain map;

f20ae09

- Add correct domain to recombinant redirect tests.

Merge branch 'master' into feature/single-instance

a7d4e53

Merge branch 'master' into feature/single-instance

9b62682

dev

32988f1

Merge branch 'master' into feature/single-instance

1c087ac

# Conflicts: # ckanext/canada/tests/test_webforms.py ### RESOLVED.

Merge branch 'master' into feature/single-instance

971b7bf

Merge branch 'master' into feature/ds-reference-tables

5ba54f3

# Conflicts: # .github/workflows/pytest.yml # ckanext/canada/assets/public/canada_public.css ### RESOLVED.

Merge branch 'master' into feature/ds-reference-tables

4dd7c21

Merge branch 'master' into feature/ds-reference-tables

aa7cbb3

Merge branch 'master' into feature/single-instance

6ee7ee1

# Conflicts: # ckanext/canada/assets/public/canada_public.css ### RESOLVED.

feat(schema): logic;

9b6def8

- Bypass validation during migrations.

fix(misc): syntax;

248e75b

- Flake8 fixes.

feat(js): exit site warning;

e366ae7

- Added script to warn of leaving canada.ca domainspace.

feat(misc): changelog;

cec1dfc

- Added change log file.

Merge branch 'master' into feature/single-instance

029e347

Merge branch 'feature/leaving-site-modal' into feature/single-instance

9fc9a6a

Merge branch 'master' into feature/single-instance

0bf7463

# Conflicts: # ckanext/canada/plugin/form_plugin.py # ckanext/canada/validators.py ### RESOLVED.

Navigation Menu

Uh oh!

Rework Code Base for Single Instance - #1583

Rework Code Base for Single Instance#1583
JVickery-TBS wants to merge 208 commits into
masterfrom
feature/single-instance

JVickery-TBS commented Jun 18, 2025

Uh oh!

codecov Bot commented Oct 7, 2025 •

edited

Loading

Uh oh!

wardi left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

JVickery-TBS commented Jun 18, 2025

Uh oh!

codecov Bot commented Oct 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ 1 Tests Failed:

Uh oh!

wardi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codecov Bot commented Oct 7, 2025 •

edited

Loading