Skip to content

Remove N+1 in sync #326

Open
Open

Description

@lmatayoshi

At the moment, N+1 queries are being triggered when fetching entries from the database. In remoteSync function, the mapping forces an individual query for each fetched record, which results in a lot of processing time. This situation becomes really annoying in environments such as Nigeria, which at the moment accounts for 70.000+ AntibioticConsumptionStats.

network_devtools

We can see in the picture taken from the Network tab in the dev tools that fetching data from the server only takes about 1 second, whereas resolving the queries takes 10 times more time: 10+ seconds.

Solution involves turning that mapping of individual queries into a single one that creates or updates all the entries at once. Double check that synchronization keeps working after the fix, because the process has its quirks and it easily gets broken

Activity

  1. added
    store (admin)Issues for maap-store admin
    collectorIssues for maap-collector
    and removed
    store (admin)Issues for maap-store admin
    on Nov 25, 2020
  2. changed the title [-]Reduce antibiotic consumption sync time[/-] [+]Remove N+1 in sync[/+] on Nov 30, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions