Add project page: Qualitative Data Categorisation Improvements#362
Open
github-actions[bot] wants to merge 2 commits into
Open
Add project page: Qualitative Data Categorisation Improvements#362github-actions[bot] wants to merge 2 commits into
github-actions[bot] wants to merge 2 commits into
Conversation
52 tasks
amaiaita
reviewed
Jul 1, 2026
Collaborator
There was a problem hiding this comment.
could you please add some alt text!
amaiaita
reviewed
Jul 1, 2026
amaiaita
left a comment
amaiaita
left a comment
Collaborator
There was a problem hiding this comment.
overall looks great, just a couple minor things!
|
|
||
| The legacy approach used a trained sklearn/BERT ensemble that required manual retraining and achieved ~0.70 weighted F1. | ||
|
|
||
| We replaced this with an LLM-based classifier using Claude Sonnet via the NHS Federated Data Platform (FDP) API. The system uses retrieval-augmented few-shot prompting — for each comment being classified, it retrieves semantically similar examples from a corpus of 13,000 labelled comments and includes them as in-context calibration. Comments are processed in batches of 50 per API call for efficiency. This approach requires no model training, and making category changes is as simple as updating a prompt. |
Collaborator
There was a problem hiding this comment.
we've been told to not mention FDP in comms stuff so maybe remove this
| title: 'Qualitative Data Categorisation Improvements' | ||
| summary: 'LLM-based multi-label text classifier for NHS patient experience comments, replacing a legacy ML model with a RAG-augmented approach requiring no model training' | ||
| origin: 'Insight and Voice' | ||
| tags: ['CLASSIFICATION', 'LARGE LANGUAGE MODELS (LLM)', 'MACHINE LEARNING', 'NATURAL LANGUAGE PROCESSING (NLP)', 'MODELLING', 'UNSTRUCTURED', 'TEXT DATA', 'PYTHON', 'IN DEVELOPMENT'] |
Collaborator
There was a problem hiding this comment.
This is entirely the fault of my template, will change it, but would you change the tag for LLM to just say LLM?
Collaborator
There was a problem hiding this comment.
and the NLP one to just say "Natural Language Processing" with no brackets
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Project Submission: Qualitative Data Categorisation Improvements
Summary
LLM-based multi-label text classifier for NHS patient experience comments, replacing a legacy ML model with a RAG-augmented approach requiring no model training
Details
docs/images/our_work/qualitative-data-categorisation-improvements/Changes
docs/our_work/qualitative-data-categorisation-improvements.mdmkdocs.yml(added to Current/Past Projects and Work Area sections)Next Steps
Closes #361