Database For UK Nurse Registration 'Completely Unacceptable' (theregister.com) 42
Lindsay Clark reports via The Register: The UK Information Commissioner's Office has received a complaint detailing the mismanagement of personal data at the Nursing and Midwifery Council (NMC), the regulator that oversees worker registration. Employment as a nurse or midwife depends on enrollment with the NMC in the UK. According to whistleblower evidence seen by The Register, the databases on which the personal information is held lack rudimentary technical standards and practices. The NMC said its data was secure with a high level of quality, allowing it to fulfill its regulatory role, although it was on "a journey of improvement." But without basic documentation, or the primary keys or foreign keys common in database management, the Microsoft SQL Server databases -- holding information about 800,000 registered professionals -- are difficult to query and manage, making assurances on governance nearly impossible, the whistleblower told us.
The databases have no version control systems. Important fields for identifying individuals were used inconsistently -- for example, containing junk data, test data, or null data. Although the tech team used workarounds to compensate for the lack of basic technical standards, they were ad hoc and known by only a handful of individuals, creating business continuity risks should they leave the organization, according to the whistleblower. Despite having been warned of the issues of basic technical practice internally, the NMC failed to acknowledge the problems. Only after exhausting other avenues did the whistleblower raise concern externally with the ICO and The Register. The NMC stores sensitive data on behalf of the professionals that it registers, including gender, sexual orientation, gender identity, ethnicity and nationality, disability details, marital status, as well as other personal information.
The whistleblower's complaint claims the NMC falls well short of [the standards required under current UK law for data protection and the EU's General Data Protection Regulation (GDPR)]. The statement alleges that the NMC's "data management and data retrieval practices were completely unacceptable." "There is not even much by way of internal structure of the databases for self-documentation, such as primary keys, foreign keys (with a few honorable exceptions), check constraints and table constraints. Even fields that should not be null are nullable. This is frankly astonishing and not the practice of a mature, professional organization," the statement says. For example, the databases contain a unique ten-digit number (or PRN) to identify individuals registered to the NMC. However, the fields for PRNs sometimes contain individuals' names, start with a letter or other invalid data, or are simply null. The whistleblower's complaint says that the PRN problem, and other database design deficiencies, meant that it was nearly impossible to produce "accurate, correct, business critical reports ... because frankly no one knows where the correct data is to be found." A spokesperson for the NMC said the register was "organized and documented" in the SQL Server database. "For clarity, the register of all our nurses, midwives and nursing practitioners is held within Dynamics 365 which is our system of record. This solution and the data held within it, is secure and well documented. It does not rely on any SQL database. The SQL database referenced by the whistleblower relates to our data warehouse which we are in the process of modernizing as previously shared."
The databases have no version control systems. Important fields for identifying individuals were used inconsistently -- for example, containing junk data, test data, or null data. Although the tech team used workarounds to compensate for the lack of basic technical standards, they were ad hoc and known by only a handful of individuals, creating business continuity risks should they leave the organization, according to the whistleblower. Despite having been warned of the issues of basic technical practice internally, the NMC failed to acknowledge the problems. Only after exhausting other avenues did the whistleblower raise concern externally with the ICO and The Register. The NMC stores sensitive data on behalf of the professionals that it registers, including gender, sexual orientation, gender identity, ethnicity and nationality, disability details, marital status, as well as other personal information.
The whistleblower's complaint claims the NMC falls well short of [the standards required under current UK law for data protection and the EU's General Data Protection Regulation (GDPR)]. The statement alleges that the NMC's "data management and data retrieval practices were completely unacceptable." "There is not even much by way of internal structure of the databases for self-documentation, such as primary keys, foreign keys (with a few honorable exceptions), check constraints and table constraints. Even fields that should not be null are nullable. This is frankly astonishing and not the practice of a mature, professional organization," the statement says. For example, the databases contain a unique ten-digit number (or PRN) to identify individuals registered to the NMC. However, the fields for PRNs sometimes contain individuals' names, start with a letter or other invalid data, or are simply null. The whistleblower's complaint says that the PRN problem, and other database design deficiencies, meant that it was nearly impossible to produce "accurate, correct, business critical reports ... because frankly no one knows where the correct data is to be found." A spokesperson for the NMC said the register was "organized and documented" in the SQL Server database. "For clarity, the register of all our nurses, midwives and nursing practitioners is held within Dynamics 365 which is our system of record. This solution and the data held within it, is secure and well documented. It does not rely on any SQL database. The SQL database referenced by the whistleblower relates to our data warehouse which we are in the process of modernizing as previously shared."
No medical bills, though. (Score:5, Insightful)
The NHS has its issues, sure. But at least nobody gets a bill, so it's still infinitely better than the US nightmare.
Re: (Score:3)
UK healthcare spending in 2021 was £4,188/person ($5,277), compared with $12,555/person in the US in 2022. So, reasonably good value for money. Being available to everybody for free (at the point of use) is a big plus. This does contribute to longer waiting times though, unless you pay cash to see a private doctor.
Re:No medical bills, though. (Score:5, Informative)
Wait times used to be very low, but after the 2008 financial crisis we got a government that decided to run the NHS into the ground and give some of its work to private contractors. It was a deliberate choice, not a necessity.
When properly funded, the NHS works well and offers short wait times. The only real issue is that people keep voting for it to get worse, because they are idiots.
Re: (Score:1)
Re: No medical bills, though. (Score:4, Informative)
A family of four. Let's turn that into the individual number. $16,000/4 = $4,000 per person. Huh, that's an interesting number, given that everyone in the UK is covered by the NHS for that.
US healthcare spending per person in 2022 was $13,493. Of that, Medicare and Medicaid (US federal government spending on healthcare that only covers part of the care for the elderly and poor) was 39%, $5262. That's right, US PUBLIC healthcare spending is higher than the UK, and a far smaller percentage of the population is covered.
Don't try to play the "but US healthcare is better" game either. US healthcare is MUCH more expensive, and not significantly better. In terms of longevity outcomes, it's significantly worse. Average life expectancy in the UK was 80.43 years in 2020, 77.41 in the US.
US healthcare wait times are less in some (but not all) cases. Getting in to see your primary care doctor might only take a week. But for some specialists it gets much worse. Where I am, a dermatologist is about 6 months. A rheumatologist, 3 months. A pulmonologist, 4 months. A cardiologist, 6 months. A neurologist, a year.
Emergency room wait times are frequently several hours if you don't go in via ambulance.
Re: (Score:2)
Long wait times are the norm even to see your primary care physician where I am. I live in a sparsely-populated region of the US and our wait times are outrageous. My friends in the UK are appalled at how long we have to wait for routine care.
And I'll never understand why vision and dental aren't considered "healthcare". The US system is a mess.
Re: (Score:2)
It tends to be worse in rural areas. The specialists don't exist at all, the primary care doctors are overloaded, and the hospitals (especially in states that didn't expand Medicaid) are closing.
I've got "good" insurance, but it's a limited network as of last year, so I had to find some new doctors. If I'm out of town, the only thing it covers is emergencies and telemedicine.
And... yeah, it's insane that your eyes and teeth aren't considered worthy of care without separate insurance that is never very goo
Re: (Score:3)
Ahh, right, the UK NHS out of pocket costs. I understand hospital parking in the UK can be quite expensive [sky.com]. And it looks like a few things [www.nhs.uk] aren't free any more - but they're still significantly cheaper than in the US. Routine eye care and dental care aren't covered for adults by US health insurance at all, you've got to buy separate insurance for those if you're over 18. Wigs aren't covered. And it looks like your prescriptions are £9.65 if you don't qualify for free prescriptions. Meanwhile, if
Re: (Score:2)
I've never seen prices that low.
Most dentists charge at least $100 for a cleaning, $300 for a filling, and $1500 for a crown.
I've never seen an optometrist for less than $100, and that was a special one of the chains was running.
And... affordable compared to FREE? Because that's what it costs in much of Europe.
There's a reason lots of Americans go to Mexico or Costa Rica for dental care.
Re: (Score:1)
Nothing is free. I think you mean $100 for a cleaning + exam + xray which is indeed the average but not the lowest if you shop around. The rest sounds about right for averages.
Which is still lower cost than NHS or most EU state coverage which is allowed once every other year, the cost of exam + cleaning varies from ~$65 out of pocket (band 1) to $150 out of pocket (band 2) + the taxes you already paid in and only if the government doctor prescribes the cleaning, regular exams and cleanings are only availabl
Re: (Score:1)
HMRC sends me a bill every year!
Re: (Score:2)
And the IRS sends me one - or rather, they make me write up my own bill, US tax returns are notoriously complicated and time consuming, the average American spends 13 hours filing federal tax returns every year.
And US public healthcare spending is significantly HIGHER than in the UK, but covers far fewer people.
Re: (Score:2)
The average American must be a total moron.
It only takes maybe an hour for me to file our taxes, and that includes gathering up all the forms that come in the mail and downloading them from wherever.
What the hell are people doing for 13+ hours?
Re: (Score:2)
You're probably not filing a schedule C.
It's not that complicated if you're just an employee. Of course, the Brits in that situation get a pre-filled form to look over, all they have to do is look over it and make corrections if there are any.
Re: (Score:2)
Even when I had a business filing as an S-Corp it didn't take me thirteen fucking hours.
Additionally, more than 50% of Americans pay NO federal income tax. They're sure as fuck not filing a schedule C.
Re: (Score:2)
Ahh, that Romney bullshit again.
100% of working Americans pay federal income tax. There's a 15.3% federal income tax that starts at the first earned dollar, they hide half of it from you by taxing it before you ever get the check. Then there's the 10% tax bracket that starts after the $13,850 standard deduction, most Americans make more than that.
The only people who pay no federal income tax are the wealthy, because they can hide their income, and when they are taxed on it, it's usually capital gains that
Re: (Score:2)
So, your basic measure of desirability is whether or not something appears to be "free" ?
Re: (Score:2)
So, your basic measure of desirability is whether or not something causes half a million bankruptcies a year [self.inc]?
Re: (Score:2)
I didn't tout bankruptcy. However, the poster I responded to did tout the appearance of being free.
Underscores the need for IT Transformation at NHS (Score:4, Interesting)
As with recent articles, they've been working on a federated data platform that would bring in all these disparate data systems under one house. They've attempted several times in the past two decades providing only half measures with many participants reverting to their own existing platforms, or seeking IT systems of their own. I hope this time they can succeed and show the world how healthcare could be managed. The pilots they've been running seem promising.
Budget (Score:1)
Not properly funding critical services will do that. https://www.independent.co.uk/... [independent.co.uk]
Re: (Score:3)
The ONS inflation-adjusted figures show government-financed healthcare expenditure increasing by 13.5% in 2020 and 9.6% in 2021:
https://www.ons.gov.uk/peoplep... [ons.gov.uk]
Obviously these increases were driven by the covid pandemic, consequently healthcare spending relative to GDP did not increase in 2022 and 2023, but it still remained higher than the pre-pandemic projected spending (£165.3B rather than £163.9B):
https://www.nuffieldtrust.org.... [nuffieldtrust.org.uk]
Also, your article was published before the Budget was announ
Re: (Score:2)
That budget announcement is 16 days and a few hours old, I don't know what improvements you expect in under three weeks.
But feel free to keep making misleading posts that agree with your narrative.
raburton's post below [slashdot.org] is far more relevant in this case.
Re: Budget (Score:5, Interesting)
Except this has nothing to do with the NHS budget. The NMC is "financed by the fees paid by nurses, midwives and nursing associates". https://www.nmc.org.uk/globala... [nmc.org.uk]
Re: (Score:1)
Yeah. It's a boring, tedious article. Ive seen a dozen databases used by different software vendors and they were all crummy in their own peculiar ways. Some were better than others, some were overengineered and needlessly complicated and difficult to extract reports from, some were denormalized with way too many columns and wasted tons of space to store gigabytes of almost nothing but zeroed (not null) values, some had no checks and some junk due to data entry errors, fields used for different purposes dep
Re: (Score:3)
This whistleblower sounded like a dweeb even before i read to the bottom of the article where there is this update:
For clarity, the register of all our nurses, midwives and nursing practitioners is held within Dynamics 365 which is our system of record," the spinner continued. "This solution and the data held within it, is secure and well documented. It does not rely on any SQL database. The SQL database referenced by the whistleblower relates to our data warehouse which we are in the process of modernizing as previously shared."
Of course the statement that their Dynamics 365 "does not rely on any SQL database" is highly unlikely, imo. Dynamics 365 is the branding of Microsoft's suite of CRM/ERP/accounting big enterprise software and while i'd never worked with it (or heard of it, tbh) of COURSE it's backed by SQL Server (what ELSE would it be backed by ? this is primarily what "SQL databases" have been used for a
Re: What else is new (Score:3, Informative)
Dynamics 365 relies on SQL Server (more accurately SQL Server Reporting Services) for data. All Dynamics is is a low code ERP/CRM platform. So yes, the data in there will be ineptly stored because it is basically your average Excel sheet manager being promoted to programmer building these things.
If you then try to bolt on real programmers to do real work, you indeed have data consistency errors. Which can be hard to work with or outright regulatory problem (eg having a unique id that is built using somethin
Re: (Score:2)
I have seen my own share of poorly structured and maintained data bases, but this specific data base is not like the ones I have dealt with. The one discussed here contains very personal and private information about a whole work force (with a massive head count), and entry of this data was not optional, but mandatory for everyone in this work force. This creates a lot higher responsibility for the operators of this DB, and therefore much harder requirements on maintainability, reliability and security. Thi
I'm not surprised (Score:2)
Gov't systems are often Fuckvilles. There are usually insufficient checks and balances to prevent short-term thinking and prevent office politics from gumming up decision-making. Fixing leaking systems has less "brag points" than some shiny executive toy, so the money goes to the toy.
Auditors who understand IT need to inspect stuff, not generic auditors.
Could be worse (Score:2)
At least there wasn't a massive irretrievable data loss https://www.youtube.com/watch?... [youtube.com]
3rd Normal Form (Score:2)
Sensitive data (Score:2)
The NMC stores sensitive data on behalf of the professionals that it registers, including gender, sexual orientation, gender identity, ethnicity and nationality, disability details, marital status, as well as other personal information.
I wonder how they explain the need for sexual orientation and gender identity. Ethnicity is also weird from some viewpoints
This is not a government issue. This is common. (Score:2)
If you are a consultant and see "enterprise" databases made by Fortune 500 companies, you'll see a lot of this "schema rot," due to patching, organizational disfunction, merging of data from old (and incompatible) legacy systems, lack of written documentation and version control, and chaos left behind by past hiring of low-cost contractors who neither understand database design nor the business domain the database should be modeling. You'll also often find that only a few people have been there long enough