It seems that a lot of users are, irrationally, opposed to the
use of optical media for long-term archival storage.
What, then, are some alternatives for the general user who does
not command the BIG $BUCKS necessary for enterprise grade solutions.
Tape? Not likely.
Note that magnetic (HDD) or IC NAND (SSD) devices are not to be
considered long-term storage. Consumer SSDs are especially not
long term.
Optical media is still by far the best in terms of cost/benefit
for the general user.
I have hundreds of optical disks that I have produced over the
past years, all with GNU/Linux, and they will last me until the
end of my time.
Nux Vomica <nv@linux.rocks> wrote:I don't trust in USB drives, since I had amounts of crashes. My DVD-RAMs
Optical disks are good if you want a handheld copy of something, but
external storage or large USB media is what I'd prefer.
It seems that a lot of users are, irrationally, opposed to the
use of optical media for long-term archival storage.
In comp.os.linux.misc Nux Vomica <nv@linux.rocks> wrote:
It seems that a lot of users are, irrationally, opposed to the
use of optical media for long-term archival storage.
Having personally experienced failures of both cd-r and dvd-r media
wherein the recorded media became unreadable in a very short timeframe
(only a few years) even with proper storage it is not at all irrational
to be skeptical of claims of significant lifetimes for optical media
(esp. the user recordable type, pressed disks are a different matter). >Existing user recordable optical systems have, so far, had a poor track >record, so any new system has a higher bar to get over before it is
trusted for any long-term archive use.
Am Fri, 27 Sep 2024 12:40:57 -0400 schrieb Joel:
Nux Vomica <nv@linux.rocks> wrote:I don't trust in USB drives, since I had amounts of crashes. My DVD-RAMs
Optical disks are good if you want a handheld copy of something, but
external storage or large USB media is what I'd prefer.
are still working.
On Fri, 27 Sep 2024 17:36:29 -0000 (UTC), Rich <rich@example.invalid>
wrote:
In comp.os.linux.misc Nux Vomica <nv@linux.rocks> wrote:
It seems that a lot of users are, irrationally, opposed to the
use of optical media for long-term archival storage.
Having personally experienced failures of both cd-r and dvd-r media
wherein the recorded media became unreadable in a very short timeframe
(only a few years) even with proper storage it is not at all irrational
to be skeptical of claims of significant lifetimes for optical media
(esp. the user recordable type, pressed disks are a different matter).
Existing user recordable optical systems have, so far, had a poor track
record, so any new system has a higher bar to get over before it is
trusted for any long-term archive use.
I back up to DVD. Some of my backups (CDs) are 30 years old,
and > 99% work perfectly. Since optical media is dirt cheap, I do
duplicate backups, and if one gives a read error, I include its better
half in my next backup.
I use USB pendrives for very short term storage, and rotate.
They tend to go bad.
As to storing sensitive data on someone else's computer, LOL,
why not give them all your passwords and company secrets too?
[]'s
Having personally experienced failures of both cd-r and dvd-r media
wherein the recorded media became unreadable in a very short timeframe
(only a few years) even with proper storage it is not at all irrational
to be skeptical of claims of significant lifetimes for optical media
(esp. the user recordable type, pressed disks are a different matter). Existing user recordable optical systems have, so far, had a poor track record, so any new system has a higher bar to get over before it is
trusted for any long-term archive use.
On 27-09-2024 20:39, Shadow wrote:
As to storing sensitive data on someone else's computer, LOL,
why not give them all your passwords and company secrets too?
I completely agree. In at least 25 years I never lost a byte. Mind
you, I make duplicates of really important data.
Am Fri, 27 Sep 2024 12:40:57 -0400 schrieb Joel:
Nux Vomica <nv@linux.rocks> wrote:I don't trust in USB drives, since I had amounts of crashes. My DVD-RAMs
Optical disks are good if you want a handheld copy of something, but >>external storage or large USB media is what I'd prefer.
are still working.
On Fri, 27 Sep 2024 17:36:29 -0000 (UTC), Rich wrote:
I always purchased Taiyo Yuden DVD's which have an excellent
reputation for longevity, but since DVDs have a small capacity
I now only use M-Disc bdr.
Of course, one must always be mindful of future technology.
The strategy is to always copy important data to improved
formats but, so far, with optical media, this is not necessary
yet.
Am Fri, 27 Sep 2024 12:40:57 -0400 schrieb Joel:
Nux Vomica <nv@linux.rocks> wrote:I don't trust in USB drives, since I had amounts of crashes. My DVD-RAMs
Optical disks are good if you want a handheld copy of something, but
external storage or large USB media is what I'd prefer.
are still working.
Regards
Ralf
If you just wanted to store a little data for distant future
generations, information etched into glass should last longer than
info on plastic M-Discs. You could use a program like Optar to
convert into a printable data representation and then use a glass
etching kit or laser to 'burn' that to a paper-sized pane of glass.
In comp.os.linux.misc Computer Nerd Kev <not@telling.you.invalid> wrote:Absolutely proven: berry juice on rock in deep rock tunnels. Lasts for
If you just wanted to store a little data for distant future
generations, information etched into glass should last longer than
info on plastic M-Discs. You could use a program like Optar to
convert into a printable data representation and then use a glass
etching kit or laser to 'burn' that to a paper-sized pane of glass.
Provided, of course, that no one along the eons drops the pane, or
drops something on the pane, shattering it into a million pieces in the process.
Do you do restore tests regularly? If not, it could be that you might not have as many backups as you might think. =/
Once you get up to a certain size, they tend to use SSD controllers
inside, then a USB converter connected to that. This can radically
improve the storage characteristic. The SSD controller has static
and dynamic wear leveling. Very few USB sticks have that in a USB
controller (but there are some).
Consumer SSDs are especially notWhy not?
long term.
Provided they are kept powered on
How many 'fixes' have been applied to hard drives?
Checksums. Bad block management, Defragmentaion.
All signs of cheap technical gimmicks intended to foist an inferior technology onto an unsuspecting public.
Once you get up to a certain size, they tend to use SSD controllers
inside, then a USB converter connected to that. This can radically
improve the storage characteristic. The SSD controller has static and
dynamic wear leveling. Very few USB sticks have that in a USB controller
(but there are some).
Rich wrote:
Computer Nerd Kev <not@telling.you.invalid> wrote:Absolutely proven: berry juice on rock in deep rock tunnels. Lasts
If you just wanted to store a little data for distant futureProvided, of course, that no one along the eons drops the pane, or
generations, information etched into glass should last longer than
info on plastic M-Discs. You could use a program like Optar to
convert into a printable data representation and then use a glass
etching kit or laser to 'burn' that to a paper-sized pane of glass.
drops something on the pane, shattering it into a million pieces in the
process.
for eons.
On Fri, 27 Sep 2024 21:32:07 -0400, Paul wrote:
Once you get up to a certain size, they tend to use SSD controllers
inside, then a USB converter connected to that. This can radically
improve the storage characteristic. The SSD controller has static and
dynamic wear leveling. Very few USB sticks have that in a USB controller
(but there are some).
The thing is, Linux has, included as standard, support for filesystems
(e.g. f2fs, ubifs) that are specifically designed for use on flash
storage, with features like wear-levelling directly built into their storage-management algorithms.
Consider that SSDs (including USB sticks) incorporate elaborate interface controllers to pretend to the OS that they are disks and can use conventional disk-centric filesystems: imagine the overhead that would simply disappear if you could go direct to the low-level storage and use
one of these purpose-built filesystems!
The faster that storage devices get, the more sensitive they
become to details. This is why I would keep you well away
from my PCIe Rev5 NVMe at 14000/12000 MB/sec. That still
needs an error corrector, and somehow keep up with the
need to correct every sector being read out. It's one
of the reasons those get so hot (and they put toy heatsinks
on top).
That's also how you can have devices like this. You would not
get these sorts of rates, without IOPs in the picture to help.
https://www.anandtech.com/show/21486/highpoint-updates-nvme-raid-cards-for-pcie-50-50-gbps-directattached-ssd-storage
On 29/09/2024 02:27, Paul wrote:
The faster that storage devices get, the more sensitive they
become to details. This is why I would keep you well away
from my PCIe Rev5 NVMe at 14000/12000 MB/sec. That still
needs an error corrector, and somehow keep up with the
need to correct every sector being read out. It's one
of the reasons those get so hot (and they put toy heatsinks
on top).
That's also how you can have devices like this. You would not
get these sorts of rates, without IOPs in the picture to help.
https://www.anandtech.com/show/21486/highpoint-updates-nvme-raid-cards-for-pcie-50-50-gbps-directattached-ssd-storage
So SSDs are safe for long term storage (say, a decade?), even if you don't access them, so long as you keep them powered on?
On Sun, 9/29/2024 2:28 AM, Lawrence D'Oliveiro wrote:
On Fri, 27 Sep 2024 21:32:07 -0400, Paul wrote:
Once you get up to a certain size, they tend to use SSD controllers
inside, then a USB converter connected to that. This can radically
improve the storage characteristic. The SSD controller has static and
dynamic wear leveling. Very few USB sticks have that in a USB
controller (but there are some).
The thing is, Linux has, included as standard, support for filesystems
(e.g. f2fs, ubifs) that are specifically designed for use on flash
storage, with features like wear-levelling directly built into their
storage-management algorithms.
Consider that SSDs (including USB sticks) incorporate elaborate
interface controllers to pretend to the OS that they are disks and can
use conventional disk-centric filesystems: imagine the overhead that
would simply disappear if you could go direct to the low-level storage
and use one of these purpose-built filesystems!
Nobody wants their CPU donating a couple cores, to make up for the ARM
cores the SSD has. One of my SSDs has a three-core ARM, two cores are
for error correction on read!
On Sun, 9/29/2024 4:23 PM, Lars Poulsen wrote:
On 29/09/2024 02:27, Paul wrote:
The faster that storage devices get, the more sensitive they
become to details. This is why I would keep you well away
from my PCIe Rev5 NVMe at 14000/12000 MB/sec. That still
needs an error corrector, and somehow keep up with the
need to correct every sector being read out. It's one
of the reasons those get so hot (and they put toy heatsinks
on top).
That's also how you can have devices like this. You would not
get these sorts of rates, without IOPs in the picture to help.
https://www.anandtech.com/show/21486/highpoint-updates-nvme-raid-cards-for-pcie-50-50-gbps-directattached-ssd-storage
So SSDs are safe for long term storage (say, a decade?), even if you don't access them, so long as you keep them powered on?
We don't know all the details of the firmware fixes, but
at least in one case where the TLC used to get "mushy", the
fix for that was selective rewriting of some kind, to "refresh"
the device. That might have been a Samsung.
They don't give us constant estimates of archival life, leaving
us to "guess" the number is ten years. After all, it takes
ten years to test :-)
NOR flash chips used to get Bit Rot, between 10 and 20 years,
but that's an example of a device with no error correction
at all. The error correction in an SSD, is "mondo-powerful",
but, it assumes random degradations, not correlated ones.
If all the floating gates head to zero volts, an ECC
can't save you then. It is the archival case, that (eventually)
has to fail.
It's like the Helium disk drives in a sense. We know Helium
will eventually all leak out of the drive. There is no spigot
on the side for refilling them. If I put a 22TB drive inside
a time capsule glass bottle, come back in 40 years, it's
a good assumption the drive will not start. Some of the drives
have a pressure sensor (it's been spotted in SMART but is
not documented). We know then, from "ground truth", a Helium drive
is not archival quality. All we can argue about, is what year
all the helium will be gone. The guarantee is for five years,
but this is not a measured quantity, and if there was any
significant field failure rate attributable to no gas left,
it hasn't made the news yet. But the details of the design,
tell you the gas cannot last forever (it is retained by a
"thick adhesive", not by a gas-tight tin -- clever
people did this). The drives have two lids, the inner lid
secured with adhesive (gas "tight"), the outer welded lid
mechanically protects the inner lid from "finger pokes".
The welded lid is not gas tight. The welds do not really
need to be all that fancy.
The flash is the same way then. We know the floating gates,
even though disallowed, the electrons will eventually leave,
and we will be left with a "deflated feeling". If you did
happen to power up the device once a year, and (somehow)
the device notices a high error corrector rate, it might
choose to rewrite the sectors behind your back. I'd leave
it powered over night, while it catches up on house cleaning.
That's for TLC or QLC. The SLC and MLC drives, might not
even have that chunk of code, for their maintenance. If they
had the code, and the TLC or QLC ones had inherited the code,
we would not have noticed a thing. The fact someone had to
add code, tells you the SLC and MLC rely on the quality of the
floating gates, to make it to ten years. Based on the NOR flash
getting the odd bit corrupted at, say, 15 years, gives you
some idea about how well the SLC device may hold up. At fifteen
years, it can use its error corrector and hide those not
very dense failures. Since TLC and QLC are constantly
using their error correctors, the behavior is not the same.
Would accelerated life testing be valid for TLC or QLC archival
parameters ? Dunno. All we know is, the physics are the same
for the floating gate, but the thresholds are a lot tighter
on the SSDs you and I own, and there HAS to be a consequence
to this. The archival just cannot be as good... unless you
power them occasionally and let them sweep the dust under
the rug. The ECC can count the number of bits in error in
the sector, and based on that, it knows how close to
"uncorrectable" it is getting -- if the power is on.
Leave it in the back yard for 40 years, the cells will be
flat, and rewrites, will not be possible.
I would say that 6TB air-breathing drives (state on the lid
"do not cover this hole), those are archival material. I
would expect to power one up 20 years from now, and it will work.
That's why I own five or six of those, but I only own one
Helium drive.
And with the right optical media choice (not the dye ones),
those could be buried in the yard as well. Just keep the
humidity down. You don't want any biological attacks
on the media. Maybe some Verbatim Gold DVDs would be
good yard material.
On 29/09/2024 02:27, Paul wrote:
The faster that storage devices get, the more sensitive they
become to details. This is why I would keep you well away
from my PCIe Rev5 NVMe at 14000/12000 MB/sec. That still
needs an error corrector, and somehow keep up with the
need to correct every sector being read out. It's one
of the reasons those get so hot (and they put toy heatsinks
on top).
That's also how you can have devices like this. You would not
get these sorts of rates, without IOPs in the picture to help.
https://www.anandtech.com/show/21486/highpoint-updates-nvme-raid-cards-for-pcie-50-50-gbps-directattached-ssd-storage
So SSDs are safe for long term storage (say, a decade?), even if you
don't access them, so long as you keep them powered on?
I would say that 6TB air-breathing drives (state on the lid
"do not cover this hole), those are archival material. I
would expect to power one up 20 years from now, and it will work.
SO ... REPLICATION ... keep MOVING yer data to the
latest/greatest and 'cloud'. Stick to compression
and encryption SURE to be supported really long term.
On 29/09/2024 22:18, Paul wrote:
I would say that 6TB air-breathing drives (state on the lid
"do not cover this hole), those are archival material. I
would expect to power one up 20 years from now, and it will work.
Well my 2TB drives only lasted about 6 years *powered on*.
I am not sanguine about the lifetime of any magnetic media.
There are hard drives from back in the 1980s that are still booted up
after years in storage. Some boot, some don't, and some are just
partially corrupted.
Magnetic fields are no more permanent than electric fields in SSDs.
SSDS are simply too new to have any reliable long term statistics under
real working conditions.
The short answer is that we are pissing in the wind when it comes to any long term digital storage.
We know paper and ink lasts, we have the dead sea scrolls..
We know that selenium treated photographs last at least 160 years, We
know that first generation colour prints are seriously degraded after
only 50...
We know that some spinning rust 40 years on is still data recoverable ,
we know that a lot is not.
Often for other reasons than magnetic corruption - corrosion on drive spindles etc. Dead capacitors in the onboard electronics
A decent cosmic ray knifing through any modern electronics will fuck the DRAM up to the point where the machine may crash.
No problem. Reboot it...
There are no perfect solutions All data is to an extent written in 'vanishing ink'
But my current best guess is that a rolling replacement of mirrored
disks (rust or SSD) as they show error counts in a 24x7 powered machine
is probably as good as it gets, and the smaller and slower the storage
is, probably the less stressed it will be. Looking at SSD current draws,
it is the cheaper slower ones that seem to draw less and run cooler.
The best news is that we have SMART. And failing but not yet failed
drives due to ageing show up in terms of parity errors. On a 24x7 system.
No one knows till they power up a 40 year old drive whether or not the
data is either still there, or is recoverable.
So my wet finger is moving towards permanently on, lower power, larger,
slow SSDS. From permanently on spinning rust.
My personal server was first built in 2000 or thereabouts. Debian Linux. It's on its 4th motherboard and its third set of hard drives, and its umpteenth OS upgrade. but the data is still there from 2000 or so.
I am constructing, slowly, a replacement based on a Raspberry PI and
twin mirrored SSDs,
When its shown to be reliable, I may switch off the *86 based one
I have had another thought, and that is why we 'archive' in the first
place. That goes back to the days when *working* storage was small, but the need was for stuff to be available for occasional use from slower
media like tape.
Today, with SSDs, our *working* storage can be enormous. And fast. We no longer need traditional data archives. Just leave it all on the running machine, and mirror it.
If you must have 'offsite storage', rsynch another portable drive every
so often and take it away to somewhere safe.
No one knows till they power up a 40 year old drive whether or not the data is either still there, or is recoverable.
Nux Vomica <nv@linux.rocks> wrote:
It seems that a lot of users are, irrationally, opposed to the
use of optical media for long-term archival storage.
What, then, are some alternatives for the general user who does
not command the BIG $BUCKS necessary for enterprise grade solutions.
Tape? Not likely.
Note that magnetic (HDD) or IC NAND (SSD) devices are not to be
considered long-term storage. Consumer SSDs are especially not
long term.
Optical media is still by far the best in terms of cost/benefit
for the general user.
I have hundreds of optical disks that I have produced over the
past years, all with GNU/Linux, and they will last me until the
end of my time.
Optical disks are good if you want a handheld copy of something, but
external storage or large USB media is what I'd prefer.
On Fri, 27 Sep 2024 12:40:57 -0400, Joel <joelcrump@gmail.com> wrote:
Nux Vomica <nv@linux.rocks> wrote:
Optical media is still by far the best in terms of cost/benefit
for the general user.
I have hundreds of optical disks that I have produced over the
past years, all with GNU/Linux, and they will last me until the
end of my time.
Optical disks are good if you want a handheld copy of something, but >>external storage or large USB media is what I'd prefer.
But "large USB media" could be optical, or magnetic, or something
else.
On 30-09-2024 13:25, The Natural Philosopher wrote:
On 29/09/2024 22:18, Paul wrote:As regards to magnetic storage: if it is a magnetic tape, its not
I would say that 6TB air-breathing drives (state on the lid
"do not cover this hole), those are archival material. I
would expect to power one up 20 years from now, and it will work.
Well my 2TB drives only lasted about 6 years *powered on*.
I am not sanguine about the lifetime of any magnetic media.
There are hard drives from back in the 1980s that are still booted up
after years in storage. Some boot, some don't, and some are just
partially corrupted.
Magnetic fields are no more permanent than electric fields in SSDs.
SSDS are simply too new to have any reliable long term statistics
under real working conditions.
The short answer is that we are pissing in the wind when it comes to
any long term digital storage.
We know paper and ink lasts, we have the dead sea scrolls..
We know that selenium treated photographs last at least 160 years, We
know that first generation colour prints are seriously degraded after
only 50...
We know that some spinning rust 40 years on is still data recoverable
, we know that a lot is not.
Often for other reasons than magnetic corruption - corrosion on drive
spindles etc. Dead capacitors in the onboard electronics
A decent cosmic ray knifing through any modern electronics will fuck
the DRAM up to the point where the machine may crash.
No problem. Reboot it...
There are no perfect solutions All data is to an extent written in
'vanishing ink'
But my current best guess is that a rolling replacement of mirrored
disks (rust or SSD) as they show error counts in a 24x7 powered
machine is probably as good as it gets, and the smaller and slower the
storage is, probably the less stressed it will be. Looking at SSD
current draws, it is the cheaper slower ones that seem to draw less
and run cooler.
The best news is that we have SMART. And failing but not yet failed
drives due to ageing show up in terms of parity errors. On a 24x7 system.
No one knows till they power up a 40 year old drive whether or not the
data is either still there, or is recoverable.
So my wet finger is moving towards permanently on, lower power,
larger, slow SSDS. From permanently on spinning rust.
My personal server was first built in 2000 or thereabouts. Debian
Linux. It's on its 4th motherboard and its third set of hard drives,
and its umpteenth OS upgrade. but the data is still there from 2000 or
so.
I am constructing, slowly, a replacement based on a Raspberry PI and
twin mirrored SSDs,
When its shown to be reliable, I may switch off the *86 based one
I have had another thought, and that is why we 'archive' in the first
place. That goes back to the days when *working* storage was small,
but the need was for stuff to be available for occasional use from
slower media like tape.
Today, with SSDs, our *working* storage can be enormous. And fast. We
no longer need traditional data archives. Just leave it all on the
running machine, and mirror it.
If you must have 'offsite storage', rsynch another portable drive
every so often and take it away to somewhere safe.
lasting that long. I had to extract some data from a several years old magnetic tape once. Fortunately I decided to read and store the whole
tape in one go. After it had passed the heads in the tape unit, the complete magnetic layer ended up as dust on the bottom of the unit...
On Fri, 27 Sep 2024 12:40:57 -0400, Joel <joelcrump@gmail.com> wrote:
Nux Vomica <nv@linux.rocks> wrote:
It seems that a lot of users are, irrationally, opposed to the
use of optical media for long-term archival storage.
What, then, are some alternatives for the general user who does
not command the BIG $BUCKS necessary for enterprise grade solutions.
Tape? Not likely.
Note that magnetic (HDD) or IC NAND (SSD) devices are not to be
considered long-term storage. Consumer SSDs are especially not
long term.
Optical media is still by far the best in terms of cost/benefit
for the general user.
I have hundreds of optical disks that I have produced over the
past years, all with GNU/Linux, and they will last me until the
end of my time.
Optical disks are good if you want a handheld copy of something, but
external storage or large USB media is what I'd prefer.
But "large USB media" could be optical, or magnetic, or something
else.
In comp.os.linux.misc Nux Vomica <nv@linux.rocks> wrote:
It seems that a lot of users are, irrationally, opposed to the
use of optical media for long-term archival storage.
Having personally experienced failures of both cd-r and dvd-r media
wherein the recorded media became unreadable in a very short timeframe
(only a few years) even with proper storage it is not at all irrational
to be skeptical of claims of significant lifetimes for optical media
(esp. the user recordable type, pressed disks are a different matter). Existing user recordable optical systems have, so far, had a poor track record, so any new system has a higher bar to get over before it is
trusted for any long-term archive use.
Frankly, I don't think today's data WILL last even
100 years. Lucky with 50. CD/DVD/BR is already
going away.
The formats will become obsolete, the devices/drivers
are already obsolete. These facts are seriously
freaking-out archivists already.
Got one of those old removable-pack hard disk units
from the late 60s ? MIGHT have moon-landing stuff
on it. TRY to find anything to READ it. I've got
some 8-inch floppies with data and some cool FORTRAN
pgms on them. What can I find to read them ? It's
a PROBLEM that's just getting worse.
What DOES last ... baked clay tablets. Still
readable after 8000 years. You can read all
about the hero Gilgamesh, note the accounting
of taxes on wheat in Uruk :-)
On Tue, 10/1/2024 1:04 AM, Steve Hayes wrote:
On Fri, 27 Sep 2024 12:40:57 -0400, Joel <joelcrump@gmail.com> wrote:
Nux Vomica <nv@linux.rocks> wrote:
It seems that a lot of users are, irrationally, opposed to the
use of optical media for long-term archival storage.
What, then, are some alternatives for the general user who does
not command the BIG $BUCKS necessary for enterprise grade solutions.
Tape? Not likely.
Note that magnetic (HDD) or IC NAND (SSD) devices are not to be
considered long-term storage. Consumer SSDs are especially not
long term.
Optical media is still by far the best in terms of cost/benefit
for the general user.
I have hundreds of optical disks that I have produced over the
past years, all with GNU/Linux, and they will last me until the
end of my time.
Optical disks are good if you want a handheld copy of something, but
external storage or large USB media is what I'd prefer.
But "large USB media" could be optical, or magnetic, or something
else.
Presumably something involving the USB Mass Storage standard :-)
(Fraudulent 16TB flash stick -- well, I only wanted to pay $10 for it)
https://i5.walmartimages.com/asr/dffbc8a8-87c8-4db7-b6e8-d94b3b2f9b8e.dae192be27efdde1869519eda80a1c9f.jpeg
There were some announcements of 16TB flash drives (2.5"), but
at least some of those designs, did not make it to stores
(controller problems, lack of qty of big flash chips). One
company might be shipping, but the price is about 2X what you
should be paying.
No, in fact, flash is not the answer. Unless you want to build
a huge USB tree of some sort, using Patriot 1TB drives until
you have enough storage.
And real flash drives, the 100TB 3.5" ones, cost as much as a car.
And they tend to work at SATA III rates. Still waiting for a
reviewer to get one (in its own Brinks truck).
Paul
Ever seen one of these ?
https://i.pinimg.com/736x/47/7b/d0/477bd0823ee461ed51192a3c530d7a49--vintage-advertisements-computers.jpg
http://archive.computerhistory.org/resources/physical-object/scopus/102626624.lg.jpg
I do ... they were the 'state of the art' when I first
got into computers. Beat the shit out of tapes.
They fit into a little 'washing machine' box. They
usually had a plastic case with a handle. You could
REMOVE them and put in another disc pack (but DO
make sure they'd stopped spinning - I remember at
least one amusing incident).
They were THE thing in the 60s and 70s - long before
before the little ones that'd fit into yer IBM-PC.
MOSTLY the read heads all moved in unison, but I saw
one at NASA that had independently-moving read/write
arms but that MAY have been one of their one-off
bits of tech. Those were big fat arms too ... nearly
a half-inch between each disk platter in the pack.
Cool to watch them work.
Now, oddly, I took a tour of a US attack sub in
the latter 80s and, in the sonar room, they STILL
had one of these things - attached to a box so
large it probably still used discrete transistors
as its CPU - but mil stuff is specced MANY years
in advance so it's NEVER the latest greatest tech
except in the movies.
Had a brief, understandably security-couched,
conversation with the sonar guy - wherein he
agreed that the best sound/pattern detector was
still gonna be between-the-ears for a LONG time
to come. The computer signal-processing stuff was
an AID, not any definitive word on anything.
Human skill was still what was needed. Our
pattern-detection/comprehension hardware STILL
exceeds most "AI" efforts even now. Darwin
did good.
On Tue, 1 Oct 2024 01:29:29 -0400, "186282@ud0s4.net"
<186283@ud0s4.net> wrote:
Frankly, I don't think today's data WILL last even
100 years. Lucky with 50. CD/DVD/BR is already
going away.
The formats will become obsolete, the devices/drivers
are already obsolete. These facts are seriously
freaking-out archivists already.
And that is the problem with digitising documents, which seems to be
all the rage.
Got one of those old removable-pack hard disk units
from the late 60s ? MIGHT have moon-landing stuff
on it. TRY to find anything to READ it. I've got
some 8-inch floppies with data and some cool FORTRAN
pgms on them. What can I find to read them ? It's
a PROBLEM that's just getting worse.
Part of the problem is planned obsolescence, and that is perhaps one
of the points in favour of open-source software. A firm goes bankrupt
and documents produced with their proprietary software becomes
unreadable.
Perhaps a law should be enacted that any computer software that is "no
longer supported" should be made open source.
I usually rec 'forward replication', keep copying
yer data to the latest/greatest media. Alas even
that seems to be running out. The Future is 'cloud',
but remember Vlad's nasty boyz might evaporate that
cloud anytime now.
Optical disks - esp M-Disks - can LAST ... but they
are LOW-DENSITY by today's defs by a BIG margin.
It's also not clear if you'd be able to FIND a
DVD reader device 25 years from now. Even most
current PCs/laptops don't COME with those anymore.
It's a SERIOUS PROBLEM. We make SO much data now,
SUCH volume, and at least SOME of it IS important
for both legal and historical reasons - yet there
are NO really good archival media.
Optical disks - esp M-Disks - can LAST ... but they
are LOW-DENSITY by today's defs by a BIG margin.
It's also not clear if you'd be able to FIND a
DVD reader device 25 years from now. Even most
current PCs/laptops don't COME with those anymore.
The Library Of Congress and Smithsonian are FREAKIN'
at this point. SO much historical data - but they
can't even find the hardware/drivers to READ the
often-proprietary media.
On 2024-10-03 12:40 a.m., Steve Hayes wrote:
Perhaps a law should be enacted that any computer software that is "no
longer supported" should be made open source.
It happens a lot but there is no law pushing for that. Nevertheless,
what you mentioned above is true albeit most hypothetical. In most
cases, even obsolete formats can be read by the new suites because there
is a converter built into the software. I imagine this was a problem in
the 90s with the format of 80s software which suddenly disappeared, but
it no longer seems to be true. Still, I agree that formats should be >open-source.
In comp.os.linux.misc 186282@ud0s4.net <186283@ud0s4.net> wrote:
Optical disks - esp M-Disks - can LAST ... but they
are LOW-DENSITY by today's defs by a BIG margin.
It's also not clear if you'd be able to FIND a
DVD reader device 25 years from now. Even most
current PCs/laptops don't COME with those anymore.
Which is exactly the problem.
Your DVD 'media' might survive 1,000 years. But if you can't buy a DVD reader for any price, all that 'media' is useless.
We've already seen that issue before. Try to find a Zip disk or
SuperDisk drive. If any could be purchased, they would all be from
places like eBay and of questionable operational quality.
The reality is the only way to 'archive' is to periodically copy one's archived data to newer storage systems, which involves at least the
work of the actual copy process (and which must occur before the
mechanical devices for the old media fail) and of verification that the
copy process did not itself corrupt the data.
On Thu, 10/3/2024 2:28 AM, 186282@ud0s4.net wrote:
Ever seen one of these ?
https://i.pinimg.com/736x/47/7b/d0/477bd0823ee461ed51192a3c530d7a49--vintage-advertisements-computers.jpg
http://archive.computerhistory.org/resources/physical-object/scopus/102626624.lg.jpg
I do ... they were the 'state of the art' when I first
got into computers. Beat the shit out of tapes.
They fit into a little 'washing machine' box. They
usually had a plastic case with a handle. You could
REMOVE them and put in another disc pack (but DO
make sure they'd stopped spinning - I remember at
least one amusing incident).
They were THE thing in the 60s and 70s - long before
before the little ones that'd fit into yer IBM-PC.
MOSTLY the read heads all moved in unison, but I saw
one at NASA that had independently-moving read/write
arms but that MAY have been one of their one-off
bits of tech. Those were big fat arms too ... nearly
a half-inch between each disk platter in the pack.
Cool to watch them work.
Now, oddly, I took a tour of a US attack sub in
the latter 80s and, in the sonar room, they STILL
had one of these things - attached to a box so
large it probably still used discrete transistors
as its CPU - but mil stuff is specced MANY years
in advance so it's NEVER the latest greatest tech
except in the movies.
Had a brief, understandably security-couched,
conversation with the sonar guy - wherein he
agreed that the best sound/pattern detector was
still gonna be between-the-ears for a LONG time
to come. The computer signal-processing stuff was
an AID, not any definitive word on anything.
Human skill was still what was needed. Our
pattern-detection/comprehension hardware STILL
exceeds most "AI" efforts even now. Darwin
did good.
Our "personal computer" design at work, had one of those
washing machines as an option. That was our departmental
file server at the time. Surprisingly, it worked well,
and no complaints about the loadable disk packs. Swap a
pack, allow the machine to purge the air for five to ten
minutes, then put it back online.
As for your paragraph:
HDDs are better - but they ARE mechanical devices
and also, esp with the high-cap ones, 'bit rot'
can become an issue. Don't expect to put 'em in
the safe for 25 years and still expect to get
data out of them.
Just be aware there are two groups of drives.
A 22TB Helium drive, is not really archival quality.
It's the unknown Helium status that is the limiting factor.
An air breathing drive at 6TB, is more likely to be running
20 years from now.
As for the "bit-rot" assertion, there is this non-authoritative source
https://datarecovery.com/rd/what-are-hard-drive-error-correction-codes-eccs/
"Most modern hard drives use the Reed-Solomon error correction algorithm,
which doesn’t require much storage space or processing power."
That means that when a sector is read, it can be:
1) Perfect
2) Correct-able
3) Uncorrect-able (returns CRC error)
Flash devices also work this way (50 byte syndrome for 512 bytes sector).
At one time, hard drives assumed single bit errors, random in nature, uncorrelated, no burst errors. The original error detection might have
been a Fire Code. You could multiply by (X+1) a couple times, when
selecting a polynomial, to allow it to handle multiple single bit errors.
I don't think there was a correction capability with that, it was just
for bit-rot. Back then, you were more likely to see "bits" in the
head signal. Today, the head signal is wavey-gravy and only DSP
techniques recover the data. A lot of items use scramblers to
reduce zeros sensitivity, so even if you attempted to "zero the drive",
if you scope the head signal, it will be a wavey gravy signal, not
a "flat signal". The signal will not be zero volts from one end of the platter to the other. The signal is scrambled (and could also be
encrypted by FDE encryption, and they would *still* scramble it).
That's why if you think anyone "looks with a microscope and
reads out your .txt file", no, it does not work that way :-)
The recovery process needs at least a computer... and a codes genius.
The FDE part, is why when repairing hard drives today, you have to swap
the ROM from the original controller, with the one on the replacement controller, because the key is stored in there. In the old days, you
just slapped on a matching controller model number, and it just worked. That's not how it happens today. There are several ROMs on the board,
but only one needs to be swapped.
On 10/3/24 1:13 PM, Rich wrote:
In comp.os.linux.misc 186282@ud0s4.net <186283@ud0s4.net> wrote:
Optical disks - esp M-Disks - can LAST ... but they
are LOW-DENSITY by today's defs by a BIG margin.
It's also not clear if you'd be able to FIND a
DVD reader device 25 years from now. Even most
current PCs/laptops don't COME with those anymore.
Which is exactly the problem.
Your DVD 'media' might survive 1,000 years. But if you can't buy a DVD
reader for any price, all that 'media' is useless.
We've already seen that issue before. Try to find a Zip disk or
SuperDisk drive. If any could be purchased, they would all be from
places like eBay and of questionable operational quality.
I actually HAVE a gen-1 ZipDisk - parallel port.
They were supposed to be The Future :-)
Now nothing even comes with a parallel port.
Does anybody sell a USB->DB25 Parallel ?
Ah :
https://www.amazon.com/C2G-16899-Parallel-Printer-Adapter/dp/B000UX21PY/ref=sr_1_6?dib=eyJ2IjoiMSJ9.1VI8pfXqbYFCqzemMpvwSeS7sWlXF85-8KCEE02Eq0U5ijmY6VQsHDd4jiHTJdbXneNSTAjv0kQYw5VX4GdbqyTSunox50IwAfWTs_yauKlzZUkYaW-1O0zu_K-T2aZ-lucb5yIPkraxSzUvcjoEALqE4U6n0B54UxrIA4O_XuR08eLCcjsrjV-zuudG7_iyjE76TOIVN5lFDy-JxuJ6Z9LWfk4rkdxNqfT1bg0KjN0.sM52_QZ_SQzD3giWHwr3zXrQaCEJ8eYQR17c0YfJzW8&dib_tag=se&keywords=parallel+to+usb+adapter&qid=1728012393&sr=8-6
Linux drivers ???????
The reality is the only way to 'archive' is to periodically copy one's
archived data to newer storage systems, which involves at least the
work of the actual copy process (and which must occur before the
mechanical devices for the old media fail) and of verification that the
copy process did not itself corrupt the data.
Exactly what I said ... "forward replication". It's
actual WORK alas .....
"Cloud" also - but remember that the cloud can
go away in a number of fashions - and not just
Vlad and his boyz.
Oh well, back to baked clay tablets - proven 9Kyear
retention and you don't need a reading device :-)
On Thu, 3 Oct 2024 09:58:04 -0400, CrudeSausage <crude@sausa.ge>
wrote:
On 2024-10-03 12:40 a.m., Steve Hayes wrote:
Perhaps a law should be enacted that any computer software that is "no
longer supported" should be made open source.
It happens a lot but there is no law pushing for that. Nevertheless,
what you mentioned above is true albeit most hypothetical. In most
cases, even obsolete formats can be read by the new suites because there
is a converter built into the software. I imagine this was a problem in
the 90s with the format of 80s software which suddenly disappeared, but
it no longer seems to be true. Still, I agree that formats should be
open-source.
I don't know about Linux, but Windows 11 cannot run programs that are
used to read or create older documents, and if you put earlier
versions on a new computer Microsoft won't let you run them. Windows
XP, the 32-bit version anyway, should be made open source.
On Thu, 10/3/2024 10:46 PM, Steve Hayes wrote:code-leak
I don't know about Linux, but Windows 11 cannot run programs that are
used to read or create older documents, and if you put earlier versions
on a new computer Microsoft won't let you run them. Windows XP, the
32-bit version anyway, should be made open source.
Wasn't the source stolen ? Article is from four years ago.
https://www.theverge.com/2020/9/25/21455655/microsoft-windows-xp-source-
All that you need, is a virtualization solution.
On Fri, 04 Oct 2024 02:19:08 -0400, Paul wrote:
On Thu, 10/3/2024 10:46 PM, Steve Hayes wrote:code-leak
I don't know about Linux, but Windows 11 cannot run programs that are
used to read or create older documents, and if you put earlier versions
on a new computer Microsoft won't let you run them. Windows XP, the
32-bit version anyway, should be made open source.
Wasn't the source stolen ? Article is from four years ago.
https://www.theverge.com/2020/9/25/21455655/microsoft-windows-xp-source-
Well, if they won't let you use the real thing, I suppose a pirate copy
will do. But it's the attitude of companies like that that makes "digitisation" a very insecure and unreliable way of archiving documents.
All that you need, is a virtualization solution.
From what I've heard from users, such solutions are clunky and
unreliable.
BTW what's the quickest way o updating the browser I'm using in Linux? It barfed on the link you gave, saying it wasn't a secure connection. Firfox
for Windows lets me override that, but Firefox for Linux obviously
doesn't, so I'd better update it.
On Thu, 3 Oct 2024 09:58:04 -0400, CrudeSausage <crude@sausa.ge>
wrote:
On 2024-10-03 12:40 a.m., Steve Hayes wrote:
Perhaps a law should be enacted that any computer software that is "no
longer supported" should be made open source.
It happens a lot but there is no law pushing for that. Nevertheless,
what you mentioned above is true albeit most hypothetical. In most
cases, even obsolete formats can be read by the new suites because there
is a converter built into the software. I imagine this was a problem in
the 90s with the format of 80s software which suddenly disappeared, but
it no longer seems to be true. Still, I agree that formats should be
open-source.
I don't know about Linux, but Windows 11 cannot run programs that are
used to read or create older documents, and if you put earlier
versions on a new computer Microsoft won't let you run them. Windows
XP, the 32-bit version anyway, should be made open source.
On Fri, 04 Oct 2024 02:19:08 -0400, Paul wrote:
On Thu, 10/3/2024 10:46 PM, Steve Hayes wrote:code-leak
I don't know about Linux, but Windows 11 cannot run programs that are
used to read or create older documents, and if you put earlier versions
on a new computer Microsoft won't let you run them. Windows XP, the
32-bit version anyway, should be made open source.
Wasn't the source stolen ? Article is from four years ago.
https://www.theverge.com/2020/9/25/21455655/microsoft-windows-xp-source-
Well, if they won't let you use the real thing, I suppose a pirate copy
will do. But it's the attitude of companies like that that makes "digitisation" a very insecure and unreliable way of archiving documents.
All that you need, is a virtualization solution.
From what I've heard from users, such solutions are clunky and
unreliable.
BTW what's the quickest way o updating the browser I'm using in Linux? It barfed on the link you gave, saying it wasn't a secure connection. Firfox for Windows lets me override that, but Firefox for Linux obviously
doesn't, so I'd better update it.
On 2024-10-03 10:46 p.m., Steve Hayes wrote:
On Thu, 3 Oct 2024 09:58:04 -0400, CrudeSausage <crude@sausa.ge>
wrote:
On 2024-10-03 12:40 a.m., Steve Hayes wrote:
Perhaps a law should be enacted that any computer software that is "no >>>> longer supported" should be made open source.
It happens a lot but there is no law pushing for that. Nevertheless,
what you mentioned above is true albeit most hypothetical. In most
cases, even obsolete formats can be read by the new suites because there >>> is a converter built into the software. I imagine this was a problem in
the 90s with the format of 80s software which suddenly disappeared, but
it no longer seems to be true. Still, I agree that formats should be
open-source.
I don't know about Linux, but Windows 11 cannot run programs that are
used to read or create older documents, and if you put earlier
versions on a new computer Microsoft won't let you run them. Windows
XP, the 32-bit version anyway, should be made open source.
There is a compatibility mode in the new versions of Windows which do indeed allow you to run old software. It was demonstrated that you can fairly easily run software made even for Windows 3.0. Please produce a few examples of old programs which don't run _at all_ in Windows 11, and I'm sure that someone with some time on their hands will not only install and run the software, but explain how to get it working.
On 04/10/2024 10:56, Steve Hayes wrote:
BTW what's the quickest way o updating the browser I'm using in Linux? ItI think there is a way because I am running latest Firefox against my
barfed on the link you gave, saying it wasn't a secure connection. Firfox
for Windows lets me override that, but Firefox for Linux obviously
doesn't, so I'd better update it.
own (non https) sites
This is probably what I disabled years ago:
Preferences ->Privacy & security->HTTPS Only Mode
and select:
Don’t enable HTTPS-Only Mode
Then instead of flat out refusing to connect it gives you other options >IIRC.
I don't know about Linux, but Windows 11 cannot run programs that are
used to read or create older documents, and if you put earlier
versions on a new computer Microsoft won't let you run them. Windows
XP, the 32-bit version anyway, should be made open source.
There is a compatibility mode in the new versions of Windows which do
indeed allow you to run old software. It was demonstrated that you can >fairly easily run software made even for Windows 3.0. Please produce a
few examples of old programs which don't run _at all_ in Windows 11, and
I'm sure that someone with some time on their hands will not only
install and run the software, but explain how to get it working.
On Fri, 10/4/2024 8:34 AM, CrudeSausage wrote:
On 2024-10-03 10:46 p.m., Steve Hayes wrote:
I don't know about Linux, but Windows 11 cannot run programs that are
used to read or create older documents, and if you put earlier
versions on a new computer Microsoft won't let you run them. Windows
XP, the 32-bit version anyway, should be made open source.
There is a compatibility mode in the new versions of Windows which do indeed allow you to run old software. It was demonstrated that you can fairly easily run software made even for Windows 3.0. Please produce a few examples of old programs which don't run _at all_ in Windows 11, and I'm sure that someone with some time on their hands will not only install and run the software, but explain how to get it working.
Windows 11 is 64 bit only, and cannot run programs with 16 bit installers
or programs with 16 bit code. This means some older games won't run.
windows 10 has a 32 bit edition, but that's a kind of "limited" OS when
you are trying to run a browser that is greedy for RAM. The 32 bit edition >can run an old copy of Doom. Maybe you install two copies of the OS,
on your disk drive (you're allowed and they use the same server-side license), >a 64-bit one (for browser work) and a 32-bit one
(where you do your WinXP era work), and then you have better odds
of getting some things to work.
Even when running the 32-bit one, as you say, the Compatibility Assistant--
can make some things work, but it doesn't always succeed.
Paul
On Thu, 3 Oct 2024 09:58:04 -0400, CrudeSausage <crude@sausa.ge>
wrote:
Perhaps a law should be enacted that any computer software that is "no
longer supported" should be made open source.
It happens a lot but there is no law pushing for that. Nevertheless,
what you mentioned above is true albeit most hypothetical. In most
cases, even obsolete formats can be read by the new suites because there >>is a converter built into the software. I imagine this was a problem in >>the 90s with the format of 80s software which suddenly disappeared, but
it no longer seems to be true. Still, I agree that formats should be >>open-source.
I don't know about Linux, but Windows 11 cannot run programs that are
used to read or create older documents, and if you put earlier
versions on a new computer Microsoft won't let you run them. Windows
XP, the 32-bit version anyway, should be made open source.
On 10/3/24 1:13 PM, Rich wrote:
In comp.os.linux.misc 186282@ud0s4.net <186283@ud0s4.net> wrote:
Optical disks - esp M-Disks - can LAST ... but they
are LOW-DENSITY by today's defs by a BIG margin.
It's also not clear if you'd be able to FIND a
DVD reader device 25 years from now. Even most
current PCs/laptops don't COME with those anymore.
Which is exactly the problem.
Your DVD 'media' might survive 1,000 years. But if you can't buy a DVD
reader for any price, all that 'media' is useless.
We've already seen that issue before. Try to find a Zip disk or
SuperDisk drive. If any could be purchased, they would all be from
places like eBay and of questionable operational quality.
I actually HAVE a gen-1 ZipDisk - parallel port.
They were supposed to be The Future :-)
Now nothing even comes with a parallel port.
Does anybody sell a USB->DB25 Parallel ?
If I were M$, I wouldn't want it [Windows NT source code] to be public domain, even
at 20+ years old. It's still the foundation of Win11Copilot, today.
Obviously, XP and other ancient OSes don't share any code with Win11 -
but the foundation of Vista was XP 64-bit/Server2003.
On 05/10/2024 in message <4ni1gjhb2adbl8uagkarl0ujb6cb3big8i@4ax.com> Joel >wrote:
Obviously, XP and other ancient OSes don't share any code with Win11 -
but the foundation of Vista was XP 64-bit/Server2003.
I can assure you that the Windows API used by Windows 98 is alive and well >and driving every Windows OS since then. Programs I wrote for Win 68 still >run perfectly well on Win 10/11. The only change has been the introduction >of 64 bit versions of the various DLLs.
"Jeff Gaines" <jgnewsid@outlook.com> wrote:
On 05/10/2024 in message <4ni1gjhb2adbl8uagkarl0ujb6cb3big8i@4ax.com> Joel >>wrote:
Obviously, XP and other ancient OSes don't share any code with Win11 - >>>but the foundation of Vista was XP 64-bit/Server2003.
I can assure you that the Windows API used by Windows 98 is alive and well >>and driving every Windows OS since then. Programs I wrote for Win 68 still >>run perfectly well on Win 10/11. The only change has been the introduction >>of 64 bit versions of the various DLLs.
I'm aware that the API isn't unique to NT.
On Thu, 3 Oct 2024 23:31:46 -0400, "186282@ud0s4.net"
<186283@ud0s4.net> wrote:
On 10/3/24 1:13 PM, Rich wrote:
In comp.os.linux.misc 186282@ud0s4.net <186283@ud0s4.net> wrote:
Optical disks - esp M-Disks - can LAST ... but they
are LOW-DENSITY by today's defs by a BIG margin.
It's also not clear if you'd be able to FIND a
DVD reader device 25 years from now. Even most
current PCs/laptops don't COME with those anymore.
Which is exactly the problem.
Your DVD 'media' might survive 1,000 years. But if you can't buy a DVD
reader for any price, all that 'media' is useless.
We've already seen that issue before. Try to find a Zip disk or
SuperDisk drive. If any could be purchased, they would all be from
places like eBay and of questionable operational quality.
I actually HAVE a gen-1 ZipDisk - parallel port.
They were supposed to be The Future :-)
Now nothing even comes with a parallel port.
Does anybody sell a USB->DB25 Parallel ?
I put all by ZipDisks on CD-R, and have a directory on my current
computer with them on it as well.
I have another computer that still runs, but the monitor died, and i
was unable to replace it. It had an SCSI connection, which was
supposed to be the future. Has anyone seen anything with a SCSI port recently?
Computer entropy is real.
Where the limitation comes in, is finding a SCSI controller chip
on a PCI Express card. My collection of SCSI cards here are
all PCI. The last machine with a PCI slot is now ten years old.
Two of my other machines had motherboard failures, which has--- Synchronet 3.20a-Linux NewsLink 1.114
really "hollowed out" my redundancy plan. I will soon
have nothing current, to use the PCI cards in the junk room.
I have a 2906 to run the old scanner, but the old scanner
needs some repair work now. I don't know if I have the skill
to figure out the root cause. It might be dirty... but I doubt it.
The "rot" is at multiple levels, not just one level.
That's why the "rot" is accelerating. There are fewer things
you can depend on. It would be OK perhaps, if only one
standard took a shit. When two or three take a shit,
now you're talking real money.
Paul
| Sysop: | DaiTengu |
|---|---|
| Location: | Appleton, WI |
| Users: | 991 |
| Nodes: | 10 (1 / 9) |
| Uptime: | 76:03:21 |
| Calls: | 12,949 |
| Calls today: | 3 |
| Files: | 186,574 |
| Messages: | 3,264,538 |
