datahoarder

they made cool workout posters, and still do, but I think they got dmca'd in 2016. the superheroes are all gone.

I have been lurking on this community for a while now and have really enjoyed the informational and instructional posts but a topic I don't see come up very often is scaling and hoarding. Currently, I have a 20TB server which I am rapidly filling and most posts talking about expanding recommend simply buying larger drives and slotting them in to a single machine. This definitely is the easiest way to expand, but seems like it would get you to about 100TB before you cant reasonably do that anymore. So how do you set up 100TB+ networks with multiple servers?

In light of some of the recent dystopian executive orders, a lot of data is being proactively taken down. I am relying on this data for a report I'm writing at work, and I suspect a lot of others may be relying on it for more important reasons. As such, I created two torrents, one for the data behind the ETC Explorer tool and another for the data behind the Climate and Economic Justice Screening Tool. Here's an article about taking down the latter. My team at work suspects the former will follow soon.

Trying to figure out if there is a way to do this without zfs sending a ton of data. I have:

Most commented pages on each site sorted from most(Aniwave) to least(Anitaku) amount of comments:

Following up from my previous post.

I'd love to know if anyone's aware of a bulk metadata export feature or repository. I would like to have a copy of the metadata and .torrent files of all items.

Reposted from lemmy.world c/politics since it violated it's rule #1 about links.

Hello,

The September 17th archive of the oldest public video on ashens' channel is saved with the comments section of a completely different video.

I did try to read the sidebar resources on https://www.reddit.com/r/DataHoarder/. They're pretty overwhelming, and seem aimed at people who come in knowing all the terminology already. Is there somewhere you suggest newbies start to learn all this stuff in the first place other than those sidebar resources, or should I just suck it up and truck through the sidebar?

I download lots of media files. So far I have been storing these files after I am done with them on a 2TB hard disk. I have been copying over the files with rsync. This has so far worked fairly well. However the hard disk I am using is starting to get close to full. Now I am needing to find a solution so I can span my files over multiple disks. If I were to continue to do it as I do now, I would end up copying over files that would already be on the other disk. Does the datahoading community have any solutions to this?

Hey everyone,

I know for photos i could throw them through something like Converseen to take them from .jpg to .jxl, preserving the quality (identical visially, even when pixel peeping), but reducing file size by around 30%. What about video? My videos are in .h265, but can i reencode them more efficiently? im assuming that if my phone has to do live encoding, its not really making it as efficient as it could. could file sizes be reduced without losing quality by throwing some processing time at it? thank you all

Yoko Taro is the creative director behind the Nier and Drakengard series, and he has released a lot of supplemental material across a variety of mediums over the years in Japan. Accord's Library is a site that is dedicated to finding this material, archiving it, and translating it.

Hi everyone!

I tried to put emphasis on the personal nature of the question in the title. I'm not asking for myself or the average individual. I also mean ideal in the way where cost is still a factor. The iPhone 16 Pro has a 1TB model but it's around $500 more expensive than the 128GB version.

I have a bunch of old VHS tapes that I want to digitize. I have never digitized VHS tapes before. I picked up a generic HDMI capture card, and a generic composite to HDMI converter. Using both of those, I was planning on hooking a VCR up to a computer running OBS. Overall, I’m rather ignorant of the process. The main questions that I currently have are as follows:

Hey folks, I’m lucky enough to be in one of the few places on the internet that uses LTO.

Years ago I came across filecoin/sia decentralized data storage and I started trying them but then I stopped due to lack of time. Some days ago I’ve heard in a podcast about a kind of NAS that does kinda the same thing: it spreads chunks of data across other devices owned by other users.

I’m entertaining the thought to write my backups onto tape storage. So my questiont to this community is: does someone know where (if any) to get cheap and simple (my requirements are “just writes & reads the data and is usable with a Linux machine”) used tape drives?

I’m celebrating my datahoarding problem.

Ensh*ttification

I’ll just save them in this folder so that I can totally come back later and read them.

I’m running my media server with a 36tb raid5 array with 3 disks, so I do have some resilience to drives failing. But currently can only afford to loose a single drive at a time, which got me thinking about backups. Normally I’d just do a backup to my NAS, but that quickly gets ridiculous for me with the size of my library, which is significantly larger than my NAS storage of only a few tb. And buying cloud storage is much too expensive for my liking with these amounts of storage.

Hey guys, so it seems that Linkwarden isn’t as good as I was hoping, since some websites will throw up a cookie popup or some other screen that basically prevents the capture.

a few days ago i saw a post on the reddit datahoarder community asking how to backup keys and other small files for a long time. it reminded me of a script i made some time ago to save my otp secrets in case of loss of device or a reenactment of the raivo otp incident, so i decided to make it public on github, hope someone here finds it useful

i want to buy a few hard drives for backups.

Are they worth considering or only worth it at certain price points?

Vimms Lair is getting removal notices from Nintendo etc. We need someone to help make a rom pack archive can you help?

With Google Workspace cracking down on storage (Been using them for unlimited storage for years now) I was lucky to get a limit of 300TBs, but now I have to actually watch what gets stored lol

I have a collection of about ~110 4K Blu-Ray movies that I’ve ripped and I want to take the time to compress and store them for use on a future Jellyfin server.

I formatted my PC recently and I’m reinstalling some stuff I forgot to make a backup of settings like yt-dlp so I started searching for what is a good config to download the best mp4 quality and found some interesting setups so I figured I’d make a thread for people to share what they use.

What in your hoard do you treasure the most? I imagine to a lot of us it is photos and videos of our families, which I’d love to hear about, but also interested in rare bits of media or information that makes your collection unique.

Hi everyone,

YouTube is cracking down Adblocker and they may never work in a year or so.

Running GParted gives me an error that says

Basically title!

I read something about once-reliable sites that would tell you the best [tech thing] now not giving legit reviews, being paid to say good things about certain companies, and I do not remember where I read that or which sites, so I figured I’d bypass the issue and ask people here. I’m pretty new to anything near the level of complexity and technical details that I see on datahoarder communities. I know about the 321 backup rule and that’s it. This is me trying to find something to hold copy 3 of my data.

They’re all available on this Tumblr page. I’d appreciate it a lot.

I was considering making a 30+ TB NAS to simplify and streamline my current setup but because it’s a relatively low priority for me I am wondering is it worth it to hold off for a year or two?

I’m looking at my library and I’m wondering if I should process some of it to reduce the size of some files.

I was so confident that WhatsApp was backing itself up to Google ever since I got my new pixel but I just wasn’t. Then yesterday I factory reset my phone to fix something else and I lost it all. Years worth of chats from so many times in my past just aren’t there, all my texts with my mom and my family, group chats with old friends… I can’t even look at the app anymore, I’ll never use Whatsapp as much as I used to. I just don’t feel right with this change. There’s no way to get those chats back and now it doesn’t feel like there’s any point backing up WhatsApp now! I really wanna cry like this is so unfair!! And all I had to do was check Whatsapp before I did a factory reset… the TINIEST THING I could have done and prevented this and I didn’t fucking do it!!!

Run this javascript code with the document open in the browser: codeberg.org/…/googleDocsRevisionDownloader.js

I found the URL format for step 2 here:

So it’s been a a few years since I’ve bought hard drives for my little home server and wanted to get a bead on what’s the target on dollar to TB in the post Covid world. Thanks!

I’m looking for EPUB formats of all the three books.

Hello, I’m wondering what do you guys use and recommend for efficient book, comic, manga and lightnovel file management, tagging, directory structures and automated tools for all that.

@datahoarder The Internet Archive Just Backed Up an Entire Caribbean Island https://www.wired.com/story/internet-archive-backed-up-aruba-caribbean-island/

Hi, anyone could point me at an ELI5 about SAS hardware? I’d like to assemble a NAS using an old HP Z200, I want SAS because I’d get also a tape drive for backups and I cannot find SATA tape drives. For example, is a Dell Perc H310 pci-e card good for me? Can I avoid hardware RAID?

While clicking through some random Lemmy instances, I found one that’s due to be shut down in about a week — dmv.social. I’m trying to archive what I can onto the Wayback Machine, but I’m not sure what the most efficient way to go about it is.

Seems the SSD sometimes heats up and the content disappears from the device, mostly from my router, sometimes from my laptop. Do you know what I should configure to put the drive to sleep or something similar to reduce the heat?

Last year Elon Musk accidentally revealed that he has a burner account on Twitter called @ErmnMusk.