UPDATE: To everyone who suggested YUNO, thank you so much. This seems like it is about to make my journey much easier. It is basically almost exactly what I was looking for, but I was unaware that it existed.
Thank you ALL for your suggestions, actually. It’s a bit overwhelming for an almost complete noobie but I an going to look into all of the suggestions in time. I just saw that there were several mentions of YUNO so I decided to make that one of the first things I investigated.
So, about two months ago, I had a very eye opening experience. As the result of a single misconfigured security setting on my Android, I was locked out of my Google Account on my phone AND all of my PCs. I had no access whatsoever to Google, or any of the literally hundreds of services that I get through Google.
This is when I realized that I relied entirely on Google/Android because those two days were actually very difficult, being cut off from media, services, passwords, everything, from the past almost twenty years of my life, could be taken away from me in an instant. The decades of my life that were locked away in my Google Account included hundreds of thousands of pictures, almost a hundred thousand audio tracks, several hundred books, several hundred apps, thousands of videos, etc. ad infinitum. Unfortunately, very little of this material was backed up at that point. That is my fault. Also, the misconfigured security setting was my fault as well.
The amount of data, media, memories, services, etc. that would have been lost is actually endless and it would have affected my life in several ridiculously negative ways.
Luckily, in the end, I was able to get my access back and then basically immediately grabbed all of the several terabytes of information and media of mine that they had, and that I was almost locked out of. I have it all in my house now on a drive in my computer, with a backup made on another disconnected disk.
I then decided that no corporation was ever going to have such an insanely high level of influence on and control over my entire life and my media ever again. That experience was actually very scary.
I’ve been trying to get into SelfHosting, but am finding it quite daunting and difficult.
There is a LOT of stuff that I have to learn, and I am mostly unsure of where to even begin. I know basically nothing about networking.
I need to learn the very basic stuff and work my way up from there, but everything that I’ve seen on the Internet assumes that the reader already has a basic to intermediate understanding of networking and the subjects that surround it. I do not, but I am going to learn.
I just need someone to show me where to start.
Thanks in advance for any assistance!
The FUTO guide is meant for total beginners.
Thank you! I will begin to look over it tomorrow!
They use OpenVPN for some reason. Wireguard is superior in every way. In case you set up a VPN.
I didn’t know I needed this. Thanks!
Yunohost should be the software you’re looking for. Install stuff by clicking. Much less terminal stuff
Oh my god, you were right. Yuno is AMAZINGLY useful for exactly what it is that I am attempting to do!
Glad you like it! If it’s useful to you, don’t forget to donate or at least say thanks to the contributors once everything is up and running and stable.
Don’t forget backups! Restic is in yunohost and should be useful for that. Yunohost has a guide.
I haven’t actually began to use it yet because first I am trying to understand the framework and fundamental basics of what it is that I am attempting to do. When I get a grasp on that, I will definitely be using YUNO and overwhelmingly likely will donate a good sum of money to that project.
You can test it in a virtual machine like virtualbox or virt-manager. Then you can get a good feel for it.
Ahhh, good point! I definitely know about Virtualbox. I shall try that. Thanks!!
Sevral people have mentioned Yuno and I’m going to look into it shortly. Thanks for the input!
There’s Yuno, CasaOS is ridiculously easy to setup, manage and maintain as well. There’s UnRaid (not free, but very good), Proxmox is extremely versatile.
I am currently running light services (caldav, carddav, PW manager, and some other lighter stuff) on an N150 mini PC, and have a hefty server for heavier services running on Proxmox.
Of course, I follow the 3-2-1 backup rule, but only for data I could never get again. Movies, Series, music, I never back up.
As someone who went through this after trumps 2nd term and power grabs i can give you my process:
- angrily unsubscribe all big tech subscriptions
- make a protonmail and tutamail account, realize I like proton suote more and decide to subscribe
- transfer all passwords to proton suite
- download all photos and other from cloud to an external drive. TURNS OUT THIS TAKES SEVERAL DAYS WTF
- angrily order a rasp-pi and an external SSD
- use step by step tutorials from pimylifeup to install docker and immich. Fall in love
- gradually (via help of google and GPTs) become confident enough with command line to start managing the server headless over SSH
Fast forward 6 months: My router is now running OpenWRT. With a few necessary exceptions my network access is always through ProtonVPN. My external devices are connected via wireguard to the router when not on home wifi and only after that reach the www. I have 24/7 access to my services from everywhere. My main server is now an old office mini pc running about 10 services. Im using borg for nightly snapshots(its a bit like apple time machine) and after that everything is backed up to another server at a friends house via rsync and ssh. I have a third mini computer whose purpose is to be my tv’s UI with access to services like the national broadcasts web ui and my own jellyfin and invidious (adless youtube client) The tv does not have an internet connection anymore. I even made a custom land page that automatically opens full screen in a browser when open my tv.
The point is: this builds gradually and you have fun doing it. …until it breaks :D The most painful parts involved networking so you can settle for LAN only at first to keep things simple
Re-investing in a new platform full of tools (Proton suite) isn’t in my opinion a rational answer. My answer is self host vaultwarden, self host your file storage, and choose between Proton and Tuta for mail, and use your own domain name so you can take your email address with you should you move.
In my opinion No-one should ever store any form of personal data implicitly on someone else’s computer.
Oh i have vaultwarden as well nowadays
You seem to imply you also want to selfhost some email service. But that’s sadly one of the few things that will always be better at a trusted third party email provider.
Besides that it seems the most important thing you want is pure data storage, and that kind of selfhosting is not hard. In many cases one would not even consider it as part of “selfhosting” as it can be as simple as a local NAS or external HDD.
So my question is what do you actually want to accomplish? Because I think for a lot of your concerns you don’t even need to go and host something.
I don’t want to replace email, per se. More so, the services that come along with it. Stuff like photo storage and sync, notes and reminder sync, calendar sync, and a lot of the hundreds of other small things that I have just been leaving up to Google that would cause a huge problem for me if I lost access.
I have the password situation handled with KeePassXC on my PCs and KeePassDX on my phone, but that’s about as far as I have come as of yet.
If you want to start cheap, I can recommend you to use an old notebook. In my opinion it’s the perfect home server for beginners.
- It’s cheap (most people have an unused laying around anyway)
- If it’s old enough to still have a dvd drive, you can replace it with a second sata ssd. There are cheap frames for this available.
- it has a battery, so it can shutdown if there is a power outage
- It’s slim. You can just throw it on your closet and forget about it
Most services don’t need much. So it’s just fine if your “server” is like 10 years old. My first notebook server had 2 cores and 4 GB ram and it run Proxmox with like 10 lxc containers just fine.
Awesome suggestion! Thank you.
The only thing to watch out for using a laptop that is plugged in 24x7 is the battery. Battery management systems are generally pretty good, but Li-ion batteries can fail catastrophically. As long as you make a point to check on it periodically it’s probably fine.
I’m using an old laptop as a local interface for my network setup, since its in my basement, and I actually pulled the battery out entirely since I have a beefy UPS powering everything. Paranoid, maybe, but a Li-ion battery sitting on top of my equipment rack could do a ton of damage if it were to fail someday.
If you have systems or services you’re dependant so strongly, always have an backup / emergency access. 3rd party or self hosted.
My 5c but I think you agree.
Point being as a decades old it professional I see design more important as the detail implementation.
Indeed, I do agree but I’ve never done anything close to this magnitude so it is kind of intimidating for me. It is a learning process though!
I’d recommend not to go containerized but that can start a flame war. I would think it easier. But best to stick to the recommended beginner tutorial that someone else posted and go along from there.
Then ask questions on the way.
Native vs containerized really depends on what it is going to be doing tbh. If it’s just downloading and/or moving files around, containerized is fine. And having your docker-compose.yml files saved somewhere external will make future hardware upgrades/recovery much easier.
There is certainly some learning curve to figure out the quirks of a compose file, but the nice part is that most services will post an example compose file for you to edit as needed. And that means learning it is basically just a matter of reading the example files and figuring out what the different fields mean; yaml is extremely easy to read, even for someone who has never looked at it before. You may have some fringe cases that need a deeper dive, but the vast majority of setups are basically just a matter of “copy the example compose, edit the volumes as needed, and fuckin send it.”
Yes indeed. However as a beginner I think it’s far easier just to install and run a daemon. But maybe that’s just me. And of course if the intended way of running it is only container by default.
I will probably get flogged by this answer but here it goes:
I’d throw you right into the deep end: get a spare machine (an old laptop or PC) and install proxmox on it. Play around, breaks shit, delete the container/VM and start over.
Grab stuff from the Community Helper Scripts and see new stuff, try alternatives, see what works for you and don’t be afraid of breaking stuff.
It takes a bit longer and some basic concepts might fly over your head, but the stuff you learn like this, you learn by heart.
It’s been a few years since I started tinkering with a laptop with a busted video output circuit. Now I serve NextCloud and Immich to my family, keep receipts and documents neatly organised on Paperless, have a decent arr stack and a bunch of extra goodies. All from “a PC without video? Might as well make a server” now with a proper machine with several drives on ZFS pools, health checks and redundancy.
Its a helluva rabbit hole.
Isn’t that how everyone does it? 😬
I didn’t start with a spare, so by the time I was semi-reliant on my self hosted stuff, a breakage was an issue. Also I started with bare Linux, then CasaOS. There was no easy rollback from snapshot/restore backup like on proxmox
I was unaware that those Community Helper Scripts existed! They should definitely be helpful at some point down the road!
Good luck!
I really wish people would realize the level of dependency, and thus leverage, these companies have encouraged us to give them, before they learn it first hand.
Yeah, it was dumb. I should have thought about it long before what occured, but I didn’t. But, in the end, I definitely learned my lesson.
You’re not dumb, we are all being brainwashed into sticking our asses in the air and convinced we won’t get fucked.
Start with a nas, the rest will naturally come when you try to access your data for outside, or organize your data, or save more data types to your nas.
Your nas should be the central device and you build the rest around it.
Now, The question is, which nas? I would recommend synology, they are not too performance, a bit expensive and the company is lately doing suspicious moves, but the sw and the hw are rock solid and they are quite good for beginners from almost all angles. Extra point for how many howtos and tutorials are present in internet.
Once you are comfortable with them, you will realize the rest
Unless you have experience with ethernet equipment and such it is probably better to start with some hosted service of an open-source app like Nextcloud or Immich or (slightly more advanced) a VPS somewhere. Doing it immediately from home with your own server has a steep learning curve.
Thank you for the advice! After I get a firm grasp of the concepts at hand, I will look into NextCloud and Immich.
At first, you have to decide what do you need. You can selfhost almost everything, but in my opinion there is no need to do so.
Second thing is hardware to host it. I saw a few comments recommending NAS. It is of course good thing, but my suggestion is just building your own NAS. You need only decent computer to do it.
The easiest way is just installing TrueNAS on it - with that you can setup file sharing and your apps via docker.
But what apps would you need/want? I can recommend a few from my stack:
- vaultwarden - for storing passwords, 2FA codes
- immich - for storing photos, videos, autoupload from phone
- adguard - for getting rid of ads, tracking They are really easy to deploy.
As an alternative to file shares via SMB, nextcloud is really good option. It’s google drive on steroids. Also includes photo gallery with great app on android/ios with autoupload option.
Nice! Yes, photo storage and backup as well as note sync, reminder sync, calendar, etc. are all very important micro services to me.
I think Nextcloud is a really good option for you. It includes everything you mentioned.
@MTZ saving this for later! I aim to go the same path soon :)
Good luck on your journey!
Yikes. Before you dip into any of the self-hosting, take and get a WD Gold drive - from Western Digital directly (wd.com) - do NOT go through Amazon or NewEgg or any third party merchant. Send in the warranty that goes with it and register the drive (this is for covering the off chance it’s a DOA unit) Then get a good quality enclosure to pop the drive into and take your time and back up EVERYTHING onto that new HD.
Don’t use an SSD.
You want a spinning platter drive, as this is backup only, so once it’s full with all of your content, it gets dated and labeled and popped into a drawer for safe keeping. If you have countless terabytes of data, get more drives and swap them into the enclosure, date and incrementally fill. A fine tip sharpie to note what’s on the drive is fine, or if you’re obsessively anal about it, make a spreadsheet with that info… If your drives are kept dry and stored with care they will last for DECADES…
The truth if being honest here - I’m a data hoarder and most of the stuff I’ve tucked away since I first came online (in 1999) is now on drives that I maybe spin up once a year. I used to have the notion that it was critical that all my shit was accessible all the time and I ended up dropping money on networked storage… and over time, realized that as long as I knew where the files were, DID have the most important stuff - family photos and scans - tucked away not only in long term storage, but on multiple drives in multiple machines, (home, work, laptop) it was okay not have it served up instantly.
Just reading your post made me go cold inside - I can only imagine what you were going through until it got sorted. From a bonafide old school data hoarder… Please, back your shit up locally. Use enterprise drives.
Then sort a self-hosting soultion.
Several detailed, easy to understand and very good pieces of advice! Thank you! I have definitely saved your comment for referencing throughout this process!
Which software do you use for backups?
I do my backups manually.
As I have run unsuported Mac installs for the last 20 years, I started a long time ago, automatically partitioning my OS drives and making storage volumes to work off of.
The storage volume in the computer will have subfolders for the type of data - music, video, photos, etc.
When my storage volumes fill, I will pull my latest backup drive out of storage, hook it up then go into each storage subfolder, sort by date and add everything that’s newer than what’s in the backup drive. (which is actually how Apple’s Time Machine backups work - incrementally sorted by date - but I’ve had this method since the start, so I just stuck with it)
I just make sure to take note of how many files/folders I’m adding to the backup drive and note what it has at the start, then at the end, as a double-check of it all, before I clear the storage drive on the computer. (I did not do this and lost almost a years worth of music rips, waay back in 2003. Rebuilt the music I lost then iTunes threw a wobbler and lost the library for me. FML…)
The longest backup will ALWAYS be the initial one if you’re dealing with a first time backup. The rest, once you work out how to organize your files, is academic.
What I’ve found is that your tastes will change, you grab content you think you’ll want to hold onto forever… and then years later, you realize it’s low-bitrate, low-resolution, too pixellated… whatever… and you decide to delete it.
With the software doing the backups for you - it’s too easy to just let it rip and go have dinner while it works and you end up with files that you’d otherwise get rid of. Part of being a data hoarder is not keeping everything forever. There’s a ton of garbage online. Tastes change as you get older… You want to curate that shit so you can keep what’s most important - like family stuff.
And really good porn.
I’m 100% self taught & was in exactly the same place. I’d never used Linux before I got my first Pi. I spent a bit of time trying to familiarise myself with & made some notes regarding command line (notes I still rely on).
There’a so many ways to achieve the goal, you’ll eventually find a way that works for you. My personal preference was Docker/Docker Compose deployed via Portainer.
Even that was confusing. Until I found this excellent video on how to read Docker requirements & apply them step by step into Portainer. He explains slowly & methodically exactly what he is doing & why.
Portainer is a method of handling Docker stacks/containers via a web UI. Both Docker & Portainer are simple to install.
It’s easier to use Docker Compose files and/or .env (environment variable) files (both are even simpler to deploy through Portainer) but this video taught me what was going on & gave me confidence to have a go. What attracted me to Docker is you can easily remove stacks/containers if/when you make a mess rather than wiping the drive & start again, which is how I went about things initially.
This gave me the tools to set up Nginx Proxy Manager & I never looked back.
As you’ve realised, a robust backup solution is essential (plus off site backup for particularly important stuff) as things will inevitably go wrong along the way (I see Borg, Restic mentioned often, I went for Kopia).
I can’t recommend highly enough making detailed notes along the way, I rely on Joplin.
If you start using Docker, dont fall into the trap of using the “latest” tag. If you know the version number you’re running its far easier to re-deploy if an update is bad.
Enjoy your new time consuming, teeth gnashingly frustrating …and yet rewarding hobby 👍
I had something similar happen with Google a few years ago. Even though I had my password and access to my email they decided I was trying to hack my own account and locked me out. Like you I immediately started to look for other solutions.
Syncthing file sharing is really easy to install and use. There are no ports to configure on your router and everything is encrypted in transit. I have my phone’s DCIM directory set up to sync to my home server and PC so new photos are backed up and available everywhere in a few seconds. I installed Syncthing intending to move to another solution eventually, but it works so well (aside from one or two files that occasionally don’t sync) that I’ve just stuck with it.
For passwords Keepass & KeepassXC work really well on just about every platform. I share the password file using Syncthing and in years of doing this I’ve never had a problem that I didn’t cause myself and those were minor.
You can get both of these up and running with very little effort and quickly limit your reliance on Google, then move to other solutions if you find they’d work better for you.
In the time since this happened, I have set up KeePassXC on all of my PCs as well as KeePassDX on my phone, and taken all of my passwords 100% out of Chrome.
I’ll absolutely look into SyncThing! I’ve heard of it many times, I just haven’t used it myself yet.
Thanks for the info!
Syncthing is incredible. I use it on my devices, and everything is also backed up to my NAS.
Second this, Syncthing rocks. Only ever have rare, minor problems in day-to-day use
