My short opinion:

Video

h264 is the best option for compatibility. There have been free software encoders and decoders forever and IIRC all patents have expired. Basically every device you will encounter and every software system can play h264 videos and the encoders are fairly good.

AV1 is the best option for quality. It is completely free and is becoming widely supported. It will likely be supported for a long time as it is the first widely available high quality free codec. It is significantly better quality than h264 so will result in smaller files for the same quality or better quality for the same file size. Hardware decoding support has only really become common in devices that hit the market in the last few years. But most new devices will have hardware decoding.

Both of these are web-compatible as well which is nice.

Audio

Opus for lossy and FLAC for lossless are both some of the best codecs in their class, completely free and widely available. There are also both web-compatible.

DVD’s max out at about 580p (for PAL, NTSC is 480p), resolutions are measured by the number of horizontal lines of pixels (counted from top to bottom of video/screen), not vertical which at 4:3 square aspect ratio on dvds does tend to be 720 pixels (by contrast full resolution HD video’s number of vertical lines is 1920 while it’s horizontal lines are of course 1080, hence 1080p). You’re not the first person to be confused by this.

Professional encoders who fully understand the encoders and the schemes in use and care about not seeing artifacting or low quality would never intentionally go as low as 300mb for a feature length movie of even an hour. Yes there are people who do such things but they’re not well regarded and it won’t look even passable on anything larger than a phone screen.

Recognized quality groups that seek low sizes might get an animated feature (less bitrate needed due to lack of fine detail in animation vs real film) in SD quality down to around that. But for most live action content the sizes I see from the best of the best concerned with smaller release sizes are in the 900mb to 1.5GB range for 60-90 minute features.

300mb for a 90 minute live action feature even in SD is just not going to look good, some of the groups who get those sizes make them look even half-passable by running pre-filters in virtualdub that smooth, reduce grain and detail, etc before passing to the encoder. That kind of thing is way beyond anything you’re going to learn in a few youtube videos though, that’s advanced stuff with scripting.

Think about it this way, if you shoot for 1GB encodes with 265 or AV1 you can store over 900 movies on a 1tb drive which can be had for well under a hundred dollars.

I would like the best and fanciest algorithms to have least dataloss.

There is no magic that will get you where you want. If you want detail preserved you need more bitrate which translates to larger sizes. Modern codecs like HEVC and AV1 mean you need as much as 1/5th the bitrate you needed with old MPEG2/4 encoding schemes used on DVDs, that’s darn good savings but it has its limits.

Do as you will but anything live action (non-animated) significantly under 1000kbps average bitrate is going to look awful on a 1080p screen and much worse than what it would look like if you popped your dvd in the disc drive and played it from there.

Opus is fine if you’re not worried about compatibility and just playing on a computer.

I usually go for VP9 and opus in a webm container. This is widely supported, because it is Google sponsored, used on YouTube and supported by all browsers. As other people told you here, DVD are not 720p, they are 480 or 576 interlaced or progressive. You can find more details on Wikipedia.

AV1 is the best compressing standard for video, but it is slower in encoding than HEVC (usually). VP9 is the predecessor, which is compresses better than AVC and comes close to HEVC. Speed wise is comparable to x265.

Opus is the best audio standard, the only reason you may want to avoid it is for incompatibilities with older hardware players (in car radios, BluRay players,…).

Removed by mod

I encode everything with AV1 and Opus, although, as others have pointed out, there is the disadvantage that older (as in “not new”) devices won’t have hardware acceleration for those. I don’t care about that tho, because 1080p runs just fine on decent CPUs and I want to encode for the future, not yesteryear.

For AV1 I use 10bit, VFR, RF25. If you want, you can play around with the performance profile to trade file size for encoding speed.

For Opus I use 320 kbit/s fixed Bitrate and 7.1 downmix. Note that when dealing with audio and subtitles, you absolutely should include tracks of ‘unknown’ language! Otherwise they are just thrown away. And of course you should include every other language you care about. You can also choose to pass through Opus encoded tracks automatically.

I prefer MKV as container format, because from all I hear it’s flexible and robust. Pretty much everything else is set to default. I’d advise to safe your settings to a preset and to briefly verify encoded videos actually work and contain the desired tracks.

AV1 and VP9 are likely going to be your highest efficiency “free” codecs. AV1 is the way to go if you mean free as in free open source. It’s not very likely to be implemented in many TVs or set-top-boxes, but VLC/ffmpeg will be able to decode any of these. Webm uses vp8 or VP9 which are “free”(made by Google) but it’s just more specific settings for sharing online/viewing in browser.

H264/H265 has license fees for non-free software and hardware, but they will be your most widely supported option. H265 is approximately twice as efficient as h264 (meaning you can get the same quality of encode from half the file size).

Regardless of preset I think you can get handbrake to encode something reasonable from any of these codecs. Especially with DVD video you’ll be able to crank through videos with modern high efficiency codecs

I’d use HEVC/H265 anyway, always, the best quality for its size. And AAC for audio

https://en.m.wikipedia.org/wiki/X265

Don’t overcomplicate things.