I think the biggest issue is IP reputation. There are good products out there to do all the heavy lifting, which prevent most of the issue with good default configuration. But no matter how good your setup is, if your IP is/was known for sending spam once, you're on some list with the big providers. If you don't already have an IP for years with a solid reputation, you're going to go through a lot of "please check your spam" communication. I know cases where even neglecting measures such as using DKIM is okay if your IP is okay. The other way around: you need to implement basically every measure possible to even get started being recognized as a trustable sender.

I went down this road and reversed.

I use cloudflare for domain reg, fastmail as my mail provider and free Gmail as my mail client.

I've had issues with my emails going to spam with Gmail and Outlook, but in my 2ish years of self-hosting email, it never got to a point where it was banned outright. All I do is tell people to check their spam folders for my email and report it as Not Spam, haven't had issues with a ban.

Something that has really helped with rejections and/or spam was having a PTR (reverse DNS) record for your server, some providers like Gmail explicitly require it. It's something that I have to pay a VPS for in order to get. But I can have that VPS with that PTR record and tunnel everything through Wireguard to my home server, all without TLS termination on the VPS.

i just use aws SES, it's already very cheap

Stalwart mail is really good and easy to setup. I register a domain name for a few bucks a year and protect it with a Lets Encrypt certificate. Free and very rewarding!!