For those of you who don't know, Linux From Scratch is a project that teaches you how to compile your own custom distro, with everything compiled from source code.
What was your experience like? Was it easier or harder than you expected? Do you run it as a daily driver or did you just do it for fun?
I did Gentoo Stage 1 (which was very similiar to what you plan to do) in 2005 with a shitty laptop. 24 hours until I had a working shell compiled. A whole week until I had a graphical desktop working properly. Stupid me didn't have enough and did it again in 2013 with better hardware within just 36 hours to the desktop.
If you seek a challenge that leaves you with angelic patience once you've overcome the never ending rages you'll encounter to push through to the end against all odds, lots of errors, bad documentation, dependencies from hell AND keeping it running, which will inevitably raise your patience muscles strength again and again, then yes, do it. Just accept that at some point, something will break inside you.
PTSD from the "before times" of Linux is the reason I only run stable builds with UI installers now. Checking a list of dependencies is not how you should spend your free time. Even if you pull it off and install it from scratch; who are you going to boast about it to? US?!
That was the worst part. You have this super-optimized build that you somehow managed to make work with an ungodly amount of personal time, effort, blood and tears. It will only work so long as the hardware survives and on this machine only, nowhere else. Any update can knock your build down, making you work on debugging anywhere from a few minutes to full-on weeks.
So you have a system that works at best as well as any other system which you could get flying within an hour with only a few clicks in the installer.
That's it. That's what you've worked for and need to continue working for.