I'm pretty sure I just isolated and fixed a problem that Lucee has had for over 3 years. It may have even been there in Railo.
It may even be the solution for the complaint made in this old ticket:
It appears the code written for the isToBig() is not reliable or not accurate somehow. It attempts to guess if there is enough heap memory left to process a huge post request using the heap instead of reverting to the alternative behavior, which is a very efficient disk based method. The disk based approach only consumes a trivial amount of heap (like up to 1mb), whereas the isToBig is able to consume ALL the heap and accidentally throw java heap space through the inaccuracy of trying to guess how much memory is needed.
This can make Lucee very unstable / able to crash. This is a serious flaw in lucee, that was easy to fix.
If we are going to persist in allowing further optimization beyond the 1mb default, I suggest that value become an Lucee admin option on the Request page named "Limit Request Upload Size" with a comment describing the behavior. I'm talking about giving us access to modify this value in FormImpl.java.
DiskFileItemFactory(DiskFileItemFactory.DEFAULT_SIZE_THRESHOLD, (File) tempDir)
Obviously the user must also configure tomcat (or other server's) connector appropriately to allow a significantly larger post then default, which I have also done. I was able to upload a file that is larger then the heap, and I also monitored the heap during processing and saw it didn't use excessive memory anymore, so basically you had the solution there, but you weren't letting it run.
I'll submit my working commit in a moment.