Temporary .upload files persisted indefinitely when using CFThread.

Description

Normally, if you upload a file to the Lucee CFML server, the `tmp-*.upload` file automatically removed once the request is processed. However, it seems that if you spawn a `CFThread` tag during the request, it does two things:

1. It no longer deletes the `.upload` files automatically.
2. It creates a copy of each `.upload` file for each instance of `CFThread`.

I've shared my findings here:

https://www.bennadel.com/blog/3889-temporary-upload-files-are-duplicated-and-persisted-when-a-request-uses-cfthread-in-lucee-cfml-5-3-6-61.htm

I suspect this related to the request cloning behavior that I previously documented:

https://www.bennadel.com/blog/3757-lucee-appears-to-incur-request-cloning-overhead-when-spawning-cfthread-tags-in-lucee-cfml-5-3-3-62.htm

I can definitely understand persisting the `.upload` file beyond the request boundary since a `CFThread` tag may want to reference; but, the fact that it creates duplicates of the temp file seems "buggy."

Environment

  • Lucee 5.3.6.61

  • Linux (4.19.76-linuxkit) 64bit

  • Java 1.8.0_242 (Oracle Corporation) 64bit

Activity

Show:
Pothys - MitrahSoft
September 7, 2020, 11:42 AM

I've checked this ticket and confirmed the issue happened on lucee latest version 5.3.8.58-SNAPSHOT also. Yes It's no longer deletes the '.upload' files automatically when using thread. But it doesn't create a copy of each `.upload` file for each instance of 'CFThread' in my local

Ben Nadel
September 10, 2020, 11:20 AM

I’ve posted some notes on how we’re handling this at work: https://www.bennadel.com/blog/3891-deleting-temporary-upload-files-in-our-k8-operational-readiness-probe-in-lucee-cfml-5-3-6-61.htm

I theorize that the biggest problem is that Lucee’s background clean-up thread is not sorting the files before trying to delete them. In our case, I believe that is causing the background thread to error and short-circuit, not allowing it to keep up with the fast-growing volume of temp-files on our servers.

My custom code attempts to add some intelligence to the clean-up by sorting the files first:

Notice the sort = "dateLastModified ASC". This attempts to delete the oldest files first, which I believe is allowing the custom code to work more safely without creating errors.

Zac Spitzer
September 10, 2020, 11:30 AM

the purge logic also enumerates the directory over and over again LDEV-3051

Ben Nadel
September 10, 2020, 12:15 PM

Ha, fun

Assignee

Michael Offner

Reporter

Ben Nadel

Priority

Minor

Labels

Fix versions

None

Affects versions

Configure