CFQuery Lazy="true", i.e. SimpleQuery is able to consume infinite heap memory during request

Description

I reviewed the new cfquery lazy="true" 5 years ago in depth and discussed it on the forum:
https://groups.google.com/forum/#!topic/railo/3xzhi6A8-94

One of my last comments on the forum was that lazy queries are able to consume all the heap memory during the request, because you gave us no way to release the query from memory. You also didn't automatically close the query when it is done looping, and choose to implement it only when PageContextImpl release function fires. This makes it very easy to run out of heap and/or crash the JVM by accident when you enable this feature everywhere. We've been forced to never use this or use it sparingly just because we can't manually close the resultset like you can in other languages.

Fortunately, the solution is absolutely easy to implement in Lucee core. I'm providing a commit for QueryClose(queryName), which will close a query if it isn't already and silence any SQLException, since we really don't need to know anything about whether it was already closed or not.

Once I used this new function in my cfml test code, the Java heap was able to garbage collect during the request, even though I had lazy="true" on. Amazing! Exciting! Please include this function in the next snapshot for the benefit of all!

lazy="true" continues to make data access up to 3 times faster especially on select * queries. An optimal Lucee application, should encourage manually closing connections for best memory usage/performance. This is the main feature that can make Lucee separate in performance from Coldfusion and surely it is not hard to remember to type QueryClose(queryName). You could even track implicitly closed lazy queries in the debugging log to help people fix their memory usage mistakes in a future version where people start using this and make more mistakes. It would be easier for people to know this feature is there if it was recommended in the admin too since this offers a bigger boost then disabling scope cascading.

Environment

None

Attachments

1

Activity

Show:

Michael Offner 28 August 2019 at 06:14

see comment in ticket

Zac Spitzer 27 August 2019 at 10:17

here you go, query.rowData not supported with cfquery lazy=true

Michael Offner 26 August 2019 at 20:07

🤪 🍻

Zac Spitzer 26 August 2019 at 19:34

Not at my desk....

Michael Offner 26 August 2019 at 19:30

stacktrace?

Fixed

Details

Assignee

Reporter

Priority

Fix versions

New Issue warning screen

Before you create a new Issue, please post to the mailing list first https://dev.lucee.org

Once the issue has been verified, one of the Lucee team will ask you to file an issue

Sprint

Affects versions

Created 26 November 2018 at 00:26
Updated 8 May 2020 at 19:08
Resolved 28 August 2019 at 06:14