Lucee admin debugging logs new scope dump feature needs setting to disable it.
Description
Activity

Michael Offner 8 February 2019 at 13:54
we remove the feature completely for the moment, for more details consult LDEV-1990
Zac Spitzer 8 December 2018 at 00:08
Being able to enable debugging logging per session would be really useful

Bruce Kirkpatrick 18 November 2018 at 05:43
I've modified the lucee debugging log feature so that it is only able to capture request data when ?luceedebug=1 is in the url. It is important to me that Lucee doesn't collect data for any request I don't want it to. I don't want to just have a way to filter the display, I must be able to prevent capture, so that's what I did.
I also prevented Lucee from storing queries for the debugger since that can cause the JVM to run out of memory too easily. I understand this breaks the unused columns feature, but it's worth it to me to be able to debug in production. I can use the debugger in production now without a risk to server stability.
I have my production fork here right now. I got it caught up to the current snapshot. This is the first time I've run 5.3 in production. 5.3.2.16 seems ready for release for everything but the debugger.
https://github.com/brucekirkpatrick/Lucee/tree/5.3.2.16-custom
Lucee hasn't failed once since resolving the release bug the other day.

Bruce Kirkpatrick 28 October 2018 at 22:18
I worked on Lucee core myself to resolve my complaints with the debugging features. The commit you did would fix the main problem, but there are still several problems with the Lucee request debugging feature.
I have public fork of Lucee and built a series of changes in this commit that resolve my complaints with the new debugger. There were also bugs with link colors, and the filter, that I didn't report but I fixed in my version by changing or removing the code.
https://github.com/brucekirkpatrick/Lucee/commit/26f0a2cfeb39fde503a5e8c94c862d254fd811ea
Feel free to incorporate anything that looks interesting to you.
The new filter feature seems incomplete or confusing. It isn't a problem how it used to be because it only shows functions that are slower and there aren't too many because they were already grouped to begin with. Grouping the files by function and hiding the method is a bad choice, since the most important information is the method name and how long it took. I don't want to dig to find the function name. A "show more" button with display none would be a simpler way to handle that if you think they are excessive sometimes or maybe javascript based table pagination.
I did summary statistics table for the queries to be able to remove the duplicates by name. I think you should consider using code like that instead of the current query log, and this kind of cfml code with arrays is better then using query of queries like you have done in the execution filter stuff. QoQ should probably be made into a feature we can turn off in a future version of the language since I wouldn't accept anyone writing code like that at my company.
I am not interested in using the query usage feature at the moment because it puts a burden on the debugging to be able to keep queries in memory longer then it normal would, so I just disabled it, but a little more work and I could keep it as an option still. I was pleased however with how efficient the sql and execution time caching code was, so I have temporarily made QueryResultEntryImpl set qr to null, so I don't have the problem, but I can still log the basic query info safely. It would make more sense to only store the query if the query usage feature is enabled instead of always, because you aren't even using it when usage is off. I guess I'd have to pull that flag out of the static cfmlengine object and do something in PageContextImpl again like I did for the existing debugging flag. My other changes don't require this to happen though.

Bruce Kirkpatrick 26 October 2018 at 13:36
Zac,
When I was using this feature, the java heap size doesn't change enough to show there is a deep copy of application scope, probably not a problem for at least that scope.
I also don't think the request dump even works and I couldn't use it if it did, since I put a ton of objects in request as well.
If there was an alternative to cfdump that was written in non-blocking code that could stream the response back a little at a time, it would protect the server better, but it would still be slow, and might crash the browser instead. I don't like how non-blocking features of Java/Lucee are so dangerous to server stability mostly due to high memory usage. I always have to split objects into smaller pieces to deal with it, but I've been working on some non-blocking Java code instead lately.
Also debugging logs will hold on to references to hundreds or thousands of queries if the request ran that many. These queries are usually garbage collected during the request and safe in the real application, but they become a way to crash the server when query debugging is enabled. It has always been impossible for me to use safely use railo/lucee query debugging in production because it holds on to these objects too long, and bulk admin tasks will definitely crash lucee if it can't garbage collect them quickly. It would be better again for the sake of stability if the analyzing of the queries was done at runtime, but that thing that counts references couldn't work that way. Just storing the SQL statement and how long it took is more important to me, then which columns were referenced. I'd build it to optimize safe performance instead of creating features that can make it unstable.
Also, I should be able to store debug logs for only one 1 ip address, but the design of the debugging doesn't operate that way. You have to turn on debugging for all requests, and then you can limit display of debugging to ip address. This is a flaw. It should allow debugging logs to only be stored for the IP address too. This would help a great deal in making the debugger "safe" to use in production since I can avoid debugging my bulk admin jobs if I know they will crash the JVM due to high memory usage while still being able to debug the other things I'm trying to fix problems with.
It would also make sense for the developer to be able to set a maximum number of queries that are logged and analyzed per debug log entry. This might allow query debugging in production to be safe to turn on as well. Many times when there are hundreds of queries that is because there is a loop and most of that data is the same anyway, so I only really need to measure how many times the query of the same name was referenced, and perhaps its average runtime.
If the Lucee debugger was rebuilt to have these features, it would be truly awesome.
Details
Assignee
UnassignedUnassignedReporter
Bruce KirkpatrickBruce KirkpatrickNew Issue warning screen
Before you create a new Issue, please post to the mailing list first https://dev.lucee.org
Once the issue has been verified, one of the Lucee team will ask you to file an issue
Affects versions
Priority
Blocker
Details
Details
Assignee
Reporter

New Issue warning screen
Before you create a new Issue, please post to the mailing list first https://dev.lucee.org
Once the issue has been verified, one of the Lucee team will ask you to file an issue
Hi,
I see the new debug log feature is able to load finally in 5.3.2.1. However, I see there is new feature that allows us to see writedumps of all the scopes. That's cool, except for it can't be disabled and it also dump the application scope automatically I think.
It is a huge mistake to have application scope automatically be loaded as a writedump in the debugging log. We have so much data in the application scope (I assume others would too), that just visiting that debugging logs page is enough to crash Lucee, or at least make it hang for an unreasonable amount of time. My application is showing readDebug.cfm taking 16 seconds and returning 17mb of data after first loading. It would continue to get bigger as more sites load until eventually it would always crash the JVM due to out of memory error.
Please allow enabling/disabling the new scopes features, or at least to let us disable application scope on action=debugging.settings. Hopefully, it isn't attempting to clone application scope per request either, since that would also crash the server.