Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

random crashes lucee 5.3.2.77 under load

Description

Hi,
we are experiencing occasional random restarts of lucee containers. The error message is the same as in LDEV-2077, which should be fixed in the version we use (5.3.2.77). It seems to happen if the system is under load, but appears to be somewhat rare (2 times so far today).

Exception in thread "http-nio-8888-exec-6" java.lang.IllegalMonitorStateException at java.base/java.util.concurrent.locks.ReentrantLock$Sync.tryRelease(ReentrantLock.java:149) at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1302) at java.base/java.util.concurrent.locks.ReentrantLock.unlock(ReentrantLock.java:439) at java.base/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:465) at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:89) at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:33) at java.base/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1053) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.base/java.lang.Thread.run(Thread.java:834) 24-Jun-2019 08:58:09.459 INFO [Thread-7] org.apache.coyote.AbstractProtocol.pause Pausing ProtocolHandler ["http-nio-8888"] 24-Jun-2019 08:58:09.461 INFO [Thread-7] org.apache.coyote.AbstractProtocol.pause Pausing ProtocolHandler ["ajp-nio-8009"] 24-Jun-2019 08:58:09.466 INFO [Thread-7] org.apache.catalina.core.StandardService.stopInternal Stopping service [Catalina] 24-Jun-2019 08:58:09.473 INFO [Thread-7] org.apache.catalina.core.StandardWrapper.unload Waiting for [1] instance(s) to be deallocated for Servlet [CFMLServlet] 24-Jun-2019 08:58:10.475 INFO [Thread-7] org.apache.catalina.core.StandardWrapper.unload Waiting for [1] instance(s) to be deallocated for Servlet [CFMLServlet] 24-Jun-2019 08:58:11.475 INFO [Thread-7] org.apache.catalina.core.StandardWrapper.unload Waiting for [1] instance(s) to be deallocated for Servlet [CFMLServlet] 24-Jun-2019 08:58:17.158 INFO [Thread-7] org.apache.coyote.AbstractProtocol.stop Stopping ProtocolHandler ["http-nio-8888"] 24-Jun-2019 08:58:22.160 WARNING [Thread-7] org.apache.tomcat.util.net.AbstractEndpoint.shutdownExecutor The executor associated with thread pool [http-nio-8888] has not fully shutdown. Some application threads may still be running. 24-Jun-2019 08:58:22.160 INFO [Thread-7] org.apache.coyote.AbstractProtocol.stop Stopping ProtocolHandler ["ajp-nio-8009"] 24-Jun-2019 08:58:22.233 INFO [Thread-7] org.apache.coyote.AbstractProtocol.destroy Destroying ProtocolHandler ["http-nio-8888"] 24-Jun-2019 08:58:22.241 INFO [Thread-7] org.apache.coyote.AbstractProtocol.destroy Destroying ProtocolHandler ["ajp-nio-8009"]

Environment

Kubernetes Cluster
using Docker image lucee/lucee:5.3.2.77-tomcat9.0-jre11

Activity

Show:

Michael Offner 2 March 2020 at 09:51

Michael Offner 2 March 2020 at 09:46

Dominic Watson 24 January 2020 at 10:45
Edited

I repeated the test I did above, with a slightly differently modified Lucee code (adding a sleep that exaggerated the issue).

I could not reproduce the issue I had previously (looking good on that front) thumbs up . I also performed the test without modifying the code, similarly no problems.

Michael Offner 24 January 2020 at 09:53

we have rewritten with the latest Lucee 5.3.5 how thread stoping works, can you still reproduce the issze with the latest version?

Dominic Watson 16 September 2019 at 22:31

Pull request submitted to allow sys admins to disable thread.stop() by setting a java property -Dlucee.confallow.thread-stop=false. This effectively stops request timeouts from working so is not set by default in the PR.

This suits us just fine, but is there any other way to make request timeouts work for people that want them (and don’t want stability issues)?

Fixed

Details

Assignee

Reporter

Priority

Fix versions

New Issue warning screen

Before you create a new Issue, please post to the mailing list first https://dev.lucee.org

Once the issue has been verified, one of the Lucee team will ask you to file an issue

Sprint

Affects versions

Created 24 June 2019 at 10:21
Updated 5 May 2023 at 07:09
Resolved 2 March 2020 at 09:53

Flag notifications