CFZip terrible performance with large (200MB+) zip archives

Description

Appending 3 files (about 2K each) to a 220MB archive using CFZip took about 1/2 hour

System resources checked while waiting - plenty of free physical memory, minimal paging

While the CFZip tag was running, I created a duplicate archive using the command line 'zip' utility (this took a few seconds) and appended the same 3 files (this took a couple of seconds)

Stack trace captured while CFZip was running...
0: ........java.io.RandomAccessFile.open0[RandomAccessFile.java:-2]
1: ........java.io.RandomAccessFile.open[RandomAccessFile.java:346]
2: ........java.io.RandomAccessFile.<init>[RandomAccessFile.java:260]
3: ........java.io.RandomAccessFile.<init>[RandomAccessFile.java:215]
4: ........net.lingala.zip4j.io.outputstream.SplitOutputStream.<init>[SplitOutputStream.java:51]
5: ........net.lingala.zip4j.tasks.AddStreamToZipTask.executeTask[AddStreamToZipTask.java:42]
6: ........net.lingala.zip4j.tasks.AddStreamToZipTask.executeTask[AddStreamToZipTask.java:18]
7: ........net.lingala.zip4j.tasks.AsyncZipTask.performTaskWithErrorHandling[AsyncZipTask.java:42]
8: ........net.lingala.zip4j.tasks.AsyncZipTask.execute[AsyncZipTask.java:36]
9: ........net.lingala.zip4j.ZipFile.addStream[ZipFile.java:396]
10: ........org.lucee.extension.zip.tag.Zip.add[Zip.java:764]
11: ........org.lucee.extension.zip.tag.Zip.actionZip[Zip.java:618]
12: ........org.lucee.extension.zip.tag.Zip.doEndTag[Zip.java:862]
[CF code calling <CFZip>]

Environment

VirtualBox VM/CentOS 7/Java 11/Apache

Activity

Tim Parker 9 July 2021 at 22:02

oops.. sorry... I hit 'send' before I was done

When everything is done, the resulting zip file is 244MB - I used
command-line tools to unzip it and then re-zip the results to ensure
that it wasn't just poor file system performance - each of these actions
took about 10 seconds, so that (at least partially) removes blame from
the file system

Also, similar actions with ACF perform as expected on a similar VM
instance, so it seems fairly clear where the problem is

Tim

Tim Parker 9 July 2021 at 21:58

I manually extracted the 2.9.0 jar from the lex file and put it into the
.../tomcat/lucee-server/context/lib directory (after moving the 2.1.2
version to ../old-lib) - this resolved the crash, but the performance
still isn't what it should be - 1.5 seconds to create a new zip with 4
small PDFs, 27+ seconds to add 113 (mostly) small files to an existing
zip file, 35+ seconds to add 3 small (2K each) XSL files to the archive...

Zac Spitzer 9 July 2021 at 21:14

Odd, looks like the wrong zip4j jar is being used, should be zip4j 2.9.0 not 2.1.2

Tim Parker 9 July 2021 at 21:11

no joy... same error

Tim Parker 9 July 2021 at 20:50

sorry... here's the missing part:

Caused by (class java.lang.NoClassDefFoundError):
Message:
net/lingala/zip4j/io/inputstream/NumberedSplitRandomAccessFile

====
Caused by (class java.lang.ClassNotFoundException):
Message:
net.lingala.zip4j.io.inputstream.NumberedSplitRandomAccessFile not found by
zip4j.2.1.2 [47]

Thread 'ajp-nio-8009-exec-5' [ID=42, State='RUNNABLE']
... request started at 2021-07-09 16:45:12.585[running 12.833 seconds]
... Request (POST) from 192.168.1.157
... Server: cs108luceerm:443 Port:443 Script:
'/issues/loader.cfm' csModule:'utilities/generic-http-callback'
[Request made from console]
...request mode: : read
minimized stack with 94 entries
0:
........org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation[BundleWiringImpl.java:1597]
1:
........org.apache.felix.framework.BundleWiringImpl.access$300[BundleWiringImpl.java:79]
2:
........org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass[BundleWiringImpl.java:1982]
3: ........java.lang.ClassLoader.loadClass[ClassLoader.java:521]
4:
........org.apache.felix.framework.BundleWiringImpl.getClassByDelegation[BundleWiringImpl.java:1375]
5:
........org.apache.felix.framework.BundleWiringImpl.searchImports[BundleWiringImpl.java:1618]
6:
........org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation[BundleWiringImpl.java:1548]
7:
........org.apache.felix.framework.BundleWiringImpl.access$300[BundleWiringImpl.java:79]
8:
........org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass[BundleWiringImpl.java:1982]
9: ........java.lang.ClassLoader.loadClass[ClassLoader.java:521]
10: ........org.lucee.extension.zip.tag.Zip.getZip[Zip.java:883]
11: ........org.lucee.extension.zip.tag.Zip.actionZip[Zip.java:597]
12: ........org.lucee.extension.zip.tag.Zip.doEndTag[Zip.java:862]
13: CFM....{calling code}

Fixed

Details

Assignee

Reporter

Priority

Labels

New Issue warning screen

Before you create a new Issue, please post to the mailing list first https://dev.lucee.org

Once the issue has been verified, one of the Lucee team will ask you to file an issue

Affects versions

Created 17 June 2021 at 17:13
Updated 20 May 2024 at 13:09
Resolved 20 May 2024 at 13:09