error parsing large json object dumps out entire json string as exception message

Description

<cfscript> filename = Expandpath('.\big.json'); timerType= "outline"; sampleJson = 'https://raw.githubusercontent.com/ozlerhakan/mongodb-json-files/master/datasets/city_inspections.json'; // 24mb //sampleJson = "https://raw.githubusercontent.com/zemirco/sf-city-lots-json/master/citylots.json"; // 189mb function say(mess, mem=false){ echo(mess & "<br>"); if(arguments.mem) writeDump(getMemoryUsage()); cfflush(); } lock name="getBigJson" throwontimeout=true timeout=2{ if (not FileExists(filename)){ setting requesttimeout=1000; http url="#sampleJson#" path="#filename#"; say("sample json downloaded"); } else { say("sample json already downloaded"); } } say("go!"); json = fileRead(filename); //json = "[1,2.3]"; timer type="#timerType#" label="now" { say(now()); } say("now done!", true); s = getTickCount(); timer type="#timerType#" label="DeserializeJson" { x = DeserializeJson(json); x = ""; } say ("DeserializeJson #getTickCount()-s#", true); </cfscript>
at lucee.runtime.interpreter.CFMLExpressionInterpreter.interpret(CFMLExpressionInterpreter.java:215) at lucee.runtime.interpreter.CFMLExpressionInterpreter.interpret(CFMLExpressionInterpreter.java:193) at lucee.runtime.functions.conversion.DeserializeJSON.call(DeserializeJSON.java:52) at lucee.runtime.functions.conversion.DeserializeJSON.call(DeserializeJSON.java:48)

https://github.com/lucee/Lucee/blob/5.3/core/src/main/java/lucee/runtime/interpreter/CFMLExpressionInterpreter.java

this behaves differently, sometimes OOM, other times it throws a syntax error which then logs out the entire json string as the exception message.

this json sample is 24mb, it should never be all logged out. the exception message should be always be short and understandable.

it's three times worse, coz the message gets logged out three times https://luceeserver.atlassian.net/browse/LDEV-2781#icft=LDEV-2781

also, I think the memory isn't being released as the exception is kept in memory with debugging enabled

also when this runs successfully, the DeserializeJson timer output doesn't work either?

Environment

None

Activity

Pothys - MitrahSoft 7 April 2021 at 15:16

The issue was solved. Now the error message is sorted to small and separated into detail. So I mark this as deployed.

Zac Spitzer 1 April 2021 at 15:16

Pothys - MitrahSoft 2 April 2020 at 07:39

, Moved the CFML snippet to exception detail.

Zac Spitzer 2 April 2020 at 07:16
Edited

can you move the cfml snippet
to the exception detail?

i.e

InterpreterException("Syntax Error, invalid Expression [" + cfml.toString().substring(0,1024) + "]");

becomes

InterpreterException("Syntax Error, invalid Expression", "[" + cfml.toString().substring(0,1024) + "]");

otherwise sensitive information may be exposed

Pothys - MitrahSoft 2 April 2020 at 06:40

Added a fix for this ticket.

Pull Request: https://github.com/lucee/Lucee/pull/912

Fixed

Details

Assignee

Reporter

Priority

Labels

Fix versions

New Issue warning screen

Before you create a new Issue, please post to the mailing list first https://dev.lucee.org

Once the issue has been verified, one of the Lucee team will ask you to file an issue

Sprint

Affects versions

Created 1 April 2020 at 10:52
Updated 27 April 2022 at 17:00
Resolved 7 April 2021 at 15:16