BigDecimal Number Parsing Fails with Invisible Unicode Characters

Description

In Lucee 6.2, numeric string parsing fails when the string contains invisible Unicode characters. This is due to the migration from Double to BigDecimal as the default number type for improved precision.

After upgrading from Lucee 6.1 to 6.2, number parsing has become more strict due to the internal switch from Double to BigDecimal. The BigDecimal constructor throws a NumberFormatException when a string contains invisible Unicode characters that were previously accepted by Double parsing.

The error message misleadingly suggests that a valid number string (e.g., "17") cannot be parsed, when in fact the string contains invisible Unicode characters not shown in the error message.

For example, when trying to parse a string that appears to be "17" but contains a zero-width space:

new BigDecimal("17\u200B", MathContext.DECIMAL128);

This throws:

NumberFormatException: Character [invisible character] is neither a decimal digit number, decimal point, nor "e" notation exponential mark.

And our error handling converts this to:

CasterException: cannot convert string [17] to a number; [error message]

Steps to Reproduce

  1. Use Lucee 6.2

  2. Try to parse a string that visually appears to be a number but contains invisible Unicode characters

  3. Example code:

<cfscript> // This string looks like "17" but has an invisible character str = "17" & chr(8203); // Adding zero-width space try { num = str * 1; // Implicit conversion will use BigDecimal in 6.2 writeOutput("Success: " & num); } catch(any e) { writeOutput("Error: " & e.message); } </cfscript>

Expected Behavior

Number parsing should either:

  1. Strip invisible Unicode characters before parsing

  2. Provide clearer error messages that indicate the presence of invisible characters

  3. Maintain backward compatibility with Lucee 6.1 behavior

Actual Behavior

Number parsing fails with a misleading error message that doesn't clearly indicate the presence of invisible characters.

Impact

High - This could silently break applications during upgrade from 6.1 to 6.2, especially those that process user input or data from external sources.

Environment

None

Activity

Pothys - MitrahSoft 
10 March 2025 at 13:24

I have tested this ticket with Lucee version 6.2.1.52-SNAPSHOT. Now, the whitespace and invisible Unicode characters are removed, and the correct number is returned. It works fine in the latest Lucee version.

Michael Offner 
10 March 2025 at 10:08

i added an additional fix and i added also test cases, please have a look
https://github.com/lucee/Lucee/commit/f3ccde777a554407c9cb627a02c73d50832a96c2

Pothys - MitrahSoft 
4 March 2025 at 13:00

I have investigated this ticket with Lucee version 6.2.1.45-SNAPSHOT. When I tried to execute the given test code, it still shows the error message without the invisible Unicode.

Fixed

Details

Assignee

Reporter

Priority

Fix versions

New Issue warning screen

Before you create a new Issue, please post to the mailing list first https://dev.lucee.org

Once the issue has been verified, one of the Lucee team will ask you to file an issue

Created 28 February 2025 at 17:15
Updated 10 March 2025 at 13:24
Resolved 10 March 2025 at 13:24