replaceNoCase not working with certain non ascii strings

Description

https://trycf.com/gist/faf488549f72d9a4324830c61ab8e8c0/lucee5?theme=monokai

<cfscript>
input = 'aaa bbb & İkra';
toReplace = 'aaa bbb & İkra';
newStr="aaa bbb;İkra";


function replaceNoCaseCustom(required string input, required string toReplace, required string newStr) {
    var offset = find( lcase( arguments.toReplace ), lcase( arguments.input ) );
    if ( offset == 0 ){
        return arguments.input;
    } else {
        if ( offset == 1 ){
           return arguments.newStr & mid( arguments.input, len( arguments.toReplace ) + 1 );
        } else {
            var s = left( arguments.input, offset - 1 ) & arguments.newStr;
            if ( ( len( arguments.toReplace ) + offset ) lt len( arguments.input ) ){
                s =s & mid( arguments.input, offset + len( arguments.toReplace ) ); 
            }
            return s;
        }
    }
}

dump(replaceNoCase(input, toReplace, newStr));  // aaa bbb & İkra
dump(replaceNoCaseCustom(input, toReplace, newStr)); // aaa bbb;İkra
</cfscript>

Linked issues

relates to

LDEV-153

replaceNoCase() fails with certain Unicode text

Activity

Show:

Michael Offner 20 February 2024 at 09:54

This is not really a fix because this not really can be fixed. Problem here is the conversion from upper case to lower case character visa versa. To make that properly, you need to define the locale of the string, because that influences the charset used and how the string properly get converted.

This “fix“ first convert everything to upper case, then it compares the byte size of the string before and after the conversion and if they differ it tries by converting everything to lower case. before it only did everything to lower case.

This solves all the examples we have so far, but in theory it does not resolve every possibility.

I also added the function lsLCase and lsUCase that takes a locale as input and can make the conversion accurate as long you give the correct locale.