Non-ASCII character support

[mertcanaltin]

Fixes #45706
Fixes #46508

• Is the character's ASCII code greater than 127? (In this case, the function returns true)
• Is the character a number? (In this case, the function returns false)
• Is the character a "#" symbol? (In this case, the function returns false)
• If no conditions are met, the function returns true.

[mertcanaltin]

test

 const  isLiteralSymbol = (char) => {
    const code = char.charCodeAt(0);

    if (code > 127) {
      return true;
    }

    if (char >= '0' && char <= 9) {
      return false;
    }

    if (code === 35) {
      return false;
    }

    return true;
  }

output

isLiteralSymbol('أهلا')
true

[mertcanaltin]

@anonrig I applied the changes, thank you very much for the review 🚀

[anonrig]

@manekinekko Can you review this?

[MoLow]

the change LGTM, but this seems to break quite a few tests

[manekinekko]

Could you:

1. group these into 2 groups: literals (true) and non-literals (false)?
2. add more samples of non-Latin characters? you can cherry-pick from this list here.

[mertcanaltin]

@manekinekko thank you so much i will fly here 🚀

[mertcanaltin]

@manekinekko would that be healthy

{
  const literals = ['A', 'a', '-', '+', 'أ', 'ت', 'ث', '讲', '演', '講'];
  const nonLiterals = ['0', '#', '\\', '+', '-'];
  
  literals.forEach((literal) => {
  assert.strictEqual(isLiteralSymbol(literal), true);
  });
  
  nonLiterals.forEach((nonLiteral) => {
  assert.strictEqual(isLiteralSymbol(nonLiteral), false);
  });
}

[mertcanaltin]

I'm so sorry I triggered you all to review, sorry for the extra notifications :/

[mertcanaltin]

@manekinekko I sent the edits thank you very much

[BridgeAR]

Should we really accept zero width characters as acceptable input? I would skip all of them. We also already have a function to check for these:

node/lib/internal/util/inspect.js

Lines 2388 to 2398 in ab064d1

	const isZeroWidthCodePoint = (code) => {
	return code <= 0x1F || // C0 control codes
	(code >= 0x7F && code <= 0x9F) || // C1 control codes
	(code >= 0x300 && code <= 0x36F) || // Combining Diacritical Marks
	(code >= 0x200B && code <= 0x200F) || // Modifying Invisible Characters
	// Combining Diacritical Marks for Symbols
	(code >= 0x20D0 && code <= 0x20FF) ||
	(code >= 0xFE00 && code <= 0xFE0F) || // Variation Selectors
	(code >= 0xFE20 && code <= 0xFE2F) || // Combining Half Marks
	(code >= 0xE0100 && code <= 0xE01EF); // Variation Selectors
	};

[mertcanaltin]

I didn't accept zero-width characters @BridgeAR i will update the code like this

  if (typeof char !== 'string' || util.inspect.isZeroWidthCodePoint(char)) {
      return false;
    }

can I do it like this?

[cjihrig]

@mertcanaltin I think that would be ok.

[MoLow]

CC @nodejs/test_runner @manekinekko additional reviews will be appreciated

[MoLow]

@anonrig can you please dismiss your review/approve?

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/49692/

[nodejs-github-bot]

Landed in 3c6547f

[mertcanaltin]

🎉