wrong codepoints for 🏎, 🏍 and maybe more

[YuriGor]

I believe correct racing car emoji lives here
But in your json, it has different codepoints
here is comparison

The first car is from unicode.org and the second one is from the latest db/emoji.json
In the earlier version of db/emoji.json the racing car symbol was correct, but then something changed.
The same issue with a motorcycle - it also has \uFE0F excessive postfix
I think you need to check all the symbols if they have such distortion.

[mislav]

You're right to spot the discrepancy, but I don't think the presence or absence of FE0F represent an issue for anyone:

>> Emoji.find_by_unicode("\u{1F3CE}")
=> #<Emoji::Character:racing_car(1f3ce-fe0f)>
>> Emoji.find_by_unicode("\u{1F3CE}\u{fe0f}")
=> #<Emoji::Character:racing_car(1f3ce-fe0f)>
>> Emoji.find_by_unicode("\u{1F3CD}\u{fe0f}")
=> #<Emoji::Character:motorcycle(1f3cd-fe0f)>
>> Emoji.find_by_unicode("\u{1F3CD}")
=> #<Emoji::Character:motorcycle(1f3cd-fe0f)>

• Both emoji characters are equally discoverable regardless of trailing FE0F or not;
• 1f3ce-fe0f is a perfectly valid and Unicode-recognized codepoint sequence that represents the racing car emoji. So is 1f3cd-fe0f for motorcycle.

Is this affecting your apps somehow?

[YuriGor]

I use javascript lib based on your JSON
And this distortion does matter because there is a simple dictionary, where Unicode strings used as keys,
so it's impossible to get emoji metadata by key in some cases.
I found a temporal workaround, but it costs performance because of instead of direct access by key I need to iterate over whole emoji collection and compare keys one by one.

Here is an issue description in PR discussion. I tried to update that lib using the most recent JSON from your lib, but it stopped to work correctly in some cases.

[mislav]

@YuriGor Okay I have regenerated our JSON to not include the \uFE0F excessive postfix except for a few dozen characters where it's necessary (without the postfix, these characters would typically get rendered as text glyphs from the pre-color emoji era).

[YuriGor]

@mislav, thank you!

[xPaw]

I think this is actually incorrect. All these emojis are in Emoji section so they require \uFE0F to show up as an emoji.

For example, this breaks emoji-regex which follows the specification by matching \p{Emoji_Presentation} or \p{Emoji}\uFE0F.

Also see #164 (comment)

[wearhere]

My findings in #178 make me suspect that in at least some cases where this library applies \uFE0F, that codepoint is superfluous. Curious for your thoughts @xPaw.

[wearhere]

After reading the spec, I now see how it's probably ok and even good for this library to apply \uFE0F to all the "base" emoji (absent skin tone modifiers), to ensure that those emoji do render as emoji and not as text. The spec even seems to recommend that implementations "fully-qualify" emoji sequences.

When applying skin-tone modifiers to base emoji, on the other hand, the variation selectors need to be stripped, but this library doesn't support that yet. I've described what I think is a correct implementation of modifiers here. Comments and questions welcome.