Skip to content

build: Auto-load ICU data from --with-icu-default-data-dir#30825

Closed
sgallagher wants to merge 1 commit intonodejs:masterfrom
sgallagher:i18n-default-dir
Closed

build: Auto-load ICU data from --with-icu-default-data-dir#30825
sgallagher wants to merge 1 commit intonodejs:masterfrom
sgallagher:i18n-default-dir

Conversation

@sgallagher
Copy link
Contributor

@sgallagher sgallagher commented Dec 6, 2019 •
edited
Loading

When compiled with --with-intl=small and
--with-icu-default-data-dir=PATH, Node.js will use PATH as a
fallback location for the ICU data.

We will first perform an access check using fopen(PATH, 'r') to
ensure that the file is readable. If it is, we'll set the
icu_data_directory and proceed. There's a slight overhead for the
fopen() check, but it should be barely measurable.

This will be useful for Linux distribution packagers who want to
be able to ship a minimal node binary in a container image but
also be able to add on the full i18n support where needed. With
this patch, it becomes possible to ship the interpreter as
/usr/bin/node in one package for the distribution and to ship the
data files in another package (without a strict dependency
between the two). This means that users of the distribution will
not need to explicitly direct Node.js to locate the ICU data. It
also means that in environments where full internationalization is
not required, they do not need to carry the extra content (with
the associated storage costs).

Related: #3460

Signed-off-by: Stephen Gallagher sgallagh@redhat.com

Attn: @srl295

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

@nodejs-github-bot nodejs-github-bot added build Issues and PRs related to build files or the CI. c++ Issues and PRs that require attention from people who are familiar with C++. v12.x labels Dec 6, 2019
srl295
srl295 requested changes Dec 6, 2019
View reviewed changes
Copy link
Member

@srl295 srl295 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor change, but the shape of this looks great to me.

  • It might be helpful to somewhere document the scenario as you did in your comment on the other issue. This could just be in the PR for now.

bnoordhuis
bnoordhuis reviewed Dec 7, 2019
View reviewed changes
@sgallagher sgallagher changed the title Auto-load ICU data from --with-icu-default-data-dir build: Auto-load ICU data from --with-icu-default-data-dir Dec 9, 2019
@sgallagher
Copy link
Contributor Author

sgallagher commented Dec 9, 2019

I updated the patch with the code review recommendations. I also changed the commit message and the first comment on this PR to match it and provide more detail on the problem this is solving.

Thank you very much for the review!

@richardlau
Copy link
Member

richardlau commented Dec 9, 2019

Shouldn't this target master?

@sgallagher sgallagher changed the base branch from v12.x to master December 9, 2019 16:29
@sgallagher
Copy link
Contributor Author

sgallagher commented Dec 9, 2019

Rebased it to master. I was working on it against v12 originally because a) that's what we have in Fedora and b) the master branch wasn't compiling successfully for me, but that's been fixed in the interim it seems.

srl295
srl295 approved these changes Dec 10, 2019
View reviewed changes
Copy link
Member

@srl295 srl295 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@srl295 srl295 added the i18n-api Issues and PRs related to the i18n implementation. label Dec 10, 2019
@srl295
Copy link
Member

srl295 commented Dec 10, 2019

fyi @nodejs/intl

bnoordhuis
bnoordhuis approved these changes Dec 12, 2019
View reviewed changes
@sgallagher
build: auto-load ICU data from --with-icu-default-data-dir
e610b98 
When compiled with `--with-intl=small` and
`--with-icu-default-data-dir=PATH`, Node.js will use PATH as a
fallback location for the ICU data.

We will first perform an access check using fopen(PATH, 'r') to
ensure that the file is readable. If it is, we'll set the
icu_data_directory and proceed. There's a slight overhead for the
fopen() check, but it should be barely measurable.

This will be useful for Linux distribution packagers who want to
be able to ship a minimal node binary in a container image but
also be able to add on the full i18n support where needed. With
this patch, it becomes possible to ship the interpreter as
/usr/bin/node in one package for the distribution and to ship the
data files in another package (without a strict dependency
between the two). This means that users of the distribution will
not need to explicitly direct Node.js to locate the ICU data. It
also means that in environments where full internationalization is
not required, they do not need to carry the extra content (with
the associated storage costs).

Refs: nodejs#3460

Signed-off-by: Stephen Gallagher <sgallagh@redhat.com>
@sgallagher
Copy link
Contributor Author

sgallagher commented Dec 12, 2019

Rebased atop the latest master and made the two minor changes @bnoordhuis requested.

@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Dec 12, 2019

CI: https://ci.nodejs.org/job/node-test-pull-request/27594/

@bnoordhuis bnoordhuis added author ready PRs that have at least one approval, no pending requests for changes, and a CI started. and removed v12.x labels Dec 12, 2019
@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Dec 13, 2019

CI: https://ci.nodejs.org/job/node-test-pull-request/27602/

@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Dec 13, 2019

CI: https://ci.nodejs.org/job/node-test-pull-request/27649/

@Trott
Copy link
Member

Trott commented Dec 14, 2019

Landed in d502b83

@Trott Trott closed this Dec 14, 2019
Trott pushed a commit that referenced this pull request Dec 14, 2019
@sgallagher @Trott
build: auto-load ICU data from --with-icu-default-data-dir
d502b83 
When compiled with `--with-intl=small` and
`--with-icu-default-data-dir=PATH`, Node.js will use PATH as a
fallback location for the ICU data.

We will first perform an access check using fopen(PATH, 'r') to
ensure that the file is readable. If it is, we'll set the
icu_data_directory and proceed. There's a slight overhead for the
fopen() check, but it should be barely measurable.

This will be useful for Linux distribution packagers who want to
be able to ship a minimal node binary in a container image but
also be able to add on the full i18n support where needed. With
this patch, it becomes possible to ship the interpreter as
/usr/bin/node in one package for the distribution and to ship the
data files in another package (without a strict dependency
between the two). This means that users of the distribution will
not need to explicitly direct Node.js to locate the ICU data. It
also means that in environments where full internationalization is
not required, they do not need to carry the extra content (with
the associated storage costs).

Refs: #3460

Signed-off-by: Stephen Gallagher <sgallagh@redhat.com>

PR-URL: #30825
Reviewed-By: Steven R Loomis <srloomis@us.ibm.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
MylesBorins pushed a commit that referenced this pull request Dec 17, 2019
@sgallagher @MylesBorins
build: auto-load ICU data from --with-icu-default-data-dir
00cbf5b 
When compiled with `--with-intl=small` and
`--with-icu-default-data-dir=PATH`, Node.js will use PATH as a
fallback location for the ICU data.

We will first perform an access check using fopen(PATH, 'r') to
ensure that the file is readable. If it is, we'll set the
icu_data_directory and proceed. There's a slight overhead for the
fopen() check, but it should be barely measurable.

This will be useful for Linux distribution packagers who want to
be able to ship a minimal node binary in a container image but
also be able to add on the full i18n support where needed. With
this patch, it becomes possible to ship the interpreter as
/usr/bin/node in one package for the distribution and to ship the
data files in another package (without a strict dependency
between the two). This means that users of the distribution will
not need to explicitly direct Node.js to locate the ICU data. It
also means that in environments where full internationalization is
not required, they do not need to carry the extra content (with
the associated storage costs).

Refs: #3460

Signed-off-by: Stephen Gallagher <sgallagh@redhat.com>

PR-URL: #30825
Reviewed-By: Steven R Loomis <srloomis@us.ibm.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
@MylesBorins MylesBorins mentioned this pull request Dec 17, 2019
Merged
targos pushed a commit that referenced this pull request Jan 14, 2020
@sgallagher @targos
build: auto-load ICU data from --with-icu-default-data-dir
67a2a47 
When compiled with `--with-intl=small` and
`--with-icu-default-data-dir=PATH`, Node.js will use PATH as a
fallback location for the ICU data.

We will first perform an access check using fopen(PATH, 'r') to
ensure that the file is readable. If it is, we'll set the
icu_data_directory and proceed. There's a slight overhead for the
fopen() check, but it should be barely measurable.

This will be useful for Linux distribution packagers who want to
be able to ship a minimal node binary in a container image but
also be able to add on the full i18n support where needed. With
this patch, it becomes possible to ship the interpreter as
/usr/bin/node in one package for the distribution and to ship the
data files in another package (without a strict dependency
between the two). This means that users of the distribution will
not need to explicitly direct Node.js to locate the ICU data. It
also means that in environments where full internationalization is
not required, they do not need to carry the extra content (with
the associated storage costs).

Refs: #3460

Signed-off-by: Stephen Gallagher <sgallagh@redhat.com>

PR-URL: #30825
Reviewed-By: Steven R Loomis <srloomis@us.ibm.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
@targos targos mentioned this pull request Jan 15, 2020
Closed
BethGriggs pushed a commit that referenced this pull request Feb 6, 2020
@sgallagher @BethGriggs
build: auto-load ICU data from --with-icu-default-data-dir
ada7624 
When compiled with `--with-intl=small` and
`--with-icu-default-data-dir=PATH`, Node.js will use PATH as a
fallback location for the ICU data.

We will first perform an access check using fopen(PATH, 'r') to
ensure that the file is readable. If it is, we'll set the
icu_data_directory and proceed. There's a slight overhead for the
fopen() check, but it should be barely measurable.

This will be useful for Linux distribution packagers who want to
be able to ship a minimal node binary in a container image but
also be able to add on the full i18n support where needed. With
this patch, it becomes possible to ship the interpreter as
/usr/bin/node in one package for the distribution and to ship the
data files in another package (without a strict dependency
between the two). This means that users of the distribution will
not need to explicitly direct Node.js to locate the ICU data. It
also means that in environments where full internationalization is
not required, they do not need to carry the extra content (with
the associated storage costs).

Refs: #3460

Signed-off-by: Stephen Gallagher <sgallagh@redhat.com>

PR-URL: #30825
Reviewed-By: Steven R Loomis <srloomis@us.ibm.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
@MylesBorins MylesBorins mentioned this pull request Feb 8, 2020
Merged
@sgallagher sgallagher deleted the i18n-default-dir branch February 14, 2020 14:55
@srl295 srl295 mentioned this pull request Mar 24, 2020
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bnoordhuis bnoordhuis bnoordhuis approved these changes

@Trott Trott Trott approved these changes

@srl295 srl295 srl295 approved these changes

@richardlau richardlau richardlau approved these changes

Assignees

No one assigned

Labels

author ready PRs that have at least one approval, no pending requests for changes, and a CI started. build Issues and PRs related to build files or the CI. c++ Issues and PRs that require attention from people who are familiar with C++. i18n-api Issues and PRs related to the i18n implementation.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@sgallagher @richardlau @srl295 @nodejs-github-bot @Trott @bnoordhuis