feat(inputs.x509_cert): Add support for JKS and PKCS#12 keystores by paulojmdias · Pull Request #16508 · influxdata/telegraf

paulojmdias · 2025-02-12T01:08:21Z

Summary

I initially thought about creating a separate input plugin for JKS & PKCS12 key stores but decided to integrate it into x509_cert since the changes were minimal. This update allows x509_cert to automatically detect and parse JKS & PKCS12 files while maintaining the existing certificate validation and metric collection logic. I also added tests to ensure everything works smoothly without overcomplicating the code.

Please evaluate if make sense and let me know what changes you feel are needed to accommodate this feature.

Checklist

No AI generated code was used in this PR

Related issues

resolves #7013

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

srebhan · 2025-02-12T15:59:26Z

@paulojmdias please check the linter issues. You can reproduce them locally using make check-deps.

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

paulojmdias · 2025-02-12T16:47:51Z

@paulojmdias please check the linter issues. You can reproduce them locally using make check-deps.

Fixed 🙌

srebhan

Thanks for your contribution @paulojmdias! This is very much appreciated! I do have some comments in the code...

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

srebhan

@paulojmdias some more comments...

srebhan · 2025-02-18T10:08:05Z

+	err = jks.Store(output, []byte("test-password"))
+	require.NoError(t, err)
+
+	return "jks://" + jksPath


This doesn't work on Windows, you should do

Suggested change

return "jks://" + jksPath

jksPath = filepath.ToSlash(jksPath)

if !strings.HasPrefix(jksPath, "/") {

jksPath = "/" + jksPath

}

return "jks://" + jksPath

Fixed also pkcs12Path

@srebhan i fixed in the commit 0824e05

Your suggestion also does not work, so I have included the following.

pkcs12Path = filepath.ToSlash(pkcs12Path) if runtime.GOOS == "windows" { pkcs12Path = strings.TrimPrefix(pkcs12Path, "/") } return "pkcs12://" + pkcs12Path

Please review and let me know.

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

…keystore_cert Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

srebhan

Thanks @paulojmdias, this looks really nice already. Some more comments regarding the test code but we are almost there.

srebhan · 2025-03-04T10:44:30Z

+	if runtime.GOOS == "windows" {
+		jksPath = strings.TrimPrefix(jksPath, "/")
+	}


Why do you need this?

@srebhan #16508 (comment)

Hmmm what if in Windows I use a path like \TEMP\foo which is an absolute path, won't this be converted to TEMP/foo instead of /TEMP/foo?

Ok I see the issue now. The problem is that you do solve something in the test which must be solved in the actual code. Imagine the user wants to specify a JKS path of C:\MyCerts\foo.jks, then he has to set

[[inputs.x509_cert]] sources = ["jks:///C:/MyCerts/foo.jks"]

and then you need to remove the leading slash in your regular code if (and only if) you get a drive letter. Please see line 282 and following for how to do this!

srebhan · 2025-03-04T10:44:38Z

+	if runtime.GOOS == "windows" {
+		pkcs12Path = strings.TrimPrefix(pkcs12Path, "/")
+	}


Why do you need this?

Answered above

srebhan · 2025-03-04T10:47:24Z

+func TestGatherKeystores(t *testing.T) {
+	pkcs12Path, jksPath := generateTestKeystores(t)
+
+	tests := []struct {
+		name     string
+		mode     os.FileMode
+		content  string
+		password string
+		error    bool
+	}{
+		{name: "valid PKCS12 keystore", mode: 0640, content: pkcs12Path, password: "test-password"},
+		{name: "valid JKS keystore", mode: 0640, content: jksPath, password: "test-password"},
+		{name: "missing password PKCS12", mode: 0640, content: pkcs12Path, error: true},
+		{name: "missing password JKS", mode: 0640, content: jksPath, error: true},
+		{name: "wrong password PKCS12", mode: 0640, content: pkcs12Path, password: "wrong-password", error: true},
+		{name: "wrong password JKS", mode: 0640, content: jksPath, password: "wrong-password", error: true},
+	}
+
+	for _, test := range tests {
+		t.Run(test.name, func(t *testing.T) {
+			if runtime.GOOS != "windows" {
+				// To be Reviewed
+				path := strings.TrimPrefix(test.content, "pkcs12://")
+				path = strings.TrimPrefix(path, "jks://")
+				require.NoError(t, os.Chmod(path, test.mode))
+			}
+
+			sc := X509Cert{
+				Sources: []string{test.content},
+				Log:     testutil.Logger{},
+			}
+
+			// Set password if provided
+			if test.password != "" {
+				sc.Password = config.NewSecret([]byte(test.password))
+			} else {
+				sc.Password = config.NewSecret(nil)
+			}
+
+			require.NoError(t, sc.Init())
+
+			acc := testutil.Accumulator{}
+			err := sc.Gather(&acc)
+
+			if (len(acc.Errors) > 0) != test.error {
+				t.Errorf("%s", err)
+			}
+		})
+	}
+}


How about splitting this function into one testing for failures (those with error: true) and another one where we don't expect errors. This has the advantage of simplifying the code and it is easier to follow as the function will set the context.

Please also test for the error message with require.ErrorContains(t, err, "essential part of the expected error message")!

@srebhan added on commit 17e0dfa

Please take a look and let me know WDYT.

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

srebhan

Thanks @paulojmdias! You need to fix the path handling on Windows. Currently your tests will pass but any real Windows path will fail as you need to remove the leading slash for the path contains a drive-letter.

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

…keystore_cert Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

telegraf-tiger · 2025-03-06T16:48:49Z

Download PR build artifacts for linux_amd64.tar.gz, darwin_arm64.tar.gz, and windows_amd64.zip.
Downloads for additional architectures and packages are available below.

☺️ This pull request doesn't significantly change the Telegraf binary size (less than 1%)

📦 Click here to get additional PR build artifacts

Artifact URLs

DEB	RPM	TAR GZ	ZIP
amd64.deb	aarch64.rpm	darwin_amd64.tar.gz	windows_amd64.zip
arm64.deb	armel.rpm	darwin_arm64.tar.gz	windows_arm64.zip
armel.deb	armv6hl.rpm	freebsd_amd64.tar.gz	windows_i386.zip
armhf.deb	i386.rpm	freebsd_armv7.tar.gz
i386.deb	ppc64le.rpm	freebsd_i386.tar.gz
mips.deb	riscv64.rpm	linux_amd64.tar.gz
mipsel.deb	s390x.rpm	linux_arm64.tar.gz
ppc64el.deb	x86_64.rpm	linux_armel.tar.gz
riscv64.deb		linux_armhf.tar.gz
s390x.deb		linux_i386.tar.gz
		linux_mips.tar.gz
		linux_mipsel.tar.gz
		linux_ppc64le.tar.gz
		linux_riscv64.tar.gz
		linux_s390x.tar.gz

srebhan

Awesome! Thanks a lot for your contribution and patience @paulojmdias!

…fluxdata#16508) Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

paulojmdias added 2 commits February 12, 2025 01:02

feat(inputs.x509_cert): add support for JKS and PKCS#12 keystors

770102b

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

fix: fix go mod

9a111ef

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

telegraf-tiger Bot added feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin plugin/input 1. Request for new input plugins 2. Issues/PRs that are related to input plugins labels Feb 12, 2025

srebhan self-assigned this Feb 12, 2025

fix: simplify code and linting

7cf57be

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

paulojmdias changed the title ~~feat(inputs.x509_cert): add support for JKS and PKCS#12 keystors~~ feat(inputs.x509_cert): add support for JKS and PKCS#12 keystores Feb 13, 2025

srebhan reviewed Feb 17, 2025

View reviewed changes

srebhan changed the title ~~feat(inputs.x509_cert): add support for JKS and PKCS#12 keystores~~ feat(inputs.x509_cert): Add support for JKS and PKCS#12 keystores Feb 17, 2025

paulojmdias added 5 commits February 17, 2025 22:35

fix: fix secret and docs

281243c

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

chore: move JKS functions to dedicated file

cbd7cfc

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

fix: move c.classification to Init()

8e7af5f

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

fix: update JKS and PKCS#12 assignments

dc8222d

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

feat: create TestGatherKeystores

25440fc

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

srebhan reviewed Feb 18, 2025

View reviewed changes

paulojmdias and others added 13 commits February 18, 2025 17:13

fix: simplify functions and code

6aaf29b

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

fix: fix docs

bd9dde8

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

Merge branch 'influxdata:master' into keystore_cert

99c88af

fix: test filepath.Abs

7998a8c

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

fix: fix linting

4720a4d

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

fix: fix windows and linting tests

d9f002d

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

fix: ignore chmod on windows

4d372bb

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

fix: fix path tests

2485669

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

fix: test fix nil windows output

bee88f4

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

fix: revert test return

3736fd2

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

fix: remove fmt

b197c90

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

chore: add debug for windows testing

d201e72

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

chore: windows test

387dcdb

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

paulojmdias and others added 11 commits February 21, 2025 01:09

fix: revert and keep debug

d35dbc5

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

fix: fix typo

2ba801b

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

fix: remove try with empty password

c80d5aa

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

feat: add normalizePathForURL func for Windows tests

f889161

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

Merge branch 'master' of https://github.com/influxdata/telegraf into …

a61aae3

…keystore_cert Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

fix: fix Windows test path

0824e05

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

fix: remove fmt

a880128

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

Merge branch 'influxdata:master' into keystore_cert

bd02ed2

Merge branch 'influxdata:master' into keystore_cert

cdce5cc

Merge branch 'influxdata:master' into keystore_cert

e9405a0

Merge branch 'influxdata:master' into keystore_cert

f82a7e0

srebhan reviewed Mar 4, 2025

View reviewed changes

paulojmdias and others added 4 commits March 4, 2025 17:37

fix: fix camel case on tests and remove c.classification init

72ce9a1

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

feat: split test functions

17e0dfa

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

fix: linting

97bfd5e

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

Merge branch 'influxdata:master' into keystore_cert

55a0e98

srebhan requested changes Mar 6, 2025

View reviewed changes

paulojmdias added 3 commits March 6, 2025 16:00

feat: improve windows parsing

12f9c9d

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

Merge branch 'keystore_cert' of github.com:paulojmdias/telegraf into …

bab4c82

…keystore_cert Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

fix: fix windows path normalize

119e239

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

srebhan approved these changes Mar 7, 2025

View reviewed changes

srebhan added the ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review. label Mar 7, 2025

srebhan assigned mstrandboge and unassigned srebhan Mar 7, 2025

mstrandboge approved these changes Mar 10, 2025

View reviewed changes

mstrandboge merged commit d231442 into influxdata:master Mar 10, 2025

github-actions Bot added this to the v1.34.0 milestone Mar 10, 2025

paulojmdias deleted the keystore_cert branch March 10, 2025 08:02

asaharn pushed a commit to asaharn/telegraf that referenced this pull request Mar 10, 2025

feat(inputs.x509_cert): Add support for JKS and PKCS#12 keystores (in…

3f06a93

…fluxdata#16508) Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

-	return "jks://" + jksPath
+	jksPath = filepath.ToSlash(jksPath)
+	if !strings.HasPrefix(jksPath, "/") {
+		jksPath = "/" + jksPath
+	}
+	return "jks://" + jksPath

Uh oh!

Conversation

paulojmdias commented Feb 12, 2025

Summary

Checklist

Related issues

Uh oh!

srebhan commented Feb 12, 2025

Uh oh!

paulojmdias commented Feb 12, 2025

Uh oh!

srebhan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

srebhan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

srebhan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

srebhan left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

telegraf-tiger Bot commented Mar 6, 2025

Artifact URLs

Uh oh!

srebhan left a comment •

edited

Loading