Conversation
| "{}.{}.{}.svc.cluster.local", | ||
| self.pod_name, self.role_group_service_name, self.namespace | ||
| ) | ||
| self.external_name.clone().unwrap_or_else(|| { |
There was a problem hiding this comment.
Not super happy with abusing the FQDN property here (or HdfsPodRef at all..?), this also breaks down if the user overrides the ports for whatever reason.
There was a problem hiding this comment.
@razvan would it make sense to split HdfsPodRef into separate types for journalnodes and namenodes? We'll probably want to only use the external name for all namenode traffic anyway, once we start doing Kerberos (since this is linked to the hostname, and HDFS nodes dislike having more than one Kerberos principal each).
rust/operator/src/hdfs_controller.rs
Outdated
| "sh".to_string(), | ||
| "-c".to_string(), | ||
| format!( | ||
| r#"kubectl get svc $POD_NAME -o json > /data/pod-svc && kubectl get node $NODE_NAME -o json > /data/pod-node && DATA_PORT=$(jq '.spec.ports[] | select(.name == "data") | .nodePort' /data/pod-svc) HTTP_PORT=$(jq '.spec.ports[] | select(.name == "http") | .nodePort' /data/pod-svc) IPC_PORT=$(jq '.spec.ports[] | select(.name == "ipc") | .nodePort' /data/pod-svc) NODE_IP=$(jq -r '.status.addresses | map(select(.type == "ExternalIP")) | .[0].address' /data/pod-node) {HADOOP_HOME}/bin/hdfs --debug datanode"# |
There was a problem hiding this comment.
Need to clean this up better
|
This has ended up becoming an example use case of stackabletech/listener-operator#1, and merging this is blocked on releasing that. |
maltesander
left a comment
maltesander
left a comment
There was a problem hiding this comment.
Need to add some rolebindings:
rolebindings.rbac.authorization.k8s.io "hdfs-datanode-55df4c3b-06cc-4311-b2a7-86bfd5d2162f" is forbidden: User "system:serviceaccount:default:hdfs-operator-serviceaccount" cannot patch resource "rolebindings" in API group "rbac.authorization.k8s.io" in the namespace "default": Forbidden
3 participants
Description
Fixes #174
This allows users to access the HDFS cluster from outside of the K8s cluster. This requires a number of changes:
LoadBalancerservices for access rather than the namenode pod hostnamesReview Checklist
Once the review is done, comment
bors r+(orbors merge) to merge. Further information