Unix sockets for host <-> container networking#75
Merged
Conversation
charliemirabile
force-pushed
the
unix_sockets
branch
from
b6d7988 to
f01b489
Compare
We can give extenginx the ability to create self signed certs for itself during the build process so that folks to not need to run the create dev keys script.
now that extenginx can create its own keys, this script is not necessary.
it is possible to just perform a bind mount to insert custom snippets at runtime whether or not a specific volume is declared in the containerfile so the complexity is unnecessary.
it is not necessary to explicitly expose the ports of extenginx since if they are mapped to host ports in the compose (as they are) they will be reachable nonetheless.
The current envsubst command was starting to reach the limits of an 80 column terminal and having it one line certainly was not helping is readability or editability.
insofar as nginx should serve only as an ssl terminating proxy for the unencrypted upstream servers, it should not be mucking around with unencrypted plain http connections. When deployed as one of multiple instances behind a main nginx proxy, that proxy can handle redirecting connections to the unencrypted port to the encrypted port (and then later to the appropriate upstream instance of extenginx)
in order to support changing from ip sockets to unix sockets, dealing explicitly with ports is not viable. Instead allow the consumer of extenginx to specify a snippet to go in the listen directive of the server.
nginx no longer binds to any host port (in fact no containers bind to any host ports at all). Instead it listens on a unix socket in a path within a bind mounted folder. This allows dev, staging, & prod to all be run with rootless podman commands, and it allows more than one instance of singularity to be running at once on the same machine (assuming each has a different folder to put its sockets in). In order to support local development and testing, introduce dev-sockets helper script. This script spawns socat processes that listen on the normal tcp ports and redirect connections to the unix sockets from extenginx. If run as root, it binds to the rootful ports for the services (443, 465, 995) and if not, it binds to those port numbers + 1000 (i.e. 1443, 1465, 1995). It is designed to be long running in the background or in a different terminal and does not need to be restarted every time the containers go up and down.
Instead of needing to run dev_sockets.sh script to proxy tcp connections to unix socket connections, curl has the ability to directly connect to a unix socket when making a request.
charliemirabile
force-pushed
the
unix_sockets
branch
from
f01b489 to
c852ad3
Compare
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is nice because we do not need to map any host ports to container ports.
Fixes: #63
Depends on: #73
Note, if you want to test these changes in the staging or prod environment, you must make sure you have updated your podman compose to the latest version of our fork because I added one commit that fixes the issue with selinux not being recognized in a verbose mount specification.
If you forget to do this, the symptom is that nginx crashes because it is unable to bind to the unix socket due to EPERM.
Also because the test improvements made curl actually perform ssl cert checking, you may run into it complaining about a hostname mismatch if the ssl certs in your ssl folder are for localhost and you are trying to do testing on staging/prod.
The fix (as a temporary workaround, this will improve after we replace bind mounts with volumes) is to extract the certs that were generated in the file system of the container image (
podman image save localhost/singularity_nginx:latest > img.tarextract the tar and find the layer that contains the ssl certs and then extract that layer into the ssl foldertar -xf some_long_hash.tar --strip-components=3 -C /path/to/the/bind/mount/for/ssl/and then runpodman exec singularity_nginx_1 nginx -s reloadso it picks up the new certs).If you forget to do this the symptom is that curl complains that it cannot verify the legitimacy of the server.