Piping using `/dev/stdin` doesn't work on Windows

[MaikuMori]

The "workaround" to get piping to work is not cross-platform.

• https://stackoverflow.com/questions/7395157/windows-equivalent-to-dev-stdin

I highly recommend handling it like kubectl and other tools where you can pass in - as file name which triggers reading from stdin.

So something like echo "asdf" | sops - or echo "asdf" | sops -f -

Is this something you would accept a PR for?

[ajvb]

@MaikuMori What is the use case for piping? Why not just write the output to a file and then run sops -e -i <filename>?

[MaikuMori]

I have a script which decodes content using sops -d enc_file and reads stdout. Then it changes the content dynamically and needs to update the original encoded file. Currently, I work around this by creating a temp file, writing the content there then running sops on it and then overwrite the original file, then delete the temp file. It would be much nicer if sops could just read stdin.

Even doing something like cat asdf | base64 | sops --output asdf.enc -e - in terminal would be nice. (Alternatively cat asdf | base64 | sops -e - > asdf.enc)

Here are other issues which mention this as preferred way:

• Make sops use config file when using STDIN #510 (mentions -)
• Editing file in-place with data from stdin #696 (could be solved with -)
• Document how to operate on stdin #330 (suggested alternative solution, but you can clearly see that /dev/stdin is not expected behavior. People expect piping to just work or work with -)

To make this even more useful sops should use config file when input is passed via stdin (tracked by #510). To decide which config to use we could use CWD or the directory of output file if specified.

Maybe also defaulting input/output type to binary unless specifically specified.

[jkroepke]

Additionally on windows, running in cygwin.

sops --decrypt --input-type yaml --output-type yaml --output /cygdrive/c/Users/RUNNER~1/AppData/Local/Temp/tmp.6bAckDpOVi/assets/values/sops/secrets.yaml.dec /cygdrive/c/Users/RUNNER~1/AppData/Local/Temp/tmp.6bAckDpOVi/assets/values/sops/secrets.yaml
#   Error: cannot operate on non-existent file

While

sops --decrypt --input-type yaml --output-type yaml - < /cygdrive/c/Users/RUNNER~1/AppData/Local/Temp/tmp.6bAckDpOVi/assets/values/sops/secrets.yaml > /cygdrive/c/Users/RUNNER~1/AppData/Local/Temp/tmp.6bAckDpOVi/assets/values/sops/secrets.yaml.dec 

Would not.

[iverberk]

I'd like to add support for this feature. It prevents the necessity of unencrypted intermediate files that might potentially linger/be exposed on Windows. We are forced to work in a Windows environment, while the app is deployed to a Linux environment via CI/CD (also Linux based). We cannot use /dev/stdin and this request would provide a clean platform-independent way of reading via stdin.

[jodem]

Just a quick comment for those who look a workaround, I used a temporary file with nanoseconds in the time to include the yaml content and remove it right after the sops execution.

if [[ $OSTYPE == 'darwin'* ]]; then
curTime=$(gdate +%s.%N)
else
curTime=$(date +%s)
fi

tempFile=./temp/${curTime}

ENCRYPTED_CONTENT=$(echo "$CONTENT" > $tempFile | sops -e --encrypted-regex "$ENCRYPTED_REGEX" --input-type yaml --output-type yaml $tempFile)

rm -f $tempFile

[erhhung]

If sops --encrypt by default outputs to stdout, why doesn't sops --decrypt read from stdin by default without a filename argument?

While the filename convention of using "-" to denote stdin is helpful, why not just read from stdin if no filename is given at all and stdin is not a tty?

[Hornwitser]

What is the use case for piping? Why not just write the output to a file and then run sops -e -i <filename>?

One use case is never writing the unencrypted content to disk. Using this as a practise when working with secret files could reduce the risk of committing the secrets unencrypted as you're (supposedly) never having unencrypted secrets in your workspace to begin with.

I really, really wish secret-generating-command | sops secret.enc.yaml would just do what I want, which is to read the secret from stdin and write it out encrypted to secret.enc.yaml (overwriting if it exists) using the rules from .sops.yaml.

[felixfontein]

Getting this fixed is still on the roadmap; the original plan was to resurrect #1104 and fix the remaining issues with it, but so far nobody got around to actually do it. (I've added this issue to the roadmap, since having the closed PR on it isn't very helpful as a reminder...)

[felixfontein]

I think we need to discuss how the interface should look like:

1. Use a magic filename -;
2. Add an option --use-stdin (or something like that) to sops encrypt and sops decrypt (which reduces the number of positional arguments expected to zero, and for sops encrypt requires the use of --filename-override).
3. Allow to provide no positional argument (instead of one), and if that's the case, read from stdin (and require the use of --filename-override when encrypting).

I'm currently tending to 2. Other opinions?

CC @getsops/maintainers