"times" behaves differently than other shells

[avih]

POSIX says this:

The times utility shall write the accumulated user and system times for the shell and for all of its child processes...

It's not obvious to me how it should behave in a subshell, and it's entirely possible that POSIX tried to document the ksh(88) behavior.

Nevertheless, in all other shells I have access to, a new subshell always resets the output of times to 0, while in ksh93[u+m] it accumulates the "main" shell even in a subshell.

For instance, this prints the "shell user seconds" component of the times output in a loop, in a subshell, and it prints ever-increasing numbers in ksh93 variants, but always 0-ish in all other shells.

ksh -c 'while x=$(times); do echo ${x%% *}; done'
0m00.003s
...
...
0m00.004s
0m00.004s
...
...
0m00.292s
0m00.292s
...
...
0m00.293s
0m00.293s
...
...

I'm intentionally not declaring it as a bug, but if compatibility with other shells is desirable for times, then it might be considered as such.

Cases where times can be useful is to measure execution duration of some function or shell snippet.

In all other shells I tried it can simply be x=$(benchmark ...; times), but in ksh93 and derivatives one would need to grab the output of times before and after running benchmark, convert them to some numeric form, and subtract the former from the latter. Or use some other methods (which do exist).

[richfelker]

I suspect ksh is trying to optimize out the subshell for builtin commands, and this is what's breaking it. Does doing something like ( /bin/true && times ) fix it?

[avih]

Does doing something like ( /bin/true && times ) fix it?

No. Same ever-increasing values.

Same ever-increasing also with ksh --posix -c ... - with or without the /bin/true && inside the subshell.

To be clear, I'm not identifying any wording in POSIX which suggest that ksh93 is wrong and all the others are right, but it is different from all the others I tried (about 20).

[McDutchie]

Yes, this is an effect of ksh having non-forking subshells, i.e., subshells that share the parent shell's process (until something makes it impossible to do so). The operating system keeps track of the times per process and resets them for a new process. If the subshell doesn't fork a new process, the times aren't reset.

The idiomatic way of forcing a subshell into its own process is the use of the ulimit command, for example, ulimit -t unlimited or ulimit -c 0. So:

$ ksh -c 'for((i=0;i<1000000;i++)); do :; done; (times; ulimit -c 0; echo forked:; times); echo end' 
0m00,319s 0m00,006s
0m00,000s 0m00,000s
forked:
0m00,000s 0m00,000s
0m00,000s 0m00,000s
end

(The final 'echo end' is needed to avoid optimising out the subshell.)

Since the times are a property of the operating system and ksh is only querying them via getrusage(2), and doesn't keep track of them itself, there's not much we can do about this.

(If you experiment with this directly on the interactive shell, then note that, as of commit d11d4c7, non-command substitution subshells always fork on interactive shells, whereas command substitutions are made impossible to suspend. This is the only way I was able to fix a whole class of fatal job control breakage. To get non-forking subshells on the interactive shell, run the whole experiment in a subshell of its own by enclosing the whole thing in another set of parentheses. Also useful: you can use ${.sh.pid} and ${.sh.stats.forks} to check if a subshell forked.)

[avih]

Thanks for the explanation.

Considering the potential issues with this optimization (which otherswise is generally effective), would it be simple to just disable this optimization when using --posix and/or with some new shell option?

EDIT: I'm not suggesting to make times force fork, because you didn't suggest that yourself despite the fact that its semantic clearly depends on forking command substitution, and code which uses it likely expect and relies on the fork behavior.

[McDutchie]

Thanks for the explanation.

Considering the potential issues with this optimization (which otherswise is generally effective), would it be simple to just disable this optimization when using --posix and/or with some new shell option?

The following test patch makes all subshells fork (without making this subject to a shell option, but that would be easy to add):

diff --git a/src/cmd/ksh93/sh/subshell.c b/src/cmd/ksh93/sh/subshell.c
index 9ab5c85b6..e3e839c43 100644
--- a/src/cmd/ksh93/sh/subshell.c
+++ b/src/cmd/ksh93/sh/subshell.c
@@ -685,6 +685,7 @@ Sfio_t *sh_subshell(Shnode_t *t, volatile int flags, int comsub)
 				else
 					sh_subfork();
 			}
+if (comsub!=2) sh_subfork();
 			sh_exec(t,flags);
 		}
 	}

If you apply this patch, build ksh, and then run the regression tests (bin/shtests -p), you'll find a number of test failures. So ksh doesn't quite work correctly in that mode. Nothing is ever simple, it seems.

It also makes many scripts run a lot more slowly, of course.

EDIT: I'm not suggesting to make times force fork, because you didn't suggest that yourself despite the fact that its semantic clearly depends on forking command substitution, and code which uses it likely expect and relies on the fork behavior.

Making times force a fork would be incorrect. The results immediately after the fork would always be zero, even if the subshell had already spent CPU time as a virtual subshell before executing times.

I don't think its semantic is very clear at all. The POSIX spec just says that it “shall write the accumulated user and system times for the shell and for all of its child processes”. The ksh version does that. Subshells aren't mentioned. There's also nothing in POSIX that says subshells have to run in separate processes.

[richfelker]

Strictly speaking, a subshell environment does not appear to be required to be its own process. Perhaps the portable way to do this, which should also work in ksh, is something like ( ... ) & wait $! which explicitly forces a process with a process id.

[avih]

Nothing is ever simple, it seems.

It also makes many scripts run a lot more slowly, of course.

Yeah...

I don't think its semantic is very clear at all.

I agree and said so twice already.

But what that statement claimed is the semantics changes depending on whether it forks or not.

So if it only sometimes fork, depending on whether the optimization kicks in or not, then it has to be considered a bug that the semantics of this command changes depending on things which are outside the control of the script author.

How would you document it? "this command sometimes counts the time in the current process, and sometimes in the parent process"?

EDIT: or "counts... in the current process, but note that whether subshell is a new process or not depends on $things".

(obviously incorrect doc either way, but you get the point).