[C++] Sporadic failures in CopyFiles test

[asfimport]

Here is an example failure on AppVeyor (Windows):
https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/35210915/job/yhdxkfu7hta3kugp#L1913

[ RUN      ] TestSubTreeFileSystem.CopyFiles
C:/projects/arrow/cpp/src/arrow/filesystem/filesystem_test.cc(608): error: Failed
'CopyFiles({{subfs_, "ab"}, {subfs_, "cd"}, {subfs_, "ef"}}, {{dest_fs, "AB/ab"}, {dest_fs, "CD/CD/cd"}, {dest_fs, "EF/EF/EF/ef"}})' failed with IOError: Path does not exist 'sub/copy/AB/ab'
[  FAILED  ] TestSubTreeFileSystem.CopyFiles (9 ms)

Similar failure on Linux, showing the line number:
https://ci.ursalabs.org/#/builders/101/builds/4412/steps/8/logs/stdio

[ RUN      ] TestSubTreeFileSystem.CopyFiles
../src/arrow/filesystem/filesystem_test.cc:608: Failure
Failed
'CopyFiles({{subfs_, "ab"}, {subfs_, "cd"}, {subfs_, "ef"}}, {{dest_fs, "AB/ab"}, {dest_fs, "CD/CD/cd"}, {dest_fs, "EF/EF/EF/ef"}})' failed with IOError: Path does not exist 'sub/copy/CD/CD/cd'
In ../src/arrow/filesystem/filesystem.cc, line 467, code: (_error_or_value13).status()
[  FAILED  ] TestSubTreeFileSystem.CopyFiles (17 ms)

The erroring line number (467) corresponds to the OpenOutputStream call below:

        auto dest_dir = internal::GetAbstractPathParent(destinations[i].path).first;
        if (!dest_dir.empty()) {
          RETURN_NOT_OK(destinations[i].filesystem->CreateDir(dest_dir));
        }

-->     ARROW_ASSIGN_OR_RAISE(
            auto destination,
            destinations[i].filesystem->OpenOutputStream(destinations[i].path));

It's not obvious why the error occurs, since the directory is created just above (race condition in the CreateDir implementation?).

Reporter: Antoine Pitrou / @pitrou
Assignee: Antoine Pitrou / @pitrou

PRs and other links:

• GitHub Pull Request #8193

_{Note: This issue was originally created as ARROW-10012. Please see the migration documentation for further details.}

[asfimport]

Antoine Pitrou / @pitrou:
I checked and LocalFileSystem::CreateDir is sane (it uses what should be a race condition-immune idiom). The only hope now is to manage to reproduce locally.

A question though: why does CopyFiles create the target directories at all? Shell commands such as cp (Unix) or copy (Windows) don't do that when copying files (rather than entire directory trees). cc @bkietz

[asfimport]

Antoine Pitrou / @pitrou:
Ah, but the CopyFiles test uses the mock filesystem. So there must be a race condition there!

[asfimport]

Antoine Pitrou / @pitrou:
Issue resolved by pull request 8193
#8193