Skip to content

Use filer.WorkspaceFilesClient instead of repofiles pkg#258

Closed
pietern wants to merge 1 commit into
mainfrom
use-filer
Closed

Use filer.WorkspaceFilesClient instead of repofiles pkg#258
pietern wants to merge 1 commit into
mainfrom
use-filer

Conversation

@pietern
Copy link
Copy Markdown
Contributor

@pietern pietern commented Mar 17, 2023

This change means we no longer do an rm -rf of a path if we encounter an error to put a file at that location. For this reason the integration test called TestAccIncrementalFileOverwritesFolder fails.

We can fix by tracking the directories we should create and remove as separate action in the diff struct and perform directory creation prior to putting files and directory removal after deleting files.

@pietern
Copy link
Copy Markdown
Contributor Author

pietern commented Mar 21, 2023

Discussion with @shreyas-goenka:

How to solve directory handling:

  1. Maintain set of directories needed for current set of files. Compare to previous set of files. This results in mkdir of added directories and rmdir of removed directories.
  2. Making new directories should happen prior to dealing with files. Removing existing directories should happen after dealing with files.
  3. Making new directories can be deduped across common prefixes where only the longest prefix is created recursively.
  4. Removing existing directories must happen sequentially, starting with the longest prefix.
  5. If removal of a directory fails, it must not be empty. We then recursively look for nested empty directories, remove those, and continue. This is necessary to remove empty directories left around by prior sync runs.

@pietern
Copy link
Copy Markdown
Contributor Author

pietern commented Jun 2, 2023

Recreated as #424.

@pietern pietern closed this Jun 2, 2023
@pietern pietern deleted the use-filer branch June 2, 2023 13:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant