Skip to content

ENH: add some more common Nodes #503

@NickCrews

Description

@NickCrews

Writing pandas dataframes to disk seems really common. I am going to have to write my own DFNode class for this. Would you like me to generalize it enough and then contribute it here?

I would imagine something like

class DFNode:

    def __init__(self, path: str | ..., kind: Literal["csv", "parquet", ...], load_kwargs: dict[str, Any = {}, save_kwargs: dict[str, Any]):
        self.path = path
        self.kind = kind
        self.load_kwargs = load_kwargs
        self.save_kwargs = save_kwargs
    
    def save(self, value: Any) -> None:
        saver = getattr(value, f"to_{self.kind}")
        saver(self.path, **self.save_kwargs)

    def load(self, is_product: bool) -> Any:
        if is_product:
            return self
        loader = getattr(pd,  f"read_{self.kind}")
        return loader(self.path, **self.load_kwargs)

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions