Is your feature request related to a problem? Please describe.
We have two separate codepaths for extract_fields and extract_columns.
Furthermore, extract_fields requires one to write the type in the extract, but we should have that information in the fn output.
I think we can simplify the code quite a bit.
Describe the solution you'd like
A single extract (name TBD) decorator that does the following:
@extract("foo", "bar")
def foo_bar(foo: int, bar: int) -> Dict[str, int]:
return {"foo": foo, "bar": bar}
@extract("foo", "bar")
def foo_bar(foo: pd.Series, bar: pd.Series) -> pd.DataFrame:
return pd.DataFrame({"foo": foo, "bar": bar})
Then it can register "extensions" to allow for us to processing different extractions. E.G. different DF types, etc...
If its a Dict[str, foo], we know it'll be a bunch of foo type outputs. If its a pd.DataFrame we know its going to be pd.Series. Etc...
Describe alternatives you've considered
Not doing this.
Additional context
Just an idea that popped up when writing some new features.
Is your feature request related to a problem? Please describe.
We have two separate codepaths for
extract_fieldsandextract_columns.Furthermore,
extract_fieldsrequires one to write the type in the extract, but we should have that information in the fn output.I think we can simplify the code quite a bit.
Describe the solution you'd like
A single
extract(name TBD) decorator that does the following:Then it can register "extensions" to allow for us to processing different extractions. E.G. different DF types, etc...
If its a
Dict[str, foo], we know it'll be a bunch offootype outputs. If its apd.DataFramewe know its going to bepd.Series. Etc...Describe alternatives you've considered
Not doing this.
Additional context
Just an idea that popped up when writing some new features.