API Reference

Functions

pandas_pyarrow.convert_to_pyarrow(df: DataFrame) → DataFrame

Apply a transformation to the dtypes of a Pandas DataFrame based on a mapping.

The function adjusts the data types of the columns in the provided DataFrame. It uses the current dtypes of the DataFrame columns, processes them through a mapping function to get the corresponding target dtypes, and applies the mapping to create a new DataFrame with updated dtypes.

Parameters:: df (pd.DataFrame) – A Pandas DataFrame whose column dtypes will be transformed.
Returns:: A new Pandas DataFrame with transformed column dtypes.
Return type:: pd.DataFrame

pandas_pyarrow.convert_to_numpy(df: DataFrame) → DataFrame

Maps the data types of a given pandas DataFrame to the target data types specified by the object’s internal mapping logic. This method processes the DataFrame by replacing its column data types according to the mapped target types and returns a new DataFrame with these updated data types. Any NaN values are handled accordingly during the process.

Parameters:: df (pd.DataFrame) – A pandas DataFrame that is to be processed.
Returns:: A new pandas DataFrame with updated column data types as per the mapping.
Return type:: pd.DataFrame

Classes

class pandas_pyarrow.PandasArrowConverter(custom_mapper: Dict[str, str] | None = None, default_target_type: str | None = 'string[pyarrow]')[source]

Bases: object

PandasArrowConverter manages the conversion of Pandas DataFrame data types to Arrow data types. :param custom_mapper: dictionary with key as the source data type and value as the target data type. Will override default mapping :param default_target_type: Optional string specifying the default data type to use if no mapping is found for a specific data type. Default is “string[pyarrow]”.

__call__(df: DataFrame) → DataFrame[source]

Apply a transformation to the dtypes of a Pandas DataFrame based on a mapping.

The function adjusts the data types of the columns in the provided DataFrame. It uses the current dtypes of the DataFrame columns, processes them through a mapping function to get the corresponding target dtypes, and applies the mapping to create a new DataFrame with updated dtypes.

Parameters:: df (pd.DataFrame) – A Pandas DataFrame whose column dtypes will be transformed.
Returns:: A new Pandas DataFrame with transformed column dtypes.
Return type:: pd.DataFrame

class pandas_pyarrow.ReversePandasArrowConverter(custom_mapper: Dict[str, str] | None = None, default_target_type: str | None = 'object')[source]

Bases: object

ReversePandasArrowConverter manages the conversion of pyarrow-backed Pandas DataFrame dtypes back to their Numpy/Pandas equivalents.

Parameters:

custom_mapper – Dictionary with key as the string-representation of the Arrow-backed dtype, and value as the desired target dtype (e.g. “object”, “int64”, etc.). This overrides default mapping returned by reverse_create_mapper().
default_target_type – Optional string specifying the default dtype to use if no mapping is found for a specific dtype. Default is “object”.

__call__(df: DataFrame) → DataFrame[source]

Maps the data types of a given pandas DataFrame to the target data types specified by the object’s internal mapping logic. This method processes the DataFrame by replacing its column data types according to the mapped target types and returns a new DataFrame with these updated data types. Any NaN values are handled accordingly during the process.

Parameters:: df (pd.DataFrame) – A pandas DataFrame that is to be processed.
Returns:: A new pandas DataFrame with updated column data types as per the mapping.
Return type:: pd.DataFrame