pipeline

Full experiment pipeline with train/test evaluation.

pipeline

@module: sce.pipeline @depends: sklearn, sce.config, sce.engine @exports: fit_context_pipeline, create_sce_pipeline @paper_ref: Section 6 (Implementation) @data_flow: raw_df -> sce_features -> model -> predictions

create_sce_pipeline

create_sce_pipeline(config: ContextConfig, model: Optional[BaseEstimator] = None) -> Pipeline

Create a scikit-learn pipeline with SCE feature engineering.

Parameters:

Name	Type	Description	Default
config	ContextConfig	SCE configuration	required
model	Optional[BaseEstimator]	Optional model to add at the end (e.g., XGBoost, LightGBM)	None

Returns:

Type	Description
Pipeline	sklearn Pipeline with SCE transformer and optional model

Example

from sklearn.ensemble import RandomForestRegressor from sce import ContextConfig, create_sce_pipeline

config = ContextConfig(categorical_cols=["region", "city"], target_col="price") pipeline = create_sce_pipeline(config, model=RandomForestRegressor()) X_train = train_df.drop(columns=["price"]) y_train = train_df["price"] pipeline.fit(X_train, y_train)

Note

The SCE step augments the feature matrix, but it does not encode raw string columns for downstream estimators. If your model cannot consume string-valued columns directly, encode or preprocess them before fitting the final model.

Source code in sce/pipeline.py

def create_sce_pipeline(config: ContextConfig, model: Optional[BaseEstimator] = None) -> Pipeline:
    """
    Create a scikit-learn pipeline with SCE feature engineering.

    Args:
        config: SCE configuration
        model: Optional model to add at the end (e.g., XGBoost, LightGBM)

    Returns:
        sklearn Pipeline with SCE transformer and optional model

    Example:
        >>> from sklearn.ensemble import RandomForestRegressor
        >>> from sce import ContextConfig, create_sce_pipeline
        >>>
        >>> config = ContextConfig(categorical_cols=["region", "city"], target_col="price")
        >>> pipeline = create_sce_pipeline(config, model=RandomForestRegressor())
        >>> X_train = train_df.drop(columns=["price"])
        >>> y_train = train_df["price"]
        >>> pipeline.fit(X_train, y_train)

    Note:
        The SCE step augments the feature matrix, but it does not encode raw string
        columns for downstream estimators. If your model cannot consume string-valued
        columns directly, encode or preprocess them before fitting the final model.
    """
    steps = [("sce", StatisticalContextEngine(config))]

    if model is not None:
        steps.append(("model", model))

    return Pipeline(steps)

fit_context_pipeline

fit_context_pipeline(df: DataFrame, config: ContextConfig, model: Optional[BaseEstimator] = None, **fit_params: Any) -> Pipeline

Convenience function to fit a complete SCE pipeline.

Parameters:

Name	Type	Description	Default
df	DataFrame	Training dataframe	required
config	ContextConfig	SCE configuration	required
model	Optional[BaseEstimator]	Optional model to include	None
**fit_params	Any	Additional parameters for model fitting	{}

Returns:

Type	Description
Pipeline	Fitted pipeline

Source code in sce/pipeline.py

def fit_context_pipeline(
    df: pd.DataFrame,
    config: ContextConfig,
    model: Optional[BaseEstimator] = None,
    **fit_params: Any,
) -> Pipeline:
    """
    Convenience function to fit a complete SCE pipeline.

    Args:
        df: Training dataframe
        config: SCE configuration
        model: Optional model to include
        **fit_params: Additional parameters for model fitting

    Returns:
        Fitted pipeline
    """
    pipeline = create_sce_pipeline(config, model)

    y = df[config.target_col]
    X = df.drop(columns=[config.target_col])

    if model is not None:
        pipeline.fit(X, y, **fit_params)
    else:
        pipeline.fit(X, y)

    return pipeline