人工智能AutoML机器学习【免费下载链接】auto-sklearnAutomated Machine Learning with scikit-learn项目地址https://gitcode.com/gh_mirrors/au/auto-sklearn点击查看免费下载auto-sklearn 基于 scikit-learn 生态其自动化能力建立在组件component体系之上每个分类器、回归器或预处理器都被包装成一个组件纳入统一的配置空间搜索。本文基于仓库文档 doc/extending.rst完整讲解如何编写自定义组件wrapper class、定义其超参数搜索空间与数据属性并通过注册 API 将组件接入 auto-sklearn最后结合仓库源码与 examples/80_extending 目录下的可运行示例给出可直接复用的实战方案。读完本文你将掌握三种组件基类分类、回归、预处理的抽象接口、get_hyperparameter_search_space()与get_properties()的完整契约、add_classifier/add_regressor/add_preprocessor注册函数的用法以及如何通过include/exclude参数控制搜索空间中的可用组件。扩展机制总览auto-sklearn 可以被非常容易地扩展出新的分类classification、回归regression和特征预处理feature preprocessing方法。整个过程分为两步实现一个 wrapper 类把现有的机器学习模型或你自己实现的算法包装成组件并实现 auto-sklearn 所需的接口注册组件通过对应的注册函数告诉 auto-sklearn 该组件的存在使其进入配置空间参与搜索。这种设计让 auto-sklearn 的核心搜索机制SMAC 优化、成功减半、集成构建等完全复用你只需要关注模型本身。从源码看注册的组件会统一进入一个全局注册表_addons并与内置组件一起参与候选构建见 autosklearn/pipeline/components/base.py。选择正确的基类根据组件的用途它必须是以下三个基类之一的子类用途基类所在模块分类器AutoSklearnClassificationAlgorithmautosklearn/pipeline/components/base.py回归器AutoSklearnRegressionAlgorithmautosklearn/pipeline/components/base.py预处理器AutoSklearnPreprocessingAlgorithmautosklearn/pipeline/components/base.py这些基类本质上是现有机器学习模型的包装器只是额外补充了 auto-sklearn 需要的功能。当然你也可以直接在组件内部实现一个全新的机器学习算法。三个基类都继承自AutoSklearnComponent它又继承自 scikit-learn 的BaseEstimator因此自定义组件天然具备set_hyperparameters()能力配置空间中的每个超参数会通过configuration.get_dictionary()读取并以setattr方式注入组件实例见 autosklearn/pipeline/components/base.py。这意味着你的__init__参数名必须与配置空间中的超参数名一一对应。每个组件必须实现三类接口返回配置空间的方法get_hyperparameter_search_space()查询组件属性的方法get_properties()任务相关方法分类/回归器实现fit()与predict()预处理器实现fit()与transform()。以下分别详细说明。定义超参数搜索空间get_hyperparameter_search_space()该方法返回一个ConfigSpace.configuration_space.ConfigurationSpace实例它描述了该组件所有可调超参数及其取值范围、分布类型和默认值。三个基类的抽象定义可参考AutoSklearnClassificationAlgorithm.get_hyperparameter_search_space()AutoSklearnRegressionAlgorithm.get_hyperparameter_search_space()AutoSklearnPreprocessingAlgorithm.get_hyperparameter_search_space()方法签名统一为get_hyperparameter_search_space(feat_typeNone, dataset_propertiesNone)其中feat_type描述特征类型如数值、类别、文本dataset_properties描述数据集属性如是否稀疏、是否为多分类二者都可作为构建条件化配置空间的依据见 autosklearn/pipeline/components/base.py。以仓库示例中的 MLP 分类器为例examples/80_extending/example_extending_classification.pyfrom ConfigSpace.configuration_space import ConfigurationSpace from ConfigSpace.hyperparameters import ( CategoricalHyperparameter, UniformIntegerHyperparameter, UniformFloatHyperparameter, ) staticmethod def get_hyperparameter_search_space(feat_typeNone, dataset_propertiesNone): cs ConfigurationSpace() hidden_layer_depth UniformIntegerHyperparameter( namehidden_layer_depth, lower1, upper3, default_value1 ) num_nodes_per_layer UniformIntegerHyperparameter( namenum_nodes_per_layer, lower16, upper216, default_value32 ) activation CategoricalHyperparameter( nameactivation, choices[identity, logistic, tanh, relu], default_valuerelu, ) alpha UniformFloatHyperparameter( namealpha, lower0.0001, upper1.0, default_value0.0001 ) solver CategoricalHyperparameter( namesolver, choices[lbfgs, sgd, adam], default_valueadam ) cs.add_hyperparameters( [hidden_layer_depth, num_nodes_per_layer, activation, alpha, solver] ) return cs配置空间还支持条件依赖即某些超参数只有在另一些超参数取特定值时才生效。例如回归示例中degree和coef0仅在kernel polynomial时才有意义examples/80_extending/example_extending_regression.pyfrom ConfigSpace.conditions import EqualsCondition degree_condition EqualsCondition(degree, kernel, polynomial) coef0_condition EqualsCondition(coef0, kernel, polynomial) cs.add_conditions([degree_condition, coef0_condition])预处理器示例中则用InCondition实现shrinkage仅在solver为 lsqr/eigen 时生效examples/80_extending/example_extending_preprocessor.py。关于如何创建ConfigurationSpace对象的更多细节可参考仓库中autosklearn/pipeline/components/classification/、regression/、feature_preprocessing/等目录下内置组件的源码它们是最佳的参照实现。声明组件能力get_properties()该方法返回一个字典描述该组件在构建机器学习流水线时能被如何使用。以下字段必须全部指定字段集合由ThirdPartyComponents.add_component()强制校验见 autosklearn/pipeline/components/base.py字段类型含义shortnamestr组件的缩写namestr组件的完整名称handles_regressionbool是否能处理回归数据handles_classificationbool是否能处理分类数据handles_multiclassbool是否能处理多分类数据handles_multilabelbool是否能处理多标签分类数据is_deterministicbool在相同随机种子下多次运行是否给出相同结果注意仓库源码还额外要求handles_multioutput字段见 autosklearn/pipeline/components/base.py回归与特征预处理的组件选择逻辑也会读取它autosklearn/pipeline/components/regression/init.py。编写自定义组件时请一并提供。input与output字段这两个字段是元组声明组件接受和产生的数据类型取值使用 autosklearn/pipeline/constants.py 中定义的常量input组件能处理的数据类型可包含多个值autosklearn.constants.DENSE稠密数据数组与SPARSE互斥autosklearn.constants.SPARSE稀疏数据矩阵与DENSE互斥autosklearn.constants.UNSIGNED_DATA无符号数据仅正输入与SIGNED_DATA互斥autosklearn.constants.SIGNED_DATA有符号数据同时包含正负输入值与UNSIGNED_DATA互斥。output组件输出的数据类型autosklearn.constants.PREDICTIONS预测结果例如分类器的输出autosklearn.constants.INPUT与输入同形式的数据autosklearn.constants.DENSE稠密数据数组与SPARSE互斥声明后稀疏数据会被转换为稠密表示autosklearn.constants.SPARSE稀疏数据矩阵与DENSE互斥声明后稠密数据会被转换为稀疏表示autosklearn.constants.UNSIGNED_DATA无符号数据与SIGNED_DATA互斥允许只能处理正数据的算法使用autosklearn.constants.SIGNED_DATA有符号数据与UNSIGNED_DATA互斥。需要注意的是在导入时这些常量可以从autosklearn.pipeline.constants导入示例代码中使用的是from autosklearn.pipeline.constants import DENSE, SPARSE, ...而文档中引用的是autosklearn.constants命名空间——两者指向同一组数值常量见 autosklearn/pipeline/constants.py。以 LDA 预处理器为例examples/80_extending/example_extending_preprocessor.pystaticmethod def get_properties(dataset_propertiesNone): return { shortname: LDA, name: Linear Discriminant Analysis, handles_regression: False, handles_classification: True, handles_multiclass: False, handles_multilabel: False, handles_multioutput: False, is_deterministic: True, input: (DENSE, UNSIGNED_DATA, SIGNED_DATA), output: (DENSE, UNSIGNED_DATA, SIGNED_DATA), }get_properties()在组件选择时会被高频调用auto-sklearn 会依据数据集属性稀疏与否、是否多分类/多标签/多输出、目标任务类型等过滤掉不合适的组件。例如AutoSklearnChoice.get_available_components()会在数据集为稀疏时剔除input中不含SPARSE的组件见 autosklearn/pipeline/components/base.pyClassifierChoice则依据handles_classification/handles_multiclass/handles_multilabel过滤autosklearn/pipeline/components/classification/init.py。因此属性声明不准确会导致组件被静默排除或产生错误结果。注册组件告诉 auto-sklearn 组件存在编写完组件类后需要按组件类型调用以下注册函数分类器autosklearn.pipeline.components.classification.add_classifier()回归器autosklearn.pipeline.components.regression.add_regressor()预处理器autosklearn.pipeline.components.feature_preprocessing.add_preprocessor()三者都是对ThirdPartyComponents.add_component()的薄封装。以add_classifier为例from typing import Type from autosklearn.pipeline.components.base import AutoSklearnClassificationAlgorithm additional_components ThirdPartyComponents(AutoSklearnClassificationAlgorithm) _addons[classification] additional_components def add_classifier(classifier: Type[AutoSklearnClassificationAlgorithm]) - None: additional_components.add_component(classifier)add_component()会做两件事见 autosklearn/pipeline/components/base.py类型校验传入对象必须是base_class的直接子类否则抛出TypeError属性校验检查get_properties()返回的键集合——既不允许出现不在白名单中的多余属性也不允许缺失shortname、name、handles_regression、handles_classification、handles_multiclass、handles_multilabel、handles_multioutput、is_deterministic、input、output中的任何一项否则抛出ValueError。注册后组件会存入ThirdPartyComponents.components一个有序字典并在构建配置空间时通过ClassifierChoice.get_components()/RegressorChoice.get_components()/FeaturePreprocessorChoice.get_components()与内置组件合并autosklearn/pipeline/components/classification/init.py。每个组件还会以自身名字作为条件分支挂入顶层__choice__分类超参数autosklearn/pipeline/components/classification/init.py从而实现组件间选择 组件内调参的两级搜索空间结构。内置组件的自动发现除了add_*注册仓库还提供了一条自动发现路径find_components()会遍历组件目录中的每个 Python 模块凡是直接继承基类的类都会被自动收集见 autosklearn/pipeline/components/base.py。这意味着把新组件文件放进autosklearn/pipeline/components/classification/、regression/或feature_preprocessing/目录即可被自动加载。对于第三方扩展add_*系列 API 是更干净、不侵入源码树的方式。编写分类组件在get_properties()和get_hyperparameter_search_space()之外分类组件还需实现fit(X, y)和predict(X)它们是 scikit-learn predictor API 的实现。此外分类器通常还应实现predict_proba(X)以支持概率输出集成与部分评价指标依赖它。仓库中AutoSklearnClassificationAlgorithm的抽象接口见 autosklearn/pipeline/components/base.py要求组件内部持有底层估计器self.estimator并通过get_estimator()返回。MLP 分类器示例的完整骨架examples/80_extending/example_extending_classification.pyfrom autosklearn.pipeline.components.base import AutoSklearnClassificationAlgorithm class MLPClassifier(AutoSklearnClassificationAlgorithm): def __init__(self, hidden_layer_depth, num_nodes_per_layer, activation, alpha, solver, random_stateNone): self.hidden_layer_depth hidden_layer_depth self.num_nodes_per_layer num_nodes_per_layer self.activation activation self.alpha alpha self.solver solver self.random_state random_state def fit(self, X, y): # 将配置空间中的超参数转换为底层模型的参数 self.num_nodes_per_layer int(self.num_nodes_per_layer) self.hidden_layer_depth int(self.hidden_layer_depth) self.alpha float(self.alpha) from sklearn.neural_network import MLPClassifier hidden_layer_sizes tuple( self.num_nodes_per_layer for i in range(self.hidden_layer_depth) ) self.estimator MLPClassifier( hidden_layer_sizeshidden_layer_sizes, activationself.activation, alphaself.alpha, solverself.solver, random_stateself.random_state, ) self.estimator.fit(X, y) return self def predict(self, X): if self.estimator is None: raise NotImplementedError() return self.estimator.predict(X) def predict_proba(self, X): if self.estimator is None: raise NotImplementedError() return self.estimator.predict_proba(X)注册并启用import autosklearn.pipeline.components.classification autosklearn.pipeline.components.classification.add_classifier(MLPClassifier) clf autosklearn.classification.AutoSklearnClassifier( time_left_for_this_task30, per_run_time_limit10, include{classifier: [MLPClassifier]}, initial_configurations_via_metalearning0, smac_scenario_args{runcount_limit: 5}, ) clf.fit(X_train, y_train)include将搜索空间限定为你注册的组件initial_configurations_via_metalearning0与smac_scenario_args{runcount_limit: 5}仅用于加速示例运行真实场景不建议照搬。编写回归组件回归组件在get_properties()和get_hyperparameter_search_space()之外需实现fit(X, y)和predict(X)同样遵循 scikit-learn predictor API。抽象接口定义在AutoSklearnRegressionAlgorithmautosklearn/pipeline/components/base.py。仓库中的核岭回归KernelRidgeRegression示例examples/80_extending/example_extending_regression.py展示了完整写法其中值得关注的点input声明为(SPARSE, DENSE, UNSIGNED_DATA, SIGNED_DATA)output为(PREDICTIONS,)表明该回归器同时兼容稠密与稀疏输入使用logTrue的UniformFloatHyperparameter在对数尺度上搜索alpha、gamma、coef0这些超参数数量级跨度大通过EqualsCondition让degree、coef0只在kernel polynomial时被激活。注册与启用方式import autosklearn.regression import autosklearn.pipeline.components.regression autosklearn.pipeline.components.regression.add_regressor(KernelRidgeRegression) reg autosklearn.regression.AutoSklearnRegressor( time_left_for_this_task30, per_run_time_limit10, include{regressor: [KernelRidgeRegression]}, initial_configurations_via_metalearning0, smac_scenario_args{runcount_limit: 5}, ) reg.fit(X_train, y_train)编写特征预处理器特征预处理器在get_properties()和get_hyperparameter_search_space()之外需实现fit(X, yNone)和transform(X)同样是 scikit-learn transformer 风格抽象定义见 autosklearn/pipeline/components/base.py。注意预处理器内部持有的是self.preprocessor而非self.estimator通过get_preprocessor()返回。LDA 预处理器示例examples/80_extending/example_extending_preprocessor.pyclass LDA(AutoSklearnPreprocessingAlgorithm): def __init__(self, solver, tol, shrinkageNone, random_stateNone): self.solver solver self.shrinkage shrinkage self.tol tol self.random_state random_state self.preprocessor None def fit(self, X, yNone): if check_none(self.shrinkage): self.shrinkage None else: self.shrinkage float(self.shrinkage) self.tol float(self.tol) import sklearn.discriminant_analysis self.preprocessor sklearn.discriminant_analysis.LinearDiscriminantAnalysis( shrinkageself.shrinkage, solverself.solver, tolself.tol, ) self.preprocessor.fit(X, y) return self def transform(self, X): if self.preprocessor is None: raise NotImplementedError() return self.preprocessor.transform(X)注册并在分类任务中强制使用该预处理器import autosklearn.pipeline.components.feature_preprocessing autosklearn.pipeline.components.feature_preprocessing.add_preprocessor(LDA) clf autosklearn.classification.AutoSklearnClassifier( time_left_for_this_task30, include{feature_preprocessor: [LDA]}, initial_configurations_via_metalearning0, smac_scenario_args{runcount_limit: 5}, ) clf.fit(X_train, y_train)FeaturePreprocessorChoice.get_available_components()会依据dataset_properties[target_type]分别检查分类/回归相关属性autosklearn/pipeline/components/feature_preprocessing/init.py所以预处理器必须正确声明handles_classification/handles_regression等字段否则会被过滤。数据预处理器data preprocessor的扩展入口除特征预处理器外auto-sklearn 还有独立的数据预处理器data preprocessing阶段其注册入口是autosklearn.pipeline.components.data_preprocessing.add_preprocessor()。仓库示例 example_extending_data_preprocessor.py 用这一入口实现了一个NoPreprocessing组件fit原样返回自身、transform直接返回输入配合include{data_preprocessor: [NoPreprocessing]}即可完全关闭默认的数据预处理步骤。这类组件的output应声明为(INPUT,)表示输出与输入同形式。限制现有组件的超参数子类化复用除了从零编写组件还可以子类化内置组件来收紧或改写其超参数空间。仓库示例 example_restrict_number_of_hyperparameters.py 演示了这一点CustomRandomForest继承AutoSklearnClassificationAlgorithm只暴露n_estimators与max_features两个超参数内置random_forest则调参更多注册后通过exclude{classifier: [random_forest]}排除内置随机森林再用clf.get_configuration_space(X_train, y_train)断言配置空间中已不再包含random_forestclf autosklearn.classification.AutoSklearnClassifier( time_left_for_this_task30, per_run_time_limit10, exclude{classifier: [random_forest]}, initial_configurations_via_metalearning0, smac_scenario_args{runcount_limit: 1}, ) clf.fit(X_train, y_train) cs clf.get_configuration_space(X_train, y_train) assert random_forest not in str(cs)这里体现了include/exclude与注册机制的组合用法include精确指定要用的组件键为classifier/regressor/feature_preprocessor/data_preprocessor值为组件类名列表exclude则剔除指定组件二者不能同时使用见 autosklearn/automl.py 的参数定义与各get_available_components()中的互斥校验。组件是如何被接入流水线的理解底层接线有助于调试扩展问题。以分类器为例autosklearn/pipeline/components/classification/init.pyClassifierChoice.get_hyperparameter_search_space()先调用get_available_components()基于get_properties()声明与数据集属性过滤候选创建顶层分类超参数__choice__默认值按random_forest→liblinear_svc→sgd→libsvm_svc的优先级从可用组件中选取为每个候选组件调用其get_hyperparameter_search_space()并以parent_hyperparameter挂载为__choice__的条件分支形成嵌套配置空间搜索到具体配置后AutoSklearnChoice.set_hyperparameters()解析__choice__并实例化对应组件把带前缀的超参数名还原后传入构造器autosklearn/pipeline/components/base.py。因此自定义组件的__init__参数名、get_hyperparameter_search_space()中的超参数名、get_properties()声明必须保持严格一致且属性声明必须真实反映组件能力——任何一个环节不匹配都会导致组件被过滤、实例化失败或运行期报错。快速自查清单编写并注册一个组件时建议逐项核对基类选择正确分类/回归/预处理分别继承对应的AutoSklearn*Algorithmget_hyperparameter_search_space()返回合法ConfigurationSpace超参数名与__init__参数一致get_properties()包含全部必填字段含handles_multioutputinput/output使用DENSE/SPARSE/PREDICTIONS/INPUT/SIGNED_DATA/UNSIGNED_DATA常量分类器实现fit/predict建议同时实现predict_proba回归器实现fit/predict预处理器实现fit/transform均返回self且符合 scikit-learn API 约定调用对应的add_classifier/add_regressor/add_preprocessor完成注册通过include/exclude验证组件确实进入或退出了搜索空间可借助get_configuration_space()打印确认。完成以上步骤后你的自定义模型就能像内置组件一样被 auto-sklearn 的元学习、贝叶斯优化SMAC、成功减半与集成构建等整套机制驱动参与到自动化机器学习流水线的搜索与评估中。赞分享人工智能AutoML机器学习【免费下载链接】auto-sklearnAutomated Machine Learning with scikit-learn项目地址https://gitcode.com/gh_mirrors/au/auto-sklearn点击查看免费下载相关推荐终极指南如何5步快速上手AI机器人群聊平台botgroup.chat终极指南如何5步快速上手AI机器人群聊平台botgroup.chat 你是否曾想过让多个AI机器人在同一个聊天室里互动交流 botgroup.chat是AI 应用前端交互助手LAVIS 处理器扩展实战为视频对话任务编写、注册并配置自定义 ProcessorLAVIS 处理器扩展实战为视频对话任务编写、注册并配置自定义 Processor 本文是 LAVISA One stop Library for Lang人工智能多模态计算机视觉NLP深度学习预训练微调sklearn-pandas扩展开发终极指南如何创建自定义转换器和特征生成器sklearn pandas扩展开发终极指南如何创建自定义转换器和特征生成器 sklearn pandas是连接Scikit Learn机器学习库与panda上一篇无需Caffe5分钟掌握OpenCV dnn_objdetect模块ONNX目标检测部署 下一篇ip2region 快速入门3 分钟搭出 10 微秒级离线 IP 定位创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考