Mock测试【免费下载链接】motoA library that allows you to easily mock out tests based on AWS infrastructure.项目地址https://gitcode.com/gh_mirrors/mo/moto点击查看免费下载本文以 Moto 官方服务文档 docs/docs/services/sagemaker.rst 为骨架结合 moto/sagemaker 模块的模型、响应分发与测试代码系统梳理该 Mock 服务当前已实现与未实现的 API 全貌、关键资源模型、端点、训练任务、流水线、实验的底层行为约定并给出可直接落地的 boto3 测试示例帮助你准确判断哪些 SageMaker 场景可以在本地离线复现、哪些必须走真实 AWS。一、服务概览Moto 中 SageMaker Mock 的定位Moto 通过统一的mock_aws装饰器拦截 boto3 对https://api.sagemaker.region.amazonaws.com的请求——这一点由 moto/sagemaker/urls.py 中的url_bases定义所有请求最终由SageMakerResponse.dispatch分发到 moto/sagemaker/responses.py 中对应的处理方法进而操作 moto/sagemaker/models.py 里的内存后端SageMakerModelBackend约 6000 行包含全部资源模型类。与真实 AWS 一样资源通过标准 ARN 标识、按区域隔离、支持标签add_tags/list_tags/delete_tags。当前文档列出约 420 个 SageMaker API 动作其中打勾[X]的即为 Moto 已实现接口。整体呈现创建/描述/列表全覆盖、更新与删除部分覆盖、高级编排功能稀疏覆盖的格局。二、模型与推理部署链路Model → EndpointConfig → Endpoint这是 Moto 中实现最完整、与 CloudFormation 集成最深的资源链路对应测试见 tests/test_sagemaker/test_sagemaker_models.py 与 tests/test_sagemaker/test_sagemaker_endpoint.py。2.1 Model模型注册已实现create_model、describe_model、list_models、delete_model。create_model接受ModelName、ExecutionRoleArn、PrimaryContainer/Containers、VpcConfig、Tags等参数返回形如arn:aws:sagemaker:region:account:model/name的ModelArn。模型类Model位于 moto/sagemaker/models.py继承了CloudFormationModel支持create_from_cloudformation_json/update_from_cloudformation_json/delete_from_cloudformation_json因此可以在 CloudFormation 模板中直接声明AWS::SageMaker::Model测试见 tests/test_sagemaker/test_sagemaker_cloudformation.py。几个可验证的边界行为对不存在的模型调用describe_model/delete_model会抛出ClientError错误码为ValidationException消息包含Could not find model由 moto/sagemaker/exceptions.py 定义测试在test_sagemaker_models.py中直接断言。list_models返回Models列表每项含ModelName与ModelArn初始为空列表。2.2 EndpointConfig端点配置已实现create_endpoint_config、describe_endpoint_config、delete_endpoint_config、list_endpoint_configs。create_endpoint_config的核心参数是ProductionVariantsMoto 支持两种形态实例型VariantNameModelNameInstanceTypeInitialInstanceCountServerless 型VariantNameModelNameServerlessConfig含MemorySizeInMB、MaxConcurrency并在创建时通过FakeEndpointConfig.validate_serverless_config做校验。同时支持AsyncInferenceConfig异步推理含ClientConfig.MaxConcurrentInvocationsPerInstance与OutputConfig.S3OutputPath、DataCaptureConfig、KmsKeyId与Tags。关键约束创建 EndpointConfig 前ProductionVariants中引用的 Model 必须先存在否则返回Could not find model ...FakeEndpointConfig.validate_production_variants校验测试见test_sagemaker_endpoint.py。EndpointConfigArn格式为arn:aws:sagemaker:region:account:endpoint-config/name。该资源同样支持 CloudFormationAWS::SageMaker::EndpointConfig。2.3 Endpoint推理端点已实现create_endpoint、describe_endpoint、delete_endpoint、list_endpoints以及update_endpoint_weights_and_capacities调整各 production variant 的权重与容量。create_endpoint绑定已存在的EndpointConfigName生成EndpointArndescribe_endpoint返回EndpointStatusMoto 模拟为InService以及经过_process_production_variants加工后的ProductionVariants包含 Desired/Current 实例计数与权重字段update_endpoint_weights_and_capacities通过DesiredWeightsAndCapacities更新变体配置并返回新的EndpointArn。三、训练、转换与处理类作业TrainingJob / TransformJob / ProcessingJob / CompilationJob3.1 TrainingJob训练任务已实现create_training_job、describe_training_job、list_training_jobsstop_training_job、delete_training_job尚未实现。训练任务模型FakeTrainingJobmoto/sagemaker/models.py 第 343 行起在创建时即完成以下同步状态机模拟状态字段TrainingJobStatus与SecondaryStatus直接置为Completed无需等待按OutputDataConfig.S3OutputPath生成模型产物路径{S3OutputPath}/{training_job_name}/output/model.tar.gzModelArtifacts.S3ModelArtifacts测试在 tests/test_sagemaker/test_sagemaker_training.py 中断言其以输出路径开头、以output/model.tar.gz结尾自动写入AlgorithmSpecification.MetricDefinitions、SecondaryStatusTransitions起始状态Starting与FinalMetricDataListCreationTime、TrainingStartTime、TrainingEndTime、LastModifiedTime均为当前 UTC 时间。create_training_job支持HyperParameters、AlgorithmSpecificationTrainingImageTrainingInputMode、InputDataConfigS3DataSource的S3Prefix/ManifestFile与ShardedByS3Key/FullyReplicated分发方式、ResourceConfig、VpcConfig、StoppingCondition、CheckpointConfig、DebugHookConfig、DebugRuleConfigurations、TensorBoardOutputConfig、ExperimentConfig、EnableManagedSpotTraining等参数。list_training_jobs支持分页MaxResults/NextToken与过滤NameContains、StatusEquals、时间范围。测试中还验证了两条严格的输入校验StatusEquals必须是枚举[Completed, Stopped, InProgress, Stopping, Failed]之一否则返回ValidationException非法的NextToken返回Invalid pagination token。3.2 TransformJob批量转换与 ProcessingJob数据处理已实现create_transform_job、describe_transform_job、list_transform_jobsstop_transform_job未实现。FakeTransformJob接收TransformInputDataSourceSplitType、TransformOutput、TransformResourcesInstanceTypeInstanceCount、BatchStrategy、MaxConcurrentTransforms、MaxPayloadInMB、DataProcessing、Environment、ExperimentConfig等。已实现create_processing_job、describe_processing_job、stop_processing_job、list_processing_jobs。FakeProcessingJob支持AppSpecification容器镜像与参数、ProcessingInputs/ProcessingOutputConfigS3 输入输出、ProcessingResources、NetworkConfig、StoppingCondition、RoleArn。两者均遵循创建即Completed/InProgress简化状态 标准 ARN 时间戳的 Moto 约定测试分别见 tests/test_sagemaker/test_sagemaker_transform.py 与 tests/test_sagemaker/test_sagemaker_processing.py。3.3 CompilationJob模型编译 / Neo已实现create_compilation_job、describe_compilation_job、list_compilation_jobs、delete_compilation_job。FakeCompilationJob支持InputConfig、OutputConfigS3OutputLocation与TargetDevice等、RoleArn、StoppingCondition、ModelPackageVersionArn、VpcConfig见 tests/test_sagemaker/test_sagemaker_compilation_jobs.py。四、MLOps 编排Pipeline、Experiment / Trial / TrialComponent、FeatureGroup4.1 Pipeline流水线Pipeline 是文档中实现动作最多的子领域之一create_pipeline、describe_pipeline、update_pipeline、delete_pipeline、list_pipelines、start_pipeline_execution、describe_pipeline_execution、describe_pipeline_definition_for_execution、list_pipeline_parameters_for_execution、list_pipeline_executions均已实现stop_pipeline_execution、retry_pipeline_execution、send_pipeline_execution_step_success/failure未实现。值得注意的底层细节create_pipeline除PipelineDefinition字符串外还支持PipelineDefinitionS3Location从 S3 读取定义moto/sagemaker/utils.py 中的load_pipeline_definition_from_s3负责加载执行状态机简化start_pipeline_execution生成PipelineExecutionArn执行状态模拟为Executing→Succeeded并可携带PipelineParameters、ParallelismConfiguration、ClientRequestToken相关辅助函数按名称/ARN 查找 pipeline 与 execution集中在 moto/sagemaker/utils.py测试见 tests/test_sagemaker/test_sagemaker_pipeline.py。4.2 Experiment / Trial / TrialComponent实验追踪三件套Experimentcreate_experiment、describe_experiment、list_experiments、delete_experimentTrialcreate_trial、describe_trial、list_trials、delete_trialTrialComponentcreate_trial_component、describe_trial_component、update_trial_component、list_trial_components、delete_trial_component、associate_trial_component、disassociate_trial_component。FakeTrialComponent支持Parameters含数值与字符串两种类型值、InputArtifacts/OutputArtifacts、MetadataProperties、Status、StartTime/EndTime等字段update_trial_component支持ParametersToRemove、InputArtifactsToRemove、OutputArtifactsToRemove的增量删除。create_trial可将 Trial 挂到 Experiment 下list_trials支持按ExperimentName/TrialComponentName过滤。测试见 tests/test_sagemaker/test_sagemaker_experiment.py、test_sagemaker_trial.py、test_sagemaker_trial_component.py。4.3 FeatureGroup特征组已实现create_feature_group、describe_feature_groupdelete_feature_group、list_feature_groups、update_feature_group未实现。创建参数包括FeatureGroupName、RecordIdentifierFeatureName、EventTimeFeatureName、FeatureDefinitions特征名 类型、OfflineStoreConfig、RoleArn、Tagsdescribe_feature_group返回创建时写入的特征定义与配置测试见 tests/test_sagemaker/test_sagemaker_feature_groups.py。五、模型治理与质量监控类作业定义Moto 实现了四类 SageMaker Clarify / 监控类的 JobDefinition 资源均遵循create_*_job_definition返回 ARN →describe/list/delete管理的模式作业定义已实现动作关键参数DataQualityJobDefinitioncreate / describe / list / deleteDataQualityAppSpecification、DataQualityJobInput、DataQualityJobOutputConfig、DataQualityBaselineConfig、JobResourcesModelQualityJobDefinitioncreate / describe / list / deleteModelQualityAppSpecification、ModelQualityJobInput、ModelQualityJobOutputConfig、ModelQualityBaselineConfig、JobResourcesModelBiasJobDefinitioncreate / describe / list / deleteModelBiasAppSpecification、ModelBiasJobInput、ModelBiasJobOutputConfig、ModelBiasBaselineConfigModelExplainabilityJobDefinitioncreate / describe / list / deleteModelExplainabilityAppSpecification、ModelExplainabilityJobInput、ModelExplainabilityJobOutputConfig、ModelExplainabilityBaselineConfig这些资源统一接收JobDefinitionName、RoleArn、JobResourcesClusterConfig、NetworkConfig、StoppingCondition、Environment、Tags返回JobDefinitionArn。对应测试test_sagemaker_data_quality_jobs_definitions.py、test_sagemaker_model_quality.py、test_sagemaker_model_bias_job_definitions.py、test_sagemaker_model_expainability.py。六、Notebook 实例与 Domain托管环境6.1 NotebookInstance已实现create_notebook_instance、start_notebook_instance、stop_notebook_instance、delete_notebook_instance、list_notebook_instances、create_notebook_instance_lifecycle_config、describe_notebook_instance_lifecycle_config、delete_notebook_instance_lifecycle_configdescribe_notebook_instance未实现列表中也无update_*。创建参数包括InstanceType、RoleArn、SubnetId、SecurityGroupIds、KmsKeyId、LifecycleConfigName、DirectInternetAccess默认Enabled、VolumeSizeInGB默认 5、AcceleratorTypes、DefaultCodeRepository、AdditionalCodeRepositories、RootAccess。后端通过validate_volume_size_in_gb与validate_instance_type做参数校验。生命周期脚本OnCreate/OnStart以{Content: base64 字符串}字典形式传入。Notebook 实例同样支持 CloudFormationAWS::SageMaker::NotebookInstance与AWS::SageMaker::NotebookInstanceLifecycleConfig测试见 tests/test_sagemaker/test_sagemaker_notebooks.py。6.2 DomainStudio 域已实现create_domain、describe_domain、list_domains、delete_domain。创建参数包括DomainName、AuthMode、DefaultUserSettings、SubnetIds、VpcId、DomainSettings、AppNetworkAccessType、HomeEfsFileSystemKmsKeyId、KmsKeyId、AppSecurityGroupManagement、DefaultSpaceSettingsdelete_domain支持RetentionPolicy保留/删除 Home EFS。测试见 tests/test_sagemaker/test_sagemaker_domains.py。七、AutoML、超参调优与 SageMaker HyperPod 集群AutoMLV2已实现create_auto_ml_job_v2、describe_auto_ml_job_v2、list_auto_ml_jobs、stop_auto_ml_job。FakeAutoMLJob支持AutoMLJobInputDataConfig、OutputDataConfig、AutoMLProblemTypeConfig、RoleArn、SecurityConfig、AutoMLJobObjective、ModelDeployConfig、DataSplitConfig测试见 tests/test_sagemaker/test_sagemaker_automljobs.py。HyperParameterTuningJob已实现create_hyper_parameter_tuning_job、describe_hyper_parameter_tuning_job、list_hyper_parameter_tuning_jobs、delete_hyper_parameter_tuning_jobstop_hyper_parameter_tuning_job未实现。支持HyperParameterTuningJobConfig策略、目标指标、资源限制、TrainingJobDefinition与TrainingJobDefinitions、WarmStartConfig、Autotune测试见 tests/test_sagemaker/test_sagemaker_hyper_parameter_tuning_job.py。HyperPod Cluster已实现create_cluster、describe_cluster、delete_cluster、list_clusters、describe_cluster_node、list_cluster_nodes。集群模型校验InstanceGroups中的节点实例类型valid_cluster_node_instance_typesdescribe_cluster_node返回节点的生命周期配置与执行角色信息测试见 tests/test_sagemaker/test_sagemaker_cluster.py。八、ModelPackage 与 ModelCard模型注册中心ModelPackageGroupcreate_model_package_group、describe_model_package_group、list_model_package_groupsput_model_package_group_policy/get_model_package_group_policy未实现。ModelPackagecreate_model_package、describe_model_package、update_model_package、list_model_packagesdelete_model_package未实现。update_model_package支持ModelApprovalStatus由 moto/sagemaker/utils.py 的validate_model_approval_status校验、ApprovalDescription、CustomerMetadataProperties的增删、AdditionalInferenceSpecificationsToAddcreate_model_package支持版本化ModelPackageGroupName与非版本化ModelPackageName两种形态。测试见 test_sagemaker_model_package_groups.py 与 test_sagemaker_model_packages.py。ModelCardcreate_model_card、update_model_card、describe_model_card、delete_model_card、list_model_cards、list_model_card_versions全部实现。ModelCard 以Content存储 JSON 字符串并按版本号递增管理支持ModelCardStatusDraft/PendingReview/Approved/Archived测试见 tests/test_sagemaker/test_sagemaker_model_cards.py。九、跨资源的 Tags 与 SearchTagsadd_tags/list_tags/delete_tags已实现作用于任意带 ARN 的资源SageMakerModelBackend._get_resource_from_arn解析 ARN 前缀定位资源是测试中给模型、端点、训练任务打标签的通用手段。Searchsearch已实现但只支持部分 SearchExpression文档明确标注 Only a few SearchExpressions are implemented. Please open a bug report if you find any issues.。从 moto/sagemaker/models.py 的compare_value/evaluate_search_expression实现看它对已注册资源执行基于运算符的字段比较覆盖的资源与算子有限复杂嵌套表达式多条件And/Or、NestedFilters、SubExpression等可能不被支持。在编写依赖 Search 的测试前建议先在 tests/test_sagemaker/test_sagemaker_search.py 中确认当前支持的表达式形态。十、已知未实现项与参数缺口除上述各节标注的未实现动作外文档明确列出了list_notebook_instances当前不生效的过滤参数CreationTimeBefore, CreationTimeAfter, LastModifiedTimeBefore, LastModifiedTimeAfter, NotebookInstanceLifecycleConfigNameContains, DefaultCodeRepositoryContains, AdditionalCodeRepositoryEquals即这些过滤条件会被静默忽略列表仅按SortBy/SortOrder/NameContains/StatusEquals生效。此外大量面向真实基础设施的动作如create_workteam、create_labeling_job、create_monitoring_schedule、create_inference_component、query_lineage、update_domain、update_feature_group、stop_training_job、stop_transform_job、create_presigned_notebook_instance_url等均未实现——它们不在SageMakerResponse中提供处理入口调用会得到标准未实现响应。完整对照以 docs/docs/services/sagemaker.rst 的勾选项为准。十一、实战在单元测试中驱动 SageMaker 主链路以下示例综合了模型、端点配置、端点与训练任务四条链路并验证标签与搜索行为可直接在本地离线运行import boto3 import pytest from moto import mock_aws ROLE_ARN arn:aws:iam::123456789012:role/FakeRole REGION us-east-1 mock_aws def test_sagemaker_main_flow(): sm boto3.client(sagemaker, region_nameREGION) # 1. 注册模型 model_arn sm.create_model( ModelNamemy-model, ExecutionRoleArnROLE_ARN, PrimaryContainer{ Image: 382416733822.dkr.ecr.us-east-1.amazonaws.com/linear-learner:1, ModelDataUrl: s3://my-bucket/model/model.tar.gz, }, )[ModelArn] assert model_arn.endswith(:model/my-model) # 2. 创建端点配置引用已存在模型否则报 ValidationException ec_arn sm.create_endpoint_config( EndpointConfigNamemy-ec, ProductionVariants[ { VariantName: AllTraffic, ModelName: my-model, InitialInstanceCount: 1, InstanceType: ml.t2.medium, } ], )[EndpointConfigArn] assert ec_arn.endswith(:endpoint-config/my-ec) # 3. 创建端点并打标签 sm.create_endpoint(EndpointNamemy-endpoint, EndpointConfigNamemy-ec) endpoint_arn sm.describe_endpoint(EndpointNamemy-endpoint)[EndpointArn] sm.add_tags(ResourceArnendpoint_arn, Tags[{Key: env, Value: test}]) assert sm.list_tags(ResourceArnendpoint_arn)[Tags] [ {Key: env, Value: test} ] # 4. 创建训练任务同步返回 Completed 并生成模型产物路径 training sm.create_training_job( TrainingJobNamemy-job, RoleArnROLE_ARN, AlgorithmSpecification{ TrainingImage: 382416733822.dkr.ecr.us-east-1.amazonaws.com/linear-learner:1, TrainingInputMode: File, }, ResourceConfig{ InstanceCount: 1, InstanceType: ml.c4.2xlarge, VolumeSizeInGB: 10, }, InputDataConfig[ { ChannelName: train, DataSource: { S3DataSource: { S3DataType: S3Prefix, S3Uri: s3://my-bucket/sagemaker/train/, S3DataDistributionType: ShardedByS3Key, } }, } ], OutputDataConfig{S3OutputPath: s3://my-bucket/sagemaker/}, StoppingCondition{MaxRuntimeInSeconds: 3600}, ) assert training[TrainingJobArn].endswith(:training-job/my-job) desc sm.describe_training_job(TrainingJobNamemy-job) assert desc[TrainingJobStatus] Completed assert desc[ModelArtifacts][S3ModelArtifacts] ( s3://my-bucket/sagemaker/my-job/output/model.tar.gz ) # 5. 列表与分页 assert len(sm.list_training_jobs()[TrainingJobSummaries]) 1 page sm.list_training_jobs(MaxResults1) assert page[NextToken] is not None # 6. 清理 sm.delete_endpoint(EndpointNamemy-endpoint) sm.delete_endpoint_config(EndpointConfigNamemy-ec) sm.delete_model(ModelNamemy-model)要点回顾TrainingJobStatus直接为CompletedModelArtifacts.S3ModelArtifacts恒为{S3OutputPath}/{TrainingJobName}/output/model.tar.gzlist_training_jobs的StatusEquals只接受Completed/Stopped/InProgress/Stopping/Failed。十二、源码导航快速定位实现与测试目的路径服务入口与 URL 匹配moto/sagemaker/urls.pyAPI 请求分发层每个已实现动作对应一个 handlermoto/sagemaker/responses.py资源模型与后端逻辑ARN、状态机、校验moto/sagemaker/models.py通用工具ARN 格式化、Pipeline/S3 加载、ModelCard 过滤、审批状态校验moto/sagemaker/utils.py参数校验辅助is_integer_between、is_one_ofmoto/sagemaker/validators.py自定义异常模型不存在等moto/sagemaker/exceptions.py全部行为测试模型、端点、训练、流水线、Search 等 28 个文件tests/test_sagemaker结语Moto 的 SageMaker Mock 以内存对象 ARN 标识 简化状态机为设计主线对训练、转换、处理、端点部署、流水线与实验追踪等高频测试场景提供了即建即用的支持同时对真实 AWS 依赖较重的高级能力标注、监控调度、Workteam、Inference Component、Lineage 查询等保持克制地未实现。写测试时建议以本文第十一节的方式先验证目标 API 是否在 docs/docs/services/sagemaker.rst 勾选清单中再结合tests/test_sagemaker下的对应用例确认参数校验与返回结构即可避免本地全绿、上云即红的落差。赞分享Mock测试【免费下载链接】motoA library that allows you to easily mock out tests based on AWS infrastructure.项目地址https://gitcode.com/gh_mirrors/mo/moto点击查看免费下载相关推荐如何按照 Unleash Client Specification 实现自定义客户端 SDK如何按照 Unleash Client Specification 实现自定义客户端 SDK 如果你的业务场景官方 SDK 无法覆盖私有语言、嵌入式环境、特殊Mock测试moto 中 AWS DirectConnect Mock 实现解析:已支持 API 清单、内部机制与使用边界moto 中 AWS DirectConnect Mock 实现解析:已支持 API 清单、内部机制与使用边界 本文基于 moto 仓库中 DirectConnMock测试moto 中 AWS Directory Service (ds) 的 Mock 实现已支持 API、底层原理与实战测试moto 中 AWS Directory Service ds 的 Mock 实现已支持 API、底层原理与实战测试 AWS Directory ServicMock测试上一篇探索洛雪音乐播放器打造个人专属音乐中心的完整指南下一篇三步搞定3DS主题管理Anemone3DS完整上手指南创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考