相关概念节点分类节点是一个独立的服务器实例或进程用于存储数据并参与集群的索引和搜索功能。每个节点都有自己的名称、角色和职责。master node负责管理集群的整体状态和执行一些关键的集群级别的操作。主节点负责将分片存储到数据节点、维护集群的状态信息、决定哪些节点是集群的一部分等。本身不负责存储实际的数据。data node数据节点负责存储实际的数据。管理索引的分片并处理与数据相关的操作如搜索、索引和删除。数据节点存储索引的一部分并通过将数据分布在多个节点上来实现水平扩展。coordinating node协调节点是可选的节点类型主要用于处理搜索请求的协调工作。不存储实际的数据而是负责接收来自客户端的搜索请求并将这些请求转发给数据节点。协调节点有助于分散搜索请求的负载特别是在大规模集群中。ingest node预处理节点负责数据的预处理如数据的转换和附加处理。这允许在将数据索引到es之前对数据进行修改或过滤。该节点通常用于日志处理和数据管道中。machine learning node用于执行与机器学习相关的任务如异常检测或趋势分析。remote-eligible node远程合格节点拥有远程集群客户端角色可以充当远程客户端。默认情况下集群内任意节点都可以作为跨集群的客户端连接到远程集群。transform节点用于执行数据转换操作例如从一个索引中提取数据并将其转换为另一个索引以便在数据仓库和分析方面进行使用。index索引是一个逻辑存储类似关系型数据库中的数据库用来存储管理相关数据数据以JSON格式存储在索引的基本单元-文档Document。shard一条索引的数据往往会被分为多个分片进行存储每个分片是一个独立的存储单元底层为一个Lucene索引可以在集群中的不同节点上分布以提供高可用性和负载均衡。分片的目的允许索引水平分割和分布存储实现数据和负载的分布。分片的数量在索引创建后不能被更改。通常每个主分片的大小应该适中以便在集群中的各个节点上均匀分布负载也有利于数据的快速查询。主分片每个索引都可以被分为一个或多个主分片。主分片存储索引的一部分数据并负责处理搜索和查询操作。主分片的数量设置在索引创建时便被定义无法进行动态修改。Replica副本是指索引的一个复制每个索引可以被配置为具有零个或多个副本。副本可以提高集群的可用性和容错性以及提高搜索性能。每个索引被划分为一个或多个主分片。主分片负责存储索引的一部分数据以及处理搜索和写入请求。为了提高可用性和容错性每个主分片可以有零个或多个副本。每个索引的副本数量是可以配置的。在创建索引时可以指定副本的数量。副本数量的选择取决于对可用性和性能的需求。mapping映射是索引中文档的规范定义了结构和字段属性描述了文档中的每个字段的数据类型、分词器、是否索引等信息。类似于关系型数据库中的Schema。举个例子PUT/book{mappings:{properties:{title:{type:text,analyzer:standard},author:{type:keyword},publish date:{type:date},views:{type:integer},tags:{type:keyword}}}}index的名称为booktitle是一个文本字段使用标准分词器author是一个关键字字段用于精确匹配publish_date是一个日期字段views是一个整数字段tags是一个关键字字段用于标签信息环境单机安全dockerpull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/opensearchproject/opensearch-dashboardsdockerpull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/opensearchproject/opensearch# 部署yaml文件mkdir-p/root/opensearch-dockercatdocker-compose.yml version:3services: opensearch: image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/opensearchproject/opensearch:latest container_name: opensearch# restart: unless-stoppedenvironment: -discovery.typesingle-node -OPENSEARCH_INITIAL_ADMIN_PASSWORDYourStrongPass123 ports: -9200:9200-9600:9600ulimits: memlock:{soft: -1, hard: -1}nofile:{soft:65536, hard:65536}volumes: - opensearch-data:/usr/share/opensearch/data opensearch-dashboards: image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/opensearchproject/opensearch-dashboards:latest container_name: opensearch-dashboards# restart: unless-stoppedports: -5601:5601volumes: - /root/opensearch-docker/config/opensearch_dashboards.yml:/usr/share/opensearch-dashboards/config/opensearch_dashboards.yml:ro - /root/opensearch-docker/certs:/usr/share/opensearch-dashboards/config/certs:ro depends_on: - opensearch volumes: opensearch-data:# 加密证书填写访问ip地址mkdir-p/root/opensearch-docker/certs openssl req-x509-newkeyrsa:2048-nodes\-keyout/root/opensearch-docker/certs/dashboards-key.pem\-out/root/opensearch-docker/certs/dashboards-crt.pem\-days365\-subj/CCN/OMyCompany/CN192.168.52.1\-addextsubjectAltNameIP:192.168.52.1,DNS:localhost# dashboard配置文件mkdir-p/root/opensearch-docker/configcatopensearch_dashboards.yml server.host:0.0.0.0server.ssl.enabled:trueserver.ssl.certificate: /usr/share/opensearch-dashboards/config/certs/dashboards-crt.pem server.ssl.key: /usr/share/opensearch-dashboards/config/certs/dashboards-key.pem opensearch.hosts:[https://opensearch:9200]opensearch.username: admin opensearch.password: YourStrongPass123 opensearch.ssl.verificationMode: none opensearch_security.cookie.secure:true# 权限chown-R1000:1000 /root/opensearch-docker/certschmod600/root/opensearch-docker/certs/dashboards-key.pemchmod644/root/opensearch-docker/certs/dashboards-crt.pem# 删除、启动docker-composedowndocker-composeup-d# 验证9200、5601均是加密访问curl-k-uadmin:YourStrongPass123 https://localhost:9200# 浏览器 https://localhost:5601多机安全# 根CAmkdir-p/root/opensearch-docker/certscd/root/opensearch-docker/certs openssl genrsa-outroot-ca-key.pem4096openssl req-x509-new-nodes\-keyroot-ca-key.pem\-sha256-days3650\-subj/CCN/OMyCompany/CNMyRootCA\-outroot-ca.pem# 一套通用节点证书CERT_DIR/root/opensearch-docker/certscd${CERT_DIR}## 私钥所有节点共用openssl genrsa-outnode-key.pem2048## CSRopenssl req-new-keynode-key.pem\-subj/CCN/OMyCompany/CNopensearch-node\-outnode.csr## SAN扩展把OpenSearch IP Dashboard IP 全部写进去catnode.extEOF subjectAltName IP:192.168.52.129, IP:192.168.52.130, IP:192.168.52.131, IP:192.168.52.1, DNS:localhost EOF## 用根CA签发openssl x509-req-innode.csr\-CA${CERT_DIR}/root-ca.pem\-CAkey${CERT_DIR}/root-ca-key.pem\-CAcreateserial\-out${CERT_DIR}/node.pem\-days365-sha256\-extfile${CERT_DIR}/node.ext## 权限chmod600${CERT_DIR}/node-key.pemchmod644${CERT_DIR}/node.pem${CERT_DIR}/root-ca.pem# root-ca-key.pem不分发# root-ca.pem、node.pem、node-key.pem手动分发私钥到各节点# 内核参数sysctl-wvm.max_map_count262144echovm.max_map_count262144/etc/sysctl.confsysctl-p# 先起node-19200端口正常再起node-2、node-3# opensearch节点部署文件services: opensearch: image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/opensearchproject/opensearch:latest container_name: opensearch network_mode:hostrestart: unless-stopped environment: -cluster.namemy-opensearch-cluster -node.namenode-1# 每台机器改-node.rolescluster_manager,data,ingest-discovery.seed_hosts192.168.52.129,192.168.52.130,192.168.52.131 -cluster.initial_cluster_manager_nodesnode-1,node-2,node-3 -OPENSEARCH_INITIAL_ADMIN_PASSWORDYourStrongPass123 -bootstrap.memory_locktrue -OPENSEARCH_JAVA_OPTS-Xms4g-Xmx4gulimits: memlock:{soft: -1, hard: -1}nofile:{soft:65536, hard:65536}volumes: - opensearch-data:/usr/share/opensearch/data - /root/opensearch-docker/certs:/usr/share/opensearch/config/certs:ro - /root/opensearch-docker/config/opensearch.yml:/usr/share/opensearch/config/opensearch.yml:ro volumes: opensearch-data:# opensearch节点配置文件cluster.name: my-opensearch-cluster node.name: node-1# 每台机器改node.roles:[cluster_manager, data, ingest]discovery.seed_hosts:[192.168.52.129,192.168.52.130,192.168.52.131]cluster.initial_cluster_manager_nodes:[node-1,node-2,node-3]bootstrap.memory_lock:truenetwork.host:0.0.0.0# Transport 层plugins.security.ssl.transport.pemcert_filepath: /usr/share/opensearch/config/certs/node.pem plugins.security.ssl.transport.pemkey_filepath: /usr/share/opensearch/config/certs/node-key.pem plugins.security.ssl.transport.pemtrustedcas_filepath: /usr/share/opensearch/config/certs/root-ca.pem plugins.security.ssl.transport.enforce_hostname_verification:false# HTTP 层plugins.security.ssl.http.enabled:trueplugins.security.ssl.http.pemcert_filepath: /usr/share/opensearch/config/certs/node.pem plugins.security.ssl.http.pemkey_filepath: /usr/share/opensearch/config/certs/node-key.pem plugins.security.ssl.http.pemtrustedcas_filepath: /usr/share/opensearch/config/certs/root-ca.pem# 可信节点证书plugins.security.nodes_dn: -CNopensearch-node,OMyCompany,CCNplugins.security.allow_default_init_securityindex:true# dashboard节点部署文件services: opensearch-dashboards: image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/opensearchproject/opensearch-dashboards:latest container_name: opensearch-dashboards network_mode:hostrestart: unless-stopped volumes: - /root/opensearch-docker/config/opensearch_dashboards.yml:/usr/share/opensearch-dashboards/config/opensearch_dashboards.yml:ro - /root/opensearch-docker/certs:/usr/share/opensearch-dashboards/config/certs:ro# dashboard节点配置文件server.host:0.0.0.0server.ssl.enabled:trueserver.ssl.certificate: /usr/share/opensearch-dashboards/config/certs/node.pem server.ssl.key: /usr/share/opensearch-dashboards/config/certs/node-key.pem opensearch.hosts:[https://192.168.52.129:9200,https://192.168.52.130:9200,https://192.168.52.131:9200]opensearch.username: admin opensearch.password: YourStrongPass123 opensearch.ssl.certificateAuthorities:[/usr/share/opensearch-dashboards/config/certs/root-ca.pem]opensearch.ssl.verificationMode: full opensearch_security.cookie.secure:true