php实现timespan技巧_Elasticsearch项目实战

目录

Elasticsearch是一个基于Apache Lucene(TM)的开源搜索引擎。
无论在开源还是专有领域，Lucene可以被认为是迄今为止最 前辈、性能最好的、功能最全的搜索引擎库。

一说到全文搜索,lucene久负盛名。
从前间,由于项目须要,打仗过一个叫盘古分词的开源项目,借助个中的分词实现了分词搜索的功能。
而盘古分词便是lucence的.NET版本。
听说这个开源项目已经规复更新并支持. NET Core,有兴趣的童鞋可以去围不雅观一下(https://github.com/LonghronShen/Lucene.Net.Analysis.PanGu/tree/netcore2.0)。

我想很多童鞋都听过ELK,ELK是Elasticsearch、Logstash、Kibana。
恰好公司运维同事引入了这样一套体系,用于建立集中式日志网络系统，将所有节点上的日志统一网络，管理，访问。
虽然能够从一定程度上办理基本的问题,但是原生的kibana界面和查询办法都不足友好,很难推向广大的开拓职员。
于是我在想，我们是否可以利用这个开源的库集成到运维自动化平台当中,让这把利剑发挥出更大的代价。

一、环境搭建

本文是基于windows 10操作系统的es环境的搭建。

由于es是java措辞开拓的,以是这里要安装java环境。

jdk下载:

https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

安装完成之后便是配置环境变量:

查看是否安装成功:

2.安装Elasticsearch

Elasticsearch版本已经比较多,初学者可能比较懵。
特殊是在安装head和Kibana的时候,如果版本不匹配,每每会导致无法利用。
这里利用的是elasticsearch-5.6.11版本。

elasticsearch-5.6.11下载:

https://www.elastic.co/downloads/past-releases/elasticsearch-5-6-11

解压到C:\ELk 备用。

3.head安装(非必需)

es 4.x 版本安装head很大略,只需下载head插件解压到指定目录即可。
es 5.x+须要借助node安装。

head下载:

https://github.com/mobz/elasticsearch-head

解压到C:\ELk\elasticsearch-5.6.11

node下载:

https://nodejs.org/dist/v8.12.0/node-v8.12.0-win-x64.zip

安装node

检讨node和npm是否安装成功

path环境变量末端 会自动增加 C:\Program Files\nodejs\

安装 phantomjs 官网:http://phantomjs.org/下载【配置环境变量】

安装grunt npm install -g grunt-cli

实行C:\ELk\elasticsearch-5.6.11\bin\elasticsearch.bat

实行命令启动 head

浏览器访问:http://localhost:9100/

4.安装kibana

导致为止,实在elasticsearch自身已经安装完成。
通过Head就能很方便的操作es,但是kibana集成了head类似功能,并供应了更加友好的访问界面。

kibana-5.6.9-windows-x86下载:

https://www.elastic.co/downloads/past-releases/kibana-5-6-9

下载之后,解压到C:\ELk\kibana-5.6.9-windows-x86

实行C:\ELk\kibana-5.6.9-windows-x86\bin\kibana.bat

浏览器访问:http://localhost:5601

二、DSL的基本利用

elasticsearch也像mysql一样供应了专门的语法来操作数据。
Elasticsearch provides a full Query DSL (Domain Specific Language) based on JSON to define queries.

PUT people/person/1?op_type=create{ \"大众user\"大众 : \公众kimchy\"大众, \"大众post_date\公众 : \"大众2009-11-15T14:12:12\"大众, \公众message\"大众 : \"大众trying out Elasticsearch\公众}修正

POST /user/guest/20/_update{ \公众doc\"大众: { \"大众RealName\"大众:\"大众LukyHuu20\"大众 }}查询

GET /user/guest/_search{ \"大众query\公众: { \公众match\公众: { \公众Id\公众:22 } }}删除

DELETE /user/guest/15{ }

三、Elasticsearch .Net

elasticsearch因此restfulAPI办法对外供应接口,并供应客户端给多种措辞利用。
Elasticsearch uses standard RESTful APIs and JSON. We also build and maintain clients in many languages such as Java, Python, .NET, SQL, and PHP. Plus, our community has contributed many more. They’re easy to work with, feel natural to use, and, just like Elasticsearch, don't limit what you might want to do with them. 参考(https://www.elastic.co/products/elasticsearch)

1.Low level client基本利用

本文是先容ES的.NET客户端,Elasticsearch .Net - Low level client[5.x]

通过引入对应的版本的客户端,便可通过C#操作ES。
参考(https://www.elastic.co/guide/en/elasticsearch/client/net-api/5.x/elasticsearch-net.html)

连接

var settings = new ConnectionConfiguration(new Uri(\"大众http://example.com:9200\公众)) .RequestTimeout(TimeSpan.FromMinutes(2));var lowlevelClient = new ElasticLowLevelClient(settings);

插入文档

var indexResponse = lowlevelClient.Index<byte[]>(\"大众user\"大众, \公众guest\公众, user.Id.ToString(), user); byte[] responseBytes = indexResponse.Body;

更新文档

var searchResponse = lowlevelClient.Update<string>(\"大众user\"大众, \"大众guest\"大众, id.ToString(), new { doc = new { RealName = realname, Description = description } });bool successful = searchResponse.Success;

查询

var searchResponse = lowlevelClient.Search<string>(\"大众user\公众, \公众guest\"大众, new { query = new { match = new { Id = id } } });bool successful = searchResponse.Success;

删除

var searchResponse = lowlevelClient.Delete<string>(\"大众user\"大众, \"大众guest\"大众, id.ToString()); bool successful = searchResponse.Success;

2.项目实战

前面大致先容了ES的安装和基本利用。
那么,如何在项目中落地呢？

利用nuget安装Elasticsearch.Net 5.6.4

Install-Package Elasticsearch.Net -Version 5.6.4

安装完后,

基本的增删该查在项目中的实现上面已经有所先容,这里重点讲一下查询:

笔者利用的.NET MVC5 Web框架,对付返回的结果笔者做了一个大略封装：

public class ESearchRoot<T> { /// <summary> /// /// </summary> public int took { get; set; } /// <summary> /// /// </summary> public string timed_out { get; set; } /// <summary> /// /// </summary> public _shards _shards { get; set; } /// <summary> /// /// </summary> public Hits<T> hits { get; set; } } public class _shards { /// <summary> /// /// </summary> public int total { get; set; } /// <summary> /// /// </summary> public int successful { get; set; } /// <summary> /// /// </summary> public int skipped { get; set; } /// <summary> /// /// </summary> public int failed { get; set; } } public class HitsItem<T> { /// <summary> /// /// </summary> public string _index { get; set; } /// <summary> /// /// </summary> public string _type { get; set; } /// <summary> /// /// </summary> public string _id { get; set; } /// <summary> /// /// </summary> public string _score { get; set; } /// <summary> /// /// </summary> public T _source { get; set; } /// <summary> /// /// </summary> public List<int> sort { get; set; } /// <summary> /// /// </summary> public Highlight highlight { get; set; } } public class Hits<T> { /// <summary> /// /// </summary> public int total { get; set; } /// <summary> /// /// </summary> public string max_score { get; set; } /// <summary> /// /// </summary> public List<HitsItem<T>> hits { get; set; } } public class Highlight { /// <summary> /// /// </summary> public List<string> Description { get; set; } }

由于soure返回的工具是不定的,以是利用了泛型。
本项目soure对应的类,user:

///<summary> /// /// </summary> public class User { /// <summary> /// /// </summary> public string Account { get; set; } /// <summary> /// /// </summary> public string Phone { get; set; } /// <summary> /// /// </summary> public string Email { get; set; } /// <summary> /// /// </summary> public string RealName { get; set; } /// <summary> /// /// </summary> public string CanReview { get; set; } /// <summary> /// /// </summary> public string CanExcute { get; set; } /// <summary> /// /// </summary> public string Avatar { get; set; } /// <summary> /// /// </summary> public string IsUse { get; set; } /// <summary> /// /// </summary> public int Id { get; set; } /// <summary> /// /// </summary> public string Name { get; set; } /// <summary> /// /// </summary> public string Description { get; set; } /// <summary> /// /// </summary> public DateTime CreateTime { get; set; } /// <summary> /// /// </summary> public DateTime ModifyTime { get; set; } }

项目利用了带条件的分页查询:

public List<AdminUser> GetBySomeWhere(string keyword, int limit, int pageSize, out int total) { List<AdminUser> users = new List<AdminUser>(); total = 0; try { var settings = new ConnectionConfiguration(new Uri(\公众http://localhost:9200/\"大众)) .RequestTimeout(TimeSpan.FromMinutes(2)); var lowlevelClient = new ElasticLowLevelClient(settings); //根据不同的参数 来构建不同的查询条件 var request = new object(); if (!String.IsNullOrEmpty(keyword)) { request = new { from = limit, size = pageSize, query = new { match = new { Description = keyword } }, highlight = new { fields = new { Description = new { } } }, sort = new { Id = new { order = \"大众desc\"大众 } } }; } else { request = new { from = limit, size = pageSize, query = new { match_all = new { } }, highlight = new { fields = new { Description = new { } } }, sort = new { Id = new { order = \"大众desc\"大众 } } }; } var searchResponse = lowlevelClient.Search<string>(\公众user\"大众, \"大众guest\"大众, request); bool successful = searchResponse.Success; var responseJson = searchResponse.Body; if (!successful) { return users; } ESearchRoot<User> root = JsonHelper.JSONStringObject<ESearchRoot<User>>(responseJson); if (root != null) { total = root.hits.total; foreach (HitsItem<User> item in root.hits.hits) { if (item._source != null) { string highlightDescription = String.Empty; StringBuilder sbDs = new StringBuilder(); if (item.highlight != null && item.highlight.Description.Count > 0) { //ighlightDescription = item.highlight.Description[0]; foreach (var d in item.highlight.Description) { sbDs.Append(d); } highlightDescription = sbDs.ToString(); } AdminUser user = new AdminUser { Id = item._source.Id, RealName = item._source.RealName, Account = item._source.Account, Email = item._source.Email, Phone = item._source.Phone, //IsUse=item._source.IsUse, Avatar = item._source.Avatar, Description = item._source.Description, HighlightDescription = highlightDescription, CreateTime = item._source.CreateTime, ModifyTime = item._source.ModifyTime }; users.Add(user); } } } return users; } catch (ElasticsearchClientException ex) { //Log4Helper.Error } return users; }

项目终极的效果如下:

四、总结

elasticsearch是很强大的开源工具,在实现全文搜索上有其独到之处,也是大数据的剖析方面利器,值得大家深入去研究和实践。

五、参考

个人微信"大众年夜众号:码农商业参谋