Elastic Search Query to Retrieve Records from Elastic Server

Elastic Search is an open-source search tool that is built on Lucene but natively it is JSON + RESTful. Elastic Search provides a JSON-style domain-specific language which can be used to execute queries, and is referred as the Query-DSL.

The search API allows us to execute a search query and get back search hits that match the query. Elastic search will fetch the records at lightning speed because of schema-less table structure. The query can either be provided using a simple query string as a parameter or using a request body.

Here I am showing how to write queries for Elastic search with some good set of standard queries as an example.

 

Basic Queries Using Only the Query String

Basic queries can be done using only query string parameters in the URL. For example, the following searches for the text ‘test’ in any field in any document and return at most 5 results:
{ElasticURL}/_search?size=5&q=test

 

Full Query API

Full Queries are powerful and complex ones which include queries that involve in faceting and statistical operations and should use the full elastic search query language and API. The queries are written as JSON structure in the query language and sent to the query endpoint (query language details are given below). There are two options to send a query to the search endpoint:

1. Either as the value of a source query parameter e.g. :

{ElasticURL}/_search?source={Query as JSON}

2. Or in the request body, e.g.,

{
     "query" : {
         "term" : { "PropertyName": "test" }
     }
 }

From & Size in Query

Pagination of results can be done by using the ‘from’ and ‘size’ parameters. The ‘from’ parameter defines the offset from the first result we want to fetch. The ‘size’ parameter allows us to configure the maximum amount of hits to be returned.

{
     "size" : 10,
     "from" : 0,
     "query" : {}
 }

Sample response from Elastic server

{
  "took": 7,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 4,
    "max_score": 4.5618434,
    "hits": [
      {
        "_index": "ph_property",
        "_type": "property",
        "_id": "10322",
        "_score": 4.5618434,
        "_source": {
          "PropertyID": 10322,
          "PropertyCode": "VTELD21NGXKK3V02GJRLPRROB",
          "BuilderCode": "BY67DP",
          "BuilderName": "Janet Spencer",
          "PropertyName": "AWS test",
          "BHK": "",
          "PropertyTypeCode": "DO20ET",
          "PropertyType": "Residential Land"
        }
      }
    ]
  }
}

Query DSL Examples

1. Match all/Find Everything

{
     "query" : {
         "match_all" : { }
     }
 }

2. Filter on one field

{
     "query" : {
         "term" : { field-name: value }
     }
 }

3. Match with a field

{
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "field": "value"
          }
        }
      ]
    }
  }
}

4. Multi-match query builds on the match query to allow multi-field queries

{
  "multi_match" : {
    "query":    "this is a test",
    "fields": [ "subject", "message" ]
  }
}

5. Find documents which consist the exact term specified in the field specified

{
  "query": {
    "bool": {
      "should": [
        {
          "term": {
            "status": {
              "value": "urgent"
            }
          }
        },
        {
          "term": {
            "status": "normal"
          }
        }
      ]
    }
  }
}

6. Find documents, where the field specified consist values (strings, numbers, or dates) in the range specified

{
  "size": "9",
  "query": {
    "bool": {
      "must": [
        {
          "range": {
            "BudgetFrom": {
              "gte": 50000
            }
          }
        },
        {
          "range": {
            "BudgetTo": {
              "lte": 2231346
            }
          }
        }
      ]
    }
  }
}

7. The filtered query is used to combine a query that is used for scoring with another query that is used for filtering the result set.

{
  "filtered": {
    "query": {
      "match": { "tweet": "full text search" }
    },
    "filter": {
      "range": { "created": { "gte": "now-1d/d" }}
    }
  }
}

8. Filters documents that only have the provided ids.

{
    "ids" : {
        "type" : "my_type",
        "values" : ["1", "4", "100"]
    }
}

9. Filters documents that are matching the provided document/mapping type.

{
    "type" : {
        "value" : "my_type"
    }
}

10. Filter on two fields

{
    "query": {
        "filtered": {
            "query": {
                "match_all": {}
            },
            "filter": {
                "and": [
                    {
                        "range" : {
                            "b" : { 
                                "from" : 4, 
                                "to" : "8"
                            }
                        },
                    },
                    {
                        "term": {
                            "a": "john"
                        }
                    }
                ]
            }
        }
    }
}

An actual example with some search parameter:

{
  "from": "0",
  "size": "9",
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "ProjectTypeCode": "EQ92JK"
          }
        },
        {
          "match": {
            "PropertyStatusCode": "AS82IZ"
          }
        },
        {
          "match": {
            "PropertyTypeCode": "SJ85GF"
          }
        },
        {
          "match": {
            "MicroMarketCode": "DX60DL"
          }
        },
        {
          "range": {
            "SizeFrom": {
              "gte": 1000
            }
          }
        },
        {
          "range": {
            "BudgetFrom": {
              "gte": 50000
            }
          }
        },
        {
          "range": {
            "BudgetTo": {
              "lte": 2231346
            }
          }
        },
        {
          "range": {
            "PossessionDate": {
              "gte": "2016-01-01"
            }
          }
        },
        {
          "match": {
            "City": "Bengaluru"
          }
        }
      ]
    }
  }
}

Conclusion

These are the few frequently used queries to retrieve the data from Elastic server. However Elastic Search response can be retrieved using various other queries like Geo queries, joining queries, compound queries, specialized queries, etc. We can even join multiple queries to get the hits from elastic server. The response we get from Elastic Server is very fast compared to MySQL/Sql Server queries and hence it is now being used widely.

FacebookTwitterGoogle+Share
About Ranjan Deka

Ranjan Deka is a full stack developer at Vmoksha Technologies. He has completed his Bachelor of Engineering from Royal School of Engineering & Technology, Guwahati, Assam. He is passionate about emerging technologies in the .Net framework and JavaScript. He is a nature lover who loves to travel different unexplored places. Along with that, He loves to play guitar, cricket, and computer games.



One comment on “Elastic Search Query to Retrieve Records from Elastic Server

  1. Pankaj

    Nice blog Ranjan !!

    Well explained and really helpful. :)

    Keep blogging. ;)

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>