Sunday, January 19, 2014

Faceting using Elasticsearch Aggregations


Facets are probably one of the most compelling reasons to use search engines for an ecommerce site, They can be used to render pages such as search results, browsing of categories etc, We will take a look at the real world example of how ES Aggregations (AKA Facets in older versions) can be used to build category navigation for an ecommerce site.
Similar results can be achieved using ES Facets, but in this blog we will look at "aggregations" which seems to be the way forward and has additional flexibility as compared to ES "Facets"

ES Aggregations will eventually replace ES Facets, but the nice thing is they have a lot in similarity so to learn or migrate to this new feature from ES "facets" to "aggregations" should not be complicated.
You can read more about the new feature on this GIT issue tracker https://github.com/elasticsearch/elasticsearch/issues/3300

Mock User Interface

The mock screen in this example is a typical category or search term navigation on an ecommerce site, at a very basic level you have 4 components of search results.

Facets - Facets help users to narrow down / or filter a search result, facet is built based on the search context.

Sort Order - Sort order impacts the search results components, it defines in what order the results should be listed on the page, for instance a user may sort by lowest to highest price or by product ratings.

Pagination of Results - Pagination component allows an user to navigate back and forth through a search results, this also guides the number of records that should be returned in ES query.

Search Result - Restricted to number of records that should be displayed on the landing page, perhaps this will be configurable based on your application needs.



Sample Data


We will begin with a schema less version of our products index, schema less support happens to be a nice thing about ES to get started quickly with design and testing, you can always add a schema latter on for production quality index and to better control the behavior, for the purpose of this demo we will go with following sample for our products index.


Sort Results

User can sort the search or category navigation results by "lowest to highest" price or by popular products, we can combine "sort" element with aggregations to achieve this.
In our example we have used "sort" by lowest to highest price as follows.

"sort" : [{"offerprice" : {"order" : "asc", "mode" : "avg", "ignore_unmapped":true, "missing":"_last"}},"_score"]

Sorting within Facets

The results within in the facets can be sorted using order types within the term definition, for instance in our example we are sorting the Brand  Facet by total count of each brand in descending order.
"order": { "_count" : "desc" }
Similarly we are sorting size facet in ascending using "order": { "_count" : "asc" }

Pagination Component

Pagination of results can be achieved by using "from" and "to" fields, these can be passed either in Query body or as a URL param, in our sample we have passed this in JSON body as follows.
"from" : 0, "size" : 5 or "from" : 5, "size" : 5 for the next page

They can also be used as URL params as follows.
curl -XGET 'http://localhost:9200/products/_search?pretty=true&from=5&to=5


Facet Selection

Search results are also influenced by the facet selection, for instance a user wants to see all products in men's category that are from Brand "diesel" and are of size "small", this can be achieved by using an "and" filter as follows.

....
"and": [
                {
                    "term": {
                        "Brand":"diesel"    
                    }
                    
                },
                {
                    "term": {
                        "size":"small"
                    }
                    
                }]
            }
...
...

What is missing

I could be completely wrong, but I have not been able to achieve following with in the ES Query, of course there are alternate ways of doing this within the application code, but I would love to see these added to ES aggregations in future.

#1 As you can see in our example, we are defining the price range in ES query, but then in a typical ecommerce model the price range may vary dynamically based on the browse category, so Instead of defining price ranges like, 1 to 5, 6 to 10.. etc. there should be a way to get an even spread by defining the number of buckets.


Wednesday, January 1, 2014

Installing Spring STS or Eclipse IDE on Linux Mint 16

The steps in this section are for Spring STS, but these can be used for eclipse as well.
The instructions involves installing Spring ide or eclipse from source and then creating a startup Menu option so it can be launched from the menu.

Download the latest version of eclipse or Spring STS

curl -O http://download.springsource.com/release/STS/3.4.0/dist/e4.3/spring-tool-suite-3.4.0.RELEASE-e4.3.1-linux-gtk-x86_64.tar.gz
tar -xvzf spring-tool-suite-3.4.0.RELEASE-e4.3.1-linux-gtk-x86_64.tar.gz

sudo mv springsource /opt
sudo chown -R root:root /opt/springsource
sudo chmod -R +r /opt/springsource
sudo cp /opt/springsource/sts-3.4.0.RELEASE/icon.xpm /usr/share/pixmaps/sts.xpm

Run this command to create a file with startup options for sts or eclipse.

sudo cat >> /usr/share/applications/sts.desktop<<EOF
[Desktop Entry]
Encoding=UTF-8
Name=Spring IDE
Comment=Spring IDE
Exec=/opt/springsource/sts-3.4.0.RELEASE/STS
Icon=/usr/share/pixmaps/sts.xpm
Terminal=false
Type=Application
Categories=GNOME;Application;Development;
StartupNotify=true
EOF


Navigate to Menu -> Programming -> Spring IDE
This should now launch eclipse or Spring ide instance.