<p><b>We recently introduced a new addition to the Search API - JSON queries. The search request can now be executed with a POST request, which includes the query-parameters within its payload. Along with this new query we also introduce a new parameter <code>SELECT</code> with the sub-parameters <code>WHERE</code> and <code>GROUPING</code>, which is equivalent to YQL.</b></p>
<p><b>The new query</b></p>
<p>With the Search APIs newest addition, it is now possible to send queries with HTTP POST. The query-parameters has been moved out of the URL and into a POST request body - therefore, no more URL-encoding. You also avoid getting all the queries in the log, which can be an advantage.</p>
<p>This is how a GET query looks like:</p>
<pre>GET /search/?param1=value1¶m2=value2&...</pre>
<p>The general form of the new POST query is:</p>
<pre>POST /search/
{
param1 : value1,
param2 : value2,
...
}
</pre>
<p>The dot-notation is gone, and the query-parameters are now nested under the same key instead.</p>
<p>Let’s take this query:</p>
<pre>GET /search/?yql=select+%2A+from+sources+%2A+where+default+contains+%22bad%22%3B&ranking.queryCache=false&ranking.profile=vespaProfile&ranking.matchPhase.ascending=true&ranking.matchPhase.maxHits=15&ranking.matchPhase.diversity.minGroups=10&presentation.bolding=false&presentation.format=json&nocache=true
</pre>
<p>and write it in the new POST request-format, which will look like this:</p>
<pre>POST /search/
{
"yql": "select * from sources * where default contains \"bad\";",
"ranking": {
"queryCache": "false",
"profile": "vespaProfile",
"matchPhase": {
"ascending": "true",
"maxHits": 15,
"diversity": {
"minGroups": 10
}
}
},
"presentation": {
"bolding": "false",
"format": "json"
},
"nocache": true
}
</pre>
<p>With Vespa running (see <a href="https://docs.vespa.ai/documentation/vespa-quick-start.html">Quick Start</a> or <a href="https://docs.vespa.ai/documentation/tutorials/blog-search.html">Blog Search Tutorial</a>), you can try building POST-queries with the <i>new querybuilder GUI</i> at <a href="http://localhost:8080/querybuilder/">http://localhost:8080/querybuilder/</a>, which can help you build queries with e.g. autocompletion of YQL:</p>
<figure data-orig-width="934" data-orig-height="1408" class="tmblr-full"><img src="https://66.media.tumblr.com/744108dea7ea2411b436bc1c439bd2d5/tumblr_inline_pd55x78hVH1vpfrlb_540.png" alt="image" data-orig-width="934" data-orig-height="1408"/></figure><p><b>The Select-parameter</b></p>
<p>The <code>SELECT</code>-parameter is used with POST queries and is the JSON equivalent of YQL queries, so they can not be used together. The <code>query</code>-parameter will overwrite <code>SELECT</code>, and decide the query’s querytree.</p>
<p><b>Where</b></p>
<p>The SQL-like syntax is gone and the tree-syntax has been enhanced. If you’re used to the query-parameter syntax you’ll feel right at home with this new language. YQL is a regular language and is parsed into a query-tree when parsed in Vespa. You can now build that tree in the <code>WHERE</code>-parameter with JSON. Lets take a look at the yql: <code>select * from sources * where default contains foo and rank(a contains "A", b contains "B");</code>, which will create the following query-tree:</p>
<figure data-orig-width="1323" data-orig-height="622" class="tmblr-full"><img src="https://66.media.tumblr.com/f25922b612d4341d275fd0b7eb1715a7/tumblr_inline_pd55yrwNbZ1vpfrlb_540.png" alt="image" data-orig-width="1323" data-orig-height="622"/></figure><p>You can build the tree above with the <code>WHERE</code>-parameter, like this:</p>
<pre><code>{
"and" : [
{ "contains" : ["default", "foo"] },
{ "rank" : [
{ "contains" : ["a", "A"] },
{ "contains" : ["b", "B"] }
]}
]
}
</code></pre>
<p>Which is equivalent with the YQL.</p>
<p><b>Grouping</b></p>
<p>The grouping can now be written in JSON, and can now be written with structure, instead of on the same line. Instead of parantheses, we now use curly brackets to symbolise the tree-structure between the different grouping/aggregation-functions, and colons to assign function-arguments.</p>
<p>A grouping, that will group first by year and then by month, can be written as such:</p>
<pre><code>| all(group(time.year(a)) each(output(count())
all(group(time.monthofyear(a)) each(output(count())))
</code></pre>
<p>and equivalentenly with the new <code>GROUPING</code>-parameter:</p>
<pre><code>"grouping" : [
{
"all" : {
"group" : "time.year(a)",
"each" : { "output" : "count()" },
"all" : {
"group" : "time.monthofyear(a)",
"each" : { "output" : "count()" },
}
}
}
]
</code></pre>
<p><b>Wrapping it up</b></p>
<p>In this post we have provided a gentle introduction to the new Vepsa POST query feature, and the <code>SELECT</code>-parameter. You can read more about writing <a href="https://docs.vespa.ai/documentation/search-api.html">POST queries in the Vespa documentation</a>. More examples of the POST query can be found in the Vespa tutorials.</p>
<p>Please share experiences. Happy searching!</p>