Simple_yql_query

We'll start by doing a simple YQL query to the Yahoo web search table which basically points you to the Yahoo search results. Now when you 'select *' from such a table the result you'll get back will look something like this:

<result xmlns="http://www.inktomi.com/">
    <abstract><![CDATA[<b>Pizza</b>]]></abstract>
    <clickurl>http://happypizzaland.com</clickurl>
    <date>2011/01/15</date>
    <dispurl><![CDATA[<b>happypizzaland.com</b>/<b>pizza</b>]]></dispurl>
    <size>138428</size>
    <title><![CDATA[<b>Pizza</b>]]></title>
    <url>http://happypizzaland.com/pizza</url>
</result>

Making the actual HTTP GET call with DTF is very straightforward and can be done using the existing http_get tag, lets put together a very simple function that will make the HTTP call and return the body of the response as the return of our DTF function:

<function name="yql">
    <param name="q" type="required"/>
    
    <urlencode result="q" source="${q}"/> 
    <http_get uri="${yql.url}?q=${q}"/>
    <return>${http.get.body}</return>
</function>   

As you can notice from the above XML we also use the urlencode method to make sure that the URL we're going pass to the HTTP libraries have been correctly encoded before making the call. Of course if you make sure that your URL data has all been properly encoded then you can avoid this call but it really helps to sometimes keep your data in a more human readable format and use this simple tag to do the job.

We can now use this new function to now send a few different queries and just make sure that the response from the server is an HTTP 200 response meaning that its a valid query being done. The actual test can be found within the tests/examples/yahoo/yql/simple_query.xml and is presented here for demonstration purposes.

<?xml version="1.0" encoding="UTF-8"?>

<script xmlns="http://dtf.org/v1" name="simple_query">
    <info>
        <author>
            <name>Rodney Gomes</name>
            <email></email>
        </author>
        <description>
        This test will simple query the various queries that are stored in the 
        ${yql.queries} property and make sure that YQL is capable of responding
        with some data. This does not of course validate the data or do any 
        additional work and is simply part of a wiki example of how to use DTF
        for testing a service like YQL.
        </description>
    </info>

    <local>
        <createstorage id="INPUT" path="${dtf.xml.path}/input"/>
        <createstorage id="OUTPUT" path="${dtf.xml.path}/output"/>

        <loadproperties uri="storage://INPUT/yql.properties"/>
    </local>

    <function name="yql">
        <param name="q" type="required"/>
        
        <urlencode result="q" source="${q}"/> 
        <http_get uri="${yql.url}?q=${q}"/>
        <return>${http.get.body}</return>
    </function>    

    <for property="query" range="${yql.queries}">
        <log>querying [${query}]</log>
        <call function="yql" result="result">
            <property name="q" value="${query}"/>
        </call>
    </for>
</script>