-
Notifications
You must be signed in to change notification settings - Fork 0
Using events
Events are another feature that distinguish DTF from the rest of the testing frameworks out there. With events you can give additional information after the execution of each action (tag) in your test and allow the test case writer to decide if he/she wants to use this additional information to make any decisions or record this additional information for post processing. Once an event is thrown in DTF (be it from the Java code written for a specific tag or because you executed the tag event within your test case) you will immediately have that same event available along with all of its fields through a property resolution. So for example if you were to throw the following event with the event tag like so:
<event name="my.event">
<attribute name="field1" value="value1"/>
<attribute name="field2" value="value2"/>
</event>Then you'd be able to access those fields immediately after just by using the property ${my.event.field} and ${my.event.field}. This is what really allows you to make events extremely useful during your test writing. Now we'll go over a few other examples of using events in the following examples.
The simplest usage of events include just validating that a certain status code was returned when doing a specific HTTP operation or we juts want to validate that the content contains an expected value. Here is a small example of how to use the events throw from the HTTP tags to check what the returned status code was.
<?xml version="1.0" encoding="UTF-8"?>
<script xmlns="http://dtf.org/v1" name="http_event_usage">
<info>
<author>
<name>Rodney Gomes</name>
<email>rlgomes@yahoo-inc.com</email>
</author>
<description>
</description>
</info>
<for property="i" range="[0..5]">
<http_get uri="http://github.com" onFailure="continue" />
<log>status code: ${http.get.status}</log>
</for>
</script>
The test just issues a few http_get calls and then prints the returned status code from that operation. All tags that throw events have this documented in the generated documentation here and you can easily find the exact event names and fields that are available after each execution of these tags.
In the previous section we showed you how to pull some of the values from events and just print them to the screen, but the true value of events is to be able to take the event data and process it immediately in order to use its values in another request or to validate the resulting event data has the expected content. The following example issues a search request through Yahoo's web search API and then validates that the search results contain a link to the www.yahoo.com website since the search term was Yahoo!.
<http_get uri="http://api.search.yahoo.com/WebSearchService/V1/webSearch?appid=dtftest&results=1&query=Yahoo!"/>
<assert>
<match source="${http.get.body}"
expression="<ClickUrl>http://www.yahoo.com/</ClickUrl>"
partial="true"/>
</assert>We very simply look for the ClickUrl element with the http://www.yahoo.com link within it to validate that we've gotten the expected response back (there is a better way of doing this using property transformations but that will be used in a more advanced tutorial).
Another great feature that events give you is the ability to measure performance because every time you generate an event you will basically be measuring the amount of time it takes to do a certain operation. When gathering these events with the available record tag and then using tags that were created to read back and do calculations with those events you're able to fully use the killer features of DTF. Lets start by recording our events to an output file before we process them and calculate statistics. Here is the example code:
<record uri="storage://OUTPUT/http_get.txt">
<for property="i" range="1..3">
<http_get uri="http://gist.github.com/raw/471044/e3ccbdc209157714625dc76e97dcc6b6d583f771/recording_events.xml" onFailure="continue"/>
</for>
</record>Now the output file http_get.txt contains all of the event data that was generated and since we didn't specify the type to use on the *record8 tag, the default is to record in a txt format which is human readable. Here is what the output looks like for those 3 events (I named the gist with a .properties extension so that it would color the gist nicely):
http.get.start=1278797995082
http.get.stop=1278797999408
http.get.body=%3Crecord+uri%3D%22storage%3A%2F%2FOUTPUT%2Fhttp_get.txt%22%3E%0A++++%3Cfor+property%3D%22i%22+range%3D%221..3%22%3E%0A++++++++%3Chttp_get+uri%3D%22http%3A%2F%2Fgithub.com%22+onFailure%3D%22continue%22%2F%3E%0A++++%3C%2Ffor%3E%0A%3C%2Frecord%3E
http.get.bodysize=166
http.get.uri=http%3A%2F%2Fgist.github.com%2Fraw%2F471044%2Fe3ccbdc209157714625dc76e97dcc6b6d583f771%2Frecording_events.xml
http.get.status=200
http.get.statusmsg=OK
http.get.headerout.Server=nginx%2F0.7.61
http.get.headerout.Date=Sat%2C+10+Jul+2010+21%3A39%3A59+GMT
http.get.headerout.Content-Type=text%2Fplain%3B+charset%3Dutf-8
http.get.headerout.Connection=keep-alive
http.get.headerout.Status=200+OK
http.get.headerout.ETag=%224edbc915896dc73ff06c90d6c1a08183%22
http.get.headerout.Content-Transfer-Encoding=binary
http.get.headerout.X-Runtime=8ms
http.get.headerout.Content-Disposition=inline
http.get.headerout.Content-Length=166
http.get.headerout.Set-Cookie=_github_ses%3DBAh7BzoLbG9jYWxlIgdlbiIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6Rmxhc2g6OkZsYXNoSGFzaHsABjoKQHVzZWR7AA%253D%253D--ca3e81ffeac9f14e935f863392bc043d41679deb%3B+path%3D%2F%3B+expires%3DWed%2C+01+Jan+2020+08%3A00%3A00+GMT%3B+HttpOnly
http.get.headerout.Cache-Control=private
http.get.start=1278797999415
http.get.stop=1278797999576
http.get.body=%3Crecord+uri%3D%22storage%3A%2F%2FOUTPUT%2Fhttp_get.txt%22%3E%0A++++%3Cfor+property%3D%22i%22+range%3D%221..3%22%3E%0A++++++++%3Chttp_get+uri%3D%22http%3A%2F%2Fgithub.com%22+onFailure%3D%22continue%22%2F%3E%0A++++%3C%2Ffor%3E%0A%3C%2Frecord%3E
http.get.bodysize=166
http.get.uri=http%3A%2F%2Fgist.github.com%2Fraw%2F471044%2Fe3ccbdc209157714625dc76e97dcc6b6d583f771%2Frecording_events.xml
http.get.status=200
http.get.statusmsg=OK
http.get.headerout.Server=nginx%2F0.7.61
http.get.headerout.Date=Sat%2C+10+Jul+2010+21%3A39%3A59+GMT
http.get.headerout.Content-Type=text%2Fplain%3B+charset%3Dutf-8
http.get.headerout.Connection=keep-alive
http.get.headerout.Status=200+OK
http.get.headerout.ETag=%224edbc915896dc73ff06c90d6c1a08183%22
http.get.headerout.Content-Transfer-Encoding=binary
http.get.headerout.X-Runtime=20ms
http.get.headerout.Content-Disposition=inline
http.get.headerout.Content-Length=166
http.get.headerout.Set-Cookie=_github_ses%3DBAh7BzoLbG9jYWxlIgdlbiIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6Rmxhc2g6OkZsYXNoSGFzaHsABjoKQHVzZWR7AA%253D%253D--ca3e81ffeac9f14e935f863392bc043d41679deb%3B+path%3D%2F%3B+expires%3DWed%2C+01+Jan+2020+08%3A00%3A00+GMT%3B+HttpOnly
http.get.headerout.Cache-Control=private
http.get.start=1278797999583
http.get.stop=1278797999762
http.get.body=%3Crecord+uri%3D%22storage%3A%2F%2FOUTPUT%2Fhttp_get.txt%22%3E%0A++++%3Cfor+property%3D%22i%22+range%3D%221..3%22%3E%0A++++++++%3Chttp_get+uri%3D%22http%3A%2F%2Fgithub.com%22+onFailure%3D%22continue%22%2F%3E%0A++++%3C%2Ffor%3E%0A%3C%2Frecord%3E
http.get.bodysize=166
http.get.uri=http%3A%2F%2Fgist.github.com%2Fraw%2F471044%2Fe3ccbdc209157714625dc76e97dcc6b6d583f771%2Frecording_events.xml
http.get.status=200
http.get.statusmsg=OK
http.get.headerout.Server=nginx%2F0.7.61
http.get.headerout.Date=Sat%2C+10+Jul+2010+21%3A39%3A59+GMT
http.get.headerout.Content-Type=text%2Fplain%3B+charset%3Dutf-8
http.get.headerout.Connection=keep-alive
http.get.headerout.Status=200+OK
http.get.headerout.ETag=%224edbc915896dc73ff06c90d6c1a08183%22
http.get.headerout.Content-Transfer-Encoding=binary
http.get.headerout.X-Runtime=8ms
http.get.headerout.Content-Disposition=inline
http.get.headerout.Content-Length=166
http.get.headerout.Set-Cookie=_github_ses%3DBAh7BzoLbG9jYWxlIgdlbiIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6Rmxhc2g6OkZsYXNoSGFzaHsABjoKQHVzZWR7AA%253D%253D--ca3e81ffeac9f14e935f863392bc043d41679deb%3B+path%3D%2F%3B+expires%3DWed%2C+01+Jan+2020+08%3A00%3A00+GMT%3B+HttpOnly
http.get.headerout.Cache-Control=private
As you can tell the output of these fields is URL encoded to avoid issues with multiple lines within the content of data but you can read the majority of the data that was returned if you needed to debug something about the resulting events.
This data wasn't generated to be read or seen by a tester anyways and that is why there are quite a few tags that can be used to read back the data and do validation, calculate statistics or drive test execution. In the next example I will be using the stats tag to calculate performance statistics for the data that we generated, here is the test:
<record uri="storage://OUTPUT/http_get.txt">
<for property="i" range="1..100">
<http_get uri="http://gist.github.com/raw/471044/e3ccbdc209157714625dc76e97dcc6b6d583f771/recording_events.xml" onFailure="continue"/>
</for>
</record>
<query uri="storage://OUTPUT/http_get.txt"
cursor="data"/>
<stats event="stats" cursor="data"/>
<log>
Stats from Gist GET Requests:
ops/sec: ${stats.avg_occ}
min duration: ${stats.min_dur}ms
max duration: ${stats.max_dur}ms
avg duration: ${stats.avg_dur}ms
</log>In the previous example we print out the various events that are generated by the stats that give us the average occurrences of the events generated per second as well as the minimum, maximum and average duration times of the total of events that were generated during our test run. Have a look at the generated documentation for more information relative to events generated by the stats tag. The output from the previous test would look like so:
[java] INFO 10/07/2010 14:51:05 Log -
[java] Stats from Gist GET Requests:
[java] ops/sec: 5.719
[java] min duration: 144ms
[java] max duration: 1135ms
[java] avg duration: 173ms
[java]
The above test was executed over a wireless network and that is why the minimum duration is in the 100ms ball park.
Events can also be used to drive test execution and by this we mean that you can use events from some write operations to drive the execution of your read operations or delete operations. Lets see how this works for the following test that will create a few directories in a box.net account using the http_request tag and then deleting those same directories using the events generated at creation time to drive the deletions. Here is what the whole test would look like:
<?xml version="1.0" encoding="UTF-8"?>
<script xmlns="http://dtf.org/v1" name="create_delete_collection_box.net">
<info>
<author>
<name>Rodney Gomes</name>
<email>rlgomes@yahoo-inc.com</email>
</author>
<description></description>
</info>
<local>
<createstorage id="INPUT" path="${dtf.xml.path}/input" />
<createstorage id="OUTPUT" path="${dtf.xml.path}/output" />
<loadproperties uri="storage://INPUT/box.net.properties" />
</local>
<http_config id="CONFIG">
<credentials username="dtftesting@gmail.com" password="pass4dtftesting" />
</http_config>
<record uri="storage://OUTPUT/mkcol_events.txt">
<for property="i" range="1..3">
<http_request method="MKCOL" uri="http://www.box.net/dav/test${i}">
<http_config refid="CONFIG" />
</http_request>
<log>created ${http.MKCOL.uri}</log>
</for>
</record>
<query uri="storage://OUTPUT/mkcol_events.txt" cursor="events" />
<iterate cursor="events">
<http_request method="DELETE" uri="${events.uri}">
<http_config refid="CONFIG" />
</http_request>
<log>deleted ${events.uri}</log>
</iterate>
</script>Quite a lot went into this test including the introduction of some new tags that allow you to configure the HTTP tags to handle Basic HTTP authentication that box.net requires. You can easily see how simple it is to really use previously recorded events to drive the deletion of the same collections that were previously created in this test. The only thing that is a bit tricky to get right is to know exactly what part of the generated events contains the information you need to be used in the subsequent event driven operations. Sometimes you'll have to generate a few events to analyze the results and then figure out if you can use an existing field or if you need to manipulate data from a previous event using property transformations.
The ability to generate your own events in the XML scripting language may not seem useful at first but after understanding that you can use an event to measure a portion of your test execution it should be obvious that you can use the event tag in your test to measure a block of different DTF tags. This allows you to easily measure the time that certain groups of tag execution take that may be used by customers of a certain API to do specific common tasks. For example the GitHub Repository API to delete a repository but the delete operation consists of two HTTP requests one to request the delete and another to confirm the delete, because of this we'd want to know the total time to delete a repository and not the time of the individual requests. This is where we can just wrap the call to the delete function with a simple event tag and name it accordingly to be able to get the event you're really interested in. The following example illustrates how to do this within the delete repository function:
<function name="delete_repository" export="true">
<param name="user" default="${github.user}" />
<param name="token" default="${github.token}" />
<param name="name" type="required" />
<property name="authdata" value="login=${user}&token=${token}" />
<event name="github.delete">
<http_post uri="${github.baseuri}/xml/repos/delete/${name}"
onFailure="continue">
<entity>${authdata}</entity>
</http_post>
<property name="dtoken"
value="${http.post.body:xpath:/delete-token/delete-token/text()}" />
<!-- confirm deletion -->
<http_post uri="${github.baseuri}/xml/repos/delete/${name}?delete_token=${dtoken}"
onFailure="continue">
<entity>${authdata}</entity>
</http_post>
</event>
</function>Any call to this function will now get back the event github.delete which can easily be used to calculate the statistics you really wanted for the complete operation of deleting a repository from GitHub.