Pulling Data Back from the LRS: Correlation for Analysis

If you’ve been following along in my series of articles,you’ve covered a lot of ground. And now, it’s time to bring it all together. Inthis final article in the series, we’re going to look at the reporting side ofthe examples used in the third article: “xAPI Can Tell You What They Learned from the Video.” In that article, we lookedat sending the statements to record when someone played a video and what partof the video that user played. We also looked at one way to report which answerthe user selected on a quiz asking about the video. Again, the question was “Whatis the first animal you see in the video?” Not exactly the deepest or mostprobing question, I know. But it does show if the person was paying attention.

In this final outing, we’re going to look at how you can pull that data back from the LRS. UsingxAPI queries, we’ll show who answered the question, what their answer was, andif they played the relevant part of the video. This way we can find out whetherthe video is effective. Remember, there are three practical results we can gethere:

1. The average user watches the video, but gets theanswer incorrect. This could potentially indicate the video is not effective.
2. The average user does not watch the video, butdoes get the answer correct. This could indicate that the question was toosimple.
3. The average user who did watch the video answersthe question correctly, and the average user who did not watch the videoanswers the question incorrectly. This would tend to indicate that both thequestion and the video are effective.

We’re hoping to see the report trend toward the thirdoption.

But I’m not a programmer/developer

As usual, I will show code examples so you can see how I didthings. But the process is what’s really important here, so I’ll do my best todescribe that outside of the code.

We have to consider what we want to know. That means knowingwhat data we need, how to get it, and how we can piece it together. At the endof the day, the code is simple. The strategy is the more difficult thing to puttogether, and that can be made much easier. (A great place to start is with afantastic book from Sean Putman and Janet Laane Effron titled Investigating Performance: Design and Outcomes with xAPI.)

Looking at the data

So, from the previous articles in this series,we’ve generated some statements. There are two we want to look at here. The firstis the information the LRS knows about when I watched the video:

	{  "verb": {    "id": "https://adlnet.gov/expapi/verbs/Play",    "display": {      "en-US": "Video Played"    }  },  "version": "1.0.0",  "timestamp": "2017-08-19T13:37:22.625732+00:00",  "object": {    "definition": {      "name": {        "en-US": "Big Buck Bunny Video"      },      "description": {        "en-US": "sample description"      }    },    "id": "https://example.com/bigbuckbunnyvid.html",    "objectType": "Activity"  },  "actor": {    "mbox": "mailto:[email protected]",    "name": "anthony",    "objectType": "Agent"  },  "stored": "2017-08-19T13:37:22.625732+00:00",  "result": {    "extensions": {      "https://example.com/xapi/period_end": 17.4,      "https://example.com/xapi/period_start": 1.5    }  },  "id": "5125d510-00c5-4431-a3aa-9918d688d437",  "authority": {    "mbox": "mailto:[email protected]",    "name": "xapi-tools",    "objectType": "Agent"  }}

Let’s take a look at what we can learn from this:

• Who played the video (in this case, it was me)
• I played the video on August 19 at about 13:37Zulu
• The video I played was my favorite example: Big Buck Bunny
• I played it from 1.5 seconds to 17.4 seconds
• I playedthe video (the specific verb is useful to know, as well!)

That’s a good bit of information. And we’ll be able to useall of it. Now, let’s look at what we know about my effort to answer the quiz question:

	{  "verb": {    "id": "https://adlnet.gov/expapi/verbs/answered",    "display": {      "en-US": "answered"    }  },  "version": "1.0.0",  "timestamp": "2017-08-19T13:38:26.793810+00:00",  "object": {    "definition": {      "name": {        "en-US": "xAPI Video Quiz"      },      "description": {        "en-US": "Correlating quiz answers to video consumption"      }    },    "id": "https://omnesLRS.com/xapi/quiz_tracker",    "objectType": "Activity"  },  "actor": {    "mbox": "mailto:[email protected]",    "name": "anthony",    "objectType": "Agent"  },  "stored": "2017-08-19T13:38:26.793810+00:00",  "result": {    "extensions": {      "https://example.com/xapi/location": "15.6"    },    "response": "Bunny"  },  "id": "88291bfb-b7fb-4131-847b-e0445826461b",  "authority": {    "mbox": "mailto:[email protected]",    "name": "xapi-tools",    "objectType": "Agent"  }}

Similar to the video statement, we can learn a bunch ofstuff from this statement:

• Someone answered the question (again, guilty)
• I answeredthe question (verbs matter)
• I answered the question at 13:38 Zulu on August19
• I answered that specific question
• “Bunny” was my answer
• The answer can be found at 15.6 seconds into thevideo

Putting all of this together

So now we have a lot of information about this one consumer.It’s fairly easy to just eyeball the two statements and see that I watched therelevant part of the video, but I did answer the question incorrectly (thecorrect answer is “Bird”). But now let’s try that for a few thousand consumers. That’s not going tobe so easy. We want a way to programmatically query all the relevant statements. 

This is where strategy takes over: What is the exact question we want answered? I cansee two possibilities:

1. Of those who played the video, who answered thequestion and how did they answer?
2. Of those who answered the question, how did theyanswer and did they play the video?

Essentially, this translates into one of two processes, eachof which requires several steps:

1. Of those who watched the video, who answered thequestion and how did they answer?
• Query for statements where someone played thatvideo. (Remember, you can’t query on extensions! So we have to get all the statements.)
• Crawl through those statements for those whoplayed the relevant bit of the video, and collect the agent IDs.
• For each agent ID we collect, query for any statementswhere that agent answered the quiz question.
2. Of those who answered the question, how did theyanswer and did they watch the video?

• Query for statements where someone answered thequestion.
• For each agent that answered the question, queryfor statements where that agent played the video.
• Crawl through the video statements to see if theconsumer played the relevant bits of the video. (Again, we can’t query forextensions; we must do this on the client side.)

At first glance, you might not see a difference betweenthose processes. But there could be a huge difference here. Specifically, there’slikely to be many more statements for the video than for the question. Consumersmay have watched the video several times. Or there may be a significant numberwho have watched the video, but haven’t progressed through the course to thequiz yet.

In the end, we may have the same amount of statements to process, but if we takethe second option, we should have fewer queriesto run. So, it’s more efficient to take the second route and start by queryingfor those who answered the test question first. And that is easily done. Usingthe ADL xAPI wrapper, the code looks like this :

var quizParams = ADL.XAPIWrapper.searchParams()quizParams['verb'] = "https://adlnet.gov/expapi/verbs/answered";quizParams['activity'] = "https://omnesLRS.com/xapi/quiz_tracker";//	Now, issue the query var ret = ADL.XAPIWrapper.getStatements(quizParams);

Looking at the code, we create a variable for the queryparameters in line 1. In lines 2 and 3, we set the verb and activity IDs forwhich we want to query. In line 6, we send the query, assigning the returnedstatements to the variable “ret.”

Once we have the statements, we crawl through them and startbuilding the next query :

		if (ret) {		for (i = 0; i < ret.statements.length; i++) {							var studentName = ret.statements[i].actor.mbox;			var studentAnswer = ret.statements[i].result.response;			var studentTS = ret.statements[i].timestamp + "";				var answerLocation = 	 	ret.statements[i].result.extensions["https://example.com/xapi/location"];

In line 1, we make sure that there were statements returned.If there are, then one at a time, we look at them and pull out the data weneed, using the “for” loop defined in line 2. We’re collecting the student’sname (in this case it’s actually their email address), the answer selected,when they answered the question, and where the answer was provided in thevideo. This last one (on line 6) is by pulling from the result extension. Noticehow we have to reference that, using the URL defined in that extension’s keyedpair from line 29 of the quiz statement seen above. And, the dotted notationfor each of the data points is easy to follow:ret.statements[i].actor.mbox mapsto the mbox defined in the actor object of the statement stored in the variable“ret.” And so on and so forth.

So now we have all the information we need for each of thestatements where someone answered the question. Now, we need to see if theywatched the video. This requires another query. Again, fairly easily built :

	var vidParams = ADL.XAPIWrapper.searchParams()	vidParams['agent'] = '{"mbox":"' + studentName + '"}';	vidParams['verb'] = "https://adlnet.gov/expapi/verbs/Play";	//Make sure you pay attention the CASE OF THE VERB!!!!!!!!!	vidParams['activity'] = "https://example.com/bigbuckbunnyvid.html";	var vidRet = ADL.XAPIWrapper.getStatements(vidParams);

Note: First andforemost, take heed of the comment on line 4. Everything is case sensitive! Youmust query for the same case as you used in the original statements! So makesure you keep all of that consistent—I can’t stress that enough! (I may havelost several hours debugging that mistake. Sigh…)

So, same as the first query, we’re setting the verb and theactivity IDs. But this time, we’re also using the user email address wecollected from the quiz statements to search for any statements where that user played the videos. This time,we’re assigning the videos to “vidRet.”

I’ll spare you the code here, as it’s a little tedious, butI’ll include the full text below for the curious and adventurous. The processis straightforward, though: Look at each statement and see if the users playedthe segment of video containing the answer. We do this by looking at the resultextensions for the period_start and period_end and comparing those to theanswerLocation variable we defined in the quiz query-results example above. Ifthe users played that segment of the video, then we set a variable calledvidWatched to “true.” And if not, then we leave that variable as “false,”meaning they didn’t play that segment of the video. Then we build that line ofthe chart and move on to the next statement in the quiz query results, and doit all over again.

The finished product will look like Table 1.

Table 1: The result of the query

If we know the correct answer (Bird), then we can see whoanswered the question correctly and if they played that segment of the video. Now,from here, we could add code to check for correct answers, and if they playedthe video, and add a column to reflect that. I did not do that here, for thesake of simplicity. There’s already a lotgoing on with this page.

In closing…

And that’s it, folks! With only a few lines of code, we’reable to bring together two completely disparate sets of data, compare them, andthen correlate that data to show if the video is effective or not. In thiscase, looking at the results, it is not!

Notice that of the six people who played the video, onlythree got the correct answer! That’s only 50 percent. That’s hardly effective. Butnow we can show that. We have the data to prove this. OK, technically not yet: We’dneed far more data, as this sample size is far too small. But you can see how thiscan scale up quickly and easily. And at the end of the day, this, more thananything else, is the most exciting part of xAPI. Before xAPI happened along,it was very difficult to build reports like this pulling together this data. Now,you can do it with the push of a button (and some code behind the scenes). Thisis what makes xAPI special. This is what makes xAPI powerful. And this is why,in this author’s humble opinion, xAPI is the way we will track our consumers inthe future. It shouldn’t be much of a stretch for you to see how, using similarqueries to those I’ve used here, you could make courses adapt to a student’sresults in another, prerequisite course. Or how you could build just about anyother report you can imagine.

I really do hope you’ve found this series of articleshelpful. I’ve tried to be as thorough as I can be without being toooverwhelming. xAPI can be a little like the game Go: a moment to learn; alifetime to master. But it doesn’t have to be. You can make it as complex or assimple as you want! It’s all up to you and your needs. All that power is therefor you to use at your will. And no matter how complicated you need it to be,just remember this…

It all starts with four lines of code.

Bonus

Below is the full text for this example. I hope you can findit useful!

Note to developers:Yes, this is a painful way to do this. It requires N-many queries based on howmany people took the quiz. If 1,000 people took the quiz, we’re running 1,000queries for the video statements. The point here is to illustrate the process,not necessarily the most efficient method. In this example, I can show twoquery processes. So it meets the needs of this article very nicely, althoughisn’t really “production ready .”

<!doctype html><head>  <meta charset="utf-8">  <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">  <title>xAPI sample page</title>  <meta name="description" content="">  <meta name="author" content="">    <style>	table, th, td {    	border: 1px solid black;    	border-collapse: collapse;	}</style>	<script type="text/javascript" src="./js/cryptojs_v3.1.2.js"></script>	<script type="text/javascript" src="./js/verbs.js"></script>	<script type="text/javascript" src="./js/xapistatement.js"></script>	<script type="text/javascript" src="./js/xapiwrapper.js"></script><script>	function config_LRS(){//  Tell the content where to send the xAPI statements			var conf = {			  "endpoint" : "https://lrs.adlnet.gov/xapi/",			  "auth" : "Basic " + toBase64('xapi-tools:xapi-tools')			};  																								function get_statements(){		config_LRS();//	Start by getting the list of statements for those who answered the quiz question//	First, we set the parameters for the query:		var quizParams = ADL.XAPIWrapper.searchParams()		quizParams['verb'] = "https://adlnet.gov/expapi/verbs/answered";		quizParams['activity'] = "https://omnesLRS.com/xapi/quiz_tracker";//	Now, issue the query 		var ret = ADL.XAPIWrapper.getStatements(quizParams); 			console.log("answered list: "); 			console.log (ret); 			var txt = " "; 			if (ret) {// walk through the list of "answer" statements to see what we have 			for (i = 0; i < ret.statements.length; i++) {				 				var studentName = ret.statements[i].actor.mbox;				var studentAnswer = ret.statements[i].result.response;				var studentTS = ret.statements[i].timestamp + "<br>";					var answerLocation = ret.statements[i].result.extensions["https://example.com/xapi/location"];//	When we find a statement for someone who answered the question, search for statements//  for when/if that user played the video//	Set the parameters for the query for "Play"								var vidParams = ADL.XAPIWrapper.searchParams()				vidParams['agent'] = '{"mbox":"' + studentName + '"}';				vidParams['verb'] = "https://adlnet.gov/expapi/verbs/Play";						// Make sure you pay attention to the CASE OF THE VERB!!!!!!!!!				vidParams['activity'] = "https://example.com/bigbuckbunnyvid.html";//	Send the query for statements where the student played this video				var vidRet = ADL.XAPIWrapper.getStatements(vidParams);				var vidWatched = false;//	Now walk through the Play statements				for (x = 0; x < vidRet.statements.length; x++) {						if (!(vidWatched)) // Look to see if the user played the relevant segment of the video						if (answerLocation > vidRet.statements[x].result.extensions["https://example.com/xapi/period_start"])							{							if (answerLocation < vidRet.statements[x].result.extensions["https://example.com/xapi/period_end"])								{								vidWatched = true;								console.log ("vidWatched = true");								}								else { vidWatched = false;									console.log ("vidWatched = " );								} 							}							else { var vidWatched = false;									console.log ("vidWatched = " );								}					}				var table = document.getElementById("result_table");				var row = table.insertRow(i + 1);    			var cell0 = row.insertCell(0);    			cell0.innerHTML = i;    			var cell1 = row.insertCell(1);    			cell1.innerHTML = studentName;    			var cell2 = row.insertCell(2);    			cell2.innerHTML = studentAnswer;    			var cell3 = row.insertCell(3);    			cell3.innerHTML = vidWatched;    													}		}	}</script></head><body><button type="button" onclick="get_statements()">Get Statements</button><br/><br/><br/><div id="results">	<table id="result_table" style="width:50%">  		<tr>    		<td> # </td>     		<td><b>Student Name</b></td>    		<td><b>Student Answer</b></td>    		<td><b>Video Watched</b></td>  		</tr>	</table></div></body></html>