Major New Official QVSource Release (1.4.2) Now Available for QlikView

We just published our latest official QVSource release (1.4.2) and it was a big one - in addition to the usual updates and improvements to the connectors we have also added a number of stability, performance and logging enhancements which should be of interest to enterprise users of QlikView.

If you are currently evaluating QVSource and especially if you are a commercial user we would strongly recommend you read this post as it almost certainly contains information which will affect you.

Much of this information was also discussed in our recent blog posts here, here, here and here, but there are also some new updates specific to this latest build.

Last Beta Version Of Certain Connectors

We are planning to take the following Connectors out of beta in the next official build of QVSource (probably in about 6 - 8 weeks time). This is particularly important to commercial users of QVSource who might currently be using these as when they are taken out of beta you will need a commercial licence to continue to use them:

Deprecated Connectors

This will be the last official build of QVSource which contains V1 of the Google Spreadsheet and Google Analytics connectors. Both of these have now been superseded by much improved 'V2' versions which we would recommend you migrate to.

Synchronous, Asynchronous and Batch Request Modes Added

This is a significant new capability in QVSource which was explained in detail in the previous blog post and so will not be covered again here. This update has the potential to give major performance improvements and also change and improve the way QVSource related QlikView script is constructed.

One thing to note is that we have now removed the previous mechanism for running asynchronous requests as the new functionality improves and supersedes this.

QVSource Windows Service

QVSource now has a dedicated Windows Service available and documented here. We have always had a well documented and stable way of configuring QVSource to run on a server environment however the new Windows Service represents an improvement over this technique and in time will likely become the recommended and only supported method.

Related to this, we have also now made it so if QVSource is running in server mode you can run QVSource desktop in order to manage connector and other settings without having to then restart the QVSource server instance for changes to be picked up.

First Version Of New Twitter Connector (V2)

This build has the first version of the new Twitter Connector V2 in which also illustrates the new UI style we are working towards as discussed here. Unfortunately Twitter will start shutting down version 1.0 of their API in early march so all users should start upgrading to this new version ASAP.

First Version of New Sentiment Analysis and Text Analytics Connector (V2)

This build has the first version of the new Sentiment Analysis and Text Analytics Connector in. This also illustrates the new UI style we are working towards as discussed here. We would also recommend users start working with this new version and reporting any issues as it will soon replace V1 of this Connector.

Checking the Status of QVSource

It has always been possible to browse to http://localhost:5555/QVSource to get a html page with various pieces of information about QVSource (e.g. the licence state, version, which Connectors were running). We have now tidied this up and added additional information. More importantly, we have added a new XML option at http://localhost:5555/QVSource?format=xml which allows you to check the exact QVSource status from your QlikView load script as explained here.

Log File & Error Page Updates

QVSource now has a much more structured XML file format with additional information added. As before, logging needs to be enabled for these to be generated as explained here. It should now be possible to load these files into QlikView directly for structured reporting.

Related to this, the HTML error page which QVSource sends back has also been enhanced and tidied.

appID Load Script Parameter

QVSource now adds an appID= parameter to the generated QlikView load script. We recommend setting a variable at the start of your load script, e.g.:

let vQVSourceAppId = 'my_app_identifier';

And then adding this variable to all of your QVSource requests, e.g:

....&appID=$(vQVSourceAppId)&....

This provides an identifier to QVSource to be used in error logging and tracing. This is explained in more detail here.

Note that in a recent 'nightly build' we actually generated the full appID=$(vQVSourceAppId) in the QVSource generated load script. Unfortunately it was pointed out that this actually breaks the QlikView 'Web File' Wizard (used for example in the General XML/JSON Connector and certain tables in other Connectors) - so now we just include an empty appID= parameter in the URL leaving it to the user to insert a value or variable in the final QlikView script as recommended above. This isn't mandatory or required and it can be left blank.

Other Connector Changes & Updates

  • The Google AdSense Connector for QlikView has been upgraded to use the latest API (version 1.2).
  • The Google Analytics Connector For QlikView has had a number of minor tweaks and fixes. These are only likely to affect you if you were using the MaxNumberOfPages parameter (i.e. not bringing back all the data for a particular query url).
  • Empty \Temp folder has been added to the distribution - this is currently the recommended location to store the parameters CSV files used by the new processParams[Sync|Async|Batch] feature.
  • The General JSON/XML API Connector now has a better preview of XML and HTML and also a new checkbox to encrypt the request URL before including in your QlikView script to help secure any sensitive credentials it might contain.
  • The Facebook Fan Pages and Insights Connectors now have buttons to 'Deauthorise' the Connectors (i.e. detach them from your Facebook account).
  • The Facebook Fan Pages Connector has a new QuestionVotes table listing the users who selected each option.
  • The Klout Connector has been tweaked to work more efficiently with the new new async feature. Note that the Klout V1 Connector for QlikView has been removed from QVSource.
  • The FreeAgent Connector has two new tables related to the Trial Balance. We have also removed V1 of this connector and recommend upgrading to the latest V2 version.
  • The first version of the Google DoubleClick For Publishers (DFP) Connector is now included. We hope to add more functionality to this soon. Please get in touch if you are interested.
  • The Semiocast Connector for QlikView now supports new languages and has some performance improvements.
  • We have upgraded the MongoDB Connector for QlikView to use the latest c# driver and also fixed a couple of minor issues.

All these recent features further enhance the speed and ease at which you can load your QlikView applications with data from a large number of web APIs.

We hope you like the updates and we are looking forward to continuing to innovate and improve QVSource - The QlikView API Connector in 2013!

New QVSource Nightly Build (1.4.0.8) Available

We have just pushed out a new nightly build of QVSource which has all of the updates noted here plus the following:

All these recent features further enhance the speed and ease at which you can load your QlikView applications with data from a large number of web APIs. We expect this will be the basis of the new official release coming in the next two weeks.

Major New QVSource Nightly Build (1.4.0.4)

Good news - We have just made a major new QVSource release available.

As there have been many significant changes and improvements this is currently only available as a 'nightly build' - this means it's not yet recommended (or supported) for critical production environments but we would recommend all QVSource users to try out this new release on their QVSource powered QlikView applications and report back any feedback or issues you are experiencing.

Where do I find this new release? If you are a QVSource customer or have requested a trial in the past you should see this new release at the personalised download link we should have sent you via email.

Note that it's possible that in the week(s) after this blog post and release is published, we will upload new nightly builds with incremental changes and fixes based on the feedback so please check the download page regularly for further updates.

Once this new release has been out for a few days to a week we will probably push out a dedicated email newsletter so that users who might not necessarily follow this blog or our Twitter, Facebook or other social media accounts, are aware of the new release.

We would then expect this to be released as the new official release some time in February.

The rest of this blog posts outlines the main changes in this latest version.

Before that - If you like what we are doing with QVSource and haven't already, please take a minute to rate us on QlikMarket or leave a recommendation on our LinkedIn product page.

Synchronous, Asynchronous and Batch Request Modes Added

This was explained in detail in the previous blog post and so will not be covered here. However this update has the potential to give major performance improvements and also change and improve the way QVSource related QlikView script is constructed. One thing to note is that we have now removed the previous mechanism for running asynchronous requests as the new functionality improves and supercedes this.

(Note: Version 1.4.0.7 released as a nightly build 26/01/13 fixes an issue in the Async feature of this).

QVSource Windows Service

QVSource now has a dedicated Windows Service available and documented here. We have always had a well documented and stable way of configuring QVSource to run on a server environment however the new Windows Service represents an improvement over this technique and in time will likely become the recommended and only supported method.

Related to this, we have also now made it so if QVSource is running in server mode you can run QVSource desktop in order to manage connector and other settings without having to then restart the QVSource server instance for changes to be picked up.

Checking the Status of QVSource

It has always been possible to browse to http://localhost:5555/QVSource to get a html page with various pieces of information about QVSource (e.g. the licence state, version, which Connectors were running). We have now tidied this up and added additional information. More importantly, we have added a new XML option at http://localhost:5555/QVSource?format=xml which allows you to check the exact QVSource status from your QlikView load script as explained here.

Log File & Error Page Updates

QVSource now has a much more structured XML file format with additional information added. As before, logging needs to be enabled for these to be generated as explained here. It should now be possible to load these files into QlikView directly for structured reporting.

Related to this, the HTML error page which QVSource sends back has also been enhanced and tidied.

appID Load Script Parameter

QVSource now adds an appID=$(vQVSourceAppId) parameter to the generated QlikView load script. We recommend setting this variable at the start of your load script (although it is not mandatory) to help provide an identifier to QVSource to be used in error logging and tracing. This is explained in more detail here.

Note that it was pointed out that appID=$(vQVSourceAppId) actually breaks the QlikView 'Web File' Wizard (used for example in the General XML/JSON Connector) - so in 1.4.0.8 and later we actually just include appID= in the URL leaving it to the user to insert a value or variable in the final QlikView script).

Connector Changes & Updates

  • The Klout V1 Connector for QlikView has been removed from QVSource, please upgrade to the new version. We have also made a couple of fixes and improvements to this new Klout Connector.
  • We have upgraded the MongoDB Connector for QlikView to use the latest c# driver and also fixed a couple of minor issues.
  • The Google AdSense Connector for QlikView has been upgraded to use the latest API (version 1.2).
  • The Semiocast Connector for QlikView now supports new languages and has some performance improvements.
  • We have removed V1 of the FreeAgent Connector for QlikView, please upgrade to the latest V2 version.
  • Added option to encrypt the request url in the General JSON/XML Web API Connector (version 1.4.0.7 released as a nightly build 26/01/13).
  • Deauthorise buttons have been added to the Facebook Fan Pages and Insights Connectors to remove the app from your Facebook profile (version 1.4.0.8 and later).
  • Empty \Temp folder has been added to the distribution - this is currently the recommended location to store the parameters CSV files used by the new processParams[Sync|Async|Batch] feature(version 1.4.0.8 and later).
  • The Web Preview window in the General JSON/XML Web API Connector should now format XML and HTML properly (version 1.4.0.8 and later).

Synchronous, Asynchronous and Batch Requests - New Performance Options for QVSource

This is the second of two posts outlining some significant changes and improvements we are making to QVSource. The first post discussed the front end UI improvements we are working on. This post covers some back end changes we are making.

This post is essentially a slightly higher level version of the more detailed documentation page you will find here.

QVSource related QlikView script is often characterised by for/next loops in which each iteration makes a request via QVSource, for example to run a search on Twitter or run a text message (Tweet, Facebook post etc.) through a text analytics or sentiment analysis API. The following illustrates a typical request:

LET noRows = NoOfRows('Tweets');
    
for i=0 to $(noRows)-1
     
     let id = peek('Search_id', $(i), 'Tweets');
     let textEncoded = peek('text_urlEncoded', $(i), 'Tweets');
     
     Sentiment:
     LOAD
        '$(id)' as Search_id,
        status as sentiment_status,
        score as sentiment_score
     FROM
     [http://localhost:5555/QVSource/TextAnalyserV2/?table=Sentiment&
api=TwitterSentiment&text=$(textEncoded)&appID=$(vQVSourceAppId)format=qvx] (qvx); next

 

The new feature in QVSource allows this code to be rewritten as the following:

let vQVSourceAppId = 'test_app_';
let vWorkingFolder = 'c:\QVSource\Temp\';
let vArgsFile = '$(vWorkingFolder)$(vQVSourceAppId)' & 'SentimentArgs.txt';
 
Params:
LOAD
    Search_id as rowKey,
    Search_text as text,
    'TwitterSentiment' as api,
resident
Tweets;
 
store Params into '$(vParamsFile)' (txt);
drop table Params; // Optional.
 
Sentiment:
LOAD
    rowKey as Search_id,
    status as Sentiment_status,
    score as Sentiment_score,
    language as Sentiment_language
FROM
[http://localhost:5555/QVSource/TextAnalyserV2/?table=Sentiment&
processParamsSync=$(vParamsFile)&appID=$(vQVSourceAppId)&format=qvx] (qvx);

 

This brings us some immediate advantages:

  • The for/next loop is completely eliminated leading, arguably perhaps, to cleaner, easier to maintain code. If the QlikView script execution was being logged to a file, for/next loops with a large number of iterations could create huge log files which is also now avoided.
  • In the above for/next script example note that the text needs to be URL encoded before being sent to the API (this is so that certain characters, for example the #, do not break the GET request to QVSource). This means either having additional complex QlikView script to URL encode the content or ensuring that it was URL encoded before loading into QlikView. This new script version makes this requirement redundant.
  • Similarly, in the above for/next example, very long messages would occasionally break the request due to some maximum URL length (which we believed QlikView to be enforcing). Again, the new functionality makes this issue redundant.
  • In the above for/next  example you are constrained to needing your list of data for the for/next loop already in the QlikView data model. With the method outlined below you can take this data from any file location or even http/https web urls.

The real power of this new approach however is that it allows us additional methods of processing the list of parameters stored in the Params table.

A bit of background on this first.

In the above for/next example, each message would be processed sequentially, meaning that the sentiment score for the next message cannot be calculated until the current request has completed. If we estimate each request takes 250ms then we could be looking at 40 minutes to score 10,000 Tweets (of Facebook posts or whatever) for sentiment.

The thing is, many APIs allow multiple requests to be made in parallel from the same client and these requests will be handled by separate processes on the server (or cloud) allowing many more requests to be processed per second. And the Asynchronous mode described below allows us to take advantage of this feature.

We have already done some testing with asynchronous/parallel requests to many of the sentiment APIs QVSource connects to and shown some huge performance increases using this technique:


Further to this, some APIs also allow multiple requests to be combined into a single API call (i.e. http request), for example sending 1000 text messages in a single request to be processed for sentiment. This is often referred to as sending requests in a batch and it is the new Batch mode, described below, which allows us to take advantage of this feature.

Asynchronous Mode

To convert the above script to process the requests asynchronously simply change the parameter name processParamsSync into processParamsAsync.

Sentiment:
LOAD
    rowKey as Search_id,
    status as Sentiment_status,
    score as Sentiment_score,
    language as Sentiment_language
FROM
[http://localhost:5555/QVSource/TextAnalyserV2/?table=Sentiment&
processParamsAsync=$(vParamsFile)&appID=$(vQVSourceAppId)&format=qvx] (qvx);


QVSource will then run each or the requests in parallel on a pool of threads, and the above QlikView request will only run when all requests have completed.

Please note, that some tables in some connector might explicitly disallow Asynchronous requests (due to lack of support from the underlying API). Please see the more detailed documentation to see how to know when a table supports this.

Batch Mode

To convert the above script to process the requests asynchronously simply change the parameter name processParamsSync into processParamsBatch.

Sentiment:
LOAD
    rowKey as Search_id,
    status as Sentiment_status,
    score as Sentiment_score,
    language as Sentiment_language
FROM
[http://localhost:5555/QVSource/TextAnalyserV2/?table=Sentiment&
processParamsBatch=$(vParamsFile)&appID=$(vQVSourceAppId)&format=qvx] (qvx);


Please note, that only some tables in some connectors will support Batch mode. Unlike the Asynchronous method the API needs to have specific mechanisms for supporting this and the QVSource Connector needs to be specifically coded to take advantage of it. Please see the more detailed documentation to see how to know when a table supports this.

Auto Mode

See the docs for a description of this.

We hope to have a new nightly build out by the end of the week exposing the initial version of this new functionality for testing. As always we would welcome your feedback and suggestions.

Don't forget that you can get a free fully functional trial of QVSource from our website.

QVSource 1.4.0 Released

The end of November marked QVSource's 1st anniversary of being officially released. It's been a fantastic year for the product, which has seen it grow and mature. The enthusiasm for from you, the QlikView community, has been amazing and helped us quickly become the #1 rated product on QlikMarket.

It's been a little while since we released anything new because we're working on the next major release, which will see an increased focus on usability and performance (we have already started talking about these here on this blog). But we thought that we'd get out a release before the Christmas break, which brings in a number of enhancements, new connectors and important fixes - please note if you're a user of the Facebook Pages Connector, then there's a very important fix for you.

Another thing we'd like to highlight is that if you're still using our Klout V1 Connector, please read on as you need to switch to our newer Klout Connector as soon as possible.

Important Facebook Pages & Groups Connector Fix!

As mentioned above there's an important fix for users of the Facebook Pages & Groups Connector - a very recent change in the Facebook API  has meant that the User and Page tables weren't working any more. Since these 2 tables are commonly used, this is likely to have an impact on most users of this connector, so we strongly recommend that you install the latest version of QVSource as soon as possible which will fix the issue.

New Connectors

  • We have the first release of a new Google+ Connector!
  • We have the first release of a brand new Boardreader Connector, which enables you to read data from message boards, online user forums, blogs, review sites and much more!
  • After a short absence, we've added the Google Calendar Connector back.
  • Although it isn't a new connector we've added some cool new features to the Notifier Connector, which have added a new dimension to it. You can now send up to 5 email attachments with your messages, so this could be log files or csv files that have been created during the script running. You can also delay sending the email by up to two minutes, which means you could even email the QlikView application itself after it's fully reloaded. We think this could be really useful!

Other Connector Updates

We also have a number of other exciting updates.

  • Klout are shutting off access to their old API at the end of this year. So, this will be the last version of QVSource that includes the Klout V1 Connector. You should migrate to the newer Klout Connector as soon as possible - this will require you to regenerate your Klout based scripts.
  • We've renamed the Facebook Pages Connector to Facebook Pages & Groups Connector, this is to make it more obvious that you can get Group pages too. We have also added some new tables to this connector.
  • The CheckAccessToken table in both the Facebook Insights and Facebook Pages & Groups Connector has a breaking change to the columns returned.
  • Users of the HelloPeter Connector will need to upgrade, because the API end point was moved by HelloPeter.
  • There was an issue with the iTunes Connect Connector where some reports couldn't be uncompressed. This has now been fixed, although you need to have 7Zip installed.
  • The Semiocast Connector now supports basic sentiment analysis in Arabic, English, French, Italian and Spanish without first providing a training model. This means that it can be used in a similar way to our other Sentiment Analysis and Text Analytics Connector - in fact we plan to merge it into the same Connector in a future release. (Note however that a trained model will give better results).
  • We have made various improvements and fixes to the AlchemyAPI and Saplo Connectivity in the Sentiment Analysis and Text Analytics Connector and would recommend regenerating your load scripts if you are using these and checking the column structure.
  • We have added a new table to the LinkedIn Connector which allows you to run any LinkedIn API REST query. This does mean you will need to spend some time understanding their API but should allow data to be extracted which isn't covered by the other tables in the Connector.

General Updates

As usual there have been a number of core performance improvements which should benefit all users and all connectors.

Staying Up To Date

Don't forget, we have altered our wiki so that new accounts are automatically activated via email and we have activated all users who have previously attempted to create an account. We recommend you use the wiki (instructions here) to receive automatic alerts when connectors, features and demo apps you are interested in are updated.

Download Version 1.4.0

If you're new to QVSource and would like to download a free trial, you can sign up here.

If you've already registered for QVSource and you trial licence has expired but you would like to give it another go, just email us a copy of your licence file and we will get you set back up again!

Connecting QlikView to the Zeit Online API Using QVSource

Zeit Online, the online version of one of Germany's leading newspapers, has released a beta version of an API to their content and I couldn't resist a quick play and blog post showing how easy it is to connect QlikView to the API using QVSource.

From their own developer website:

Via the API you can access articles and corresponding metadata from the ZEIT archive dating back to 1946 and from ZEIT ONLINE. We invite everybody interested in programming, data analysis and visualization to explore the index of hundreds of thousands articles. The API can be used for free for non-commercial use cases. Please refer to the terms of use (German only at the moment).

An API key can be generated in seconds - they simply ask for your name and email address and once generated their online API explorer makes it quick and easy to build and test queries.

I started with a simple query by selecting content from the drop down of options in the API explorer (click on the word after GET/ to see the drop down) and entering merkel for the search query as highlighted below. Clicking the Send request button then quickly returned back some results in json.

A little bit of research shows that only the parameters I have filled out get passed to the API - so the actual request is:
http://api.zeit.de/content?q=merkel

And scrolling to the bottom of the results reveal there are actually 5376 results in total but only 10 returned:

To get all the results, we simply update the limit parameter to something which will capture everything:

 

The request to the API has now become:
http://api.zeit.de/content?q=merkel&limit=6000

We can now use the QVSource Web Connector to connect QlikView to this API. There is just one more thing we should be aware of which is that the API requires that the API key is passed either as an X-Authorization header or via a api_key parameter. The former is preferred by the API but the second is supported and makes our job even easier - so our API request now becomes:

http://api.zeit.de/content?q=merkel&limit=6000&api_key=[INSERT YOUR API KEY HERE]

And you can even test this in a browser.

We now open up the Web Connector in QVSource and enter this url as a GET request:

And on the Data tab we can now check the RawResponse table to check the data is coming back as expected.

There are two ways we can get this data into QlikView now. The first is to use the JsonToXmlRaw table - this uses QVSource to convert the JSON to XML and then provides a new URL for the request which can be used in QlikView as a Web File data source. This is shown below:

We can simply copy the URL highlighted above and use it in the QlikView load script Web File dialog as shown below:

A more elegant solution is to use the QVSource JsonToTable table. To use this we must first do a quick simple analysis of the JsonToXmlRaw request, to do this we can take the URL generated above and browse to it in Internet Explorer where we get a nicely formatted XML response:

From this is should be quite clear that each row in the table we want is an instance of the matches element, and the XPATH to select these elements is then DATA/matches. To use this we need to enter it into the config screen of the connector as shown:

Now we have done this we can use the JsonToTable table in QVSource and get a nicely formatted table:

And the auto generated script to load this into QlikView as a QVX file:

We can now easily load this into QlikView and start analysing:

(Note that I actually changed the search term to 'angela merkel' for this final chart).

This is really just the start of what could be done with QlikView + QVSource. For example, QVSource also has a powerful Text Analytics and Sentiment Analysis Connector which could be used to score the article snippets and headlines for sentiment, or to extract entities such as categories and concepts.

And of course QVSource has connectors to Facebook, Twitter, YouTube and a ton of other sources which would allow you to bring in and mash up data from all sorts from other places.

Zeit are inviting users to show what can be done with their API - the their words:

We invite you to share your apps, experiments and experiences with the API. We will highlight interesting projects in the App Gallery section. Wir laden Sie ein, uns Ihre Umsetzungen, Experimente und Erfahrungen mit der API mitzuteilen.

So it would be great to see what the QlikView community could do with the API. Don't forget you can register for a free fully functional evaluation edition of QVSource here.

A major milestone but no time to stand still

We achieved a major milestone this month which marked the end of our first year of providing QVSource commercially.

We are considering the first year a great success and are delighted that already all of our earliest customers (excluding one desktop only user) have renewed for the second year - thank you!

We have also maintained the highest rated position on QlikMarket and currently have 18 5-star reviews - if you are a user and like the product please take a minute to rate us.

But it's no time to get complacent - we have been connecting QlikView to APIs since the end of 2010 when QVSource development began, and with all the experience we have gained and lessons we have learned we know we can still do many things better. Year one was about getting the basics right, next we will be focusing on usability and performance.

So this is half of a two part post on some of the improvements we are working on. In this post I'll look at our front end (user interface) and how it's changing and in the next post I'll look at our 'back end' QlikView scripting and how we are planning to simultaneously reduce complexity and improve performance.

Our current connector config UI is perfectly functional and gets the job done but there are a few challenges it still presents:

  • It shows all the input fields all at once, even though some may not be relevant to all the tables you are interested in.
  • It lacks supportive information about the fields and tables.
  • Each connector's UI is manually built which leads to some inconsistencies between connectors.
  • It presents all the tables all of the time, even though 80% of users will typically only use a common subset of tables.

A case in point is the Twitter Connector - here's a screen shot of the current version:

 

Current users will no doubt recognise this and some of the issues outlined above which it presents. I am not picking on this connector, I think all but the simplest of connectors have a similar issue.

So we are now re-developing the connector config user interfaces to have a layout which looks more like this (note this is still in development so the soon to be released version might differ slightly): 

 

A few things to point out.

At the top left a new drop down now allows the user to see different groups of tables. This is because for many connectors, 80% of users will typically only use a common subset of tables so now the first time they use the connector they are presented only the most common or popular ones to avoid information overload. The drop down allows you to select different grouping and of course 'All Tables' will be an option for power users: 

 

Below the table list we now have a new area where additional information about the selected table which could be relevant to the user is shown - here we see an example for the Twitter search table:

 

This will often essentially be us mapping relevant documentation from the underlying API to the QVSource user. Hyperlinks can also be included here to link to our docs or other relevant web pages.

The final big change is the input parameter section - this should now only show input parameters for the currently selected table:

 

Which should make the connector far more intuitive to use. This interface is also now 'auto generated' which means that there will be a much greater level of visual consistency between the connectors and also makes our SDK easier to use as you simply need to define which parameters each table needs and let the framework take care of the rest.

You should also notice that we now have tool tips on some of the fields - where necessary this allows additional helpful information about the fields to be show.

Finally, when the table's data is requested we now have much improved parameter validation before the request even gets to the connector. Here we see what happens when the Search table is requested without entering a query.

 

We hope you like the new changes, as they are still being worked on please let us know any further suggestions or concerns you might have and we will try to address them in the coming weeks.

In the next blog post I'll be discussing some of the changes we are making to simplify and improve the performance of the QlikView scripting for QVSource.

Script Updates In New Twitter Demo Application For QlikView

In our last post we discussed the front end changes in our template QlikView application for Twitter analysis. We have also made a number of changes and improvements to the load script which I wanted to briefly outline.

  • The sentiment analysis can now be run asynchronously which can give huge performance boosts (up to a 90% reduction in load time of the sentiment in some of our tests as discussed here). This is turned on using the vAsyncSentiment variable in the config file.
  • New variables have been added to the config file to turn off sentiment analysis (vGetSentiment) and Klout scoring (vGetKloutScores) as we have seen some use cases where the user only wishes to load matching Tweets into the application.

As usual the full list of changes to the application can be seen on the first tab of the load script:

This latest version of this template application (version 1.0.5.1) is available for download on our wiki here. We would also recommend reading the contents of config.txt which is fully documented and explains some of the other advanced options which this application provides.

You will need QVSource to be able to reload it with your own data and you can register for a free fully functional trial on our website.

New Twitter Demo For QlikView Available Online And For Download

We've just published online a new version of one of our Twitter applications, which is tracking Tweets based around 3 popular web browsers: Google Chrome, Internet Explorer and Mozilla Firefox. In terms of a data model it's very similar to the previous version but we've given it a fresh new look and made it a bit more iPad friendly i.e. not too wide and easy on the eye.

With this application we tried to look at some interesting metrics that we hadn't explored before, such as at what rate are people joining the conversation, the share of voice each day across the search terms and what are the most influencial people talking about. But also simple things such as identifying and pulling out images that are in the Tweets and the most mentioned users.

We also wanted to improve the message that QlikView and QVSource give users a unique ability to understand what's going on in this relatively new area of analytics

Chained Reloads

This application is a little different to our normal demos because its load script doesn't actually contain any QVSource based script. Instead, it relies on another QlikView application (called the Twitter Demo - Load Script & Data Model application) to do the work using the Twitter Connector and then enriching it for Klout scores and sentiment & text analytics. This 'Primary' application creates the basic data model and generates a number of QVDs that represent different parts of the data model, such as the Klout scores or Twitter accounts being tracked.

The idea we had was that we could build other applications that use this base data but then manipulate, create tables or ignore some data so that new metrics could be developed to meet the specific dashboard needs without having to make unnecessary calls to Twitter or trying to recreate the data model every time for slightly different requirements. 

Since we want the data to be up-to-date we reload the primary application every 10 minutes. But how do you know when to reload the dependant application? Every 10 minutes, but 5 minutes after the first one started? Depending on what you're trying to track this might be fine i.e. low volume Tweets. But it's also risky - what if there's a surge in data and the primary application takes 11 minutes to load? Could the dependant application start to load too soon?

Well, thankfully QlikView has a really simple function that only allows an application to start a reload once another application it depends on has successfully completed its own reload - we call this Chained Reloading.

To do this, you need to go to your QliKView Server Management Console and set up your primary application to reload whenever you need it to.

Then, select the application from the list which you want to reload when the other's finished and select from the scheduled reload options "On Event of Another Task", then "Successful", then the primary application. It's as simple as that!

Get the Application

You can view this application online here or you can download the Twitter pack, which includes the Load Script & Data Model application and instructions to get your data in to it here.

Facebook Health Checker Template Application for QlikView Updated

We have just given our template application for Facebook social media analysis in QlikView a review and overhaul, making a number of improvements and fixes.

You can see a live version of this application on our website here:
http://www.qvsource.com/QlikView-Live-Demos/Live-Facebook-Pages-Demo

Where we are running daily reloads of Facebook data for three of the top browsers as an example. But QVSource users can reload this with data from any Facebook Fan Pages or Groups.

So for example if you are a QlikView user with responsibility for your social media strategy you could use this dashboard to monitor data for your brand and those of your top competitors, comparing engagement, sentiment and large number of other metrics which the data model makes available.

It really is that easy - a small config file allows you to specify the Facebook Fan Pages you would like to load data for, as well as a number of other settings including how far you would like to go back in time, which sentiment API(s) you would like to use etc. The application is then designed to be reloaded as often as you like (we often use daily reloads), intelligently accumulating new data into QVDs and providing a rich data model from which to base your own KPIs and dashboards.

(BTW - We also have similar applications for Twitter, Google Analytics, YouTube and a number of other APIs on our wiki).

One of the main changes to this version is the ability to load sentiment and user info asynchronously which can give big performance boosts - we also plan to allow this for posts and comments in the next version as well.

The list of updates and changes are found in the first tab of the load script and also copied below:
* Option added in config to load sentiment asynchronously for better performance (vAsyncSentiment).
* Option added in config to load user info asynchronously for better performance (vLoadPostersAndCommenterInfoAsync).
* Sentiment now only loaded when status = 'OK'.
* Commenter_name not Commenter_username now used in AllUsers data set which fixes an issue where the name for some commenters was empty (when loaded from QVDs).
* AllUsers_language and AllUsers_country added (parsed from locale) (when loaded from QVDs).
* Added Page_loaddate_Month to FB_Page (when loaded from QVDs).
* Comments Per Post vs Date no longer accumulative.
* Likes Per Post vs Date no longer accumulative.
* Removed print and export to Excel icons on charts.
* Added country and language charts for All Users.
* Fixed Sentiment Vs Gender Chart.
* Fixed error in summary count of posts and comments on intro tab.
* Various UI rearrangements and improvements on all tabs and many charts.

QVSource allows QlikView users to connect to literally hundreds of APIs quickly and easily - this application uses the Facebook Fan Pages & Groups Connector and the Sentiment Analysis & Text Analytics Connector. You can grab a free fully functional evaluation of QVSource on our website.

Page List