New QVSource Release (1.4.9) Available

We have just pushed out a major new release of QVSource with a large number of updates including four exciting new connectors for QlikView.

If you are currently evaluating QVSource or using it commercially we would definitely recommend you start using this new version (after reading the notes below carefully).

In particular, if you are a commercial user, please see if any of the MailChimp, Klout, Twitter or Facebook related fixes below affect you - if they do you should test your apps with the new version and upgrade.

In addition to the highlighted changes below, please check the change logs on our wiki for any connectors you are using as there have been a number of other minor updates and improvements.

New Connectors

In alphabetical order:

  • We have a new connector to Blue Yonder who provide a service and API for forward demanding prediction and planning. See our sample app which shows how to use this Connector. 
  • We have a new File Transfer Connector which allows you to both download and upload files from FTP and Secure FTP (SFTP) locations into/from QlikView. 
  • We have a new connector to the Google Prediction API which lets you 'train' your own categorization or regression prediction models based on your own training data set. The Prediction API uses patterns it finds in your training data to either find the closest category for the submitted example (if this is a categorical model) or estimate a value for the example (if this is a regression model) and returns the category or value. One potential use of this QlikView connector is to create your own sentiment API (we give an example here) but Google also give a few other examples on their website:
    • Document and Email Classification
    • Recommendation Systems
    • Spam Detection
    • Language Detection
    • Upsell Opportunity Analysis
    • Diagnostics
    • Suspicious Activity Detection
    • Churn Analysis 
  • We have added a new connector to HEIDI (the Higher Education Information Database for Institutions) for QlikView which should be of interest to a number of UK universities and other higher education institutions.

 Connector Updates

  • Version 3 of the Google Analytics Connector for QlikView has been added. This new version has all the features of the previous version, plus new support for real-time data and Multi-Channel Funnel Reporting! The other main change is that we have upgraded it to the new UI and workflow style (as we plan to do to all connectors in due course). The previous version (V2) of this connector should now be considered deprecated and will be removed in March 2014 giving existing users several months to make the (simple) upgrade.
  • Similarly, we have also upgraded the Google AdWords and General Web (JSON/XML/SOAP) Connectors for QlikView to use the new UI and workflow and the previous versions should be considered deprecated.
  • The General Web (JSON/XML/SOAP) Connector now supports:
    • Saving the response to a local file. This has been found useful for example, for downloading image files which can then be loaded into the actual QlikView application for usage offline and also better performance.
    • Loading local files - this is useful for example if you have a file with JSON formatted data and wish to load it into QlikView.
  • We have improved and moved the Google Spreadsheet Connector functionality into the Google Drive Connector for QlikView where we feel it makes more logical sense. The dedicated Google Spreadsheet Connector should now be considered deprecated and will be removed in March 2014 giving existing users several months to make the (simple) upgrade. Note that users who have purchased a commercial licence for the Google Spreadsheet Connector will be automatically transferred over to the Google Drive Connector.
    • We have also added a new DownloadRawFile table to the Google Drive Connector.
  • We have added a number of new tables to the LinkedIn Connector for accessing a person's companies and company statistics. The Skills table has also been renamed MySkills and only returns skills for the authenticated user.
    • Note that we are aware of a number of issues with this connector which appear to be due to changes in the underlying API. We are currently considering working on a new V2 of this Connector.
  • We have fixed a bug in the Facebook Fan Pages Connector for QlikView which caused the has_likes and has_comments columns to be swapped in some cases.
  • Twitter Connector for QlikView
    • BREAKING CHANGE: We have also made a very minor breaking change in the Twitter Connector for QlikView by removing the cursor column from the FollowerIds table. We are investigating a new way to properly support paging through some of the Twitter tables and currently you will see a new 'Post_FollowerIds_Info (Beta)' table and you will find some notes here regarding this (although this still isn't working how we would like).
    • We have also added a new Favorites table to this Connector and fixed an issue with the CustomRequest table.
  • Google DoubleClick For Publishers (DFP) Connector for QlikView
    • This is now out of beta.
    • It will also now wait up to 20 minutes for the report generation to complete and also has a few more progress update messages (when running in the UI) so you can get a better idea of how long each phase of the report request is taking.
    • We have also upgraded it to use the very latest DFP API (201311).
  • We have fixed a bug in the CampaignEmailStats table of the MailChimp Connector (it looks like a minor change in the API response started causing an issue in the connector) and the email address parameter should now be correctly URL encoded.
  • We have fixed a bug with the monthChange column showing true/false instead of numeric value in the Klout Connector and also added a new column for the score bucket.
  • We have upgraded the Google Plus Connector for QlikView to use the new UI layout and also made a couple of improvements to the Connector.
  • The MailBox Connector has been upgraded to use the new UI style and also has new tables to list and download email attachments. 
  • The Repustate option in the Text Analytics & Sentiment Analysis Connector for QlikView now supports Chinese and sentiment by topic.
  • The Google AdSense, Google DFPGoogle CalendarGoogle AdWords, Google DriveGeneral JSON/XML/SOAP and OData Connectors are now out of beta. If you have been using any of these connectors and wish to continue to do so when upgrading QVSource please contact us or your QVSource partner for licensing details.

Removed Connectors

Other Updates

  • You will probably notice we have a new product logo on our website, wiki, blog and other online locations. This is also now used in the product.
  • We have moved most of our demo QlikView applications over to our site on GitHub. Using GitHub makes it much easier to track changes that are made between versions. Note that you do not need to know how to use GitHub - if you want to download an application just look for the 'Download ZIP' button.

NOTE: You can access this latest version from the link you received when signing up for QVSource.

We really hope you are enjoying the updates!

If you like using QVSource, please take 30 seconds to help us by rating us on QlikMarket.

New Release Of QVSource (1.4.7) - The QlikView API Connector - Now Available

Today we've released a new version of QVSource, which has a large number of exciting enhancements, updates and fixes across a number of the connectors.

Of particular importance are a number or changes to the Facebook Fan Pages & Groups Connector - some of them breaking changes. So if you are using this connector please take note and upgrade and test your application scripts appropriately.

In a follow up to the last post, some more connectors have also been removed, but we've also added some new ones!

A New Way to Contact us for Support

If you've read any reviews about us (like the 36 5-star reviews on QlikMarket), you'll quickly see that we take supporting you very seriously and strive to give the very best service. To help us improve our customer service even further we've started to use Zendesk, (if, like us, you want to get Zendesk data into QlikView with QVSource, see below!) So if you do need to get in touch for support, if having read our documentation you're still stuck, please contact us at support[at]qvsource.com or qvsource.zendesk.com. That way you can be confident the right person will respond to you as quickly as possible.

Updates to Connectors

  • Sentiment & Text Analytics V2 Connector
    • Chatterbox has now been added to the connector.
    • The Repustate option now supports Italian.
  • General Web Connector
    • A comma separated list of http Headers can now be specified for requests as well as the accept type.
    • We often get asked if our General Web Connector can work with certain APIs, which in many cases it can. So, we've put together a few examples on how to use this connector with ZendeskRome2Rio (an API that gives routing between 2 points), desk.com and how to use it with a SOAP web service
  • Mashape Connector
    • The authentication process for Mashape has changed and the connector has been updated to reflect this.
  • Facebook Pages & Groups Connector
    • BREAKING CHANGE: Due to API changes, the FeedItems and Posts tables can no longer show the like count. We have added a new table LikeCountForItem table to retrieve this. If you are using the Facebook Pages Application, then you will now see zero for the count. An updated application with the new table will be released soon. If you are using this connector we would strongly recommend you test your QlikView applications against this new version and adjust your load scripts as necessary before deploying to production.
    • A new Albums table has been added.
    • A new PromotablePosts table has been added. This is similar to Posts except it includes 'the Page's own posts, including unpublished and scheduled posts' and has an additional is_published column.
    • The cache is now cleared when the user re-authenticates.
    • We have tidied up some labels and other minor issues.
  • Twitter Connector
    • We have fixed the info labels on UserLookup table.
    • A new FollowersId table has been added.
    • There is now better handling when the quota limit is reached on the Follower and Following tables.
  • Mailbox Connector
    • Fixed issue where exceptions requesting table would sometimes crash QVSource due to invalid XML characters in the stack trace.
    • Better checking of nullable values when downloading Imap mails (this could have caused an incomplete download in some instances).
    • The Imap download should no longer fail completely if a single email fails.
    • For Imap, it now handles case where To field is a MailGroup and shows the group name.
  • iTunes Connector
    • BREAKING CHANGES: There have been a number of updates and improvements to this connector which will mean any load scripts using the connector will need to be re-generated. Please see the change log for details.
  • MongoDB Connector
    • Added a new input for Database name (this can also take a comma separated list). This is useful in scenarios where the user does not have permission to enumerate the Database names.
  • General updates

Connectors Removed

  • The Solve360, Freshbooks and FreeAgent Connectors have now been removed, which as far as we are aware were not being used and, as we mentioned in our last blog post, we want to focus our efforts on connectors which are more popular. As before, if you are using these and feel we have made a mistake do get in touch and let us know.

Starter Applications & Examples

In a recent Facebook update comments can have replies against them, which was quickly added to the Facebook Page & Groups Connector. We have now incorporated this into our Facebook Pages Starter QlikView Application. There have also been some other breaking changes to the Facebook API (see above) which we are working on incorporating into the demo application.

The Google Analytics Starter Application has also had a few minor fixes.

Moving Starter/Demo QlikView Applications to GitHub

We have also thought about how we can improve the delivery and maintenance of these Starter/Demo applications to you as it's not always clear what changes we make, so we are starting to trial GitHub as a place to do this. For most users you won't really notice much change in how you get the starter applications, but for those who want to understand the changes and improvements we make, it'll become much more transparent, which we think is a good thing.

You can read a wiki page with more information on how this works here.

New YouTube Analytics Connector

In addition to the YouTube Data Connector where you can get largely public-based data on videos, we now have a YouTube Analytics Connector for those of you that need a much greater insight into your videos.

Try QVSource

If you are a customer or trialling QVSource and would like to get these enhancements as well as some other general bug fixes and performance improvements, simply go to the download link that you would have received in your QVSource download email. You can find upgrade instructions here.

If you are new to QVSource but are a QlikView user interested in getting data from Web APIs into your applications, why not get your free trial of QVSource? It's quick to do and in a few minutes you can get your data into QlikView with one of our starter applications.

If you've already trialled QVSource and would like an extension, please get in touch via support[at]qvsource.com (attaching your current licence file) and we'll happily extend it for you.

New QVSource Release (1.4.4) Available

It's been six weeks since we released 1.4.3.0 and as usual we've been working hard to make QVSource even better to give QlikView users even more power and flexibility in connecting to Web APIs and other data sources.

We are pleased to announce that a new version is now available with quite a few interesting enhancements and also a couple of fixes which may have affected some users:

  • We have fixed a couple of important stability issues:
    • Some users were finding that the connectors were not loaded/showing until the 'Include Beta' checkbox had been toggled at least once.
    • The main settings file was occasionally getting reset and losing, for example, the fact that the EULA had been accepted. This was causing issues especially for server based deployments.
  • We have added a new ListFileRevisions table to the Google Drive Connector.
  • The MongoDB Connector has had some massive performance boosts and can now be queried using JSON.
  • The Twitter Connector Search is no longer limited to a maximum of 15 pages of result (i.e. 15 x 100 = 1500 Tweets).
  • Three new tables (GroupMembers, Files and Events) have been added to the Facebook Pages & Groups Connector. We will be working on support for the new comment replies soon.
  • We have updated the DoubleClick for Publishers (DFP) Connector to use the new API (v201302).
  • BREAKING CHANGE: The Accounts table has been removed from the Facebook Insights connector (this is dues to removal of this feature from the API itself).
  • A new 'Include Depreciated Connectors' checkbox has been added and deprecated connectors are no longer shown by default.

If you are a customer or trialling QVSource and would like to get these enhancements as well as some other general bug fixes and performance improvements, simply go to the download link that you would have received in your QVSource download email.

Or, if you've never tried QVSource and would like to, then you can get a free trial by completing this form.

If you like what we do with QVSource then we'd really appreciate it if you could take a few moments to either rate us on QlikMarket or recommend us on our LinkedIn page. If you've already done this, then thank you!

Don't forget you can always keep up to date with what we're doing by following us on Twitter, Facebook and Google+.

Major New Official QVSource Release (1.4.2) Now Available for QlikView

We just published our latest official QVSource release (1.4.2) and it was a big one - in addition to the usual updates and improvements to the connectors we have also added a number of stability, performance and logging enhancements which should be of interest to enterprise users of QlikView.

If you are currently evaluating QVSource and especially if you are a commercial user we would strongly recommend you read this post as it almost certainly contains information which will affect you.

Much of this information was also discussed in our recent blog posts here, here, here and here, but there are also some new updates specific to this latest build.

Last Beta Version Of Certain Connectors

We are planning to take the following Connectors out of beta in the next official build of QVSource (probably in about 6 - 8 weeks time). This is particularly important to commercial users of QVSource who might currently be using these as when they are taken out of beta you will need a commercial licence to continue to use them:

Deprecated Connectors

This will be the last official build of QVSource which contains V1 of the Google Spreadsheet and Google Analytics connectors. Both of these have now been superseded by much improved 'V2' versions which we would recommend you migrate to.

Synchronous, Asynchronous and Batch Request Modes Added

This is a significant new capability in QVSource which was explained in detail in the previous blog post and so will not be covered again here. This update has the potential to give major performance improvements and also change and improve the way QVSource related QlikView script is constructed.

One thing to note is that we have now removed the previous mechanism for running asynchronous requests as the new functionality improves and supersedes this.

QVSource Windows Service

QVSource now has a dedicated Windows Service available and documented here. We have always had a well documented and stable way of configuring QVSource to run on a server environment however the new Windows Service represents an improvement over this technique and in time will likely become the recommended and only supported method.

Related to this, we have also now made it so if QVSource is running in server mode you can run QVSource desktop in order to manage connector and other settings without having to then restart the QVSource server instance for changes to be picked up.

First Version Of New Twitter Connector (V2)

This build has the first version of the new Twitter Connector V2 in which also illustrates the new UI style we are working towards as discussed here. Unfortunately Twitter will start shutting down version 1.0 of their API in early march so all users should start upgrading to this new version ASAP.

First Version of New Sentiment Analysis and Text Analytics Connector (V2)

This build has the first version of the new Sentiment Analysis and Text Analytics Connector in. This also illustrates the new UI style we are working towards as discussed here. We would also recommend users start working with this new version and reporting any issues as it will soon replace V1 of this Connector.

Checking the Status of QVSource

It has always been possible to browse to http://localhost:5555/QVSource to get a html page with various pieces of information about QVSource (e.g. the licence state, version, which Connectors were running). We have now tidied this up and added additional information. More importantly, we have added a new XML option at http://localhost:5555/QVSource?format=xml which allows you to check the exact QVSource status from your QlikView load script as explained here.

Log File & Error Page Updates

QVSource now has a much more structured XML file format with additional information added. As before, logging needs to be enabled for these to be generated as explained here. It should now be possible to load these files into QlikView directly for structured reporting.

Related to this, the HTML error page which QVSource sends back has also been enhanced and tidied.

appID Load Script Parameter

QVSource now adds an appID= parameter to the generated QlikView load script. We recommend setting a variable at the start of your load script, e.g.:

let vQVSourceAppId = 'my_app_identifier';

And then adding this variable to all of your QVSource requests, e.g:

....&appID=$(vQVSourceAppId)&....

This provides an identifier to QVSource to be used in error logging and tracing. This is explained in more detail here.

Note that in a recent 'nightly build' we actually generated the full appID=$(vQVSourceAppId) in the QVSource generated load script. Unfortunately it was pointed out that this actually breaks the QlikView 'Web File' Wizard (used for example in the General XML/JSON Connector and certain tables in other Connectors) - so now we just include an empty appID= parameter in the URL leaving it to the user to insert a value or variable in the final QlikView script as recommended above. This isn't mandatory or required and it can be left blank.

Other Connector Changes & Updates

  • The Google AdSense Connector for QlikView has been upgraded to use the latest API (version 1.2).
  • The Google Analytics Connector For QlikView has had a number of minor tweaks and fixes. These are only likely to affect you if you were using the MaxNumberOfPages parameter (i.e. not bringing back all the data for a particular query url).
  • Empty \Temp folder has been added to the distribution - this is currently the recommended location to store the parameters CSV files used by the new processParams[Sync|Async|Batch] feature.
  • The General JSON/XML API Connector now has a better preview of XML and HTML and also a new checkbox to encrypt the request URL before including in your QlikView script to help secure any sensitive credentials it might contain.
  • The Facebook Fan Pages and Insights Connectors now have buttons to 'Deauthorise' the Connectors (i.e. detach them from your Facebook account).
  • The Facebook Fan Pages Connector has a new QuestionVotes table listing the users who selected each option.
  • The Klout Connector has been tweaked to work more efficiently with the new new async feature. Note that the Klout V1 Connector for QlikView has been removed from QVSource.
  • The FreeAgent Connector has two new tables related to the Trial Balance. We have also removed V1 of this connector and recommend upgrading to the latest V2 version.
  • The first version of the Google DoubleClick For Publishers (DFP) Connector is now included. We hope to add more functionality to this soon. Please get in touch if you are interested.
  • The Semiocast Connector for QlikView now supports new languages and has some performance improvements.
  • We have upgraded the MongoDB Connector for QlikView to use the latest c# driver and also fixed a couple of minor issues.

All these recent features further enhance the speed and ease at which you can load your QlikView applications with data from a large number of web APIs.

We hope you like the updates and we are looking forward to continuing to innovate and improve QVSource - The QlikView API Connector in 2013!

Connecting QlikView to the Zeit Online API Using QVSource

Zeit Online, the online version of one of Germany's leading newspapers, has released a beta version of an API to their content and I couldn't resist a quick play and blog post showing how easy it is to connect QlikView to the API using QVSource.

From their own developer website:

Via the API you can access articles and corresponding metadata from the ZEIT archive dating back to 1946 and from ZEIT ONLINE. We invite everybody interested in programming, data analysis and visualization to explore the index of hundreds of thousands articles. The API can be used for free for non-commercial use cases. Please refer to the terms of use (German only at the moment).

An API key can be generated in seconds - they simply ask for your name and email address and once generated their online API explorer makes it quick and easy to build and test queries.

I started with a simple query by selecting content from the drop down of options in the API explorer (click on the word after GET/ to see the drop down) and entering merkel for the search query as highlighted below. Clicking the Send request button then quickly returned back some results in json.

A little bit of research shows that only the parameters I have filled out get passed to the API - so the actual request is:
http://api.zeit.de/content?q=merkel

And scrolling to the bottom of the results reveal there are actually 5376 results in total but only 10 returned:

To get all the results, we simply update the limit parameter to something which will capture everything:

 

The request to the API has now become:
http://api.zeit.de/content?q=merkel&limit=6000

We can now use the QVSource Web Connector to connect QlikView to this API. There is just one more thing we should be aware of which is that the API requires that the API key is passed either as an X-Authorization header or via a api_key parameter. The former is preferred by the API but the second is supported and makes our job even easier - so our API request now becomes:

http://api.zeit.de/content?q=merkel&limit=6000&api_key=[INSERT YOUR API KEY HERE]

And you can even test this in a browser.

We now open up the Web Connector in QVSource and enter this url as a GET request:

And on the Data tab we can now check the RawResponse table to check the data is coming back as expected.

There are two ways we can get this data into QlikView now. The first is to use the JsonToXmlRaw table - this uses QVSource to convert the JSON to XML and then provides a new URL for the request which can be used in QlikView as a Web File data source. This is shown below:

We can simply copy the URL highlighted above and use it in the QlikView load script Web File dialog as shown below:

A more elegant solution is to use the QVSource JsonToTable table. To use this we must first do a quick simple analysis of the JsonToXmlRaw request, to do this we can take the URL generated above and browse to it in Internet Explorer where we get a nicely formatted XML response:

From this is should be quite clear that each row in the table we want is an instance of the matches element, and the XPATH to select these elements is then DATA/matches. To use this we need to enter it into the config screen of the connector as shown:

Now we have done this we can use the JsonToTable table in QVSource and get a nicely formatted table:

And the auto generated script to load this into QlikView as a QVX file:

We can now easily load this into QlikView and start analysing:

(Note that I actually changed the search term to 'angela merkel' for this final chart).

This is really just the start of what could be done with QlikView + QVSource. For example, QVSource also has a powerful Text Analytics and Sentiment Analysis Connector which could be used to score the article snippets and headlines for sentiment, or to extract entities such as categories and concepts.

And of course QVSource has connectors to Facebook, Twitter, YouTube and a ton of other sources which would allow you to bring in and mash up data from all sorts from other places.

Zeit are inviting users to show what can be done with their API - the their words:

We invite you to share your apps, experiments and experiences with the API. We will highlight interesting projects in the App Gallery section. Wir laden Sie ein, uns Ihre Umsetzungen, Experimente und Erfahrungen mit der API mitzuteilen.

So it would be great to see what the QlikView community could do with the API. Don't forget you can register for a free fully functional evaluation edition of QVSource here.

Page List