Scout Analytics is not the first to monitor account usage and flag unlicensed use. There are commercial and homegrown account analysis tools out there. Most of these tools try to draw correlations between time (concurrency), IP address, and browser information. A recent set of questions from customers were:
- How good can these tools be?
- What is the difference in performance from Scout Analytics?
Investigation
Using real world data, we looked at different techniques to analyze the IP addresses such as: total count, average interval, frequency, smallest interval, and others. All the techniques computed an IP-related metric for an account (such as the total count of distinct IP addresses) and compared that measure to a threshold. Next we compared the percentile of shared accounts above the threshold to the percentile of non-shared accounts above the threshold. This gives us an idea of how effective a measure is.
The best results for detecting unlicensed use from IP address was measuring the smallest interval between logins originating from different IP addresses. Unfortunately, the technique has a high false positive rate associated with it. To detect 25% of the accounts with unlicensed use, the technique would have a false positive rate of 10%! Dropping the false positive rate to 1% would only allow 4% of the violators to be identified. Overall the performance of IP address techniques is poor.
Thoughts
Why do IP address related measures perform so poorly? IP address counts have a big range in values for a single-user account (e.g., mobile workforce). Some accounts used a single address, others used over 80! This range overshadows the vast majority of differences that might be caused by having multiple users in an account vs. only having a single user.
About Scout by ServiceSource
Contact
