API Operations

using 3scale's Account and Analytics APIs

This article showcases how you can create scripts that extend the current capabilities of the built-in 3scale Analytics.

By using the Account Management and Analytics API you can create scripts that get the custom information that you need on the format that you prefer. In the article we describe a particular use-case but the same approach can be used to get whatever data you need out of 3scale.

3scale continously improves the features available on your API Admin Portal. But many times you might be ahead of our developement plans and have a very specific need that it's not yet supported.

This could be very frustrating if there was no way around it, but fortunately there is since 3scale always gives your the tools to access all your data. The mantra would be: what 3scale does not do [yet], you can always do it yourself :-)

Granted that DIY has some costs, some resources have to be put in place to write the scripts. But the costs is not that high as we will show in this article, and what's more important, DIY gives you total flexibility and control to implement any use case you can think of.

Recently one customer came to us with a very specific need that could not be solved with 3scale in a streamlined way.

They are onboarding thousands of new developers per week, which is fantastic news! They are able to survive to such a success thanks to 3scale's API Management platform. On-boarding so many developers is a daunting task unless you rely on a API management solution such as 3scale where key provisioning, sign-up workflows, email communication, etc. is all automated. So far so good.

There was, however, something that was not possible to do with 3scale and it was quite important for them. Since they are onboarding so many people, they needed a straight-forward way to classify the new developers based on their engagement with their API so that their operations and marketing teams can interact with the new developers more effectively.

Such a feature, at least on the level of detail, is not yet available in 3scale's built-in Analytics tools. But the all the data, therefore, it can be extracted by using 3scale's Account and Analytics API.

They would like to know how many new developers have signed up for the Free Evaluation Plan in the last 10 days splitted if different ways.

First they would like to know how many developers signed-up and did not ever used their API. What they want to do with this information is not the scope of the article, but it's clear that this is valuable information that can help adoption of your API.

Second they would like to split the developers that have had used their API in two groups:

  • developers that used for a period of time, let's say the first half of the 10 days, and then stopped using the API. These developers tried it out, but they have become inactive.
  • developers that have been using the API consistently, for those, they would like to know the percentual growth (or decline).

This information is actually available on 3scale's built-in analytics, the problem is that there is no view to show it aggregated, which makes the whole experience quite cumbersome.

A typical answer to this problem would be: "This classification will be part of the upcoming Analytics revamp". Funny thing is that's actually the case. But that does not solve your problem, you need it now. We want to give you the tools to do all what you need to do without depending on our release schedule.

We can share the recipe that usually gives better results into achieving custom work.

  • Tinker a bit with 3scale's ActiveDocs. 3scale has all its API available as ActiveDocs so that you can try them out from your browser. This allows you to find out the request that best serves your needs, get the request (curl-like) and get a grasp of the response. See an example...

    This is the ActiveDocs for the API request to fetch all applications that will be used on the script to extend the analytics.

  • Once you have done the research with the ActiveDocs. Put down the request on your scripting language of choice (ours is pedestrian ruby). Thanks to the request and responses provided by ActiveDocs it has never been easier to explore what the API does.
  • Repeat until you have a script that does exactly what you need. For the example of the extended analytics the script is available as a gist, you can try it out in your account.

As pedestrian as it sounds this is the easiest way to proceed. The ActiveDocs let you quickly get a grasp on what the API is able to do. Then, it's just a matter of finding which 3 or 4 requests are needed for the task you want to do and put a script together.

To achieve the custom analytics that our customer wanted those were the steps taken:

  • Retrieve the full list of applications. This operations requires pagination, but as you can see it's dead simple.

  • Filter the applications that do not meet the criteria, i.e. the plan must be Evaluation and they have to be newer than 10 days.

  • Then for each application that meets the criteria get its usage, meaning how many hits the application has had in the last 10 days.

  • Finally you must cross reference the applications to the accounts because the information of the developers is stored in the account object.

And that's pretty much it. Now you only need to put everything together (you can get the full script as a gist) and you are done. You have a script that gets that information that was not yet available on 3scale's built-in analytics.

3scale is very DIY friendly. We believe that anyone should be able to extend the current capabilities of any service that he has contracted. As much as we would like to think that we can cover all the bases we will always be a bit behind in certain aspects. Whether because it's a super special need or simply because we are busy building other features. Whatever the case, when something is not possible right way we do not want to block you. We give you full access to your data and a complete API toolset to work with it.

If you have scripts that you want to share with us and the rest of 3scale's users please send them to us. We would be happy to collect them and publish them.

Each application using the API has a traffic trace in the 3scale system and this can be viewed from the management console as well as recovered by API.

This is part 3 of a 3 part series on application control. See the other sections:

  1. Finding an Application
  2. Suspending an Application
  3. This How to.

At a end of this how to will be able find information on an application including the current traffic the application is generating.

You can find the application from the Accounts or Applications tabs or by searching as described in the appropriate How to.

Once you have located the application you’ll see an overview screen of information on the application as shown in the following image.

The Items labelled in the diagram correspond to the following information:

  1. The name of the application given by the developer.
  2. Meta data captured for the application (learn how to set which data to capture in the advanced section).
  3. The Status of the application – is it Live or Suspended?
  4. The current API Identifiers, Keys and certificates which the application has (this view varies depending on what authentication pattern is in use.
  5. A summary of traffic statistics for the application.
  6. Information on the application plan the application is on and the rate limits which apply.

Using the menu labelled item 7 in the image above click on Stats and this will take you to the statistics view for the application as shown in the image below.

Controlling the Metrics, Methods and time range allow you to check different types of data about the application.

If during plugin configuration, logging to 3scale of calls was enabled then clicking on the *API Request Log" Menu item shows the latest logged transactions for the application. These may be all transactions or a subset (such as errors) depending on configuration.

During API operations you may need to communicate urgently with the developers using the API.

Once you’ve gone through this HowTo you’ll know how to find the developer account which manages a particular application and communicate with them – both via 3scale and externally.

If you already know the Account and Developer who manages the Application in question you can navigate to their account from the Accounts tab as shown below:

Accounts View

If you have only the application ID or API key, you can use the search box on the Accounts tab to find the relevant account account. More information on locating applications is provided here.

Once you are on the account profile page as shown below, click on the message Icon.

Send Message Button

The message created here will be sent both to the account system dashboard where all developers on the account will see it and also by email the persons on the buyer account which have admin status within the account.

If email is unlikely to be fast enough for your purposes (in case of emergency for example), then you can also use the contact information submitted by the developer at signup time which is available:

  • On the company account page (generally general contact information but may include a phone number.
  • Developer/User specific information on the users’ own file.

Note that if you desire you can make contact phone numbers a required field upon signup.

During API operations you may often need to quickly find information on an application that is accessing the API – either for support purposes, to change configuration or potentially because the application is misbehaving and needs to be disabled.

This is part 1 of a 3 part series on application control. See the other sections:

  1. This how to
  2. Suspending An Application
  3. Checking Traffic for an Application

At the end of this HowTo will be able to quickly locate an application in the dashboard based on either it’s name, an API key or an Application identifier.

To find an application you need to know something about it. This may be the name of the account it belongs to or the application's name in which case locating it is straightforward from the Accounts or Applications tab in the Dashboard. However, often you have no information other than the Application identifier or it’s API Key (e.g. if you are seeing this information in your own access logs).

If you are searching by identifier then for the different authentication types you need the following information:

  • For API Key only authentication patterns: the API Key.
  • For App Id + App Key authentication patterns: the App Identifier (search on the App Key is not supported).
  • For oAuth authentication patterns: the Client_iD (search on the secret is not supported).

Armed with whatever data you have, head to the Accounts area of the dashboard and use the search box on the page (shown in the image below).

Once the results are returned, click on the application you’d like to access and you’ll be taken to that application’s homepage which shows information such as that shown in the image below.

If an application is misusing the API and affecting other traffic you may need to quickly suspend it’s operations before contacting the developer involved to ask them to amend their code or configuration

This is part 2 of a 3 part series on application control. See the other sections:

  1. Finding an Application
  2. This How to.
  3. Checking Traffic for an Application

With this HowTo you’ll know how to disable all keys and access tokens for an application.

You can find the application from the Accounts or Applications tabs or by searching as described in the appropriate How to.

Once you have located the application and see the application summary page. Click the suspend Icon below the name of the application. This action will immediately disable the application from the API and suspend all keys from working. Calls with these application keys will be rejected by the control system (*).

The application can be unsuspended using the same button once the problematic behaviour has been rectified.

  • Note that if you use caching in your Agents suspension may not be immediate but require a short time out.

How you contact the developer of the application will depend on your workflow and policy. One option for doing this from the same page is by clicking on the Account name underneath the submenu.

This takes you to the account view, from here you can identify the key administrator of the account which owns the application and can contact them either by email or by clicking on the send message button as shown – this will generate a dashboard message for the user concerned.

The use of Webhooks allows you to tightly integrate 3scale with your back office workflow. When specified events happen within the 3scale system your applications will be notified with a webhook message, and you can use the data e.g. from a new account signup to populate your CRM system.

By the end of this section you will be able to configure and take action on the webhooks for your portal.

A Webhook is a custom HTTP callback triggered by an event. In the 3scale system all the possible events are displayed in the screenshot below:

Howtos Webhooks Overview

When one of these events occurs, the 3scale system makes an HTTP (or HTTPS) request to the URI configured in the Webhooks section. On your end you can configure the listener to invoke some desired behavior like event tracking, etc.

The remaining two checkboxes on the screenshot turn on webhooks ('Web hooks enabled? box') and allow webhooks to also be fired by actions in the Admin console. The default (unchecked) behavior is to trigger webhooks only from within the Developer portal (bear in mind that this means not all events can be triggered).

The format of the webhook is always the same. It makes a post to the endpoint with an XML document of the following structure:

The <type> gives you the subject of the event i.e. "application", "account", etc. The <action> - what has been done, i.e. "updated", "created", "deleted". Finally the <object> is the XML object itself in the same format that is returned by the Account Management API. To check the this you can use our interactive ActiveDocs.

If you need to provide assurance the webhook was fired by 3scale, this can be done as follows: Expose a https webhook url and add a custom parameter to your webhook declaration in 3scale, e.g. https://your-webhook-endpoint?someSecretParameterName=someSecretParameterValue. You decide on the parameter name and value. Then inside your webhook endpoint check for the presence of this parameter value.

If you want to experiment with the webhooks, or troubleshoot issues then you may find RequestBin a great (and free) service to view the result of the webhooks: http://requestb.in/

If you experience an outage for your listening endpoint, you can recover failed deliveries. 3scale will consider a webhook delivered if your endpoint responds with a 200 code. Otherwise it will retry 5 times with a 60 seconds gap. Therefore after any recovery from an outage, or periodically, you should run a check, and if applicable clean up the queue. You can find more in the ActiveDocs for the two methods:

  • Webhooks List Failed Deliveries
  • Webhooks Delete Failed Deliveries