SDK for the Brandwatch API: Demo

Introduction

The goal of this notebook is to demonstrate the capabilities of the Python Software Development Kit for Brandwatch's API. The SDK was designed to address many of the challenges involved in building complex applications which interact with RESTful API's in general and Brandwatch's API in particular:

• The SDK's object hierarchy roughly mirrors the API's resource hierarchy, making the code intuitive for those familiar with the Brandwatch platform
• All required parameters are enforced, and most optional parameters are supported and documented
• Typical Brandwatch workflows are supported behind the scenes; for instance, one can validate, upload, and backfill a query with a single function call
• The SDK is designed to support simple and readable code: sensible defaults are chosen for rarely used parameters and all resource IDs are handled behind the scenes

From the user's perspective, the basic structure of the SDK is as follows. One first creates an instance of the class BWProject; this class handles authentication (via a user name and password or API key) and keeps track of project-level data such as the project's ID. (Behind the scenes, the user-level operations are handled by the class BWUser from which BWProject is inherited.) One passes BWProject instance as an argument in the constructor for a series of classes which manage the various Brandwatch resources: queries, groups, tags, categories, etc. These resource classes manage all resource-level operations: for example a single BWQueries instance handles all HTTP requests associated with queries in its attached project.

Typically, you'd import only the classes you plan on using, but for this demo all classes are listed except for superclasses which you do not use explicitly)

The SDK uses the Python logging module to tell you what it's doing; if desired you can control what sort of output you see by uncommenting one of the lines below:

Project

When you use the API for the first time you have to authenticate with Brandwatch. This will get you an access token. The access token is stored in a credentials file (tokens.txt in this example). Once you've authenticated your access token will be read from that file so you won't need to enter your password again.

You can authenticate from command line using the provided console script bwapi-authenticate:

$ bwapi-authenticate
Please enter your Brandwatch credentials below
Username: example@example
Password:
Authenticating user: example@example
Writing access token for user: example@example
Writing access token for user: example@example
Success! Access token: 00000000-0000-0000-0000-000000000000

Alternatively, you can authenticate directly:

Now you have authenticated you can load your project:

Before we really begin, please note that you can get documentation for any class or function by viewing the help documentation

Queries

Now we create some objects which can manipulate queries and groups in our project:

Let's check what queries already exist in the account

We can also upload queries directly via the API by handing the "name", "searchTerms" and "backfillDate" to the upload funcion. If you don't pass a backfillDate, then the query will not backfill.

The BWQueries class inserts default values for the "languages", "type", "industry", and "samplePercent" parameters, but we can override the defaults by including them as keyword arguments if we want.

Upload accepts two boolean keyword arguments - "create_only" and "modify_only" (both defaulting to False) - which specifies what API verbs the function is allowed to use; for instance, if we set "create_only" to True then the function will post a new query if it can and otherwise it will do nothing. Note: this is true of all upload functions in this package.

If you're uploading many queries at a time, you can upload in batches. This saves API calls and allows you to just pass in a list rather than iterating over the upload function.

Channels will be shown as queries and can be deleted as queries, but must be uploaded differently. You must be authenticated in the app to upload channels.

In order to upload a channel you must pass in the name of the channel, the handle you'd like to track and the type of channel. As with keyword queries, we can upload channels individually or in batches.

Note: Currently we can only support uploading Twitter channels through the API.

We can delete queries one at a time, or in batches.

Groups

You'll notice that a lot of the things that were true for queries are also true for groups. Many of the functions are nearly identical with any adaptations necessary handled behind the scenes for ease of use.

Again (as with queries), we need to create an object with which we can manipulate groups within the account

And can check for exisiting groups in the same way as before.

Now let's check which queries are in each group in the account

We can easily create a group with any preexisting queries.

(Recall that upload accepts two boolean keyword arguments - "create_only" and "modify_only" (both defaulting to False) - which specifies what API verbs the function is allowed to use; for instance, if we set "create_only" to True then the function will post a new query if it can and otherwise it will do nothing.)

Or upload new queries and create a group with them, all in one call

We can either delete just the group, or delete the group and the queries at the same time.

Downloading Mentions (From a Query or a Group)

You can download mentions from a Query or from a Group (the code does not yet support Channels)

There is a function get_mentions() in the classes BWQueries and in BWGroups. They are used the same way.

Be careful with time zones, as they affect the date range and alter the results. If you're using the same date range for all your operations, I reccomend setting some variables at the start with dates and time zones.

Here, today is set to the current day, and start is set to 30 days ago. Each number is offset by one to make it accurate.

To use get_mentions(), the minimum parameters needed are name (query name in this case, or group name if downloading mentions from a group), startDate, and endDate

There are over a hundred filters you can use to only download the mentions that qualify. see the full list in the file filters.py

Here, different filters are used, which take different data types. filters.py details which data type is used with each filter. Some filters, like sentiment and xprofession below, have a limited number of settings to choose from.

You can filter many things by inclusion or exclusion. The x in xprofession stands for exclusion, for example.

To filter by tags, pass in a list of strings where each string is a tag name.

You can filter by categories in two differnt ways: on a subcategory level or a parent category level. To filter on a subcategory level, use the category keyword and pass in a dictionary, where each the keys are the parent categories and the values are lists of the subcategories. To filter on a parent category level, use the parentCategory keyword and pass in a list of parent category names.

Note: In the following call the parentCategory filter is redundant, but executed for illustrative purposes.

Categories

Instantiate a BWCategories object by passing in your project as a parameter, which loads all of the categories in your project.

Print out ids to see which categories are currently in your project.

Upload categories individually with upload(), or in bulk with upload_all(). If you are uploading many categories, it is more efficient to use upload_all().

For upload(), pass in name and children. name is the string which represents the parent category, and children is a list of dictionaries where each dictionary is a child category- its key is "name" and its value is the name of the child category.

By default, a category will allow multiple subcategories to be applies, so the keyword argument "multiple" is set to True. You can manually set it to False by passing in multipe=False as another parameter when uploading a category.

For upload_all(), pass in a list of dictionaries, where each dictionary corrosponds to one category, and contains the parameters described above.

Let's upload a category and then check what's in the category.

Now let's upload a few categories and then check what parent categories are in the system

To add children/subcategories, call upload() and pass in the parent category name and a list of the new subcategories to add.

If you'd like to instead overwrite the existing subcategories with new subcategories, call upload() and pass in the parameter overwrite_children = True.

To rename a category, call rename(), with parameters name and new_name.

You can delete categories either individually with delete(), or in bulk with delete_all().

You also have the option to delete the entire parent category or just some of the subcategories.

To delete ALL CATEGORIES in a project, call clear_all_in_project with no parameters. Be careful with this one, and do not use unless you want to delete all categories in the current project.

First let's delete just some subcategories.

Tags

Instantiate a BWTags object by passing in your project as a parameter, which loads all of the tags in your project.

Print out ids to see which tags are currently in your project.

There are two ways to upload tags: individually and in bulk. When uploading many tags, it is more efficient to use upload_all.

In upload, pass in the name of the tag.

In upload_all, pass in a list of dictionaries, where each dictionary contains "name" as the key and the tag name as the its value

To change the name of a tag, but mantain its id, upload it with keyword arguments name and new_name.

As with categories, there are three ways of deleting tags.

Delete one tag by calling delete and passing in a string, the name of the tag to delete

Delete multiple tags by calling delete_all and passing in a list of strings, where each string is a name of a tag to delete

To delete ALL TAGS in a project, call clear_all_in_project with no parameters. Be careful with this one, and do not use unless you want to delete all tags in the current project

Brandwatch Lists

Note: to avoid ambiguity between the python data type "list" and a Brandwatch author list, site list, or location list, the latter is referred to in this demo as a "Brandwatch List."

BWAuthorLists, BWSiteLists, BWLocationLists work almost identically.

First, instantiate your the object which contains the Brandwatch Lists in your project, with your project as a the parameter. This will load the data from your project so you can see what's there, upload more Brandwatch Lists, edit existing Brandwatch Lists, and delete Brandwatch Lists from your project

Printing out ids will show you the Brandwatch Lists (by name and ID) that are currently in your project.

To upload a Brandwatch List, pass in a name as a string and the contents of your Brandwatch List as a list of strings. The keyword "authors" is used for BWAuthorLists, shown below. The keyword "domains"is used for BWSiteLists. The keyword "locations" is used for BWLocationLists.

To see the contents of a Brandwatch List, call get_list with the name as the parameter

Uploading is done with either a POST call, for new Brandwatch Lists, or a PUT call, for existing Brandwatch Lists, where the ID of the Brandwatch Lists is mantained, so if you upload and then upload a list with the same name and different contents, the first upload will create a new Brandwatch List, and the second upload will modify the existing list and keep its ID. Similarly, you can change the name of an existing Brandwatch List by passing in both "name" and "new_name"

To add items to a Brandwatch List without reentering all of the existing items, call add_items

To delete a Brandwatch List, pass in its name. Note the ids before the Brandwatch List is deleted, compared to after it is deleted. The BWLists object is updated to reflect the Brandwatch Lists in the project after each upload and each delete

The only difference between how you use BWAuthorlists compared to how you use BWSiteLists and BWLocationLists is the parameter which is passed in.

BWAuthorlists:

authors = ["edward albee", "tenessee williams", "Anna Deavere Smith"]

BWSiteLists:

domains = ["github.com", "stackoverflow.com", "docs.python.org"]

*BWLocationLists:

locations = [{"id": "mai4", "name": "Maine", "type": "state", "fullName": "Maine, United States, North America"},
{"id": "verf", "name": "Vermont", "type": "state", "fullName": "Vermont, United States, North America"},
{"id": "rho4", "name": "Rhode Island", "type": "state", "fullName": "Rhode Island, United States, North America"} ]

*Requires dictionary of location data instead of a string

Rules

Instantiate a BWRules object by passing in your project as a parameter, which loads all of the rules in your project.

Print out names and IDs to see which rules are currently in your project.

Every rule must have a name, an action, and filters.

The first step to creating a rule through the API is to prepare filters by calling filters().

If your desired rules applies to a query (or queries), include queryName as a filter and pass in a list of the queries you want to apply it to.

There are over a hundred filters you can use to only download the mentions that qualify. See the full list in the file filters.py. Here, different filters are used, which take different data types. filters.py details which data type is used with each filter. Some filters, like sentiment and xprofession below, have a limited number of settings to choose from. You can filter many things by inclusion or exclusion. The x in xprofession stands for exclusion, for example.

If you include search terms, be sure to use nested quotes - passing in "cat food" will result in a search that says cat food (i.e. cat AND food)

The second step is to prepare the rule action by calling rule_action().

For this function, you must pass in the action and setting. Below I've used examples of adding categories and tags, but you can also set sentiment or workflow (as in the front end).

If you pass in a category or tag that does not yet exist, it will be automatically uploaded for you.

The last step is to upload!

Pass in the name, filters, and action. Scope is optional - it will default to query if queryName is in the filters and otherwise be set to project. Backfill is also optional - it will default to False.

The upload() function will automatically check the validity of your search string and give a helpful error message if errors are found.

You can also upload rules in bulk. Below we prepare a bunch of filters and actions at once.

When uploading in bulk, it is helpful (but not necessary) to use the rules() function before uploading in order to keep the dictionaries organized.

As with other resources, we can delete, delete_all or clear_all_in_project

Signals

Instantiate a BWSignals object by passing in your project as a parameter, which loads all of the signals in your project.

Print out ids to see which signals are currently in your project.

Again, we can upload signals individually or in batch.

You must pass at least a name, queries (list of queries you'd like the signal to apply to) and subscribers. For each subscriber, you have to pass both an emailAddress and notificationThreshold. The notificationThreshold will be a number 1, 2 or 3 - where 1 means send all notifications and 3 means send only high priority signals.

Optionally, you can also pass in categories or tags to filter by. As before, you can filter by an entire category with the keyword parentCategory or just a subcategory (or list of subcategories) with the keyword category. An example of how to pass in each filter is shown below.

Signals can be deleted individually or in bulk.

Patching Mentions

To patch the metadata on mentions, whether those mentions come from queries or from groups, you must first instantiate a BWMentions object and pass in your project as a parameter.

if you don't want to upload your tags and categories ahead of time, you don't have to! BWMentions will do that for you, but if there are a lot of differnet tags/categories, it's definitely more efficient to upload them in bulk ahead of time

For this example, i'm arbitrarily patching a few of the mentions, rather than all of them