Conductor Searchlight API

The Searchlight API provides secure, programmatic access to data in Searchlight accounts. This API allows users to see the current configuration of their accounts and request reports related to those accounts.

Users can access the Searchlight API through HTTPS requests against the endpoints described at the end of this document.

Contents

Notes on Using the Searchlight API

Before you get started, review the following information to help make the process go smoothly.

How do I Authenticate My Requests to the Searchlight API?

The Conductor Searchlight API uses a signed URL parameter to securely authenticate users. Here’s what you need to compute and submit a signed API request:

  • API URL. This is the URL your client code is trying to access. This is typically received by navigating through results returned by other API calls, beginning with the “Accounts List” call (more on this later).
  • API Key. This parameter identifies a unique user, similar to a username. Authorized Searchlight users can obtain API keys when they register with the Searchlight API at http://developers.conductor.com/.
    The API key is secured by SSL as long as communication occurs over HTTPS.
  • Signature. This parameter, computed by and attached to each API request by the client code, is based on a temporary shared secret. Authorized Searchlight users obtain the shared secret at the same time they obtain an API key after registering with the Searchlight API at http://developers.conductor.com/.

Typical client code reads the API Key and shared secret from their own local database or other configuration source (config file, environment variables, etc.).

Building a Signed URL

Searchlight cannot authenticate a request made to an API endpoint without a signed URL built by the client code. For example, a request to the “Accounts List” endpoint at https://api.conductor.com/v3/accounts would need to be made using a URL like the following: https://api.conductor.com/v3/accounts?apiKey=3487yuyb9sjvb7634987y&sig=jkadscjhbkuqywgqwe98134098124.

Building a signed URL like the one above includes:

  1. Starting with the base URL
  2. Appending the apiKey parameter to the URL with a value equal to the user’s API key
  3. Computing a signature from the shared secret and the current system time
  4. Appending this signature as the value of the sig parameter to the URL

Completing steps 1, 2, and 4 is straightforward and can be done with a simple string concatenation or with your programming environment’s URL manipulation libraries.

Next, let’s focus on generating the signature described in step 3.

Generating the Signature

A signature is a one-way MD5 hash of three values:

  • The user's API key (apiKey)
  • The user's shared API secret (sharedSecret)
  • The current time expressed as seconds since the UNIX epoch (unixEpochTimeSeconds)

Your programming environment must provide the ability to do the following, which is part of the standard library for most programming languages:

  • Get the current UNIX time in seconds.
  • Compute MD5 hashes.

To allow for reasonable clock drift between clients and Conductor's servers, the Searchlight API considers a signature matching if it was signed within 5 minutes of the time on Conductor's servers.

Example Code

Review the samples below for demonstrations of how to compute the signature to use in step 3 above:

Python

import time

import hashlib

import sys

def gen_api_sig(api_key, shared_secret):

m = hashlib.md5()

m.update(api_key)

m.update(shared_secret)

epoch_sec = int(time.time())

m.update(str(epoch_sec))

digest = m.digest()

hex_digest = digest.encode('hex')

return hex_digest

#This is simply for demonstration of how to invoke this from the command line

#under the assumption that this is invoked with the arguments "api_key" and

#"shared_secret". In production scenarios, this is probably not advisable

#because in standard system configurations it will leave the shared secret

#visible in the process list and / or command history, accessible

#to any user on the system with appropriate privileges

if __name__ == '__main__':

api_key = sys.argv[1]

shared_secret = sys.argv[2]

sig = gen_api_sig(api_key, shared_secret)

print "apiKey=%s&sig=%s" % (api_key, sig)

Java

import java.security.MessageDigest;

import java.security.NoSuchAlgorithmException;

String generateSignature(final String apiKey, final String sharedSecret)

throws NoSuchAlgorithmException {

final String stringToHash = apiKey + sharedSecret + reqEpochSec;

final MessageDigest md = MessageDigest.getInstance("MD5");

final byte[] digestBytes = md.digest(stringToHash.getBytes());

// this avoids dependency on commons.codec.binary,

//which could have classpath issues

final StringBuffer sb = new StringBuffer();

for (int i = 0; i < digestBytes.length; i++) {

sb.append(Integer.toString((digestBytes[i] & 0xff) + 0x100, 16).substring(1));

}

final String md5 = sb.toString();   return md5;

}

Other Java Examples

Refer to the Conductor java api-examples repo on Github

What Kind of Content Types Should I Expect from the Searchlight API?

Responses through the Searchlight API are available as JSON. To achieve the highest performance and minimize unnecessary requests, clients should do the following, unless otherwise specified:

  • Set their “Accept” header to “application/json”.
  • Expect responses to have the “Content-Type” header with value “application/json.” Responses might include standard HTTP Cache-related or ETag headers. 

What about Redirects?

For maximum flexibility, reliability, and performance, the Searchlight API may use standard HTTP redirect functionality, such as 301 or 302 redirects. Any HTTP clients should be configured to allow for these redirects.

Best Practices for Using the Searchlight API

Data Warehousing

The Searchlight v3 API is a data warehousing API. It is optimized for bulk transfers of data from one system to another where the destination system, such as a MySQL database, is capable of processing that data without loading it all into memory. As such, it is not well suited for online in-memory processing of that data, such as an "interactive app" use case.

We recommend clients use a mechanism for transferring bytes between systems without materializing or parsing it.

For example, you might use curl or wget, and pipe the output to a file. The pattern here is a stream processor that loads data into a backing store (like a relational database) for further querying and analysis, instead of a system that attempts to materialize and work with the entire dataset in memory. Using Postman or most other naive JSON parsers that attempt to parse or materialize the entire dataset is generally an anti-pattern, and might cause client performance issues.

Below are some example scripts demonstrating possible approaches:

Bash

export API_KEY=<<insert your Conductor API key here>>; export API_SECRET=<<insert your Conductor API secret here>>;

function fetch_conductor_api_url() {     URL="$1";     EPOCH_SEC=$(date +%s) SIG=$(echo -n ${API_KEY}${API_SECRET}${EPOCH_SEC} | md5)

SIGNED_URL="${URL}?apiKey=${API_KEY}&sig=${SIG}";

curl ${SIGNED_URL} | jq '.'

}

Java

Refer to the Conductor java api-examples repo on Github

Suggested Procedure

To support the kind of data warehouse use case we suggest you use, we recommend the following procedure, weekly:

  1. Retrieve the following endpoints and upsert them into corresponding tables using each item’s identifier string as a unique key:
    • Rank source
    • Location
    • Device lists
    Although items will never be removed from these lists, new items may be added and existing descriptive information may change.
  2. Retrieve the account list, and for each active account you want to reference, use the provided webPropertyList URL to retrieve the corresponding client web properties. Upsert them into a corresponding table using their identifier strings as unique keys.
    If you want, you can maintain their respective comparison web properties, each specific to a given rank source, in separate tables. Note that the same comparison web property may appear under multiple combinations of client web property and rank source, and with different labels.
  3. For each client web property, use the included trackedSearchList URL to retrieve the currently configured tracked searches. Use the rank source, location, and device lists to interpret the data associated with each tracked search.
  4. For each combination of client web property and rank source, use the included webPropertyRankReport URL to access the Client Web Property Rank Report. Use the tracked search and web property lists to interpret the data associated with each report item.
    The Endpoint section of this document discusses the format and interpretation of the fact data.

We've put together sample Java API code that demonstrates the points above. Find that here.

Interactive Endpoint Documentation

You can use our interactive API documentation to discover, test, and debug live calls. You’ll also find links to detailed endpoint documentation, which you can also review below.

Endpoints

Below are links to documentation for each endpoint.

About Conductor

Conductor is the global leader in SEO technology. Thousands of brands and enterprise marketers use its cloud-based SEO platform, Conductor Searchlight™ to measure, improve and manage their natural search efforts to stay ahead of the competition, increase site traffic and boost revenues.

One of New York City’s fastest growing and most admired technology companies, Conductor is routinely recognized for excellence in business and corporate culture. To learn more, go to www.conductor.com or follow Conductor on Twitter @conductor.

Learn More