Legiscan API - INFO-664 notebooks

The Legiscan API provides access to legislative data, such as bills, sponsors, hearing information, etc, from all 50 states. To use the Legiscan API, you’d first need to make an account on Legiscan, and request an API key. The key (which we will incorporate into our API call) gives you authentication to access the data on Legiscan.

After getting our API key, we can create our API call. The first step is import our libraries that we need in order to use the API. Then, we can construct our API call, putting together the root, key and query.

import requests
import pandas as pd
import time

# the components of the API call, which make up the "request" variable
url = 'https://api.legiscan.com/?key='
key = # insert your key here
page = 1
request = requests.get(url + key + '&op=getSearch&state=ALL&query=transgender' + '&page=' + str(page))

# to print out the full URL, and we can paste in the browser to get 
# an interactive look at the raw results
print(url + key + '&op=getSearch&state=ALL&query=transgender' + '&page=' + str(page))

Now we can make the API call, using the “request” object that we created above. From there, we call the .json() method, to navigate through the results, which are in json format.

Accessing items in json involves using brackets to indicate the keys.

# get the page_total and the count from the request summary
page_total = request.json()['searchresult']['summary']['page_total']
count = request.json()['searchresult']['summary']['count']
print('Page total: ' + str(page_total) + '\n' + 'Total results: ' + str(count))

Page total: 13
Total results: 604

Now we write a few loops. The first one gathers all the data from each page of the results. The second loop parses that data into a dataframe object (a tabular or spreadsheet format) so we can examine it and eventually save it to a csv file.

# request the additional pages of the query by adding 1 to the 'page' 
# parameter until it reaches the page_total. Store each page of  
# requests in a list. Wait 3 seconds between each request to avoid 
# overloading the API

pages = []
for i in range(page_total):
    page = i + 1
    request = requests.get(url + key + '&op=getSearch&state=ALL&query=transgender' + '&page=' + str(page))
    time.sleep(3)
    pages.append(request.json())

# for each page of the request, parse the results and add them to a 
# dataframe. each page is a json file with individual results labeled
# '0' through '49' and # 'summary' nested under searchresult. Ignore 
# the summary and use pandas.concat to add each of the results from 
# the request in a dataframe

df = pd.DataFrame()
for page in pages:
    results = page['searchresult']
    # if the page has no results, skip it
    for i in range(50):
        if str(i) in results:
            df = pd.concat([df, pd.DataFrame(results[str(i)], index=[i])])
        else:
            continue

Now we have our data in a tabular structure, thanks to the DataFrame that we got from a different library called pandas. This DataFrame format enables us to examine our data in a spreadsheet.

df

df.info()

<class 'pandas.core.frame.DataFrame'>
Int64Index: 604 entries, 0 to 3
Data columns (total 11 columns):
 #   Column            Non-Null Count  Dtype 
---  ------            --------------  ----- 
 0   relevance         604 non-null    int64 
 1   state             604 non-null    object
 2   bill_number       604 non-null    object
 3   bill_id           604 non-null    int64 
 4   change_hash       604 non-null    object
 5   url               604 non-null    object
 6   text_url          604 non-null    object
 7   research_url      604 non-null    object
 8   last_action_date  604 non-null    object
 9   last_action       604 non-null    object
 10  title             604 non-null    object
dtypes: int64(2), object(9)
memory usage: 56.6+ KB

df.to_csv('legiscan_api_results.csv')

That’s it!