JSON

Following on from this post, I thought it might be good to think about JavaScript Object Notation (JSON).

Despite its name, JSON has evolved to be agnostic and is arguably the standard for exchanging information because its human readable and easy for computers to parse.

JSON is two things: a collection of key value pairs, and an ordered list of values. The former is easily represented with objects, dictionaries, etc. and he latter with arrays, lists, or sequences.

  • Values are string, number, object, array, true/false, or null surrounded by white-space.
  • Objects are unordered key-value pairs, represented by colon separated key-value pairs and wrapped in curly brackets.
  • Arrays are comma separated ordered collections wrapped in square brackets.
  • Comments, trailing commas, and single quoted strings are not supported.
{
    "name" : "Dougie",
    "languages" : [ "python", "java" ]
}

Python has a standard module, json, making it simple to convert Python and JSON data types. Serialisation is converting to JSON, but take care with de-serialisation as it is not quite the reverse. Not all Python data types can be converted to JSON values, so objects may not retain their original type.

This can introduce subtle problems when serialising and de-serialising:

  • JSON requires keys to be strings, so where another type is given during serialisation, it will be a string when de-serialised.
  • True serialises to true, and None serialises to null. The reverse is also true.
  • Tuple are serialised as arrays, there is no way to tell when de-serialising so you will get a list.

Serialising is done by dumping:

json.dumps(python_data)

Deserialising JSON is done by loading:

json.loads(json_data)

JSON is human readable but can be made easier to read by specifying indentation level:

json.dumps(sample_json, indent=2)

Interact with API in Python

Overview of REST

Representational State Transfer (REST) is a client server communication pattern, which defines some constraints:

  • Stateless
  • Decouple server from client
  • Should be cacheable.
  • Uniform interface
  • Layered (access directly or indirectly via proxy or load balancer).

Any service that adhere to this is known as a REST web service and provides access via REST API, which listens for HTTP methods and responds with an HTTP response. These API expose public URI, or endpoints, performing different actions dependent on the method.

Python and REST

I’ll do a separate post on writing API, because its more complicated and there are several frameworks (Flask and FastAPI for example), today we’ll focus on consuming API.

Pre-requisites

We’ll be using the requests library. As usual we’ll create a virtual environment:

python -m venv rest_env
source vest_env/bin/activate
python -m pip install requests

Now we need an API to consume, so lets use NASA’s public API. Discovering API isn’t something we can easily do programmatically, so we’re reliant on the API’s documentation. It’s also likely that responses will be JavaScript Object Notation (JSON) and we’ll consider how we parse that data a bit later on.

We’re always going to need import requests and specify a base URL, we’ll use Astronomy Picture of the Day (APOD):

import requests
api_url = "https://api.nasa.gov/planetary/apod"

Now we create a response object depending on the HTTP method:

response = requests.get(api_url)

We want to check the response:

>>> response.status_code
403

But wait, we want 200 (success) and we got 403 (forbidden) – the API requires authentication parameters!

Query string

This is the part of the URL that assigns values to a parameter. The APOD API requires a key as a parameter, you can request one here but I’ll use the demo key here. We’ll also request high definition images while we’re at it:

api_key = 'DEMO_KEY'
params = {
    'api_key':api_key,
    'hd':'True'
}
response.get(api_url, params=params)

Now when we check the status:

>>> response.status_code
200

Success!

Parsing the response

How do we get a specific attribute? Our response is in JSON. We can make this easier to work with by converting the response to a dictionary:

>>> import json
>>> response_dict = response.json()
>>> print(json.dumps(response_dict, indent=4, sort_keys=True))
{
    "date": "2025-09-14",
    "explanation": "How does your favorite planet spin? Does it spin rapidly around a nearly vertical axis, or horizontally, or backwards?  The featured video animates NASA images of all eight planets in our Solar System to show them spinning side-by-side for an easy comparison. In the time-lapse video, a day on Earth -- one Earth rotation -- takes just a few seconds.  Jupiter rotates the fastest, while Venus spins not only the slowest (can you see it?), but backwards.  The inner rocky planets across the top underwent dramatic spin-altering collisions during the early days of the Solar System.  Why planets spin and tilt as they do remains a topic of research with much insight gained from modern computer modeling and the recent discovery and analysis of hundreds of exoplanets: planets orbiting other stars.",
    "media_type": "video",
    "service_version": "v1",
    "title": "Planets of the Solar System: Tilts and Spins",
    "url": "https://www.youtube.com/embed/my1euFQHH-o?rel=0"
}

Now that we’ve a dictionary, getting the required attribute sis trivial:

>>> print(response_dict['url'])
https://www.youtube.com/embed/my1euFQHH-o?rel=0

POST and PUT

POST requests a new resource but PUT replaces a resource and creates it if it doesn’t exist, in other words PUT is idempotent where as POST is not.

JSON is also how we provide payloads. Lets take a look at an example that expects a product name and a quantity:

payload={"product_name": "Milk", "quantity": 1}
response = requests.put(api_url, json=payload)

Next time

We’ll follow this post up with a look at authentication, as most API will require some form of authentication.

Setup a Multipass CDK Environment

I want to be able to connect to the environment using Visual Studio Code, so first we need to create a SSH key:

ssh-keygen -t rsa

We need a configuration YAML, replace <generated ssh-rsa key> with the above key, saved as cloud-init.yaml:

groups:
  - vscode
runcmd:
  - adduser ubuntu vscode
ssh_authorized_keys:
  - ssh-rsa <generated ssh-rsa key>

Assuming you’ve got Multipass installed (if not sudo snap install multipass) then:

multipass launch mantic --name ubuntu-cdk --cloud-init

We’ll come back to Visual Studio Code later but first lets set everything up in the VM. We need to install aws-cli which I want to use with SSO (hence why we installed Mantic).

multipass shell ubuntu-cdk
sudo apt install awscli
aws configure sso

Follow the prompts and sign in to AWS as usual. Then install CDK:

sudo apt install nodejs npm
sudo npm install -g aws-cdk

Almost there, lets bootstrap1 (provisioning resources needed to make deployments) substituting the relevant values:

cdk bootstrap aws://<account>/<region> --profile <profile>

You should see a screen like this:

Create a new CDK application by creating a new folder, changing into it and initialising CDK:

cdk init app --language python
source .venv/bin/activate
python -m pip install -r requirements.txt

And that’s about it, except for Visual Studio Code. You’ll need to install Microsoft’s Remote-SSH extension:

You can get the IP address from multipass list, then in Code add a new SSH connection using ubuntu@<ip>:

Accept the various options presented and you’re there!

VSCode
  1. Bootstrapping provisions resources in your environment such as an Amazon Simple Storage Service (Amazon S3) bucket for storing files and AWS Identity and Access Management (IAM) roles that grant permissions needed to perform deployments. These resources get provisioned in an AWS CloudFormation stack, called the bootstrap stack. It is usually named CDKToolkit. Like any AWS CloudFormation stack, it will appear in the AWS CloudFormation console of your environment once it has been deployed. ↩︎