Found it, here is the code in case anyone runs into the same issue:
private loadBase64Atlas(key: string, data: string, json: object): void {
const imageElement = new Image();
imageElement.onload = () => {
this.scene.textures.addAtlas(key, imageElement, json);
this.onAssetLoaded();
};
const spriteBlob = this.base64ToBlob(data.split(',')[1], 'image/png');
imageElement.src = URL.createObjectURL(spriteBlob);
}
See here on how to create the blob from base64. The JSON passed into the function can just be a normal JSON object.