Audio works on web but not iOS

Hi there

We are seeing the following errors when calling play()

   Audio cache entry missing: music1
   ...
   TypeError: undefined is not an object (evaluating 'this.currentConfig.seek=0')

We are using

  "@capacitor/ios": "^1.1.1",
  "phaser": "^3.19.0"

And we have this scene

  preload() {
      console.log("musicOn assets/audio/example3.wav")
        this.load.audio('music1', 'assets/audio/example3.wav');
        this.load.on('fileprogress', function (file) {
          console.log("fileprogress file="+file.key);
         }, this);

         // remove progress bar when complete
        this.load.on('complete', function () {
          console.log("loading complete")
        }, this);
   }

   create() {
      console.log("audio1")
      this.music1 = this.sound.add('music1');
      this.music1.play();
   }

This works fine on the browser. But when running it on the iOS device and simulator, I am seeing these errors

** [log] - musicOn assets/audio/example3.wav
** [log] - fileprogress file=music1
** [log] - loading complete
** [log] - audio1
** [warn] - Audio cache entry missing: music1
** [error] - ...
** ------ STARTUP JS ERROR ------
** TypeError: undefined is not an object (evaluating 'this.currentConfig.seek=0')
..

We tried different format mp3, ogg, m4a, wav. Nothing works.
Anyone knows what this could be? thanks

I got it working. I needed to add the following in game.js

  audio: {
    disableWebAudio: true
  },

Old topic, but for whomever ends hereā€¦ still broken in Q2 2021.
Disabling WebAudio to use HTML5 audio has a perf / stability hit. Moreover, playing multiple files seems bogus here.

Found this stackoverflow topic, seems the last answer might lead somewhere. Still looking to implement the ā€œsimple fetch & decodeā€ solution thoughā€¦

Open an issue with test case if you can.

Itā€™s strange, it really looks like you should get a Error decoding audio ā€¦ error in the console if decodeAudioData() fails.

Still trying to create a way to reproduce this in the browser. I can only make it work when the game runs on the device (through capacitor). Iā€™ll try to implement a workaround and share the result.

seems this issue is affecting other people using capacitor to wrap the Phaser game:

I hope Iā€™ll manage to at least write an ā€œalternateā€ loader. Looking for a way to bypass the Fileā€™s loading mechanism and have control over the ā€œfetchā€ method (an alternative to the XHR ā€œoldā€ way).

@samme Iā€™m making some progress here. Maybe this will give you a hint as per what goes wrong? Thank anyway for your attentionā€¦

This game has 8 separate mp3 sound files. Iā€™ve added some console logs to ensure they are present in the sound cache as the preload is complete:

create(config: any) {
    console.log(">>> Create start, count: ", this.game.cache.audio.entries.values().length);

On the mac (Chrome, Safari) I get the proper count as expected:

>>> Create start, count: 8

Capacitor allows two modes:

  • ā€œdev modeā€ where resources are served from the dev computer and the iOs app just loads them over the network.
  • ā€œnormalā€ mode where the iOs app contains the phaser game and its resources in a standalone package like those found on an app store.

While dev mode reports the correct number, the normal build doesnā€™t manage to load them. No errors, it just fails silently:

>>> Create start, count: 0

Now it gets interresting:

Iā€™m manually forcing to load the file using fetch in my scene preload:

if (this.sound instanceof Phaser.Sound.WebAudioSoundManager) {
  const that = this.sound;

  this.sound.on("decoded", (audiokey) => {
    console.log(">>> decode success of key " + audiokey);
    console.log("->>> new count: ", this.game.cache.audio.entries.values().length);
  });

  console.log(">>> Preload start, count: ", this.game.cache.audio.entries.values().length);
  fetch("assets/commons/sounds/default.mp3")
    .then((response) => response.arrayBuffer())
    .then((response) => {
      that.decodeAudio("debugme", response);
    });
}

Then I can play that specific sound, and itā€™s the only one to appear in the cache. The 8 other files are still missing:

  create(config: any) {
    console.log(">>> Create start, count: ", this.game.cache.audio.entries.values().length);

    setInterval(() => {
      this.sound.play("debugme"); // this plays on the iphone using the standalone build
    }, 1000);

And the console reports:

:zap: WebView loaded
:zap: [log] - >>> Preload start, count: 0
:zap: [log] - >>> decode success of key debugme
:zap: [log] - ->>> new count: 1
:zap: [log] - >>> Create start, count: 1

So using ā€œfetchā€ is a workaround but I donā€™t understand why it solves it. Moreover, my method just loads in parallel to the preload methodā€™s internals.

Let me know what you think!

Hi!

I opened an issue yesterday related with this topic: Web Audio does not work on iOS Safari Ā· Issue #5685 Ā· photonstorm/phaser Ā· GitHub

In my case it was because I was using ogg files, and iOS does not support this file type. Using ogg and MP3 as fallback works perfectly.

Maybe this can help you.

Best regards!

1 Like

Hi @albert-gonzalez thanks for the info. Iā€™m indeed working with mp3 files (also tried m4a, same result). In my case Capacitor is used to package the phaser game for iOS. And when the mp3 files are loaded using the standard File / MultiFile loader, XHR is used andā€¦ that fails. I canā€™t yet understand why.

How do you get your game to run on iOS - via browser, cordova/capacitor, orā€¦ ?

Ok so here is my workaround. I donā€™t call that a solution since it would imply writing a proper MultiFile loader. Still busy trying to figure that one out. At least it unblocks me releasing a beta demo of my little gameā€¦

TL;DR;

Short version:

  1. donā€™t use loader.audio if Webkit is detected
  2. donā€™t allow update to process anything until further notice.
  3. on each audio asset: use a fetch ā†’ convert to buffer ā†’ decodeAudio as shown below.
  4. register a listener: this.sound.on(ā€˜decodedā€™) to detect when resources are loaded. keep track of decoded count before allowing the update loop to process anything.
  5. donā€™t forget to unregister your ā€˜decodedā€™ event in the shutdown sequence of your scene.

Sorry @rich for desacrating your splendid code, but Iā€™m not good enough (yet?) with Phaser to write a proper MultiFile implementation that properly works during the preload phaseā€¦ I hope I can book a support session with you to help me implement a proper FileTypesManager.register("iOsCapacitorSafeAudio", ...)


Now the long version.

detecting webkit (capacitor or not, desktop or mobile, just does the trick):

window && typeof window.webkitConvertPointFromNodeToPage === "function"

ok,

This hack should help as a quick fix though.

Add these 5 members to the scene:

  private doneNormalPreload = false;
  private doneUglyPreload = false;
  private countToDecode = 0;
  private decodedCount = 0;
  private canLoop = false;

preload

Preloading happens like usual: images, jsons, tilesets, etc. except for sounds, if Webkit is detected (canā€™t detect capacitor, so, this does the trick).

it must however also reset the following and register an event listener on the sceneā€™s sound system:

  this.doneNormalPreload = false;
  this.doneUglyPreload = false;
  this.decodedCount = 0;
  this.canLoop = false;

  // every time an audio file is decoded we increment the counter till we reach the desired amount
  this.sound.on("decoded", (audiokey) => {
    this.decodedCount++;
    if (this.decodedCount === this.countToDecode) {
      this.uglyLoadComplete(); 
    }
  });

  // filter out the audio assets that are already loaded, otherwise the "decoded" event handler will run into trouble
  const newAudioAssets = audioAssets.filter((asset) => !webAudiosoundManager.game.cache.audio.entries.entries[asset.key]);

  // just in case the audio asset count to preload has dropped to 0
  this.countToDecode = newAudioAssets.length;
  if (this.countToDecode === 0) {
    this.uglyLoadComplete();
  }

Preloading what the audio files needs the fetch call as follows

  newAudioAssets.map(({ key, url }) =>
    fetch(url[0])
      .then((response) => response.arrayBuffer())
      .then((audioAsBuffer) => {
        // console.log("decoding " + key);
        return webAudiosoundManager.decodeAudio(key, audioAsBuffer);
      })
  );

create method

Move the contents of your create method in a new ā€œcreateAfterSafeAudioPreloadā€ method.

Create should only contain this:

    this.doneNormalPreload = true;
    if (this.doneUglyPreload) {
      this.createAfterSafeAudioPreload();
    }

Now both doneNormalPreload and doneUglyPreload must have been set to true in order to call that createAfterSafeAudioPreload

Add the ugly load complete method to the sceneā€™s body

This one makes sure to only start the game when normal preloading is done AND this hackish preloading is done:

  private uglyLoadComplete() {
    this.doneUglyPreload = true;
    if (this.doneNormalPreload) {
      this.createAfterSafeAudioPreload();
    }
  }

** Prevent the update() method from doing anything until told so

  update(t: number, dt: number) {
    if (this.canLoop) {
      super.update(t, dt);
      this.animatedTiles.updateAnimatedTiles();
      this.playerInput.update();
      this.processIrq();
    }
  }

This is because the create() method is called by the normal/clean preloader of Phaser and that schedules the scene update. So I made this ugly boolean check waste cycles every tick to prevent issuesā€¦

** Donā€™t forget to unregister the listener **

In the shutdown method, donā€™t forget to unregister on the sound member the following:

this.sound.off("decoded");

NB make sure to do it on this.sound, not this.events !!!

1 Like

Thank you so much for a long & detail workaround.
Iā€™ve a similar problem on Android. My game works on Android when using Cordova but not Capacitor.

Update: It works well on Andoird (both Capacitor & Cordova). Sorry for not upgrading my phaser to new version.

Well thatā€™s good to know; Iā€™m not upgrading from 3.24 yet as I need to stabilize my codebase. Seems however itā€™s going to be release in the next version as a fixed was merged a few weeks ago:

then thereā€™ll be no need for this workaround!

1 Like

upgrading is not always possible - especially in late stages of a project development. TBH I plan to amortise the dev costs (aka time) of my project to build a few other titles on its codebase, evolving a few features (minor upgrades, staying on 3.24 to avoid migrating custom plugins & co).

I only plan to upgrade my Phaser dependency in my ā€œnext genā€ engine based on Entiy Compo (and a better hindsight of ā€œlessons learnedā€ & co derived from my ongoing projects) with Phaser 4ā€¦ long term thing :wink:

1 Like