Dynamically update a sprite from a websocket

Hi everyone,

I’m facing an issue that I can’t figure how to sort it out. The context is this: I have setup a websocket which receives an image encoded in base64 (inside a JSON). When I receive this image what I would like to do is to update the previous sprite content with the image that I’ve just received. I know that in Phaser I’m supposed to load all my assets before the whole game runs, but for this particular use case I really need to re-display a map that is computed elsewhere and is asynchronously sent to the frontend. Here is a snippet that only works the first time the map is received (this snippet is part of a larger class, but the relevant code is shown below):

onMapUpdate(message) {
    let map = null;

    try {
        map = JSON.parse(message.data)['map'];
    } catch (error) {
        console.error(`Error - Couldn't parse JSON: ${message.data}. Details: ${error}.`);
    }

    if (map) {
        this.scene.textures.addBase64('map', `data:image/png;base64,${map}`);

        if (this.scene.mapSprite !== null) {
            this.scene.mapSprite.setTexture('map');
        } else {
            this.scene.mapSprite = this.scene.add.sprite(
                this.scene.settings.grid.center.x,
                this.scene.settings.grid.center.y,
                'map'
            );
        }
    }
}

The onMapUpdate method is called every time the websocket receives a message. The first time this code runs it displays the map, but this only happens once as the next time the method is called this.scene.textures.addBase64 tries to create another texture under the ‘map’ key (which is duplicated from the previous call), and Phaser gives back a duplicated key error message in the console which is reasonable. Now to the above snippet what I’ve done to try to fix the duplicated texture is this:

onMapUpdate(message) {
    let map = null;

    try {
        map = JSON.parse(message.data)['map'];
    } catch (error) {
        console.error(`Error - Couldn't parse JSON: ${message.data}. Details: ${error}.`);
    }

    if (map) {
        if (this.scene.textures.exists('map')) {
            this.scene.textures.remove('map');
        }

        this.scene.textures.addBase64('map', `data:image/png;base64,${map}`);

        if (this.scene.mapSprite !== null) {
            this.scene.mapSprite.setTexture('map');
        } else {
            this.scene.mapSprite = this.scene.add.sprite(
                this.scene.settings.grid.center.x,
                this.scene.settings.grid.center.y,
                'map'
            );
        }
    }
}

which basically checks for an existing ‘map’ texture and deletes it if that’s the case and then it does exactly the same logic presented in the first snippet (which creates or updates a sprite accordingly). If I do this, then I don’t get any errors in the console but the rendering is broken as Phaser shows me a small black box with a green line and a green border. I do know that the image I’m sending is fine because it is displayed correctly in the first snippet and also because I have taken the base64 payload and loaded manually into an img element in the DOM and it renders ok. I’m running on Firefox v101.0.1 and using Phaser version 3.55.2. I found little or no information on internet regarding manipulating sprites this way.

Two things I forgot to mention. In the above snippets the type of this.scene is Phaser.Scene and the this.scene.mapSprite sprite object is initialized to null.

At this point my guess is that this could be a rendering issue maybe due to the asynchronous behavior when receiving the image but I’m not sure because if that were the case then I’d expect to have the same little black box with the green border on both snippets. Has anyone faced this use case before? By this I mean receiving images from a websocket and update them on the fly in Phaser. Maybe I’m missing something silly but from the texture object documentation I couldn’t spot anything that could be of use to solve this problem.

Thanks everyone for your time,
Lucas.

addBase64() doesn’t complete right away, so you have to do something like

// once() should be fine if adding one texture at a time
this.scene.textures.once('addtexture', (key) => {
  if (key === 'map') {
    this.scene.mapSprite.setTexture('map');
  } else {
    throw new Error('Wrong key: ' + key);
  }
});

(Corrected: 'addtexture' event)

Actually the load event seems better:

this.scene.textures.once('onload', () => {
  this.scene.mapSprite.setTexture('map');
});
1 Like

Hi @samme!

Thanks for your reply, that actually seems to be working (for a few seconds!) :smiley:. One small correction to your snippet, I think this.scene.textures.once should be called with: addtexture instead of add. Also you added the “…should be fine if adding one texture at a time”, so can I step into troubles if doing this for more textures? In the last snippet you actually suggest to listen for the onload event, but it seems that it doesn’t receive the key of the texture that was added, I’m just wondering if this event fires on any texture that was added… Why do you consider it better?

Note aside: Ok, I did a bit of testing and both solutions seem to blow up Phaser. In Firefox after a few seconds the console displays:

Uncaught TypeError: frame.source is null
    batchSprite http://localhost:1880/static/js/robot_view.js:8559
    SpriteWebGLRenderer http://localhost:1880/static/js/robot_view.js:56896
    render http://localhost:1880/static/js/robot_view.js:27771
    render http://localhost:1880/static/js/robot_view.js:51269
    render http://localhost:1880/static/js/robot_view.js:16226
    render http://localhost:1880/static/js/robot_view.js:33256
    step http://localhost:1880/static/js/robot_view.js:53958
    step http://localhost:1880/static/js/robot_view.js:29824
    step http://localhost:1880/static/js/robot_view.js:29885
    ....

Brave displays another error:

robot_view.js:8559 Uncaught TypeError: Cannot read properties of null (reading 'isGLTexture')
    at MultiPipeline2.batchSprite (robot_view.js:8559:54)
    at Sprite2.SpriteWebGLRenderer [as renderWebGL] (robot_view.js:56896:28)
    at WebGLRenderer2.render (robot_view.js:27771:25)
    at CameraManager2.render (robot_view.js:51269:30)
    at Systems2.render (robot_view.js:16226:30)
    at SceneManager2.render (robot_view.js:33256:25)
    at Game2.step (robot_view.js:53958:28)
    at TimeStep2.step (robot_view.js:29824:22)
    at step (robot_view.js:29885:25)

It looks like something does not get updated properly…

Do you think it could be missing something else?

Again thanks for your insight!

Yes, it should be 'addtexture', thank you.

You can use the key argument in the onload event also, if you like. onload is emitted only for Base64 textures, so it’s a little safer for once(). once() should work fine as long as there are no overlapping calls to addBase64().

For the error, you need to do something like

this.scene.mapSprite.setTexture('default');

when you remove the texture. If you need it more seamless then you should probably use a dynamic texture key so you can swap them immediately:

this.scene.textures.remove('map1');
this.scene.mapSprite.setTexture('map2');
1 Like

Thanks once again for taking the time to reply @samme! Actually what you’re describing I tried out yesterday, but of course as I was getting the same initial error, I discarded that alternative prematurely. I will give another try to this “double-buffering” (if it could be correctly described as that). I’m wondering why Phaser won’t automatically set the default texture for those sprites that are still referencing a texture that was removed, as (I’m assuming Phaser) knows all its objects. It’s working as expected now. Once again thank you very much for your time!

Regards,
Lucas.

1 Like