Image

The standard API for working with images on canvas is rather laborious, and often takes the fun out of creative coding. In Pts, the Img class simplifies the common use cases, from loading and displaying static images to generating dynamic textures, so that you can get started quickly. Let's take a look.

Loading and Displaying Images

We will start a minimalistic example: Load an image and display it on canvas. This can be done in 2 lines of code:

const img = await Img.load( "/assets/demo.jpg" );
space.add( time => form.image( space.pointer, img ) );

Image credit: "C 50 Last Birds And Flowers" by Kurt Schwitters

The above example uses the static function Img.load, which returns a Promise that resolves to the loaded image, and then uses CanvasForm's image function to display it. A load failure rejects the Promise.

You can also create an Img instance yourself and call the instance function load, which is handy when you want to configure the instance first. An example:

(async function() {
  let img = await new Img().load( "/assets/img_demo.jpg" );
  space.add( time => form.image( space.pointer, img ) );
})();

Once the image is loaded, you can access its properties like width and height and manipulate its data. We will be discussing these advanced use cases next.

In this example, we access the image's original width and height after it's loaded, and then rescale it to fit the canvas size.

Editing Images

When you create an Img instance with its editable parameter set to true, it will hold an internal canvas to support image manipulations. It will also match the pixel-density of your display. An example:

// Create an editable img with the current space's pixelScale
let img = new Img( { editable: true, pixelScale: space.pixelScale } );
img.load( "/assets/demo.jpg" ).then( ... );

// Alternatively, pass the options to the static load function
let img2 = await Img.load( "/assets/demo.jpg", { editable: true, pixelScale: space.pixelScale } );

You can do a lot with an editable image. Let's cover a couple common use cases.

Get Pixels and Crop Regions

The pixel function supports a very common use case: specify a pixel position on the image, get its RGBA color values, and do something with it. A wide range of visual possibilities may open up if you use this simple function creatively.

Try scribbling in different regions of the image to change it. This demo combines Create.delaunay with Img.pixel.

Another common use case is to crop a region of the image. The crop function takes a bounding box and returns an ImageData. You can then use CanvasForm's imageData to draw the region.

form.imageData( [0, 0], img.crop( bound ) );

Let's try this in a demo:

Click to cut out a region in the image. Move pointer to shift its position.

It's more efficient to draw ImageData directly on canvas. If needed, you can also export it to a blob using Img.imageDataToBlob and then load it into an image again.

Edit and Sync

Since an editable Img stores an internal canvas, you can leverage CanvasForm's many drawing functions to draw directly on it. It's that easy!

After the image is loaded, you can use getForm to create a CanvasForm for its internal canvas. For example:

const img = await Img.load( "demo.jpg", { editable: true } );
const imgForm = img.getForm();
if (!imgForm) throw new Error( "Expected an editable image" );
...
imgForm.fill("#f00").rect( rect );

The following is a demo of drawing rectangles with matching pixel colors on the image canvas.

Move pointer to draw patches on the image canvas.

Additionally, the filter function supports image filter effects like desaturation and blur (See the full list supported by canvas). Note that some effects may not work in mobile browsers.

img.filter( "blur(10px) contrast(20%) saturate(0%)" )

To display the edited image, use CanvasForm's image function and pass img.current as the image source.

// draw internal image canvas
form.image( [0, 0], img.current );

As we are only editing an internal canvas, the original image is unchanged until it's explicitly updated. Use sync, which returns a Promise, to update the original image when needed: await img.sync().

You can also work at the pixel level: setPixel writes a color into the cached pixel data, updatePixels applies those changes onto the canvas, and loadPixels refreshes the cache after you've drawn on the canvas directly. When you're done with an Img, call dispose to release its resources.

Patterns

In a similar way, you can treat an image (or an image canvas) as a pattern to fill an area. One difference is that we'll get a CanvasPattern instance for use in form.fill(...), instead of an image for form.image(...).

const pattern = await Img.loadPattern( "tile.jpg", space );
...
form.fill( pattern ).rect( rect );

Loading an image and filling it as a pattern.

A pattern can be transformed via the standard canvas API pattern.setTransform. However, its documentation is confusing and incomplete. Pts provides an easy way to create a DOMMatrix for this use case.

const m = new Mat().translate2D( ... ).rotate2D( ... ).domMatrix;
pattern.setTransform( m );

Applying transforms to the pattern. Hover to rotate the pattern.

All together, the Img class offers a wide range of potential creative expressions. For example, you can create a dynamic image and use it as a pattern fill (As shown in this demo).

It's now your turn to experiment!

Tips and Tricks

• Anticipate screens with different pixel density. You can pass a CanvasSpace's pixelScale when creating an Img instance. (See example in Cheatsheet below)

• You can load an image from a base64 string or an url, or from a blob via the fromBlob function. To export the current image, use toBase64 or toBlob functions.

• CanvasForm's image drawing function can take either an Img instance or a CanvasImageSource which includes various kinds of image objects like HTML Image or Canvas.

• Typically, you can't load an image from another domain due to security concerns. But if the image server allows for it and you want to do it, you can set the crossOrigin parameter to true when creating an Img instance. More details here.

Cheatsheet

Creating, loading, displaying

// Simplest way
let img = await Img.load( "demo.jpg");

// Load an editable image that matches the screen's resolution
let img = await Img.load("demo.png", { editable: true, pixelScale: space.pixelScale } );

// Equivalent, creating the instance first
let img = new Img( { editable: true, pixelScale: space.pixelScale } );
await img.load("demo.png")

// Display an image automatically when it's loaded
form.image( [0,0], img );

// Load a pattern and use it as fill
let pattern = await Img.loadPattern( "tile.jpg", space );
form.fill( pattern ).rect( rect );

// Get a pattern from an Img instance
const pattern = img.pattern();
form.fill( pattern ).rect( rect );

Useful properties

img.loaded; // true if the image is loaded
img.image; // the original image
img.canvas; // the internal canvas of an editable Img
img.ctx; // the context which can be used to create a CanvasForm
img.pixelScale; // pixel density which usually matches the space's

Editing an image

img.crop( rect )
img.resize( [0.5, 0.5], true );
img.filter( "blur(10px) contrast(200%)" );
img.pixel( space.pointer );

// Draw on image
let imgForm = new CanvasForm( img.ctx );
imgForm.fill( "#f00" ).point( space.pointer, 20 );

// Export as base64 string
img.toBase64();

// Getting a DOMMatrix instance for pattern transforms
const m = img.scaledMatrix.rotate2D(...).domMatrix;
pattern.setTransform( m );
form.fill( pattern ).rect( rect );